Chat/Commands: Increase output token limit for PLG #3797
Conversation
| : // Minus the character limit reserved for the answer token | ||
| this.chatModel.maxInputChars - tokensToChars(ANSWER_TOKENS) |
There was a problem hiding this comment.
This logic only makes sense for older GPT models where the input+output limit is calculated together.
For enterprise we will need to address this
umpox
changed the title
lib/shared/src/models/dotcom.ts
Outdated
| maxToken: 1800, | ||
| maxInputToken: 1800, |
There was a problem hiding this comment.
unrelated but it seems like we can use the "standard" 7k now :) https://fireworks.ai/models/fireworks/mixtral-8x22b-instruct-preview
vscode/src/models/utils.ts
Outdated
| /** | ||
| * @deprecated Use `inputTokens` instead. | ||
| */ | ||
| tokens?: number |
There was a problem hiding this comment.
We could just remove it since this was not included in any release yet?
There was a problem hiding this comment.
Yep I can do, my only concern is that this might break any internal stuff we've configured but I guess it's still fresh so that's not much of a worry
|
|
|
@sourcegraph/cody-security Guessing this was triggered because of the merge commit? Maybe we should exclude those? |
|
@umpox Ah no this is because we look at the files and not just the modified ranges within those files. Shoots. This actually needs a much more sophisticated approach (we need to know which ranges were modified and only emit linter issues in those places). Hmmm maybe as a temporary workaround we can extract the I will research how other linters do that. Sorry for the churn @umpox and please ignore it for now in this PR 🥺 🙏 |
|
|
1 similar comment
|
|
|
:D @philipp-spiess No probs |
umpox
force-pushed
the
tr/increase-output-tokens
branch
from
6138f91
to
9b8a4af
Compare
Description
closes #3648
closes #3795
closes https://github.com/sourcegraph/cody-issues/issues/14
closes #3779
closes #3786
closes #3784
closes #3721
closes #3571
closes #3472
closes #3536
This PR:
1000to4000tokensNote
This PR requires that https://github.com/sourcegraph/sourcegraph/pull/61872 is merged first
Test plan