Skip to content

Commit

Permalink
more stable autocomplete (#442)
Browse files Browse the repository at this point in the history
Makes autocomplete not jitter as much as you type, with a cache and a
few other techniques to help increase the (1) cache hit rate and (2) VS
Code's ability to reuse and not invalidate existing completion results.

Why this change?

- Extract the non-VS Code-specific parts of `getInlineCompletions` so we
can separately test *it* and the `InlineCompletionItemProvider` logic.
- All test cases were ported to the new `getInlineCompletions` (and all
pass without modification in substance).
  - This also helps with extracting completions to the agent.
- Make the cache and RequestManager simpler and dumber by exploiting VS
Code's existing behavior (of showing w/o jitter a full-line inline
completion while it is refreshed in the background).
- Remove the existing cache entirely and replace it with a simpler cache
that just memoizes calls (inside RequestManager) and does not have any
other text logic.
- Make the RequestManager not need to support `logId` rewriting or
retesting caches for a document (see below).
- Separately handle the case of "typing the last suggestion" (e.g.,
typing as suggested by the ghost text) vs. the general case of cached
network requests. This case is the only one where we want to reuse the
previous `logId`, and we can do so simply instead of (if the 2 cases are
conflated) requiring the cache to support `logId` rewriting.

Examples of bugs or undesirable behaviors this fixes:

- Absolute minimum jitter when deleting or typing ghost text.
- As long as you are still typing ghost text exactly as suggested, it
won't replace it underneath you with a new completion.
- If you delete through ghost text past the initial trigger point, it
will always suggest something new.
- When you press delete on an empty but indented line with ghost text,
the ghost text remains in place (no jitter) until you type something
different.

## Test plan

Use completions and watch the logIds. Try typing or deleting over ghost
text.
  • Loading branch information
sqs authored Aug 2, 2023
1 parent ba5dcc5 commit 1f04aaa
Show file tree
Hide file tree
Showing 26 changed files with 2,168 additions and 1,803 deletions.
1 change: 0 additions & 1 deletion lib/shared/src/configuration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ export interface Configuration {
| 'unstable-azure-openai'
autocompleteAdvancedServerEndpoint: string | null
autocompleteAdvancedAccessToken: string | null
autocompleteAdvancedCache: boolean
autocompleteAdvancedEmbeddings: boolean
autocompleteExperimentalCompleteSuggestWidgetSelection?: boolean
pluginsEnabled?: boolean
Expand Down
5 changes: 0 additions & 5 deletions vscode/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -885,11 +885,6 @@
"type": "string",
"markdownDescription": "Overwrite the access token used for code autocomplete. This is only supported with a provider other than `anthropic`."
},
"cody.autocomplete.advanced.cache": {
"type": "boolean",
"default": true,
"markdownDescription": "Enables caching of code autocomplete."
},
"cody.autocomplete.advanced.embeddings": {
"order": 99,
"type": "boolean",
Expand Down
97 changes: 0 additions & 97 deletions vscode/src/completions/cache.test.ts

This file was deleted.

104 changes: 0 additions & 104 deletions vscode/src/completions/cache.ts

This file was deleted.

Loading

0 comments on commit 1f04aaa

Please sign in to comment.