Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indexing: improve skipped doc handling #687

Merged
merged 2 commits into from
Nov 13, 2023
Merged

Indexing: improve skipped doc handling #687

merged 2 commits into from
Nov 13, 2023

Conversation

jtibshirani
Copy link
Member

@jtibshirani jtibshirani commented Nov 10, 2023

This change makes a couple small improvements to how we handle skipped docs:

  • Immediately skip ctags parsing if the content is nil
  • Always sort skipped docs to the end of the shard. This seems like a nice
    invariant. And generally it's good for performance to group data that is
    expected to be accessed together and has similar content.

Follow up to #686. I didn't see a nice opportunity for a refactor and instead
fixed the obvious issues.

Copy link
Member

@keegancsmith keegancsmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

build/ctags.go Outdated
@@ -49,7 +49,7 @@ func ctagsAddSymbolsParserMap(todo []*zoekt.Document, languageMap ctags.Language
var tagsToSections tagsToSections

for _, doc := range todo {
if doc.Symbols != nil {
if doc.Content == nil || doc.Symbols != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: rather check that len(doc.Content) == 0. Means the same thing but you get the benefit of avoiding ctags for empty content.

@jtibshirani jtibshirani merged commit e068116 into main Nov 13, 2023
8 checks passed
@jtibshirani jtibshirani deleted the jtibs/skip-docs branch November 13, 2023 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants