score: boost exported go ident and downrank _test.go #675
Conversation
There was a problem hiding this comment.
It feels surprising/ tricky to be detecting this here within symbol scoring. If we think that test files should be down-ranked more aggressively, it seems like we should just increase the contribution of "doc order" in the overall ranking, by increasing the weight. This could really help across languages too.
Since we don't reliably have code intel "file ranks" (PageRank-inspired ranking), I've been thinking we should come up with an alternative for "file importance" that's super similar to "doc order" and give it a big weight just like we did for file ranks. In addition to checks for vendored/ generated/ test, I found the "many symbols" signal to be super useful, and highly correlated with PageRank!
I think my issue with doc-order is it is completely based on the size of the shard/etc. This makes it quite hard to reason about when comparing across shards. Doc-order is really useful within a shard to try and ensure we search more important documents first (in case we hit limits). But otherwise I think we do need some sort of score value that we mix in. I agree this feels wrong to be do at scoring time, and I think as we evolve this it will end up at indexing time. One advantage right now to doing it at scoring time is we don't need to reindex so is great for experimentation.
More symbols means more important or the other way around? For example generated files have a tendancy to have lots of symbols. But I think with proper detection of generated files/etc then this likely is a good sign. eg in our codebases the types.go or api.go files are generally a good document. |
|
Should we consider introducing something like a document category to replace or augment doc order? So instead of using the position of the document we use its category which would make it comparable across shards. |
|
I filed https://github.com/sourcegraph/sourcegraph/issues/57950, we could continue the conversation there! |
Right now our symbol analyser doesn't tell us if a symbol is exported. We add a go specific tweak here to boost those results. Ideally this could be something that is encoded in the symbol information.
Additionally we do downrank _test.go files via the doc-order. But in the case of symbol matches the boosting overweighs doc order signficantly. I found the extra downraking quite useful when experimenting.
Test Plan: lots of manual testing on the keyword branch