Support VectorDBQA or other for indexed project?

[tavurth]

I notice in cursor they use a form of VectorDBQA to index the entire project filesystem.

Any plans on something like this for gptel?

Perhaps, allowing more escape hatches for functional returns could be useful, allowing Claude to call a local function which would return the relevant parts of the cached index by VectorDBQA lookup?

Searching by embeddings

Searching by regexp

[karthink]

Any plans on something like this for gptel?

Not at the moment, no. There are many tasks remaining before I can work on local RAG, which is how I understand what you're describing. For example, I need to add support for tool-use/function-calling first.

Moving to a discussion since this is not planned for gptel.

---

I'm interested in understanding what's happening in those menu screenshots above (from Cursor, I assume). What does "search behavior" mean? Are you specifying the LLM chain/pipeline for requests here?

[karthink]

Perhaps relevant: Elisa (by the author of Ellama) appears to provide something like this, although I'm not sure exactly what it does either.

[tavurth]

So in Cursor when you load a project it will ask you if you'd like to compile the entire project to "embeddings", which is basically a space that Claude can access through tool use.

In the other box, there's the option to rebuild the embeddings for just one task, giving the option to use the LLM for reasoning about which parts may be important (much more slow).

The VectorDBQA is generally a pre-compiled embedding space which uses local CPU.

This embeddings database can then be given to claude for use as a "Tool". Basically, claude will decide to use the tool, and call-back to the caller with "Request tool".

The local machine will then search the DBQA for references claude would like, and then return them to him as part of a second chat message.