Trying to build a Chatbot

[v4rm3t]

Hello, everyone!

I am trying to build a chatbot based on some documentation.

Currently, I am trying to run a gpt4all-j model with embeddings. However, after trying llama-cpp-python, I get a very slow speed for returning response.

I saw that people mentioned they have improved speed by using BERT embeddings. I am new to the space and I don't understand how I can use it to embed my documents and use that embeddings for chat completion endpoint?

Currently, I am running it on a Mac Mini i7, 32gb RAM. I am planning to deploy it to a higher resource (vRAM) cloud server in future (if it can improve the response speed).