diff --git a/docs/serverless/assistant/connect-to-byo-llm.mdx b/docs/serverless/assistant/connect-to-byo-llm.mdx index 77414912fa..ca4aa9e05c 100644 --- a/docs/serverless/assistant/connect-to-byo-llm.mdx +++ b/docs/serverless/assistant/connect-to-byo-llm.mdx @@ -90,14 +90,7 @@ First, install [LM Studio](https://lmstudio.ai/). LM Studio supports the OpenAI One current limitation of LM Studio is that when it is installed on a server, you must launch the application using its GUI before doing so using the CLI. For example, by using Chrome RDP with an [X Window System](https://cloud.google.com/architecture/chrome-desktop-remote-on-compute-engine). After you've opened the application the first time using the GUI, you can start it by using `sudo lms server start` in the CLI. -Once you've launched LM Studio, select a model: - - - - - -For security reasons, before downloading a model, verify that it is from a trusted source. It can be helpful to review community feedback on the model (for example using a site like Hugging Face). - +Once you've launched LM Studio: 1. Go to LM Studio's Search window. 1. Search for an LLM (for example, `Mixtral-8x7B-instruct`). @@ -108,6 +101,12 @@ For security reasons, before downloading a model, verify that it is from a trust * Red for "Likely too large for this machine", which typically will not work. 1. Download one or more models. + +For security reasons, before downloading a model, verify that it is from a trusted source. It can be helpful to review community feedback on the model (for example using a site like Hugging Face). + + + + In this example we used [`TheBloke/Mixtral-8x7B-Instruct-v0.1.Q3_K_M.gguf`](https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF). It has 46.7B total parameters, a 32,000 token context window, and uses GGUF [quanitization](https://huggingface.co/docs/transformers/main/en/quantization/overview). For more information about model names and format information, refer to the following table. | Model Name | Parameter Size | Tokens/Context Window | Quantization Format |