From a34282f8f94b6260adceac9d0c1916bb580cc907 Mon Sep 17 00:00:00 2001 From: Benjamin Ironside Goldstein Date: Thu, 4 Jul 2024 18:26:50 -0700 Subject: [PATCH] minor fixes --- docs/serverless/assistant/connect-to-byo-llm.mdx | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/serverless/assistant/connect-to-byo-llm.mdx b/docs/serverless/assistant/connect-to-byo-llm.mdx index 736feada4a..9f9c2dcdd5 100644 --- a/docs/serverless/assistant/connect-to-byo-llm.mdx +++ b/docs/serverless/assistant/connect-to-byo-llm.mdx @@ -94,14 +94,14 @@ Once you've launched LM Studio, select a model: For security reasons, before downloading a model, verify that it is from a trusted source. It can be helpful to review community feedback on the model (for example using a site like Hugging Face). -1. Go to LM Studio's Search window. -2. Search for an LLM (for example, `Mixtral-8x7B-instruct`). -3. Your chosen model must include `instruct` in its name in order to work with Elastic. -4. Filter your search for "Compatibility Guess" to optimize results for your hardware. Results will be color coded: +. Go to LM Studio's Search window. +. Search for an LLM (for example, `Mixtral-8x7B-instruct`). +. Your chosen model must include `instruct` in its name in order to work with Elastic. +. Filter your search for "Compatibility Guess" to optimize results for your hardware. Results will be color coded: .. Green means "Full GPU Possible", which yields the best results. .. Blue means "Partial GPU offload possible", which may work. .. Red for "Likely too large for this machine", which typically will not work. -5. Download one or more models. +. Download one or more models. In this example we used [`TheBloke/Mixtral-8x7B-Instruct-v0.1.Q3_K_M.gguf`](https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF). It has 46.7B total parameters, a 32,000 token context window, and uses GGUF [quanitization](https://huggingface.co/docs/transformers/main/en/quantization/overview). For more information about model names and format information, refer to the following table: @@ -119,7 +119,7 @@ After downloading a model, you'll need to load it in LM Studio using the GUI or It is a best practice to use the GUI to download at least one model from the marketplace first, and then load/unload the model(s) using the CLI. The GUI allows you to search for models, whereas the CLI only allows you to import specific paths, but the CLI provides a good interface for loading/unloading. -Use the following commands in your CLI: +Use the following commands in your CLI. 1. Verify LM Studio: `lms` 2. Check LM Studio's status: `lms status`