From 68a7b3628c6bd437e7d749b827843e51636a1038 Mon Sep 17 00:00:00 2001 From: Meng Zhang Date: Sun, 29 Oct 2023 01:22:13 -0700 Subject: [PATCH] docs: Update faq.mdx on multiple GPU utilization --- website/docs/faq.mdx | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/website/docs/faq.mdx b/website/docs/faq.mdx index 3425c37cbe23..089a613c53d4 100644 --- a/website/docs/faq.mdx +++ b/website/docs/faq.mdx @@ -24,11 +24,7 @@ import CodeBlock from '@theme/CodeBlock';
How to utilize multiple NVIDIA GPUs?
-

Tabby supports replicating models on multiple GPUs to increase throughput. You can specify the devices for model replication by using the --device-indices option.

- - # Replicate model to GPU 0 and GPU 1.{'\n'} - tabby serve ... --device-indices 0 --device-indices 1 - +

Tabby only supports the use of a single GPU. To utilize multiple GPUs, you can initiate multiple Tabby instances and set CUDA_VISIBLE_DEVICES accordingly.

@@ -38,4 +34,4 @@ import CodeBlock from '@theme/CodeBlock';

Follow the instructions provided in the Model Spec.

Please note that the spec is unstable and does not adhere to semver.

- \ No newline at end of file +