Skip to content

Commit

Permalink
docs: Update faq.mdx on multiple GPU utilization
Browse files Browse the repository at this point in the history
  • Loading branch information
wsxiaoys authored Oct 29, 2023
1 parent 7bd99d1 commit 68a7b36
Showing 1 changed file with 2 additions and 6 deletions.
8 changes: 2 additions & 6 deletions website/docs/faq.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,7 @@ import CodeBlock from '@theme/CodeBlock';
<details>
<summary>How to utilize multiple NVIDIA GPUs?</summary>
<div>
<p>Tabby supports replicating models on multiple GPUs to increase throughput. You can specify the devices for model replication by using the <b>--device-indices</b> option.</p>
<CodeBlock language="bash">
# Replicate model to GPU 0 and GPU 1.{'\n'}
tabby serve ... --device-indices 0 --device-indices 1
</CodeBlock>
<p>Tabby only supports the use of a single GPU. To utilize multiple GPUs, you can initiate multiple Tabby instances and set CUDA_VISIBLE_DEVICES accordingly.</p>
</div>
</details>

Expand All @@ -38,4 +34,4 @@ import CodeBlock from '@theme/CodeBlock';
<p>Follow the instructions provided in the <a href="https://github.com/TabbyML/tabby/blob/main/MODEL_SPEC.md">Model Spec</a>.</p>
<p>Please note that the spec is unstable and does not adhere to semver.</p>
</div>
</details>
</details>

0 comments on commit 68a7b36

Please sign in to comment.