Skip to content

Commit

Permalink
Update build from source
Browse files Browse the repository at this point in the history
  • Loading branch information
nv-hwoo committed Jul 19, 2024
1 parent b9482c2 commit af79608
Showing 1 changed file with 11 additions and 13 deletions.
24 changes: 11 additions & 13 deletions src/c++/perf_analyzer/genai-perf/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,24 +86,22 @@ genai-perf --help

<summary>Alternatively, to install from source:</summary>

### From Source
Since GenAI-Perf depends on Perf Analyzer,
you'll need to install Perf Analyzer binary:

GenAI-Perf depends on Perf Analyzer. Here is how to install Perf Analyzer:
### Install Perf Analyzer (Ubuntu, Python 3.8+)

#### Install Perf Analyzer (Ubuntu, Python 3.8+)

Note: you must already have CUDA 12 installed.
**NOTE**: you must already have CUDA 12 installed.

```bash
pip install tritonclient

apt update && apt install -y --no-install-recommends libb64-0d libcurl4
```

Alternatively, you can install Perf Analyzer
[from source](../docs/install.md#build-from-source).
You can also build Perf Analyzer [from source](../docs/install.md#build-from-source) as well.

#### Install GenAI-Perf from source
### Install GenAI-Perf from source

```bash
git clone https://github.com/triton-inference-server/client.git && cd client
Expand Down Expand Up @@ -387,7 +385,7 @@ COMMAND LINE OPTIONS

Show the help message and exit.

## Endpoint Options:
### Endpoint Options:

##### `-m <list>`
##### `--model <list>`
Expand Down Expand Up @@ -432,7 +430,7 @@ An option to enable the use of the streaming API. (default: `False`)

URL of the endpoint to target for benchmarking. (default: `None`)

## Input Options
### Input Options

##### `-b <int>`
##### `--batch-size <int>`
Expand Down Expand Up @@ -498,7 +496,7 @@ data. (default: `550`)
The standard deviation of number of tokens in the generated prompts when
using synthetic data. (default: `0`)

## Profiling Options
### Profiling Options

##### `--concurrency <int>`

Expand All @@ -523,7 +521,7 @@ stable. The measurement is considered as stable if the ratio of max / min from
the recent 3 measurements is within (stability percentage) in terms of both
infer per second and latency. (default: `999`)

## Output Options
### Output Options

##### `--artifact-dir`

Expand All @@ -542,7 +540,7 @@ exported to `<profile_export_file>_genai_perf.csv`. For example, if the profile
export file is `profile_export.json`, the genai-perf file will be exported to
`profile_export_genai_perf.csv`. (default: `profile_export.json`)

## Other Options
### Other Options

##### `--tokenizer <str>`

Expand Down

0 comments on commit af79608

Please sign in to comment.