Skip to content

Commit

Permalink
Some updates to C++ benchmark (#708)
Browse files Browse the repository at this point in the history
- Fix whitespace in usage text
- Fix memory usage output on MacOS
- Add readme
  • Loading branch information
edgchen1 authored Jul 18, 2024
1 parent cb313b4 commit 60269a0
Show file tree
Hide file tree
Showing 3 changed files with 23 additions and 2 deletions.
2 changes: 1 addition & 1 deletion benchmark/c/options.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ namespace {
s << "Usage: " << program_name << " -i <model path> <other options>\n"
<< " Options:\n"
<< " -i,--input_folder <path>\n"
<< " Path to the ONNX model directory to benchmark, compatible with onnxruntime-genai.\n "
<< " Path to the ONNX model directory to benchmark, compatible with onnxruntime-genai.\n"
<< " -b,--batch_size <number>\n"
<< " Number of sequences to generate in parallel. Default: " << defaults.batch_size << "\n"
<< " -l,--prompt_length <number>\n"
Expand Down
8 changes: 7 additions & 1 deletion benchmark/c/posix/resource_utils.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,13 @@ size_t GetPeakWorkingSetSizeInBytes() {
throw std::runtime_error("getrusage failed with error code " + std::to_string(errno));
}

return static_cast<size_t>(rusage.ru_maxrss * 1024L);
#if defined(__APPLE__)
constexpr size_t kBytesPerMaxRssUnit = 1;
#else
constexpr size_t kBytesPerMaxRssUnit = 1024;
#endif

return static_cast<size_t>(rusage.ru_maxrss) * kBytesPerMaxRssUnit;
}

} // namespace benchmark::utils
15 changes: 15 additions & 0 deletions benchmark/c/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# model_benchmark

`model_benchmark` is an end-to-end benchmark program for ONNX Runtime GenAI models.
It is written in C++ and built as part of the ONNX Runtime GenAI build (e.g., via [build.py](../../build.py)).

It is an alternative to the [Python benchmark script](../python/benchmark_e2e.py) that can be run in environments where Python is not available.

Example usage:
```
model_benchmark -i <path to model directory>
```

Run with `--help` to see information about additional options.

Note: On some platforms, such as Android, you may need to set the environment variable `LD_LIBRARY_PATH` to the directory containing the onnxruntime shared library for `model_benchmark` to be able to run.

0 comments on commit 60269a0

Please sign in to comment.