Skip to content

Commit

Permalink
[chore] Set bool config param to accept only true/false - compat brea…
Browse files Browse the repository at this point in the history
…king
  • Loading branch information
kthui committed Nov 26, 2024
1 parent b6bd649 commit a26a083
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 8 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,11 +307,11 @@ or left empty (false by default) in [model.json](https://github.com/triton-infer
*Note:* vLLM metrics are not reported to Triton metrics server by default
due to potential performance slowdowns. To enable vLLM model's metrics
reporting, please add following lines to its config.pbtxt as well.
```bash
```
parameters: {
key: "REPORT_CUSTOM_METRICS"
value: {
string_value:"yes"
string_value: "true"
}
}
```
Expand Down
10 changes: 5 additions & 5 deletions ci/L0_backend_vllm/metrics_test/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -98,14 +98,14 @@ parameters: {
" >> models/vllm_opt/config.pbtxt
run_test VLLMTritonMetricsTest.test_vllm_metrics_disabled

# Test vLLM metrics reporting with parameter "REPORT_CUSTOM_METRICS" set to "yes" in config.pbtxt
# Test vLLM metrics reporting with parameter "REPORT_CUSTOM_METRICS" set to "true" in config.pbtxt
copy_model_repository
cp ${SAMPLE_MODELS_REPO}/vllm_model/config.pbtxt models/vllm_opt
echo -e "
parameters: {
key: \"REPORT_CUSTOM_METRICS\"
value: {
string_value:\"yes\"
string_value: \"true\"
}
}
" >> models/vllm_opt/config.pbtxt
Expand All @@ -120,7 +120,7 @@ echo -e "
parameters: {
key: \"REPORT_CUSTOM_METRICS\"
value: {
string_value:\"yes\"
string_value: \"true\"
}
}
" >> models/vllm_opt/config.pbtxt
Expand All @@ -134,7 +134,7 @@ echo -e "
parameters: {
key: \"REPORT_CUSTOM_METRICS\"
value: {
string_value:\"yes\"
string_value: \"true\"
}
}
" >> models/vllm_opt/config.pbtxt
Expand All @@ -146,7 +146,7 @@ echo -e "
parameters: {
key: \"REPORT_CUSTOM_METRICS\"
value: {
string_value:\"yes\"
string_value: \"true\"
}
}
" >> models/vllm_opt/config.pbtxt
Expand Down
2 changes: 1 addition & 1 deletion src/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ def init_engine(self):
def _get_bool_config_param(self, param_name: str) -> bool:
return (param_name in self.model_config["parameters"]) and (
self.model_config["parameters"][param_name]["string_value"].lower()
in ["yes", "true"]
== "true"
)

def setup_lora(self):
Expand Down

0 comments on commit a26a083

Please sign in to comment.