From d88685d39cb7c2fe7bec8bad776e7bdac5008e2f Mon Sep 17 00:00:00 2001 From: Indrajit Bhosale Date: Fri, 16 Aug 2024 16:16:22 -0700 Subject: [PATCH] Docs for GRPC Error codes (#775) Docs for GRPC Error codes in Case of Streaming. --- README.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/README.md b/README.md index bfe36fb77..2c15237bf 100644 --- a/README.md +++ b/README.md @@ -550,7 +550,28 @@ server side. If writing your own gRPC clients in the language of choice consult gRPC guide on [cancellation](https://grpc.io/docs/guides/cancellation/#cancelling-an-rpc-call-on-the-client-side). +### GRPC Status Codes + +Starting from release 24.08, Triton server introduces support for gRPC error +codes in streaming mode for all clients enhancing error reporting capabilities. When +this feature is enabled, the Triton server will return standard gRPC error codes +and subsequently close the stream after delivering the error. This feature is +optional can be enabled by adding header with `triton_grpc_error` key and `true` as +value. See [grpc error +codes](https://github.com/triton-inference-server/server/tree/main#GRPC-Status-Codes) in the server to learn about how this is handled on the server side. See gRPC +guide on [status-codes](https://grpc.io/docs/guides/status-codes/) for more details. +Below is a Python snippet to enable the feature. Without this header Triton server +will continue streaming in default mode returning error message and status inside +`InferenceServerException` object within the callback provided. +```python + triton_client = grpcclient.InferenceServerClient(triton_server_url) + # New added header key value + metadata = {"triton_grpc_error": "true"} + triton_client.start_stream( + callback=partial(callback, user_data), headers=metadata + ) +``` ## Simple Example Applications This section describes several of the simple example applications and