Skip to content

Commit

Permalink
Update doc
Browse files Browse the repository at this point in the history
  • Loading branch information
@krishung5
krishung5 committed May 21, 2024
1 parent a593f2f commit 9e7e1bd
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -599,6 +599,11 @@ Starting from 23.10, request cancellation can be checked directly on the
the TRITONSERVER_RESPONSE_COMPLETE_FINAL flag at the end of response is still
needed even the request is cancelled.

*Note:*: If `response_sender` is reused across multiple requests, the model must
ensure that the `response_sender` is properly cleaned up when the
`pb_utils.TRITONSERVER_RESPONSE_COMPLETE_FINAL` flag has not been sent for the
request and the request is rescheduled or cancelled.

##### Use Cases

The decoupled mode is powerful and supports various other use cases:
Expand Down

0 comments on commit 9e7e1bd

Please sign in to comment.