From 17ae25c2c7864c37dba2975b1a7a92a2a4ed500c Mon Sep 17 00:00:00 2001
From: kthui <18255193+kthui@users.noreply.github.com>
Date: Mon, 2 Dec 2024 18:06:28 -0800
Subject: [PATCH] [docs] Add logprobs and num of input tokens to docs

---
 docs/additional_outputs.md | 25 ++++++++++++++++++++-----
 1 file changed, 20 insertions(+), 5 deletions(-)

diff --git a/docs/additional_outputs.md b/docs/additional_outputs.md
index 5c103e89..4c531a18 100644
--- a/docs/additional_outputs.md
+++ b/docs/additional_outputs.md
@@ -46,8 +46,6 @@ for more details.
 To enable, set `return_finish_reason` input tensor to `True`. The reason will be
 sent as a string on the `finish_reason` output tensor.
 
-Supported since r24.12.
-
 ### Cumulative Log Probabilities
 
 The cumulative log probability of the generated output text. See
@@ -57,7 +55,26 @@ for more details.
 To enable, set `return_cumulative_logprob` input tensor to `True`. The floating
 point value will be sent on the `cumulative_logprob` output tensor.
 
-Supported since r24.12.
+### Log Probabilities
+
+The log probabilities of the top probability words at each position if the
+[logprobs](https://github.com/vllm-project/vllm/blob/v0.6.3.post1/vllm/sampling_params.py#L146-L152)
+are requested. Only the log probabilities of the new tokens generated since the
+last response are returned on each new response. See
+[here](https://github.com/vllm-project/vllm/blob/v0.6.3.post1/vllm/outputs.py#L24-L25)
+for more details on the log probabilities.
+
+To enable, set `return_logprobs` input tensor to `True`. The log probabilities
+will be sent on the `logprobs` output tensor as JSON.
+
+### Number of Input Tokens
+
+The number of token IDs of the prompt. See
+[here](https://github.com/vllm-project/vllm/blob/v0.6.3.post1/vllm/outputs.py#L79-L81)
+for more details.
+
+To enable, set `return_num_input_tokens` input tensor to `True`. The unsigned
+integer value will be sent on the `num_input_tokens` output tensor.
 
 ### Number of Output Tokens
 
@@ -71,8 +88,6 @@ for more details on the token IDs of the generated output text.
 To enable, set `return_num_output_tokens` input tensor to `True`. The unsigned
 integer value will be sent on the `num_output_tokens` output tensor.
 
-Supported since r24.12.
-
 ## Examples
 
 ### Add Finish Reason to Outputs