From debc89b71265dd16b06ed036062f8d6f59623914 Mon Sep 17 00:00:00 2001 From: Nat Kershaw Date: Thu, 14 Mar 2024 13:19:38 -0700 Subject: [PATCH] Add more definitions --- docs/genai/reference/config.md | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/docs/genai/reference/config.md b/docs/genai/reference/config.md index 19ee41fbf7bc4..0f13fc84cdfda 100644 --- a/docs/genai/reference/config.md +++ b/docs/genai/reference/config.md @@ -126,7 +126,7 @@ Then For each model in the pipeline there is one section, named by the model. * **_min_length_**: The minimum length that the model will generate. -* **_do_sample_**: +* **_do_sample_**: Enables Top P / Top K generation. * **_num_beams_**: The number of beams to apply when generating the output sequence using beam search. If num_beams=1, then generation is performed using greedy search. @@ -136,16 +136,33 @@ Then For each model in the pipeline there is one section, named by the model. * **_temperature_**: The temperature value scales the probability of each token so that probable tokens become more likely while less probable ones become less likely. This value can have a range 0 < `temperature` ≤ 1. When temperature is equal to `1`, it has no effect. -* **_top_k_**: Only includes tokens that do fall within the list of the `K` most probable tokens. +* **_top_k_**: Only includes tokens that do fall within the list of the `K` most probable tokens. Range is 1 to the vocabulary size. -* **_top_p_**: Only includes the most probable tokens with probabilities that add up to `P` or higher. Defaults to `1`, which includes all of the tokens. +* **_top_p_**: Only includes the most probable tokens with probabilities that add up to `P` or higher. Defaults to `1`, which includes all of the tokens. Range is 0 < top P <= 1. * **_repetition_penalty_**: Discounts the scores of previously generated tokens if set to a value greater than `1`. Defaults to `1`. * **_length_penalty_**: Controls the length of the output generated. Value less than `1` encourages the generation to produce shorter sequences. Values greater than `1` encourages longer sequences. Defaults to `1`. -* **_diversity_penalty_**: +* **_diversity_penalty_**: Not implemented. -* **_no_repeat_ngram_size_**: +* **_no_repeat_ngram_size_**: Not implemented. -* **_past_present_share_buffer_**: If set to true, the past and present buffer are shared for efficiency. +* **_past_present_share_buffer_**: If set to true, the past and present buffer are shared for efficiency. + +## Search combinations + +1. Beam search + + - num beams > 1 + - do_sample = False + +2. Greedy search + + - num_beams = 1 + - do_sample = False + +3. Top P / Top K + + - do_sample = True + \ No newline at end of file