[widgets] use chatCompletionStream
#664
Conversation
mishig25
force-pushed
the
use_chatCompletionStream
branch
2 times, most recently
from
f51c8c7
to
949a5c7
Compare
mishig25
force-pushed
the
use_chatCompletionStream
branch
5 times, most recently
from
448b308
to
0aabba9
Compare
mishig25
requested review from
osanseviero,
gary149,
Wauplin and
julien-c
as code owners
mishig25
force-pushed
the
use_chatCompletionStream
branch
from
0aabba9
to
643991a
Compare
|
ready for review ! |
|
nice @mishig25 LGTM! one curious question, how are you differentiating the models supporting chatCompletion API versus simple text generation? this https://huggingface.co/bigscience/bloom-560m from https://huggingface.co/HuggingFaceH4/zephyr-7b-beta? Because the |
| let chatText; | ||
| try { | ||
| chatText = compiledTemplate.render({ | ||
| messages, | ||
| add_generation_prompt: true, | ||
| ...specialTokensMap, | ||
| }); | ||
| } catch (e) { | ||
| error = `An error occurred while rendering the chat template: "${(e as Error).message}"`; | ||
| return; | ||
| } | ||
| const previousMessages = [...messages]; | ||
|
|
||
| const input: TextGenerationInput & Required<Pick<TextGenerationInput, "parameters">> = { | ||
| inputs: chatText, | ||
| parameters: { | ||
| return_full_text: false, | ||
| }, |
There was a problem hiding this comment.
given that's recent code from @Wauplin and @SBrandeis, i would like a review from them on this
|
@radames i think the conversational widget would not be displayed on pure text-generation models like https://huggingface.co/bigscience/bloom-560m |
yes that's right, I'm just curious how on differentiate these models since they're all |
|
@radames there's a notion of a If a model repo is cc @osanseviero who implemed |
Thanks to #645, it is now possible to use
chatCompletionStreamin Conversational Widget