Use --instruction or set a context before chat

[sazearte]

Hello, I would like to add a context to the model before launching a chat session.

It is an internal doc of about 100 pages.

I was thinking of giving it an argument-instruction but the python binding doesn't seem to support it,

I wonder, is it possible to add this context (my 100 pages doc) or is it better to create a new model from my 100 pages file ?

[abdeladim-s]

Hello @sazearte,

You can set the context via the prompt_context parameter on model instantiation.
However 100 pages is a lot and I don't think that any current language model is supporting a context of this size!

Hope this helps!

[sazearte]

hum it's not work for me i have this code:

from pygpt4all import GPT4All

prompt_context = """
toto is a blue room in Alaska
"""


prompt = """
what is toto ?
"""

if __name__ == "__main__":
    model = GPT4All('/model/model.bin', prompt_context=prompt_context)
    output = ""
    for token in model.generate(prompt):
        output = output + token
    print(output)

i have this output:
I'm sorry, but as an AI language model, I don't have enough context about "toto" or who/what it belongs to. Can you provide me with more information so that I can help?

can you give me an example that how tu use the prompt_context ?

[abdeladim-s]

What you want to accomplish is not a straightforward task, and you may need to finetune the language model to make "it learn" your data. the prompt_context is just to condition the language model to follow a pattern.

You can see the example here where you can attribute a persona to the language model by using the prompt_context.