pygpt4all - output full response as string and suppress model parameters?

[gykung]

How do i export the full response from gpt4all into a single string? And how do i suppress the model parameters (gptj_generate... and gptj_model_load...) from being printed? Thank you.

For example, i'm just looking to get the following variable...
response="German beer is a very popular beverage all over the world. The brewing process and the ingredients used are very well studied and have been proven to be very effective in keeping the body healthy. German beer is also very popular because it is brewed with only water and malted barley, which are very natural ingredients, thus maintaining a healthy lifestyle."

code:

from pygpt4all.models.gpt4all_j import GPT4All_J
def new_text_callback(text):
    print(text, end="")
model = GPT4All_J('./models/ggml-gpt4all-j-v1.3-groovy.bin')
model.generate("What do you think about German beer?",new_text_callback=new_text_callback)

response

What do you think about German beer?
German beer is a very popular beverage all over the world. The brewing process and the ingredients used are very well studied and have been proven to be very effective in keeping the body healthy. German beer is also very popular because it is brewed with only water and malted barley, which are very natural ingredients, thus maintaining a healthy lifestyle.<|endoftext|>


gptj_generate: mem per token = 15478000 bytes
gptj_generate:     load time =     0.00 ms
gptj_generate:   sample time =     0.47 ms
gptj_generate:  predict time =  9726.31 ms / 1215.79 ms per token
gptj_generate:    total time = 14902.11 ms
gptj_model_load: loading model from './models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx   = 2048
gptj_model_load: n_embd  = 4096
gptj_model_load: n_head  = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot   = 64
gptj_model_load: f16     = 2
gptj_model_load: ggml ctx size = 4505.45 MB
gptj_model_load: memory_size =   896.00 MB, n_mem = 57344
gptj_model_load: ................................... done
gptj_model_load: model size =  3609.38 MB / num tensors = 285
gptj_generate: seed = 1683099839
gptj_generate: number of tokens in prompt = 8

[abdeladim-s]

@gykung,

You can export it by simply storing the tokens in a string variable, like the following:

from pygpt4all import GPT4All_J

model = GPT4All_J('path/to/ggml-gpt4all-j-v1.3-groovy.bin')

response = ""
for token in model.generate("What do you think about German beer?\n"):
    response += token

print(response)

Please note that the parameters are printed to stderr from the c++ side, it does not affect the generated response

do not forget to update the package to the latest version

pip install -U pygpt4all

[kim-em]

The problem is that the parameters are actually printed to stdout on the c++ side, not stderr, and so the response will end up mixed up either way. Currently I am resorting to printing the response on stderr, and then my application just reads from that instead... 🤷‍♀️

[abdeladim-s]

I really do not understand what you are trying to achieve, but no, the response is not mixed up with the what is printed from the c++ side.