Can you use this for Alpaca

[XanderTheDev]

I saw a post in a different repository of a person thinking it should work with the weights of alpaca with this python package, but when it was still called pyllamacpp. Is there still a way to do it or is there a different python package I can use, because I can't find anything about this.

I've got this as code, but it literally does nothing. Also no errors.

from pyllamacpp.model import Model

def new_text_callback(text: str):
    print(text, end="", flush=True)

model = Model(ggml_model='./ggml-alpaca-7b-q4.bin', n_ctx=512)
generated_text = model.generate("Once upon a time, ", n_predict=55)
print(generated_text)

and yes I've installed it via pip install pyllamacpp

used this as the weights: https://huggingface.co/Sosaka/Alpaca-native-4bit-ggml/blob/main/ggml-alpaca-7b-q4.bin

[abdeladim-s]

@XanderTheDev, yes, you can still use this package through the GPT4ALL class to run the model because the backend is still pyllamacpp, otherwise if you prefer to jst use pyllamacpp you can still find it here.

The API has changed a bit though, you can use it now as shown here.
You can also use the old approach but you have to replace the generate method with cpp_generate.