Bug: Unable to specify seed from cli

[mjpowersjr]

Contact Details

[email protected]

What happened?

When attempting to use the -seed argument, e.g.

./mistral-7b-instruct-v0.2.Q4_0.llamafile --temp 0.7 -seed -1 -p '[INST]Write a story about llamas[/INST]'

I receive the following error:

error: unknown argument: -seed

I've tried this with a few different models, same results. If this is expected behavior, maybe updating the help documentation with more details about when the --seed argument can be used would be helpful?

Thanks!

Version

llamafile v0.8.5

What operating system are you seeing the problem on?

Linux

Relevant log output

No response

[belisarius222]

@jalehman and I just reproduced this on MacOS, but changing the argument to --seed instead of -seed fixed it, so I don't think this is unintended behavior.

[mjpowersjr]

@belisarius222 - Thanks for looking into this - just realized I missed the second dash in the submitted example. I actually tried a few other combinations of arguments including --seed before ending up here.

1. It seems like --seed is only intended to be used when supplying a prompt via the command line (not when using the --server flag. It might be nice to include a hint about that in the docs or error msg.)

2. After a second look, I think there are still some edge cases with argument parsing. It seems using the full form --prompt argument trips up the parser.

Works!

./mistral-7b-instruct-v0.2.Q4_0.llamafile --temp 0.7 --seed -1 -p '[INST]Write a story about llamas[/INST]'

Fails

./mistral-7b-instruct-v0.2.Q4_0.llamafile --temp 0.7 --seed -1 --prompt '[INST]Write a story about llamas[/INST]'

error: unknown argument: --temp

Fails

./mistral-7b-instruct-v0.2.Q4_0.llamafile --prompt '[INST]Write a story about llamas[/INST]'

error: unknown argument: --prompt

[Djip007]

Mistral-7B-Instruct-v0.2-llamafile
This model is "old" use an old version of llamafile...
did you have the same with https://huggingface.co/Mozilla/Mistral-7B-Instruct-v0.3-llamafile/tree/main

you can try with the last llamafile from: https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.8.13 too

(ref: #543 (comment))