Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

backend: rebase llama.cpp on upstream as of Sep 26th #2998

Merged
merged 5 commits into from
Sep 27, 2024

Conversation

cebtenzzre
Copy link
Member

@cebtenzzre cebtenzzre commented Sep 26, 2024

Update our llama.cpp dependency, and update the backend code to be compatible with the changes:

  • There are new translation units (cpp files) in llama.cpp, now added to our llama.cpp.cmake script.
  • The model state serialization functions have been renamed, and they have a more robust API that does bounds checking and computes the state size exactly.
  • The sampler API has been changed from a procedural API to a more object-oriented one, with a chain of sampler objects that is built in advance. (llama : refactor sampling v2 ggerganov/llama.cpp#9294)
  • The log level constants have been renumbered for some reason. Now they are not in any particular order, and care must be taken in the log callback to check the log level of a message against each one individually, and handle GGML_LOG_LEVEL_CONT specially.

Support for the new model architectures is not included in this PR.

TODO: Update the changelog for the python bindings.

This is mostly from the GGML-based model implementations, which are now
gone.

Signed-off-by: Jared Van Bortel <[email protected]>
Signed-off-by: Jared Van Bortel <[email protected]>
@cebtenzzre cebtenzzre marked this pull request as ready for review September 26, 2024 22:45
@cebtenzzre cebtenzzre merged commit f9d6be8 into main Sep 27, 2024
2 of 3 checks passed
@cebtenzzre cebtenzzre deleted the update-llamacpp-sep2024 branch September 27, 2024 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants