Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ollama: add cudart_static to cuda dependencies #303230

Merged
merged 1 commit into from
Apr 11, 2024

Conversation

abysssol
Copy link
Contributor

@abysssol abysssol commented Apr 10, 2024

Description of changes

For some currently unknown reason, the cuda enabled ollama build has recently started requiring libcudart_static.a, so for now just add the static version to the cuda environment, which appears to make ollama compile properly. I don't know if ollama actually uses cuda at runtime, though. According to @wyndon, this appears to properly use cuda at runtime.

Fixes #303046

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@wyndon
Copy link
Contributor

wyndon commented Apr 10, 2024

I can confirm this fixes the build and actually uses CUDA at runtime. Great!

@drupol drupol merged commit ababd9c into NixOS:master Apr 11, 2024
29 checks passed
@abysssol abysssol deleted the ollama-cudart-static branch April 11, 2024 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Build failure: ollama with CUDA
3 participants