Skip to content

Commit

Permalink
docs(ollama): Document how to allow unfree packages (in this case, fo…
Browse files Browse the repository at this point in the history
…r proprietary nvidia packages)
  • Loading branch information
shivaraj-bh committed Jun 24, 2024
1 parent b291b6f commit fb6de01
Showing 1 changed file with 18 additions and 0 deletions.
18 changes: 18 additions & 0 deletions doc/ollama.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,24 @@ By default Ollama uses the CPU for inference. To enable GPU acceleration:

For NVIDIA GPUs.

Firstly, allow unfree packages:

```nix
# Inside perSystem = { system, ... }: { ...
{
imports = [
"${inputs.nixpkgs}/nixos/modules/misc/nixpkgs.nix"
];
nixpkgs = {
hostPlatform = system;
# Required for CUDA
config.allowUnfree = true;
};
}
```

And then enable CUDA acceleration:

```nix
# In `perSystem.process-compose.<name>`
{
Expand Down

0 comments on commit fb6de01

Please sign in to comment.