v3.4.0

3.4.0 (2025-01-08)

Features

• token prediction (speculative decoding) (#405) (632a7bf) (documentation: Token Prediction)
• controlledEvaluate (#405) (632a7bf) (documentation: Low Level API)
• evaluateWithMetadata (#405) (632a7bf) (documentation: Low Level API)
• reranking (#405) (632a7bf) (documentation: Reranking Documents)
• token confidence (#405) (632a7bf) (documentation: Low Level API)
• experimentalChunkDocument (#405) (632a7bf)
• build on arm64 using LLVM (#405) (632a7bf)
• try compiling with LLVM on Windows x64 when available (#405) (632a7bf)
• minor: dynamically load llama.cpp backends (#405) (632a7bf)
• minor: more token values support in SpecialToken (#405) (632a7bf)
• minor: improve memory usage estimation (#405) (632a7bf)

Bug Fixes

• check for Rosetta usage on macOS x64 when using the inspect gpu command (#405) (632a7bf)
• detect running under Rosetta on Apple Silicone and show an error message instead of crashing (#405) (632a7bf)
• switch from "nextTick" to "nextCycle" for the default batch dispatcher (#405) (632a7bf)
• remove deprecated CLS token (#405) (632a7bf)
• pipe error logs in inspect gpu command (#405) (632a7bf)

---

Shipped with llama.cpp release b4435

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

v3.3.2

3.3.2 (2024-12-27)

Bug Fixes

• embedding context deadlock (#402) (e2c5c3f)

---

Shipped with llama.cpp release b4291

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

v3.3.1

3.3.1 (2024-12-09)

Bug Fixes

• align embedding input with WPM vocabulary type models (#393) (28c7984)

---

Shipped with llama.cpp release b4291

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

v3.3.0

3.3.0 (2024-12-02)

Bug Fixes

• improve binary compatibility testing on Electron apps (#386) (97abbca)
• too many abort signal listeners (#386) (97abbca)
• log level of some lower level logs (#386) (97abbca)
• context window missing response during generation on specific extreme conditions (#386) (97abbca)
• adapt to breaking llama.cpp changes (#386) (97abbca)
• automatically resolve compiler is out of heap space CUDA build error (#386) (97abbca)

Features

• Llama 3.2 3B function calling support (#386) (97abbca)
• use llama.cpp backend registry for GPUs instead of custom implementations (#386) (97abbca)
• getLlama: build: "try" option (#386) (97abbca)
• init command: --model flag (#386) (97abbca)
• JSON Schema grammar: array prefixItems, minItems, maxItems support (#388) (4d387de)
• JSON Schema grammar: object additionalProperties, minProperties, maxProperties support (#388) (4d387de)
• JSON Schema grammar: string minLength, maxLength, format support (#388) (4d387de)
• JSON Schema grammar: improve inferred types (#388) (4d387de)
• function calling: params description support (#388) (4d387de)
• function calling: document JSON Schema type properties on Functionary chat function types (#388) (4d387de)

---

Shipped with llama.cpp release b4234

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

v3.2.0

3.2.0 (2024-10-31)

Bug Fixes

• Electron crash with some models on macOS when not using Metal (#375) (ea12dc5)
• adapt to llama.cpp breaking changes (#375) (ea12dc5)
• support rejectattr in Jinja templates (#376) (ea12dc5)
• build warning on macOS (#377) (6405ee9)

Features

• chat session response prefix (#375) (ea12dc5)
• improve context shift strategy (#375) (ea12dc5)
• use RAM and swap sizes in memory usage estimations (#375) (ea12dc5)
• faster building from source (#375) (ea12dc5)
• improve CPU compatibility score (#375) (ea12dc5)
• inspect gguf command: print a single key flag (#375) (ea12dc5)

---

Shipped with llama.cpp release b3995

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

v3.1.1

3.1.1 (2024-10-06)

Features

• minor: reference common classes on the Llama instance (#360) (8145c94)

---

Shipped with llama.cpp release b3889

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

v3.1.0

3.1.0 (2024-10-05)

Bug Fixes

• improve metadata read times (#351) (4ee10a9)
• hide internal type (#351) (4ee10a9)

Features

• resolveModelFile method (#351) (4ee10a9)
• hf: URI support (#351) (4ee10a9)

---

Shipped with llama.cpp release b3887

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

v3.0.3

✨ node-llama-cpp 3.0 is here! ✨

Read about the release in the blog post

---

3.0.3 (2024-09-25)

Bug Fixes

• adapt to llama.cpp breaking change (#344) (2e751c8)

---

Shipped with llama.cpp release b3825

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

v3.0.2

✨ node-llama-cpp 3.0 is here! ✨

Read about the release in the blog post

---

3.0.2 (2024-09-25)

Bug Fixes

• node template: bug (#342) (1291b97)
• use a compressed logo image for README.md (#340) (8ab983b)

---

Shipped with llama.cpp release b3821

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

v3.0.1

✨ node-llama-cpp 3.0 is here! ✨

Read about the release in the blog post

---

3.0.1 (2024-09-24)

Bug Fixes

• deploy docs website (#337) (ec45bbf)
• release create-command package (#335) (51f4622)

---

Shipped with llama.cpp release b3808

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)