Notes on Tasks to be Done (03‐15‐2024)

Here are some open tasks. If you'd like to pick up a task, open an issue (if the issue doesn't exist, if the issue does exist add a comment that you'll be working on it) and/or join discord here https://discord.gg/H5jCBAWxAe for discussion.

Incremental Library Improvements

• MQA Implementation for 2B models (WIP at https://github.com/google/gemma.cpp/pull/114)
• Refactor terminal repl code out of run.cc into a reusable header
• Toy example of an alternative model in examples/

Large Refactors

• Pub/Sub message queue as interface between application and runtime for I/O, introspection
• Error handling as return values so that applications can handle failure states in their own ways.
• Move model configuration/selection to runtime
• Add GPU backend (webgpu?)

Google Workflow Items

• Incorporate copybara:end into copybara transformations

Features / Research

• %save / %load KV cache (see comment https://github.com/google/gemma.cpp/issues/60#issuecomment-1971268148)
• self-extend capability https://github.com/google/gemma.cpp/issues/60
• self-consistency decoding https://arxiv.org/abs/2203.11171
• start work towards webgpu backend
• FTXUI based tui