The goal of this repository is to make the code for RWKV more easily understood. There's a few things I plan to do with this:
- Make the code readable. The code in the main repository is a big mess--it's got short, undescriptive names, hacks like "getattr" and "setattr", etc.
- Add documentation. Explain what the different parts of the model are.
- Write up a paper describing the algorithm in math terms.
Right now, all three of these still need to be done. Point (1) is the current priority.