Pinned Loading

1. RWKV-Infer RWKV-Infer Public

   A large-scale RWKV v6 inference with FLA . Capable of inference by combining multiple states(Pseudo MoE). Easy to deploy on docker. Supports true multi-batch generation and dynamic State switching.…

   Python 16 1

2. RWKV-LM-RLHF RWKV-LM-RLHF Public

   Reinforcement Learning Toolkit for RWKV. Distillation,SFT,RLHF(DPO,ORPO), infinite context training, Aligning Let's boost the model's intelligence! currently under construction:)

   Python 19

3. RWKV-infctx-trainer-LoRA RWKV-infctx-trainer-LoRA Public

   RWKV v5, v6 infctx LoRA trainer with 4bit quantization,Cuda and Rocm supported, for training arbitary context sizes, to 10k and beyond!

   Python 8 2

4. RWKV-LM-State-4bit-Orpo RWKV-LM-State-4bit-Orpo Public

   State tuning with Orpo of RWKV v6 can be performed with 4-bit quantization. Every model can be trained with Orpo on Single 24GB GPU!

   Python 6 1

5. RWKV5-LM-LoRA RWKV5-LM-LoRA Public

   RWKV v5,v6 LoRA Trainer on Cuda and Rocm Platform. RWKV is a RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN an…

   Python 11 3

6. RWKV-LM-RLHF-DPO-LoRA RWKV-LM-RLHF-DPO-LoRA Public

   Forked from Triang-jyed-driung/RWKV-LM-RLHF-DPO

   Direct Preference Optimization LoRA for RWKV, aiming for RWKV-5 and 6.

   Python 1