-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The convert.py may not fit the rwkv-4-world-169m. #8
Comments
Hi @kip234 I am glad to tell you that the reason you cannot convert the v4 world model is due to the fact it has been deprecated in favor of the v5 world model (Also known as the Eagle models)! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I have tried to convert the weight file which i download from https://huggingface.co/RWKV/rwkv-4-world-169m with
convert.py
. I found the following problems.convert.py
..att.
inconvert.py
should be replaced with.attention.
.rwkv.blocks.0.attention.time_decay
is a 1d tensor. Theheadsnume
andheadsize
inconvert.py
can't unpacke fromw[f"rwkv.blocks.0.attention.time_decay"].shape
I don't know how to fix the third problem.
The text was updated successfully, but these errors were encountered: