Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Looking for LLaMA 2? #452

Open
carmocca opened this issue Aug 25, 2023 · 3 comments
Open

Looking for LLaMA 2? #452

carmocca opened this issue Aug 25, 2023 · 3 comments

Comments

@carmocca
Copy link
Contributor

carmocca commented Aug 25, 2023 •
edited
Loading

Meta AI has since released LLaMA 2. Additionally, new Apache 2.0 licensed weights are being released as part of the Open LLaMA project.

To run LLaMA 2 weights, Open LLaMA weights, or Vicuna weights (among other LLaMA-like checkpoints), check out the Lit-GPT repository.
@carmocca carmocca pinned this issue Aug 25, 2023
@Leo-T-Zang
Copy link

Hi @carmocca,
I wonder if it supports Full Parameter Supervised Training for LLaMA2 on customized dataset?
Thank you.

@rasbt
Copy link
Contributor

rasbt commented Aug 30, 2023 •
edited
Loading

Yes, full finetuning is supported via finetune/full.py script given a Llama 2 model provided via the --checkpoint_dir in Lit-GPT.

You can also use a custom dataset given that you prepare it in the right format. You can see the prepare_*.py scripts here for guidance.

@cabal-daniel
Copy link

Is this repo still intended to be supported given since it seems like the lit-gpt repo supports more and newer models.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@rasbt @carmocca @cabal-daniel @Leo-T-Zang