Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add more HF models with bug fixes. #13

Merged
merged 52 commits into from
Jul 2, 2024
Merged

Add more HF models with bug fixes. #13

merged 52 commits into from
Jul 2, 2024

Commits on Jun 21, 2024

  1. add qwen2 and hf test

    @garrett4wade
    garrett4wade committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    d346bff View commit details
    Browse the repository at this point in the history

  2. support gemma and tied embedding

    @garrett4wade
    garrett4wade committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    bf6906c View commit details
    Browse the repository at this point in the history

  3. add gpt

    @garrett4wade
    garrett4wade committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    97b2107 View commit details
    Browse the repository at this point in the history

  4. format

    @garrett4wade
    garrett4wade committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    14e4a03 View commit details
    Browse the repository at this point in the history

  5. rename gpt to gpt2

    @garrett4wade
    garrett4wade committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    85f70b9 View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. add opt support

    @garrett4wade
    garrett4wade committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    5bdd061 View commit details
    Browse the repository at this point in the history

  2. format

    @garrett4wade
    garrett4wade committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    946e8af View commit details
    Browse the repository at this point in the history

  3. add distributed save load test

    @garrett4wade
    garrett4wade committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    5aaa5df View commit details
    Browse the repository at this point in the history

  4. format

    @garrett4wade
    garrett4wade committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    9b6fedb View commit details
    Browse the repository at this point in the history

  5. fix distributed load

    @garrett4wade
    garrett4wade committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    b6b1f32 View commit details
    Browse the repository at this point in the history

  6. fix critic share embedding save load

    @garrett4wade
    garrett4wade committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    144f047 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. fix param realloc with shared embedding

    @garrett4wade
    garrett4wade committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    11fcd72 View commit details
    Browse the repository at this point in the history

  2. fix training on half devices

    @garrett4wade
    garrett4wade committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    c6dedc2 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. rebase finished

    @nuzant
    nuzant committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    a671d32 View commit details
    Browse the repository at this point in the history

  2. cuda graph pp=1 bug fix

    @nuzant
    nuzant committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    54da9db View commit details
    Browse the repository at this point in the history

  3. pipe generate cuda graph untested

    @nuzant
    nuzant committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    a0480c4 View commit details
    Browse the repository at this point in the history

  4. remove some debug prints

    @nuzant
    nuzant committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    48182ec View commit details
    Browse the repository at this point in the history

  5. test failed

    @nuzant
    nuzant committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    c922a19 View commit details
    Browse the repository at this point in the history

  6. cuda graph tested

    @nuzant
    nuzant committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    118385b View commit details
    Browse the repository at this point in the history

  7. format

    @nuzant
    nuzant committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    fa18662 View commit details
    Browse the repository at this point in the history

  8. fixed tests

    @nuzant
    nuzant committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    dad8f7c View commit details
    Browse the repository at this point in the history

  9. tested ppo and gen exp

    @nuzant
    nuzant committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    80c3b71 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'open-main' into mzy-cudagraph

    @nuzant
    nuzant committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    0488245 View commit details
    Browse the repository at this point in the history

  11. format cpp

    @nuzant
    nuzant committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    c688d7d View commit details
    Browse the repository at this point in the history

  12. Merge pull request #146 from garrett4wade/mzy-cudagraph 

    Add CUDAGraph option for generation
    @nuzant
    nuzant authored Jun 26, 2024
    Configuration menu
    Copy the full SHA
    37df40e View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. fix typos and confusing arguments

    @nuzant
    nuzant committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    dc3f9b0 View commit details
    Browse the repository at this point in the history

  2. algo debugging

    @garrett4wade
    garrett4wade committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    4d80626 View commit details
    Browse the repository at this point in the history

  3. maybe fixed

    @garrett4wade
    garrett4wade committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    352a2c0 View commit details
    Browse the repository at this point in the history

  4. emergent fix contiguous param

    @garrett4wade
    garrett4wade committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    6df5e7d View commit details
    Browse the repository at this point in the history

  5. Merge branch 'dev' of github.com:garrett4wade/distributed_llm into fi… 

    …x-contiguous-param
    @garrett4wade
    garrett4wade committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    02442b6 View commit details
    Browse the repository at this point in the history

  6. format

    @garrett4wade
    garrett4wade committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    cd37867 View commit details
    Browse the repository at this point in the history

  7. fix megatron contiguous param mapping

    @garrett4wade
    garrett4wade committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    539bacf View commit details
    Browse the repository at this point in the history

  8. debug rw

    @garrett4wade
    garrett4wade committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    ff4905e View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. run

    @garrett4wade
    garrett4wade committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    fb865cf View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. minor change

    @garrett4wade
    garrett4wade committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    a42f2a9 View commit details
    Browse the repository at this point in the history

  2. refactored tests

    @nuzant
    nuzant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    666e673 View commit details
    Browse the repository at this point in the history

  3. .

    @nuzant
    nuzant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    e40775a View commit details
    Browse the repository at this point in the history

  4. fix dataloader seeding

    @garrett4wade
    garrett4wade committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    2b1d4e7 View commit details
    Browse the repository at this point in the history

  5. fix generation configs

    @nuzant
    nuzant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    799a6ba View commit details
    Browse the repository at this point in the history

  6. move GenerationHyperparameters to realhf.api.core.model_api

    @nuzant
    nuzant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    c8e200b View commit details
    Browse the repository at this point in the history

  7. Merge pull request #148 from garrett4wade/mzy-cudagraph 

    Merge updates from public repository.
    @nuzant
    nuzant authored Jul 1, 2024
    Configuration menu
    Copy the full SHA
    ccfe95c View commit details
    Browse the repository at this point in the history

  8. .

    @garrett4wade
    garrett4wade committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    d27ad7d View commit details
    Browse the repository at this point in the history

  9. format

    @garrett4wade
    garrett4wade committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    7964892 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'main' of github.com:openpsi-project/realhf into fix-con… 

    …tiguous-param
    @garrett4wade
    garrett4wade committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    43e13a1 View commit details
    Browse the repository at this point in the history

  11. fix fast pytest

    @garrett4wade
    garrett4wade committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    b1e88e3 View commit details
    Browse the repository at this point in the history

  12. debug

    @garrett4wade
    garrett4wade committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    0633ed1 View commit details
    Browse the repository at this point in the history

  13. debug

    @garrett4wade
    garrett4wade committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    4aaf456 View commit details
    Browse the repository at this point in the history

  14. Merge branch 'main' of github.com:garrett4wade/distributed_llm into f… 

    …ix-contiguous-param
    @garrett4wade
    garrett4wade committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    d3bf732 View commit details
    Browse the repository at this point in the history

  15. fix manual ppo example

    @garrett4wade
    garrett4wade committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    0897e10 View commit details
    Browse the repository at this point in the history

  16. fix cuda graph

    @garrett4wade
    garrett4wade committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    211e0aa View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. fix 1

    @garrett4wade
    garrett4wade committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    7f0d846 View commit details
    Browse the repository at this point in the history

  2. fix 2

    @garrett4wade
    garrett4wade committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    f48130f View commit details
    Browse the repository at this point in the history