Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLHF] Fix bugs, tested #4860

Closed
wants to merge 55 commits into from
Closed

[RLHF] Fix bugs, tested #4860

wants to merge 55 commits into from

Commits on Sep 28, 2023

  1. add new branch

    @YeAnbang
    YeAnbang committed Sep 28, 2023
    Configuration menu
    Copy the full SHA
    c5eac98 View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2023

  1. fix bugs, ppo succeeded, reward from -0.2 to 0.075

    @YeAnbang
    YeAnbang committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    6f6c132 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2023

  1. V1 RLHF varified

    @YeAnbang
    YeAnbang committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    8e4eddb View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2023

  1. test huggingface rm

    @YeAnbang
    YeAnbang committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    6f6de60 View commit details
    Browse the repository at this point in the history

  2. fix kl

    @YeAnbang
    YeAnbang committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    59eec57 View commit details
    Browse the repository at this point in the history

  3. fix loading actor

    @YeAnbang
    YeAnbang committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    8916b17 View commit details
    Browse the repository at this point in the history

  4. tune hyperparameters

    @YeAnbang
    YeAnbang committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    8ed034a View commit details
    Browse the repository at this point in the history

  5. adjust dropout

    @YeAnbang
    YeAnbang committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    769f8cb View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2023

  1. add rlhf-ppo: V0.1

    @YeAnbang
    YeAnbang committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    d10b545 View commit details
    Browse the repository at this point in the history

  2. fix style

    @YeAnbang
    YeAnbang committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    3a3bf1d View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into rlhf_v1

    @YeAnbang
    YeAnbang authored Oct 6, 2023
    Configuration menu
    Copy the full SHA
    97a920d View commit details
    Browse the repository at this point in the history

  4. fix benchmark_opt_lora

    @YeAnbang
    YeAnbang committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    f797bd7 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'rlhf_v1' of https://github.com/YeAnbang/ColossalAI into… 

    … rlhf_v1
    @YeAnbang
    YeAnbang committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    355cbb9 View commit details
    Browse the repository at this point in the history

  6. remove unused file

    @YeAnbang
    YeAnbang committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    8808524 View commit details
    Browse the repository at this point in the history

  7. fix ci tests~

    @YeAnbang
    YeAnbang committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    d617ed2 View commit details
    Browse the repository at this point in the history

  8. fix ci test model

    @YeAnbang
    YeAnbang committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    fe1c970 View commit details
    Browse the repository at this point in the history

  9. fix model example ci tests

    @YeAnbang
    YeAnbang committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    426297f View commit details
    Browse the repository at this point in the history

  10. fix ci test_experience

    @YeAnbang
    YeAnbang committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    8e00e35 View commit details
    Browse the repository at this point in the history

  11. fix making experience ci test

    @YeAnbang
    YeAnbang committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    c082c67 View commit details
    Browse the repository at this point in the history

  12. fix ci test making experience

    @YeAnbang
    YeAnbang committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    fb10ff3 View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2023

  1. fix ci OOM

    @YeAnbang
    YeAnbang committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    d9f1829 View commit details
    Browse the repository at this point in the history

  2. fix ci directory not found error

    @YeAnbang
    YeAnbang committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    668a853 View commit details
    Browse the repository at this point in the history

  3. uncomment sft tests

    @YeAnbang
    YeAnbang committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    6dee6e6 View commit details
    Browse the repository at this point in the history

  4. fix model loading saving

    @YeAnbang
    YeAnbang committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    87db16d View commit details
    Browse the repository at this point in the history

  5. fix ci PPO OOM

    @YeAnbang
    YeAnbang committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    71b8f54 View commit details
    Browse the repository at this point in the history

  6. fix ci test cmd line input

    @YeAnbang
    YeAnbang committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    4820c9a View commit details
    Browse the repository at this point in the history

  7. fix bloom crititc model path

    @YeAnbang
    YeAnbang committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    f6f136d View commit details
    Browse the repository at this point in the history

  8. fix unit test remove tests for ChatGLM

    @YeAnbang
    YeAnbang committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    a1f78c6 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2023

  1. fix ci OOM

    @YeAnbang
    YeAnbang committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    c4ec11a View commit details
    Browse the repository at this point in the history

  2. skip ci test bloom gemini

    @YeAnbang
    YeAnbang committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    3140850 View commit details
    Browse the repository at this point in the history

  3. disable all bloom tests to fix ci OOM

    @YeAnbang
    YeAnbang committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    e7fc7c4 View commit details
    Browse the repository at this point in the history

  4. remove opt from stage 3 ci test due to OOM

    @YeAnbang
    YeAnbang committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    ae25276 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2023

  1. fix code refactor

    @YeAnbang
    YeAnbang committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    c650d39 View commit details
    Browse the repository at this point in the history

  2. fix code refactor

    @YeAnbang
    YeAnbang committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    36ef239 View commit details
    Browse the repository at this point in the history

  3. fix code refactor

    @YeAnbang
    YeAnbang committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    1601284 View commit details
    Browse the repository at this point in the history

  4. fix test_train style

    @YeAnbang
    YeAnbang committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    70e7e84 View commit details
    Browse the repository at this point in the history

  5. fix test train style

    @YeAnbang
    YeAnbang committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    991e0f1 View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2023

  1. fix scripts, try debug ci path not found

    @YeAnbang
    YeAnbang committed Oct 13, 2023
    Configuration menu
    Copy the full SHA
    9753395 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2023

  1. Merge branch 'main' into rlhf_v1

    @YeAnbang
    YeAnbang authored Oct 23, 2023
    Configuration menu
    Copy the full SHA
    a7b03aa View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2023

  1. fix bugs: remove duplicated no_grad and fix reward model training mode

    @YeAnbang
    YeAnbang committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    48ba0e4 View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2023

  1. refactor all dataset class

    @YeAnbang
    YeAnbang committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    2fa72ec View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2023

  1. fix class name

    @YeAnbang
    YeAnbang committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    ab71956 View commit details
    Browse the repository at this point in the history

  2. change schema format

    @YeAnbang
    YeAnbang committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    77a38fe View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2023

  1. fix datasets

    @YeAnbang
    YeAnbang committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    f87a38a View commit details
    Browse the repository at this point in the history

  2. fix datasets

    @YeAnbang
    YeAnbang committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    5e846ff View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2023

  1. fix coati, need refactor

    @YeAnbang
    YeAnbang committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    16d27f0 View commit details
    Browse the repository at this point in the history

  2. fix ci

    @YeAnbang
    YeAnbang committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    bf7f372 View commit details
    Browse the repository at this point in the history

  3. Update test_train.sh

    @YeAnbang
    YeAnbang authored Nov 2, 2023
    Configuration menu
    Copy the full SHA
    f0ee7e2 View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2023

  1. fix training scripts

    @YeAnbang
    YeAnbang committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    f134817 View commit details
    Browse the repository at this point in the history

  2. fix training scripts

    @YeAnbang
    YeAnbang committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    07a3a98 View commit details
    Browse the repository at this point in the history

  3. fix license, small updates

    @YeAnbang
    YeAnbang committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    98c8616 View commit details
    Browse the repository at this point in the history

  4. resolve conversation

    @YeAnbang
    YeAnbang committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    3f19ea7 View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2023

  1. initialize critic with reward

    @YeAnbang
    YeAnbang committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    49dfb0b View commit details
    Browse the repository at this point in the history

  2. initialize critic with reward

    @YeAnbang
    YeAnbang committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    881a0b5 View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2023

  1. fix ci tests

    @YeAnbang
    YeAnbang committed Nov 10, 2023
    Configuration menu
    Copy the full SHA
    21a6819 View commit details
    Browse the repository at this point in the history