Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sync/main #5424

Merged
merged 39 commits into from
Mar 4, 2024
Merged

Sync/main #5424

merged 39 commits into from
Mar 4, 2024

Commits on Jan 31, 2024

  1. [extension] fixed exception catch (hpcaitech#5342)

    @FrankLeeeee
    FrankLeeeee authored Jan 31, 2024
    Configuration menu
    Copy the full SHA
    abd8e77 View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2024

  1. [Chat] fix sft loss nan (hpcaitech#5345) 

    * fix script

* fix script

* fix chat nan

* fix chat nan
    @YeAnbang
    YeAnbang authored Feb 1, 2024
    Configuration menu
    Copy the full SHA
    c523984 View commit details
    Browse the repository at this point in the history

  2. [checkpointio] fix gemini and hybrid parallel optim checkpoint (hpcai… 

    …tech#5347)

* [checkpointio] fix hybrid parallel optim checkpoint

* [extension] fix cuda extension

* [checkpointio] fix gemini optimizer checkpoint

* polish code
    @ver217
    ver217 authored Feb 1, 2024
    Configuration menu
    Copy the full SHA
    ffffc32 View commit details
    Browse the repository at this point in the history

Commits on Feb 2, 2024

  1. [fix] remove unnecessary dp_size assert (hpcaitech#5351) 

    * fix: remove unnecessary assert

* test: add more 3d plugin tests

* fix: add warning
    @CWHer
    CWHer authored Feb 2, 2024
    Configuration menu
    Copy the full SHA
    1c790c0 View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2024

  1. [gemini] fix param op hook when output is tuple (hpcaitech#5355) 

    * [gemini] fix param op hook when output is tuple

* [gemini] fix param op hook
    @ver217
    ver217 authored Feb 4, 2024
    Configuration menu
    Copy the full SHA
    2dd01e3 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2024

  1. [llama] fix dataloader for hybrid parallel (hpcaitech#5358) 

    * [plugin] refactor prepare dataloader

* [plugin] update train script
    @ver217
    ver217 authored Feb 5, 2024
    Configuration menu
    Copy the full SHA
    6c0fa7b View commit details
    Browse the repository at this point in the history

  2. [llama] update training script (hpcaitech#5360) 

    * [llama] update training script

* [doc] polish docstr
    @ver217
    ver217 authored Feb 5, 2024
    Configuration menu
    Copy the full SHA
    73f9f23 View commit details
    Browse the repository at this point in the history

  3. [llama] add flash attn patch for npu (hpcaitech#5362)

    @ver217
    ver217 authored Feb 5, 2024
    Configuration menu
    Copy the full SHA
    a4cec17 View commit details
    Browse the repository at this point in the history

  4. [llama] fix neftune & pbar with start_step (hpcaitech#5364)

    @Camille7777
    Camille7777 authored Feb 5, 2024
    Configuration menu
    Copy the full SHA
    44ca61a View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2024

  1. [eval] update llama npu eval (hpcaitech#5366)

    @Camille7777
    Camille7777 authored Feb 6, 2024
    Configuration menu
    Copy the full SHA
    a5756a8 View commit details
    Browse the repository at this point in the history

  2. [llama] polish training script and fix optim ckpt (hpcaitech#5368)

    @ver217
    ver217 authored Feb 6, 2024
    Configuration menu
    Copy the full SHA
    eb4f2d9 View commit details
    Browse the repository at this point in the history

  3. [lr-scheduler] fix load state dict and add test (hpcaitech#5369)

    @ver217
    ver217 authored Feb 6, 2024
    Configuration menu
    Copy the full SHA
    c53ddda View commit details
    Browse the repository at this point in the history

  4. [llama] fix memory issue (hpcaitech#5371) 

    * [llama] fix memory issue

* [llama] add comment
    @ver217
    ver217 authored Feb 6, 2024
    Configuration menu
    Copy the full SHA
    084c912 View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2024

  1. [moe] init mixtral impl

    @oahzxl @ver217
    oahzxl authored and ver217 committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    7d8e033 View commit details
    Browse the repository at this point in the history

  2. [moe] update capacity computing (hpcaitech#5253) 

    * [moe] top2 allow uneven input

* [moe] update capacity computing

* [moe] remove debug info

* [moe] update capacity computing

* [moe] update capacity computing
    @ver217
    ver217 committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    c904d2a View commit details
    Browse the repository at this point in the history

  3. [moe] support mixtral (hpcaitech#5309) 

    * [moe] add mixtral block for single expert

* [moe] mixtral block fwd support uneven ep

* [moe] mixtral block bwd support uneven ep

* [moe] add mixtral moe layer

* [moe] simplify replace

* [meo] support save sharded mixtral

* [meo] support load sharded mixtral

* [meo] support save sharded optim

* [meo] integrate moe manager into plug

* [meo] fix optimizer load

* [meo] fix mixtral layer
    @ver217
    ver217 committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    da39d21 View commit details
    Browse the repository at this point in the history

  4. [moe] fix mixtral checkpoint io (hpcaitech#5314)

    @ver217
    ver217 committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    b60be18 View commit details
    Browse the repository at this point in the history

  5. [moe] fix mixtral forward default value (hpcaitech#5329)

    @ver217
    ver217 committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    956b561 View commit details
    Browse the repository at this point in the history

  6. [moe] fix mixtral optim checkpoint (hpcaitech#5344)

    @ver217
    ver217 committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    65e5d6b View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2024

  1. [moe] fix tests

    @ver217
    ver217 committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    06db94f View commit details
    Browse the repository at this point in the history

  2. Merge pull request hpcaitech#5377 from hpcaitech/example/llama-npu 

    [llama] support npu for Colossal-LLaMA-2
    @FrankLeeeee
    FrankLeeeee authored Feb 8, 2024
    Configuration menu
    Copy the full SHA
    4c03347 View commit details
    Browse the repository at this point in the history

  3. Merge pull request hpcaitech#5372 from hpcaitech/exp/mixtral

    @FrankLeeeee
    FrankLeeeee authored Feb 8, 2024
    Configuration menu
    Copy the full SHA
    efef43b View commit details
    Browse the repository at this point in the history

  4. [release] update version (hpcaitech#5380)

    @ver217
    ver217 authored Feb 8, 2024
    Configuration menu
    Copy the full SHA
    adae123 View commit details
    Browse the repository at this point in the history

Commits on Feb 19, 2024

  1. [llama] fix training and inference scripts (hpcaitech#5384) 

    * [llama] refactor inference example to fit sft

* [llama] fix training script to fit gemini

* [llama] fix inference script
    @ver217
    ver217 authored Feb 19, 2024
    Configuration menu
    Copy the full SHA
    7303801 View commit details
    Browse the repository at this point in the history

  2. [doc] Fix typo (hpcaitech#5361)

    @yixiaoer
    yixiaoer authored Feb 19, 2024
    Configuration menu
    Copy the full SHA
    69e3ad0 View commit details
    Browse the repository at this point in the history

  3. [doc] updated installation command (hpcaitech#5389)

    @FrankLeeeee
    FrankLeeeee authored Feb 19, 2024
    Configuration menu
    Copy the full SHA
    705a62a View commit details
    Browse the repository at this point in the history

  4. [hotfix] fix variable type for top_p (hpcaitech#5313) 

    Co-authored-by: binmakeswell <[email protected]>
    @CZYCW @binmakeswell
    CZYCW and binmakeswell authored Feb 19, 2024
    Configuration menu
    Copy the full SHA
    b833153 View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2024

  1. [hotfix] Fix wrong import in meta_registry (hpcaitech#5392)

    @stephankoe
    stephankoe authored Feb 20, 2024
    Configuration menu
    Copy the full SHA
    5d380a1 View commit details
    Browse the repository at this point in the history

Commits on Feb 26, 2024

  1. [extension] hotfix jit extension setup (hpcaitech#5402)

    @ver217
    ver217 authored Feb 26, 2024
    Configuration menu
    Copy the full SHA
    95c21e3 View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2024

  1. [example] reuse flash attn patch (hpcaitech#5400)

    @ver217
    ver217 authored Feb 27, 2024
    Configuration menu
    Copy the full SHA
    d882d18 View commit details
    Browse the repository at this point in the history

  2. [fsdp] impl save/load shard model/optimizer (hpcaitech#5357)

    @ericxsun
    ericxsun authored Feb 27, 2024
    Configuration menu
    Copy the full SHA
    bf34c6f View commit details
    Browse the repository at this point in the history

  3. [setup] fixed nightly release (hpcaitech#5388)

    @FrankLeeeee
    FrankLeeeee authored Feb 27, 2024
    Configuration menu
    Copy the full SHA
    dcdd8a5 View commit details
    Browse the repository at this point in the history

  4. [shardformer]gather llama logits (hpcaitech#5398) 

    * gather llama logits

* fix
    @flybird11111
    flybird11111 authored Feb 27, 2024
    Configuration menu
    Copy the full SHA
    0a25e16 View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2024

  1. update requirements (hpcaitech#5407)

    @TongLi3701
    TongLi3701 authored Feb 28, 2024
    Configuration menu
    Copy the full SHA
    a28c971 View commit details
    Browse the repository at this point in the history

Commits on Feb 29, 2024

  1. [workflow] added pypi channel (hpcaitech#5412)

    @FrankLeeeee
    FrankLeeeee authored Feb 29, 2024
    Configuration menu
    Copy the full SHA
    2461f37 View commit details
    Browse the repository at this point in the history

  2. [doc] fix blog link

    @binmakeswell @liwenjuna
    binmakeswell authored and liwenjuna committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    5de940d View commit details
    Browse the repository at this point in the history

  3. [doc] fix blog link

    @binmakeswell @liwenjuna
    binmakeswell authored and liwenjuna committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    a1c6cdb View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2024

  1. fix sft single turn inference example (hpcaitech#5416)

    @Camille7777
    Camille7777 authored Mar 1, 2024
    Configuration menu
    Copy the full SHA
    4b8312c View commit details
    Browse the repository at this point in the history

Commits on Mar 4, 2024

  1. Merge branch 'main' into sync/main

    @FrankLeeeee
    FrankLeeeee committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    0310b76 View commit details
    Browse the repository at this point in the history