-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[fp8] Merge feature/fp8_comm to main branch of Colossalai #6016
Commits on Jun 24, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 82aecd6 - Browse repository at this point
Copy the full SHA 82aecd6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4b59d87 - Browse repository at this point
Copy the full SHA 4b59d87View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0b2d627 - Browse repository at this point
Copy the full SHA 0b2d627View commit details -
Configuration menu - View commit details
-
Copy full SHA for f3de5a0 - Browse repository at this point
Copy the full SHA f3de5a0View commit details
Commits on Jun 27, 2024
-
Configuration menu - View commit details
-
Copy full SHA for c8d1b4a - Browse repository at this point
Copy the full SHA c8d1b4aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8aad064 - Browse repository at this point
Copy the full SHA 8aad064View commit details -
Configuration menu - View commit details
-
Copy full SHA for 384c640 - Browse repository at this point
Copy the full SHA 384c640View commit details -
Configuration menu - View commit details
-
Copy full SHA for afa5306 - Browse repository at this point
Copy the full SHA afa5306View commit details -
Configuration menu - View commit details
-
Copy full SHA for b117274 - Browse repository at this point
Copy the full SHA b117274View commit details
Commits on Jun 28, 2024
-
Configuration menu - View commit details
-
Copy full SHA for e752776 - Browse repository at this point
Copy the full SHA e752776View commit details -
Configuration menu - View commit details
-
Copy full SHA for a8af6cc - Browse repository at this point
Copy the full SHA a8af6ccView commit details -
Configuration menu - View commit details
-
Copy full SHA for ff53520 - Browse repository at this point
Copy the full SHA ff53520View commit details
Commits on Jul 5, 2024
-
[shardformer] DeepseekMoE support (hpcaitech#5871)
* [Feature] deepseek moe expert parallel implement * [misc] fix typo, remove redundant file (hpcaitech#5867) * [misc] fix typo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [Feature] deepseek support & unit test * [misc] remove debug code & useless print * [misc] fix typos (hpcaitech#5872) * [Feature] remove modeling file, use auto config. (hpcaitech#5884) * [misc] fix typos * [Feature] deepseek support via auto model, remove modeling file * [misc] delete useless file * [misc] fix typos * [Deepseek] remove redundant code (hpcaitech#5888) * [misc] fix typos * [Feature] deepseek support via auto model, remove modeling file * [misc] delete useless file * [misc] fix typos * [misc] remove redundant code * [Feature/deepseek] resolve comment. (hpcaitech#5889) * [misc] fix typos * [Feature] deepseek support via auto model, remove modeling file * [misc] delete useless file * [misc] fix typos * [misc] remove redundant code * [misc] mv module replacement into if branch * [misc] add some warning message and modify some code in unit test * [misc] fix typos --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 3420921 - Browse repository at this point
Copy the full SHA 3420921View commit details -
[Hoxfix] Fix CUDA_DEVICE_MAX_CONNECTIONS for comm overlap
Co-authored-by: Edenzzzz <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 8ec24b6 - Browse repository at this point
Copy the full SHA 8ec24b6View commit details
Commits on Jul 8, 2024
-
[Feat] Diffusion Model(PixArtAlpha/StableDiffusion3) Support (hpcaite…
…ch#5838) * Diffusion Model Inference support * Stable Diffusion 3 Support * pixartalpha support
Configuration menu - View commit details
-
Copy full SHA for cba2052 - Browse repository at this point
Copy the full SHA cba2052View commit details -
[HotFix] CI,import,requirements-test for hpcaitech#5838 (hpcaitech#5892)
* [Hot Fix] CI,import,requirements-test --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 66abf1c - Browse repository at this point
Copy the full SHA 66abf1cView commit details
Commits on Jul 9, 2024
-
[Feature] Enable PP + SP for llama (hpcaitech#5868)
* fix cross-PP-stage position id length diff bug * fix typo * fix typo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * use a one cross entropy func for all shardformer models --------- Co-authored-by: Edenzzzz <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for fbf33ec - Browse repository at this point
Copy the full SHA fbf33ecView commit details
Commits on Jul 10, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 16f3451 - Browse repository at this point
Copy the full SHA 16f3451View commit details -
[ShardFormer] Add Ulysses Sequence Parallelism support for Command-R,…
… Qwen2 and ChatGLM (hpcaitech#5897)
Configuration menu - View commit details
-
Copy full SHA for 669849d - Browse repository at this point
Copy the full SHA 669849dView commit details -
add benchmark for sft, dpo, simpo, orpo. Add benchmarking result. Sup…
…port lora with gradient checkpoint
Configuration menu - View commit details
-
Copy full SHA for d888c37 - Browse repository at this point
Copy the full SHA d888c37View commit details -
Configuration menu - View commit details
-
Copy full SHA for f6ef5c3 - Browse repository at this point
Copy the full SHA f6ef5c3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 33f1520 - Browse repository at this point
Copy the full SHA 33f1520View commit details -
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Configuration menu - View commit details
-
Copy full SHA for 8a9721b - Browse repository at this point
Copy the full SHA 8a9721bView commit details
Commits on Jul 11, 2024
-
Merge pull request hpcaitech#5850 from hpcaitech/rlhf_SimPO
[Chat] Rlhf support SimPO
Configuration menu - View commit details
-
Copy full SHA for dd9e1cd - Browse repository at this point
Copy the full SHA dd9e1cdView commit details -
Configuration menu - View commit details
-
Copy full SHA for e7a8634 - Browse repository at this point
Copy the full SHA e7a8634View commit details -
Configuration menu - View commit details
-
Copy full SHA for 115c4cc - Browse repository at this point
Copy the full SHA 115c4ccView commit details -
[zero] support all-gather overlap (hpcaitech#5898)
* [zero] support all-gather overlap * [zero] add overlap all-gather flag * [misc] fix typo * [zero] update api
Configuration menu - View commit details
-
Copy full SHA for c068ef0 - Browse repository at this point
Copy the full SHA c068ef0View commit details
Commits on Jul 15, 2024
-
Configuration menu - View commit details
-
Copy full SHA for b3594d4 - Browse repository at this point
Copy the full SHA b3594d4View commit details -
[Auto Parallel]: Speed up intra-op plan generation by 44% (hpcaitech#…
…5446) * Remove unnecessary calls to deepcopy * Build DimSpec's difference dict only once This change considerably speeds up construction speed of DimSpec objects. The difference_dict is the same for each DimSpec object, so a single copy of it is enough. * Fix documentation of DimSpec's difference method
Configuration menu - View commit details
-
Copy full SHA for 45c49dd - Browse repository at this point
Copy the full SHA 45c49ddView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1c961b2 - Browse repository at this point
Copy the full SHA 1c961b2View commit details
Commits on Jul 16, 2024
-
Merge pull request hpcaitech#5901 from hpcaitech/colossalchat
[Chat] fix eval: add in training evaluation, fix orpo sft loss bug
Configuration menu - View commit details
-
Copy full SHA for d8bf7e0 - Browse repository at this point
Copy the full SHA d8bf7e0View commit details -
[compatibility] support torch 2.2 (hpcaitech#5875)
* Support Pytorch 2.2.2 * keep build_on_pr file and update .compatibility
Configuration menu - View commit details
-
Copy full SHA for 2e28c79 - Browse repository at this point
Copy the full SHA 2e28c79View commit details -
Configuration menu - View commit details
-
Copy full SHA for 530283d - Browse repository at this point
Copy the full SHA 530283dView commit details -
[misc] support torch2.3 (hpcaitech#5893)
* [misc] support torch2.3 * [devops] update compatibility ci * [devops] update compatibility ci * [devops] add debug * [devops] add debug * [devops] add debug * [devops] add debug * [devops] remove debug * [devops] remove debug
Configuration menu - View commit details
-
Copy full SHA for 27a72f0 - Browse repository at this point
Copy the full SHA 27a72f0View commit details
Commits on Jul 17, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 73494de - Browse repository at this point
Copy the full SHA 73494deView commit details
Commits on Jul 18, 2024
-
[plugin] support all-gather overlap for hybrid parallel (hpcaitech#5919)
* [plugin] fixed all-gather overlap support for hybrid parallel
Configuration menu - View commit details
-
Copy full SHA for e861279 - Browse repository at this point
Copy the full SHA e861279View commit details -
Configuration menu - View commit details
-
Copy full SHA for 09d5ffc - Browse repository at this point
Copy the full SHA 09d5ffcView commit details -
Configuration menu - View commit details
-
Copy full SHA for 845ea72 - Browse repository at this point
Copy the full SHA 845ea72View commit details -
Configuration menu - View commit details
-
Copy full SHA for 544b7a3 - Browse repository at this point
Copy the full SHA 544b7a3View commit details
Commits on Jul 19, 2024
-
[Examples] Add lazy init to OPT and GPT examples (hpcaitech#5924)
Co-authored-by: Edenzzzz <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 8cc8f64 - Browse repository at this point
Copy the full SHA 8cc8f64View commit details -
[ColossalChat] Hotfix for ColossalChat (hpcaitech#5910)
* add ignore and tiny llama * fix path issue * run style * fix issue * update bash * add ignore and tiny llama * fix path issue * run style * fix issue * update bash * fix ddp issue * add Qwen 1.5 32B
Configuration menu - View commit details
-
Copy full SHA for f585d4e - Browse repository at this point
Copy the full SHA f585d4eView commit details -
Configuration menu - View commit details
-
Copy full SHA for d08c99b - Browse repository at this point
Copy the full SHA d08c99bView commit details -
Configuration menu - View commit details
-
Copy full SHA for d49550f - Browse repository at this point
Copy the full SHA d49550fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 150505c - Browse repository at this point
Copy the full SHA 150505cView commit details
Commits on Jul 21, 2024
-
[FIX BUG] UnboundLocalError: cannot access local variable 'default_co…
…nversation' where it is not associated with a value (hpcaitech#5931) * cannot access local variable 'default_conversation' where it is not associated with a value set default value for 'default_conversation' * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 4ec17a7 - Browse repository at this point
Copy the full SHA 4ec17a7View commit details
Commits on Jul 22, 2024
-
Configuration menu - View commit details
-
Copy full SHA for c5f582f - Browse repository at this point
Copy the full SHA c5f582fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 12fe8b5 - Browse repository at this point
Copy the full SHA 12fe8b5View commit details -
Configuration menu - View commit details
-
Copy full SHA for b0e15d5 - Browse repository at this point
Copy the full SHA b0e15d5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9688e19 - Browse repository at this point
Copy the full SHA 9688e19View commit details
Commits on Jul 23, 2024
-
Configuration menu - View commit details
-
Copy full SHA for a521ffc - Browse repository at this point
Copy the full SHA a521ffcView commit details
Commits on Jul 24, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5fb958c - Browse repository at this point
Copy the full SHA 5fb958cView commit details
Commits on Jul 25, 2024
-
[Hotfix] Fix ZeRO typo hpcaitech#5936
Co-authored-by: Edenzzzz <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 2069472 - Browse repository at this point
Copy the full SHA 2069472View commit details
Commits on Jul 26, 2024
-
[Feature] Add a switch to control whether the model checkpoint needs …
…to be saved after each epoch ends (hpcaitech#5941) * Add a switch to control whether the model checkpoint needs to be saved after each epoch ends * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for ad35a98 - Browse repository at this point
Copy the full SHA ad35a98View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8a3ff4f - Browse repository at this point
Copy the full SHA 8a3ff4fView commit details -
Configuration menu - View commit details
-
Copy full SHA for de1bf08 - Browse repository at this point
Copy the full SHA de1bf08View commit details
Commits on Jul 29, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 6fd9e86 - Browse repository at this point
Copy the full SHA 6fd9e86View commit details -
Configuration menu - View commit details
-
Copy full SHA for c8332b9 - Browse repository at this point
Copy the full SHA c8332b9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9664b1b - Browse repository at this point
Copy the full SHA 9664b1bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7b38964 - Browse repository at this point
Copy the full SHA 7b38964View commit details
Commits on Jul 30, 2024
-
[Feat] Distrifusion Acceleration Support for Diffusion Inference (hpc…
…aitech#5895) * Distrifusion Support source * comp comm overlap optimization * sd3 benchmark * pixart distrifusion bug fix * sd3 bug fix and benchmark * generation bug fix * naming fix * add docstring, fix counter and shape error * add reference * readme and requirement
Configuration menu - View commit details
-
Copy full SHA for bcf0181 - Browse repository at this point
Copy the full SHA bcf0181View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0608921 - Browse repository at this point
Copy the full SHA 0608921View commit details
Commits on Jul 31, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 09c5f72 - Browse repository at this point
Copy the full SHA 09c5f72View commit details -
[Chat] Fix lora (hpcaitech#5946)
* fix merging * remove filepath * fix style
Configuration menu - View commit details
-
Copy full SHA for 30f4e31 - Browse repository at this point
Copy the full SHA 30f4e31View commit details -
Configuration menu - View commit details
-
Copy full SHA for 66fbf2e - Browse repository at this point
Copy the full SHA 66fbf2eView commit details -
[hotfix] Remove unused plan section (hpcaitech#5957)
* remove readme * fix readme * update
Configuration menu - View commit details
-
Copy full SHA for 1aeb5e8 - Browse repository at this point
Copy the full SHA 1aeb5e8View commit details
Commits on Aug 1, 2024
-
Configuration menu - View commit details
-
Copy full SHA for f9b6fcf - Browse repository at this point
Copy the full SHA f9b6fcfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0b76b57 - Browse repository at this point
Copy the full SHA 0b76b57View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8ae8525 - Browse repository at this point
Copy the full SHA 8ae8525View commit details -
Configuration menu - View commit details
-
Copy full SHA for a249e71 - Browse repository at this point
Copy the full SHA a249e71View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0fad23c - Browse repository at this point
Copy the full SHA 0fad23cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 46c069b - Browse repository at this point
Copy the full SHA 46c069bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 37443cc - Browse repository at this point
Copy the full SHA 37443ccView commit details -
Configuration menu - View commit details
-
Copy full SHA for b5bfeb2 - Browse repository at this point
Copy the full SHA b5bfeb2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 13b48ac - Browse repository at this point
Copy the full SHA 13b48acView commit details -
Configuration menu - View commit details
-
Copy full SHA for fe24789 - Browse repository at this point
Copy the full SHA fe24789View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5ed5e8c - Browse repository at this point
Copy the full SHA 5ed5e8cView commit details -
Configuration menu - View commit details
-
Copy full SHA for e28e053 - Browse repository at this point
Copy the full SHA e28e053View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9b9b76b - Browse repository at this point
Copy the full SHA 9b9b76bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 014faf6 - Browse repository at this point
Copy the full SHA 014faf6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8dbb868 - Browse repository at this point
Copy the full SHA 8dbb868View commit details -
Configuration menu - View commit details
-
Copy full SHA for 102b784 - Browse repository at this point
Copy the full SHA 102b784View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0b5bbe9 - Browse repository at this point
Copy the full SHA 0b5bbe9View commit details -
Configuration menu - View commit details
-
Copy full SHA for dc583aa - Browse repository at this point
Copy the full SHA dc583aaView commit details -
Configuration menu - View commit details
-
Copy full SHA for 74eccac - Browse repository at this point
Copy the full SHA 74eccacView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3e2b613 - Browse repository at this point
Copy the full SHA 3e2b613View commit details -
[Feature] MoE Ulysses Support (hpcaitech#5918)
* moe sp support * moe sp bug solve * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 404b16f - Browse repository at this point
Copy the full SHA 404b16fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 09d6280 - Browse repository at this point
Copy the full SHA 09d6280View commit details -
Configuration menu - View commit details
-
Copy full SHA for 877d94b - Browse repository at this point
Copy the full SHA 877d94bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2cddeac - Browse repository at this point
Copy the full SHA 2cddeacView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7077d38 - Browse repository at this point
Copy the full SHA 7077d38View commit details -
Configuration menu - View commit details
-
Copy full SHA for 803878b - Browse repository at this point
Copy the full SHA 803878bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 46037c2 - Browse repository at this point
Copy the full SHA 46037c2View commit details -
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Configuration menu - View commit details
-
Copy full SHA for 52d346f - Browse repository at this point
Copy the full SHA 52d346fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 70c9924 - Browse repository at this point
Copy the full SHA 70c9924View commit details -
Configuration menu - View commit details
-
Copy full SHA for 74b03de - Browse repository at this point
Copy the full SHA 74b03deView commit details -
Configuration menu - View commit details
-
Copy full SHA for 067e18f - Browse repository at this point
Copy the full SHA 067e18fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 96d0fbc - Browse repository at this point
Copy the full SHA 96d0fbcView commit details -
Configuration menu - View commit details
-
Copy full SHA for b2952a5 - Browse repository at this point
Copy the full SHA b2952a5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6c39f0b - Browse repository at this point
Copy the full SHA 6c39f0bView commit details -
Configuration menu - View commit details
-
Copy full SHA for c3dc9b4 - Browse repository at this point
Copy the full SHA c3dc9b4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 59bcf56 - Browse repository at this point
Copy the full SHA 59bcf56View commit details -
Configuration menu - View commit details
-
Copy full SHA for 034020b - Browse repository at this point
Copy the full SHA 034020bView commit details -
Configuration menu - View commit details
-
Copy full SHA for cb01c0d - Browse repository at this point
Copy the full SHA cb01c0dView commit details -
Revert "[moe] implement submesh initialization"
This reverts commit 2f9bce6.
Configuration menu - View commit details
-
Copy full SHA for 5b4c123 - Browse repository at this point
Copy the full SHA 5b4c123View commit details -
Configuration menu - View commit details
-
Copy full SHA for 606b089 - Browse repository at this point
Copy the full SHA 606b089View commit details -
Configuration menu - View commit details
-
Copy full SHA for 12d043c - Browse repository at this point
Copy the full SHA 12d043cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 70793ce - Browse repository at this point
Copy the full SHA 70793ceView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7e737df - Browse repository at this point
Copy the full SHA 7e737dfView commit details -
Configuration menu - View commit details
-
Copy full SHA for f7c5485 - Browse repository at this point
Copy the full SHA f7c5485View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7bedd03 - Browse repository at this point
Copy the full SHA 7bedd03View commit details -
Configuration menu - View commit details
-
Copy full SHA for 65daa87 - Browse repository at this point
Copy the full SHA 65daa87View commit details -
Configuration menu - View commit details
-
Copy full SHA for d1d1ab8 - Browse repository at this point
Copy the full SHA d1d1ab8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 62cdac6 - Browse repository at this point
Copy the full SHA 62cdac6View commit details
Commits on Aug 2, 2024
-
[feat] Dist Loader for Eval (hpcaitech#5950)
* support auto distributed data loader * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * support auto distributed data loader * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix tp error * remove unused parameters * remove unused * update inference * update docs * update inference --------- Co-authored-by: Michelle <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 19d1510 - Browse repository at this point
Copy the full SHA 19d1510View commit details -
[lora] lora support hybrid parallel plugin (hpcaitech#5956)
* lora support hybrid plugin * fix * fix * fix * fix
Configuration menu - View commit details
-
Copy full SHA for 75c9636 - Browse repository at this point
Copy the full SHA 75c9636View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0b2d55c - Browse repository at this point
Copy the full SHA 0b2d55cView commit details -
Merge pull request hpcaitech#5962 from hpcaitech/colossalchat
[Chat] Support overall loss, update KTO logging
Configuration menu - View commit details
-
Copy full SHA for fe71917 - Browse repository at this point
Copy the full SHA fe71917View commit details
Commits on Aug 7, 2024
-
Co-authored-by: Edenzzzz <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 9179d40 - Browse repository at this point
Copy the full SHA 9179d40View commit details
Commits on Aug 8, 2024
-
[Hotfix] README link (hpcaitech#5966)
* update ignore * update readme * run style * update readme
Configuration menu - View commit details
-
Copy full SHA for ad3fa4f - Browse repository at this point
Copy the full SHA ad3fa4fView commit details
Commits on Aug 9, 2024
-
[Hotfix] Avoid fused RMSnorm import error without apex (hpcaitech#5985)
Co-authored-by: Edenzzzz <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for b4d2377 - Browse repository at this point
Copy the full SHA b4d2377View commit details
Commits on Aug 12, 2024
-
[Chat] fix readme (hpcaitech#5989)
* fix readme * fix readme, tokenization fully tested * fix readme, tokenization fully tested * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: root <root@notebook-8f919155-6035-47b4-9c6f-1be133b9e2c9-0.notebook-8f919155-6035-47b4-9c6f-1be133b9e2c9.colossal-ai.svc.cluster.local> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for ed97d3a - Browse repository at this point
Copy the full SHA ed97d3aView commit details
Commits on Aug 14, 2024
-
Configuration menu - View commit details
-
Copy full SHA for ceb1e26 - Browse repository at this point
Copy the full SHA ceb1e26View commit details
Commits on Aug 15, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 406f984 - Browse repository at this point
Copy the full SHA 406f984View commit details -
[pre-commit.ci] pre-commit autoupdate (hpcaitech#5995)
updates: - [github.com/psf/black-pre-commit-mirror: 24.4.2 → 24.8.0](psf/black-pre-commit-mirror@24.4.2...24.8.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 4dd0399 - Browse repository at this point
Copy the full SHA 4dd0399View commit details -
Configuration menu - View commit details
-
Copy full SHA for 887d2d5 - Browse repository at this point
Copy the full SHA 887d2d5View commit details
Commits on Aug 16, 2024
-
[Feature] Zigzag Ring attention (hpcaitech#5905)
* halfway * fix cross-PP-stage position id length diff bug * fix typo * fix typo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * unified cross entropy func for all shardformer models * remove redundant lines * add basic ring attn; debug cross entropy * fwd bwd logic complete * fwd bwd logic complete; add experimental triton rescale * precision tests passed * precision tests passed * fix typos and remove misc files * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add sp_mode to benchmark; fix varlen interface * update softmax_lse shape by new interface * change tester name * remove buffer clone; support packed seq layout * add varlen tests * fix typo * all tests passed * add dkv_group; fix mask * remove debug statements --------- Co-authored-by: Edenzzzz <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for f5c84af - Browse repository at this point
Copy the full SHA f5c84afView commit details -
[misc] update compatibility (hpcaitech#6008)
* [misc] update compatibility * [misc] update requirements * [devops] disable requirements cache * [test] fix torch ddp test * [test] fix rerun on address in use * [test] fix lazy init
Configuration menu - View commit details
-
Copy full SHA for 26493b9 - Browse repository at this point
Copy the full SHA 26493b9View commit details
Commits on Aug 17, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 4cf79fa - Browse repository at this point
Copy the full SHA 4cf79faView commit details -
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Configuration menu - View commit details
-
Copy full SHA for 81272e9 - Browse repository at this point
Copy the full SHA 81272e9View commit details
Commits on Aug 19, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 02636c5 - Browse repository at this point
Copy the full SHA 02636c5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 52289e4 - Browse repository at this point
Copy the full SHA 52289e4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1a5847e - Browse repository at this point
Copy the full SHA 1a5847eView commit details -
overlap kv comm with output rescale (hpcaitech#6017)
Co-authored-by: Edenzzzz <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for f1c3266 - Browse repository at this point
Copy the full SHA f1c3266View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3353042 - Browse repository at this point
Copy the full SHA 3353042View commit details -
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Configuration menu - View commit details
-
Copy full SHA for 64aad96 - Browse repository at this point
Copy the full SHA 64aad96View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4c82bfc - Browse repository at this point
Copy the full SHA 4c82bfcView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0d8e82a - Browse repository at this point
Copy the full SHA 0d8e82aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 12b4401 - Browse repository at this point
Copy the full SHA 12b4401View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2eb3683 - Browse repository at this point
Copy the full SHA 2eb3683View commit details -
Configuration menu - View commit details
-
Copy full SHA for 88b3f06 - Browse repository at this point
Copy the full SHA 88b3f06View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1f703e0 - Browse repository at this point
Copy the full SHA 1f703e0View commit details
Commits on Aug 20, 2024
-
[misc] Use dist logger in plugins (hpcaitech#6011)
* use dist logger in plugins * remove trash * print on rank 0 --------- Co-authored-by: Edenzzzz <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for dcc44aa - Browse repository at this point
Copy the full SHA dcc44aaView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5382311 - Browse repository at this point
Copy the full SHA 5382311View commit details -
Configuration menu - View commit details
-
Copy full SHA for f7acfa1 - Browse repository at this point
Copy the full SHA f7acfa1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2ee6235 - Browse repository at this point
Copy the full SHA 2ee6235View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2e4cbe3 - Browse repository at this point
Copy the full SHA 2e4cbe3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2d362ac - Browse repository at this point
Copy the full SHA 2d362acView commit details
Commits on Aug 21, 2024
-
Configuration menu - View commit details
-
Copy full SHA for eb5ba40 - Browse repository at this point
Copy the full SHA eb5ba40View commit details -
Configuration menu - View commit details
-
Copy full SHA for 193030f - Browse repository at this point
Copy the full SHA 193030fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6aface9 - Browse repository at this point
Copy the full SHA 6aface9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 698c8b9 - Browse repository at this point
Copy the full SHA 698c8b9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8b8e282 - Browse repository at this point
Copy the full SHA 8b8e282View commit details