Integrate DeepSpeed's new HF interface for sequence parallelism#3299

Open

ronald-d-rogers wants to merge 14 commits intohuggingface:mainfrom ronald-d-rogers:ds-ulysses

+82-4

Commits on Jul 15, 2024

Commits on Nov 27, 2024

Merge branch 'pass-mpu-into-deepspeed' of https://github.com/zeyugao/accelerate into ds-ulysses
AWS Sagemaker Bot
committed
use deepspeed mpu
AWS Sagemaker Bot
committed

Commits on Dec 6, 2024

adds sequence parallel state deepspeed plugin
Ronald Rogers
committed

Commits on Dec 11, 2024

sharding tensors in deepspeed dataloader
Ronald Rogers
committed
removingg Pipfile
Ronald Rogers
committed

Commits on Dec 14, 2024

cleanup and bug fixes from testing
Ronald Rogers
committed
Merge branch 'main' of https://github.com/huggingface/accelerate into ds-ulysses
Ronald Rogers
committed
adopt more of zerygao's changes
Ronald Rogers
committed

Commits on Dec 15, 2024

remove sequence sharding from dataloader
Ronald Rogers
committed

Commits on Dec 16, 2024

undo unnecessary change to import
Ronald Rogers
committed
more cleanup
Ronald Rogers
committed