Code

Improve universal checkpoint#5289

Code

Merged

tohtana merged 21 commits intomicrosoft:mastermicrosoft/DeepSpeed:masterfrom tohtana:tohtana/unit_test_univ_cptohtana/DeepSpeed:tohtana/unit_test_univ_cpCopy head branch name to clipboardMar 28, 2024

+359-106

Commits

Commits on Mar 14, 2024

• run conversion script

  tohtana

  committed
  7684ae6

  Copy full SHA for 7684ae6

• add no-parallel path

  tohtana

  committed
  1ada96d

  Copy full SHA for 1ada96d

Commits on Mar 16, 2024

• improve restoring step from universal checkpoint

  tohtana

  committed
  fe61652

  Copy full SHA for fe61652

• sort keys when comparing state dicts

  tohtana

  committed
  2b15116

  Copy full SHA for 2b15116

• add unit test for universal checkpoint

  tohtana

  committed
  575665b

  Copy full SHA for 575665b

• fix loading with load_optimizer_states=False

  tohtana

  committed
  c8c0498

  Copy full SHA for c8c0498

Commits on Mar 17, 2024

• add dp scaling test

  tohtana

  committed
  8d2dbaa

  Copy full SHA for 8d2dbaa

• remove pad for comparison

  tohtana

  committed
  3547b4e

  Copy full SHA for 3547b4e

• refactor test conditions

  tohtana

  committed
  2b6f694

  Copy full SHA for 2b6f694

• Merge branch 'master' into tohtana/unit_test_univ_cp

  tohtana

  committed
  099133c

  Copy full SHA for 099133c

• fix for torch adam

  tohtana

  committed
  ccfba1a

  Copy full SHA for ccfba1a

Commits on Mar 18, 2024

• simplify argument

  tohtana

  committed
  24484ad

  Copy full SHA for 24484ad

Commits on Mar 19, 2024

• fix for optimizer that doesn't have step in optimizer states

  tohtana

  committed
  13effa1

  Copy full SHA for 13effa1

• add api to load global state to BF16 optimizer for compatibility

  tohtana

  committed
  101e90c

  Copy full SHA for 101e90c

Commits on Mar 20, 2024

• restore all fields in param group

  tohtana

  committed
  76aca75

  Copy full SHA for 76aca75

Commits on Mar 26, 2024

• Merge branch 'master' into tohtana/unit_test_univ_cp

  tohtana

  authored
  c4b2aaa

  Copy full SHA for c4b2aaa

Commits on Mar 28, 2024

• Merge branch 'master' into tohtana/unit_test_univ_cp

  tohtana

  committed
  140d704

  Copy full SHA for 140d704

• move loading function to ZeROOptimizer

  tohtana

  committed
  9909bd3

  Copy full SHA for 9909bd3

• refactor to avoid circular import

  tohtana

  committed
  5114233

  Copy full SHA for 5114233

• fix format

  tohtana

  committed
  301e79e

  Copy full SHA for 301e79e

• fix method calls

  tohtana

  committed
  802397d

  Copy full SHA for 802397d