Code

[WIP] Fused CEL#457

Code

Draft

jeromeku wants to merge 48 commits intounslothai:mainunslothai/unsloth:mainfrom jeromeku:fused_cel_wipjeromeku/unsloth:fused_cel_wipCopy head branch name to clipboard

+1,994-23

Commits

Commits on May 9, 2024

• initial commit

  jeromeku

  committed
  4b6dff8

• add trainer utils

  jeromeku

  committed
  a6d8300

• temp

  jeromeku

  committed
  022c3d3

• rebase

  jeromeku

  committed
  5b438be

• add utils

  jeromeku

  committed
  2af523d

Commits on May 10, 2024

• edit cel

  jeromeku

  committed
  1116c7c

• add reshaping to fused cel

  jeromeku

  committed
  b2e5377

• add logits warning

  jeromeku

  committed
  e6d449e

• add shifting

  jeromeku

  committed
  6c3c90d

• fused out test

  jeromeku

  committed
  ac67948

• fix loss dtype

  jeromeku

  committed
  6312670

• add cel test

  jeromeku

  committed
  9ba1f81

• refactor callback

  jeromeku

  committed
  382a8ec

• fix grad reshape issue

  jeromeku

  committed
  71889ea

• add benchmarking script

  jeromeku

  committed
  4088670

Commits on May 11, 2024

• clean up bench script

  jeromeku

  committed
  e2b74c1

• fix sanity check

  jeromeku

  committed
  3304895

• fix gradient scaling

  jeromeku

  committed
  a25b9be

• change label shifting

  jeromeku

  committed
  a54425a

• change n_loop_iters impl

  jeromeku

  committed
  6c537e9

• add lm head test

  jeromeku

  committed
  c777c50

• add gitignore

  jeromeku

  committed
  e1e55ce

• remove pycache

  jeromeku

  committed
  7ec969e

Commits on May 12, 2024

• fix test

  jeromeku

  committed
  b83c8e8

• fix test

  jeromeku

  committed
  e39ace8

• add more test configs

  jeromeku

  committed
  5a0a703

• add fixture

  jeromeku

  committed
  291e3e2

• add benchmark

  jeromeku

  committed
  46d0e7a

• add benchmark scripts

  jeromeku

  committed
  adaebbf

• add pytest config

  jeromeku

  committed
  91511c5

• adjust testing tolerances

  jeromeku

  committed
  e1a77c9

• initial unsloth bench

  jeromeku

  committed
  cd22ece

• add benchmark hf only script

  jeromeku

  committed
  842ba0d

• add debugging

  jeromeku

  committed
  37e6e92

• add notes to test

  jeromeku

  committed
  736786e

• fix fused config

  jeromeku

  committed
  a21f92a

• refactor test utils

  jeromeku

  committed
  a204950

• add simplified hf benchmark

  jeromeku

  committed
  2da14c4

• clean up hf test bench

  jeromeku

  committed
  ec7e5dd

Commits on May 13, 2024

• add num trainable parameters

  jeromeku

  committed
  bca8815

• add readme

  jeromeku

  committed
  c053af5

• update readme

  jeromeku

  committed
  1a8d836

• remove notebooks

  jeromeku

  committed
  10a0b57

• reorganize tests

  jeromeku

  committed
  fae9e81

• add cel test utils

  jeromeku

  committed
  6fa0846

• remove pycache

  jeromeku

  committed
  f047019

• revert llama

  jeromeku

  committed
  fd92b6b

• remove formatting

  jeromeku

  committed
  f4cd689