Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds AdeMAMix Optimizer to contrib #1104

Open
wants to merge 33 commits into
base: main
Choose a base branch
from
Open

Adds AdeMAMix Optimizer to contrib #1104

Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
daed085
getting ademamix docs and notebook running
mathDR Oct 13, 2024
c47261c
fixed imports
mathDR Oct 13, 2024
bf2d4a8
fixed linting errors
mathDR Oct 14, 2024
7cb270a
ran notebook in order
mathDR Oct 14, 2024
1c8f9de
Merge branch 'google-deepmind:main' into main
mathDR Oct 14, 2024
c252b51
implementing pr feedback
mathDR Oct 18, 2024
f9b6559
updated ademamix with author docstrings
mathDR Oct 21, 2024
4b621aa
added docstrings and matched adamw api
mathDR Oct 21, 2024
1a51332
removed unneeded alpha scheduler
mathDR Oct 21, 2024
4eb6065
added alpha as a scheduler
mathDR Oct 21, 2024
4eb618a
removed b3_scheduler
mathDR Oct 21, 2024
9442085
removed b3_scheduler
mathDR Oct 21, 2024
420771f
fixing tests with new docstrings
mathDR Oct 21, 2024
9690311
fixed docstring
mathDR Oct 21, 2024
f892e29
updated notebook
mathDR Oct 22, 2024
81da0e4
fixed import ordering
mathDR Oct 22, 2024
94e3f0a
updated references using rst format
mathDR Oct 24, 2024
40c0e6e
updated docstrings
mathDR Oct 24, 2024
fb095e1
fixed linting
mathDR Oct 24, 2024
af8f22d
synced ademamix api to adamw
mathDR Oct 24, 2024
47e4248
added defaults to scale_by_ademamix
mathDR Oct 24, 2024
934761a
fixed syntaxerror
mathDR Oct 24, 2024
1b72988
updated docstrings
mathDR Oct 24, 2024
3e0699b
fixed typo
mathDR Oct 24, 2024
3ff8aba
reformatting note
mathDR Oct 24, 2024
c933d17
fixing formatting issues
mathDR Oct 24, 2024
bdeb3e2
reformatting note
mathDR Oct 24, 2024
8d38058
reformatting note
mathDR Oct 24, 2024
6f4ec8a
fixed ademamix docstring
mathDR Oct 24, 2024
e8ba763
fixed notebook ordering and line lengths
mathDR Oct 24, 2024
c84ce49
added docs image for ademamix
mathDR Oct 30, 2024
dac4260
added ademamix example to gallery
mathDR Oct 30, 2024
7c57abd
reran notebook with colab link
mathDR Oct 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/api/contrib.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ Experimental features and algorithms that don't meet the

.. autosummary::
acprop
ademamix
cocob
COCOBState
dadapt_adamw
Expand Down Expand Up @@ -37,6 +38,12 @@ Experimental features and algorithms that don't meet the
split_real_and_imaginary
SplitRealAndImaginaryState

AdEMAMix
~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. autofunction:: ademamix
.. autofunction:: scale_by_ademamix
.. autoclass:: ScaleByAdemamixState

Asynchronous-centering-Prop
~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. autofunction:: acprop
Expand Down
Loading
Loading