Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add transformer class for review #11491

Draft
wants to merge 23 commits into
base: main
Choose a base branch
from
+698 −0
Draft

add transformer class for review #11491

Changes from 5 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
1aa3ee3
add transformer class for review
paarthneekhara Dec 5, 2024
bc30ac9
Apply isort and black reformatting
paarthneekhara Dec 5, 2024
2878850
Addressed most comments, tested it with t5 training locally, but woul…
paarthneekhara Dec 14, 2024
8c2443d
Apply isort and black reformatting
paarthneekhara Dec 14, 2024
53827ed
rename
blisc Dec 16, 2024
c9c2365
rename ConvNorm; add docstrings; unest context_xattn params dict; rem…
blisc Dec 16, 2024
5bafb67
remove init_gain from ConvolutionLayer since it gets over-written in …
blisc Dec 16, 2024
6be554a
Apply isort and black reformatting
blisc Dec 16, 2024
fdc28a6
style
blisc Dec 16, 2024
8bf3736
replace non_linearity options from string to callable
blisc Dec 16, 2024
b3171f1
Apply isort and black reformatting
blisc Dec 16, 2024
62b3490
remove init_weight_method
blisc Dec 16, 2024
147be91
Merge branch 'tts_new_transformer' of github.com:paarthneekhara/NeMo …
blisc Dec 16, 2024
074e192
change copyright header
blisc Dec 16, 2024
d4793c3
fixed learnable position embedding, pos embedding args, removed x-att…
paarthneekhara Dec 19, 2024
ca6b8e7
Apply isort and black reformatting
paarthneekhara Dec 19, 2024
bb26ec9
Apply suggestions from code review from Xuesong
paarthneekhara Dec 20, 2024
3a28bf4
remove flash attn, fix residual attn bug, remove rope, test with t5tts
paarthneekhara Dec 20, 2024
4d44246
Apply isort and black reformatting
paarthneekhara Dec 20, 2024
b1cd511
Update nemo/collections/tts/modules/transformer_2412.py
XuesongYang Dec 21, 2024
98c9496
fixed typos, add type hints.
XuesongYang Dec 21, 2024
76595ed
split Self and Cross Attention
blisc Jan 3, 2025
b5a2e63
Apply isort and black reformatting
blisc Jan 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Loading