Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add transformer class for review #11491

Closed
wants to merge 78 commits into from
+1,144 −0
Closed

add transformer class for review #11491

Changes from 14 commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
1aa3ee3
add transformer class for review
paarthneekhara Dec 5, 2024
bc30ac9
Apply isort and black reformatting
paarthneekhara Dec 5, 2024
2878850
Addressed most comments, tested it with t5 training locally, but woul…
paarthneekhara Dec 14, 2024
8c2443d
Apply isort and black reformatting
paarthneekhara Dec 14, 2024
53827ed
rename
blisc Dec 16, 2024
c9c2365
rename ConvNorm; add docstrings; unest context_xattn params dict; rem…
blisc Dec 16, 2024
5bafb67
remove init_gain from ConvolutionLayer since it gets over-written in …
blisc Dec 16, 2024
6be554a
Apply isort and black reformatting
blisc Dec 16, 2024
fdc28a6
style
blisc Dec 16, 2024
8bf3736
replace non_linearity options from string to callable
blisc Dec 16, 2024
b3171f1
Apply isort and black reformatting
blisc Dec 16, 2024
62b3490
remove init_weight_method
blisc Dec 16, 2024
147be91
Merge branch 'tts_new_transformer' of github.com:paarthneekhara/NeMo …
blisc Dec 16, 2024
074e192
change copyright header
blisc Dec 16, 2024
d4793c3
fixed learnable position embedding, pos embedding args, removed x-att…
paarthneekhara Dec 19, 2024
ca6b8e7
Apply isort and black reformatting
paarthneekhara Dec 19, 2024
bb26ec9
Apply suggestions from code review from Xuesong
paarthneekhara Dec 20, 2024
3a28bf4
remove flash attn, fix residual attn bug, remove rope, test with t5tts
paarthneekhara Dec 20, 2024
4d44246
Apply isort and black reformatting
paarthneekhara Dec 20, 2024
b1cd511
Update nemo/collections/tts/modules/transformer_2412.py
XuesongYang Dec 21, 2024
98c9496
fixed typos, add type hints.
XuesongYang Dec 21, 2024
76595ed
split Self and Cross Attention
blisc Jan 3, 2025
b5a2e63
Apply isort and black reformatting
blisc Jan 3, 2025
9bcc15f
add more documentation
blisc Jan 7, 2025
96b66d6
Apply isort and black reformatting
blisc Jan 7, 2025
4a826ad
fix is_causal param; remove post layernorm
blisc Jan 8, 2025
1a25a3b
fixed typos and remove redudant vairable declaration.
XuesongYang Jan 9, 2025
f533749
bugfix: missing B, T when self.is_causal is False.
XuesongYang Jan 9, 2025
7d32c0c
combine the addition and multiplication ops into a single one.
XuesongYang Jan 9, 2025
894aaee
Update nemo/collections/tts/modules/transformer_2412.py
XuesongYang Jan 10, 2025
aed46c9
Update nemo/collections/tts/modules/transformer_2412.py
XuesongYang Jan 10, 2025
66a1709
flip mask
blisc Jan 18, 2025
20b7f8d
Selfvc ICML (#26)
paarthneekhara May 26, 2024
ca90fe5
self vc updates
paarthneekhara May 26, 2024
752c14f
ci: Fix release tag (#10402)
ko3n1g Sep 8, 2024
ad5b4b9
[🤠]: Howdy folks, let's bump `Dockerfile.ci` to 73e7b58 ! (#10779)
ko3n1g Oct 7, 2024
5f21a9d
fix: Resolve mutable default issue in MultiModalSampleConfig dataclas…
michal2409 Oct 28, 2024
006c6df
add transformer class for review
paarthneekhara Dec 5, 2024
ce184c6
Apply isort and black reformatting
paarthneekhara Dec 5, 2024
83c864a
Addressed most comments, tested it with t5 training locally, but woul…
paarthneekhara Dec 14, 2024
8f2d95b
Apply isort and black reformatting
paarthneekhara Dec 14, 2024
0e87ca4
rename
blisc Dec 16, 2024
769a5a8
rename ConvNorm; add docstrings; unest context_xattn params dict; rem…
blisc Dec 16, 2024
4c96021
remove init_gain from ConvolutionLayer since it gets over-written in …
blisc Dec 16, 2024
1781f20
Apply isort and black reformatting
blisc Dec 16, 2024
1e57504
style
blisc Dec 16, 2024
9e62b1c
replace non_linearity options from string to callable
blisc Dec 16, 2024
d6fb4b0
remove init_weight_method
blisc Dec 16, 2024
3937d8a
Apply isort and black reformatting
blisc Dec 16, 2024
209ae01
change copyright header
blisc Dec 16, 2024
3a20080
fixed learnable position embedding, pos embedding args, removed x-att…
paarthneekhara Dec 19, 2024
2608548
Apply isort and black reformatting
paarthneekhara Dec 19, 2024
63f71ec
Apply suggestions from code review from Xuesong
paarthneekhara Dec 20, 2024
354916b
remove flash attn, fix residual attn bug, remove rope, test with t5tts
paarthneekhara Dec 20, 2024
70d68b6
Apply isort and black reformatting
paarthneekhara Dec 20, 2024
39e96d0
Update nemo/collections/tts/modules/transformer_2412.py
XuesongYang Dec 21, 2024
762de59
fixed typos, add type hints.
XuesongYang Dec 21, 2024
1203bf9
split Self and Cross Attention
blisc Jan 3, 2025
a013a2c
Apply isort and black reformatting
blisc Jan 3, 2025
16a994c
add more documentation
blisc Jan 7, 2025
2ba67c2
Apply isort and black reformatting
blisc Jan 7, 2025
64f9205
fix is_causal param; remove post layernorm
blisc Jan 8, 2025
be0f2a7
fixed typos and remove redudant vairable declaration.
XuesongYang Jan 9, 2025
b971ed6
bugfix: missing B, T when self.is_causal is False.
XuesongYang Jan 9, 2025
06a02a1
combine the addition and multiplication ops into a single one.
XuesongYang Jan 9, 2025
debab72
Update nemo/collections/tts/modules/transformer_2412.py
XuesongYang Jan 10, 2025
4f23bbe
Update nemo/collections/tts/modules/transformer_2412.py
XuesongYang Jan 10, 2025
34b8bf9
flip mask
blisc Jan 18, 2025
0573be8
Apply isort and black reformatting
blisc Jan 18, 2025
1aad73e
Update nemo/collections/tts/modules/transformer_2412.py
XuesongYang Jan 18, 2025
d6b05ac
add unit tests.
XuesongYang Jan 14, 2025
fdc2fa8
Apply isort and black reformatting
XuesongYang Jan 18, 2025
f9bf5b6
rm files that were accidentally added when rebasing.
XuesongYang Jan 18, 2025
7fa7295
revert modified files when rebasing
XuesongYang Jan 18, 2025
59f137c
bug fix and multi-encoder test comments
paarthneekhara Jan 19, 2025
42c7ee3
Apply isort and black reformatting
paarthneekhara Jan 19, 2025
d2c6c7d
merge
blisc Jan 21, 2025
dcda0a2
move mask to front
blisc Jan 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Loading