Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GroupNorm Sharded. support #4945

Merged
merged 2 commits into from
Feb 1, 2024
Merged

GroupNorm Sharded. support #4945

merged 2 commits into from
Feb 1, 2024

Conversation

yugaoTT
Copy link
Contributor

@yugaoTT yugaoTT commented Jan 25, 2024

  1. height and block sharing
  2. row major and tile layout

@yugaoTT yugaoTT self-assigned this Jan 25, 2024
@yugaoTT yugaoTT force-pushed the yugao/groupnorm_sharded branch 2 times, most recently from cae0ead to a6c3dbc Compare January 29, 2024 22:26
arakhmati
arakhmati suggested changes Jan 29, 2024
View reviewed changes
tt_eager/tt_dnn/op_library/groupnorm/groupnorm_op.hpp Outdated Show resolved Hide resolved
tt_eager/tt_dnn/op_library/groupnorm/groupnorm_op.hpp
struct GroupNormShardedMultiCoreProgramConfig {
CoreCoord compute_with_storage_grid_size;
MathFidelity math_fidelity;
DataType im_data_format;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im_data_format means image_data_format or is this a typo and should be in_data_format?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is the intermediate data format, for the intermediate cbs

mywoodstock
mywoodstock approved these changes Jan 30, 2024
View reviewed changes
Copy link
Contributor

@mywoodstock mywoodstock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! minor comments.
could you also add testing for batch = 2?

tt_eager/tt_dnn/op_library/groupnorm/groupnorm_op.hpp Outdated Show resolved Hide resolved
tests/tt_eager/python_api_testing/unit_testing/test_bert_ops.py Outdated
function_level_defaults,
):
in0_shape = [1, 1, M, K]
in1_shape = [1, 1, K, N]
bias_shape = [1, 1, N]
grid_size = (8, 7)
grid_size = (1, 1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not using multicore?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reverted it back to original code, was a mistake

tests/tt_eager/python_api_testing/unit_testing/test_groupnorm_sharded.py
ttl.tensor.TensorMemoryLayout.HEIGHT_SHARDED,
),
],
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you also test for batch = 2?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@yugaoTT yugaoTT force-pushed the yugao/groupnorm_sharded branch 2 times, most recently from 7738d7c to 7815e6e Compare January 30, 2024 16:07
@yugaoTT yugaoTT requested a review from tt-aho January 31, 2024 15:39
@yugaoTT yugaoTT merged commit 1741a55 into main Feb 1, 2024
4 checks passed
@yugaoTT yugaoTT deleted the yugao/groupnorm_sharded branch March 11, 2024 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@arakhmati arakhmati arakhmati approved these changes

@mywoodstock mywoodstock mywoodstock approved these changes

@TT-BrianLiu TT-BrianLiu Awaiting requested review from TT-BrianLiu

@tt-aho tt-aho Awaiting requested review from tt-aho

Assignees

@yugaoTT yugaoTT

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@yugaoTT @mywoodstock @arakhmati @tt-aho