Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Layer norm fusion deepspeed stage3 changes #17614

Merged
merged 12 commits into from
Sep 21, 2023
Prev Previous commit
Next Next commit
add information on the test
  • Loading branch information
ajindal1 committed Sep 21, 2023
commit 5670213bd85e06d197566b0625ff7cf17ff3337c
2 changes: 2 additions & 0 deletions onnxruntime/test/optimizer/graph_transform_test_layernorm.cc
Original file line number Diff line number Diff line change
@@ -429,6 +429,8 @@ TEST_F(GraphTransformationTests, SimplifiedLayerNormFusionTest) {
}
}

// It tests the scenario when scale or bias are not Graph Inputs and not initialized in Graph
// To test this added a Identity node after Scale and Bias terms to ensure LayerNormFusion works properly
TEST_F(GraphTransformationTests, LayerNormScaleBiasTest) {
constexpr const ORTCHAR_T* model_uri = MODEL_FOLDER "fusion/layer_norm_fusion_scale_bias.onnx";
ajindal1 marked this conversation as resolved.
Show resolved Hide resolved
std::shared_ptr<Model> p_model;