Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DML EP Register Opset 21 #22547

Merged
merged 4 commits into from
Oct 25, 2024
Merged

DML EP Register Opset 21 #22547

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6c65e4f
add castlike
dtang317 Oct 21, 2024
489b4f0
add flatten, pad, transpose
dtang317 Oct 21, 2024
2757079
remove int4/unit4 from data types
dtang317 Oct 23, 2024
a7fd118
update OperatorKernels.md
dtang317 Oct 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions onnxruntime/core/providers/dml/DmlExecutionProvider/src/Operators/DmlOperatorCast.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,5 +50,6 @@ class DmlOperatorCast : public DmlOperator
DML_OP_DEFINE_CREATION_FUNCTION(Cast, DmlOperatorCast);
DML_OP_DEFINE_CREATION_FUNCTION(CastLike15, DmlOperatorCast);
DML_OP_DEFINE_CREATION_FUNCTION(CastLike19, DmlOperatorCast);
DML_OP_DEFINE_CREATION_FUNCTION(CastLike21, DmlOperatorCast);
fdwr marked this conversation as resolved.
Show resolved Hide resolved

} // namespace Dml
1 change: 1 addition & 0 deletions onnxruntime/core/providers/dml/DmlExecutionProvider/src/Operators/DmlOperatorPadding.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,5 +123,6 @@ DML_OP_DEFINE_CREATION_FUNCTION(Pad11, VersionedKernel<DmlOperatorPadding, 11>);
DML_OP_DEFINE_CREATION_FUNCTION(Pad13, VersionedKernel<DmlOperatorPadding, 13>);
DML_OP_DEFINE_CREATION_FUNCTION(Pad18, VersionedKernel<DmlOperatorPadding, 18>);
DML_OP_DEFINE_CREATION_FUNCTION(Pad19, VersionedKernel<DmlOperatorPadding, 19>);
DML_OP_DEFINE_CREATION_FUNCTION(Pad21, VersionedKernel<DmlOperatorPadding, 21>);

} // namespace Dml
13 changes: 13 additions & 0 deletions onnxruntime/core/providers/dml/DmlExecutionProvider/src/Operators/OperatorRegistration.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,7 @@ DML_OP_EXTERN_CREATION_FUNCTION(Pad11);
DML_OP_EXTERN_CREATION_FUNCTION(Pad13);
DML_OP_EXTERN_CREATION_FUNCTION(Pad18);
DML_OP_EXTERN_CREATION_FUNCTION(Pad19);
DML_OP_EXTERN_CREATION_FUNCTION(Pad21);
DML_OP_EXTERN_CREATION_FUNCTION(SpaceToDepth);
DML_OP_EXTERN_CREATION_FUNCTION(DepthToSpace);
DML_OP_EXTERN_CREATION_FUNCTION(Sqrt);
Expand Down Expand Up @@ -445,6 +446,7 @@ DML_OP_EXTERN_CREATION_FUNCTION(DynamicQuantizeMatMul);
DML_OP_EXTERN_CREATION_FUNCTION(Cast);
DML_OP_EXTERN_CREATION_FUNCTION(CastLike15);
DML_OP_EXTERN_CREATION_FUNCTION(CastLike19);
DML_OP_EXTERN_CREATION_FUNCTION(CastLike21);
DML_OP_EXTERN_CREATION_FUNCTION(MemcpyFromHost);
DML_OP_EXTERN_CREATION_FUNCTION(MemcpyToHost);
DML_OP_EXTERN_CREATION_FUNCTION(TopK7);
Expand Down Expand Up @@ -599,6 +601,7 @@ constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListMaxPoo
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListMaxUnpool = {SupportedTensorDataTypes::Float16to32, SupportedTensorDataTypes::Int64};
constexpr static std::array<SupportedTensorDataTypes, 1> supportedTypeListIndices = { SupportedTensorDataTypes::Int32|SupportedTensorDataTypes::Int64 };
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListCast = { SupportedTensorDataTypes::AllScalars, SupportedTensorDataTypes::AllScalars };
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListCast21 = { SupportedTensorDataTypes::AllScalars, SupportedTensorDataTypes::UInt4 | SupportedTensorDataTypes::Int4 };
fdwr marked this conversation as resolved.
Show resolved Hide resolved
constexpr static std::array<SupportedTensorDataTypes, 1> supportedTypeListScalars8to32 = { SupportedTensorDataTypes::Scalars8to32 };
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListScatterGather = { SupportedTensorDataTypes::AllScalars, SupportedTensorDataTypes::Int32 | SupportedTensorDataTypes::Int64 };
constexpr static std::array<SupportedTensorDataTypes, 1> supportedTypeListScatterGatherND = { SupportedTensorDataTypes::AllScalars };
Expand All @@ -612,6 +615,7 @@ constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListDequan
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListIsNan = { SupportedTensorDataTypes::Float16to32, SupportedTensorDataTypes::Bool };
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListIsInf = { SupportedTensorDataTypes::Float32, SupportedTensorDataTypes::Bool };
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListConstantOfShape = { SupportedTensorDataTypes::Int64, SupportedTensorDataTypes::AllScalars };
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListConstantOfShape21 = { SupportedTensorDataTypes::Int64 | SupportedTensorDataTypes::Int4 | SupportedTensorDataTypes::UInt4, SupportedTensorDataTypes::AllScalars };
dtang317 marked this conversation as resolved.
Show resolved Hide resolved
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListWhere = { SupportedTensorDataTypes::Bool, SupportedTensorDataTypes::AllScalars };
constexpr static std::array<SupportedTensorDataTypes, 3> supportedTypeListOneHot = /* indices, depth, values */ { SupportedTensorDataTypes::Ints32to64, SupportedTensorDataTypes::AllScalars, SupportedTensorDataTypes::AllScalars };
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListLogicalComparison7 = /* A&B,C */ { SupportedTensorDataTypes::Float16to32, SupportedTensorDataTypes::Bool };
Expand All @@ -630,6 +634,9 @@ constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListLayerN
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListShape = {SupportedTensorDataTypes::All, SupportedTensorDataTypes::Int64};
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListSize = {SupportedTensorDataTypes::All, SupportedTensorDataTypes::Int64};
constexpr static std::array<SupportedTensorDataTypes, 1> supportedTypeListQLinearSigmoid = {SupportedTensorDataTypes::UInt8 | SupportedTensorDataTypes::Int8};
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListFlatten21 = {SupportedTensorDataTypes::AllScalars, SupportedTensorDataTypes::Int4 | SupportedTensorDataTypes::UInt4};
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListPad21 = {SupportedTensorDataTypes::AllScalars, SupportedTensorDataTypes::Int4 | SupportedTensorDataTypes::UInt4};
constexpr static std::array<SupportedTensorDataTypes, 2> supportedTypeListTranspose21 = {SupportedTensorDataTypes::AllScalars, SupportedTensorDataTypes::Int4 | SupportedTensorDataTypes::UInt4};

constexpr static std::array<SupportedTensorDataTypes, 4> supportedTypeListQAttention = {
SupportedTensorDataTypes::Ints8Bit,
Expand Down Expand Up @@ -792,6 +799,7 @@ constexpr static OperatorRegistrationInformation operatorRegistrationInformation
{REG_INFO_VER( 18, Split, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported, requiredConstantCpuInputs(1))},
{REG_INFO( 7, Transpose, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported)},
{REG_INFO( 13, Transpose, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported)},
{REG_INFO( 21, Transpose, typeNameListDefault, supportedTypeListTranspose21, DmlGraphSupport::Supported)},
dtang317 marked this conversation as resolved.
Show resolved Hide resolved
{REG_INFO( 7, Concat, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported)},
{REG_INFO( 11, Concat, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported)}, // Adds negative axis.
{REG_INFO( 13, Concat, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported)}, // Adds negative axis.
Expand All @@ -804,6 +812,7 @@ constexpr static OperatorRegistrationInformation operatorRegistrationInformation
{REG_INFO_VER( 11, Pad, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported, requiredConstantCpuInputs(1, 2) /*pads, value*/)}, // https://microsoft.visualstudio.com/OS/_workitems/edit/26007728
{REG_INFO_VER( 13, Pad, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported, requiredConstantCpuInputs(1, 2) /*pads, value*/)}, // https://microsoft.visualstudio.com/OS/_workitems/edit/26007728
{REG_INFO_VER( 18, Pad, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported, requiredConstantCpuInputs(1, 2, 3) /*pads, value, axes*/)},
{REG_INFO_VER( 21, Pad, typeNameListDefault, supportedTypeListPad21, DmlGraphSupport::Supported, requiredConstantCpuInputs(1, 2, 3) /*pads, value, axes*/)},

#if DML_TARGET_VERSION >= 0x6400
{REG_INFO_VER( 19, Pad, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported, requiredConstantCpuInputs(1, 2, 3) /*pads, value, axes*/)},
Expand All @@ -819,6 +828,7 @@ constexpr static OperatorRegistrationInformation operatorRegistrationInformation
{REG_INFO( 8, Expand, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported, requiredConstantCpuInputs(1))},
{REG_INFO( 13, Expand, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported, requiredConstantCpuInputs(1))},
{REG_INFO( 9, ConstantOfShape, typeNameListConstantOfShape, supportedTypeListConstantOfShape, DmlGraphSupport::Supported, requiredConstantCpuInputs(0))},
{REG_INFO( 21, ConstantOfShape, typeNameListConstantOfShape, supportedTypeListConstantOfShape21, DmlGraphSupport::Supported, requiredConstantCpuInputs(0))},
{REG_INFO( 7, Gather, typeNameListScatterGather, supportedTypeListScatterGather, DmlGraphSupport::Supported)},
{REG_INFO( 11, Gather, typeNameListScatterGather, supportedTypeListScatterGather, DmlGraphSupport::Supported)},
{REG_INFO( 13, Gather, typeNameListScatterGather, supportedTypeListScatterGather, DmlGraphSupport::Supported)},
Expand Down Expand Up @@ -853,6 +863,7 @@ constexpr static OperatorRegistrationInformation operatorRegistrationInformation
{REG_INFO_COPY( 9, Flatten, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported)},
{REG_INFO_COPY(11, Flatten, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported)},
{REG_INFO_COPY(13, Flatten, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported)},
{REG_INFO_COPY(21, Flatten, typeNameListDefault, supportedTypeListFlatten21, DmlGraphSupport::Supported)},
{REG_INFO_COPY( 7, Squeeze, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported)},
{REG_INFO_COPY(11, Squeeze, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported)},
{REG_INFO_COPY(13, Squeeze, typeNameListDefault, supportedTypeListAllScalars, DmlGraphSupport::Supported, requiredConstantCpuInputs(1))},
Expand Down Expand Up @@ -1087,6 +1098,7 @@ constexpr static OperatorRegistrationInformation operatorRegistrationInformation
{REG_INFO( 21, Cast, typeNameListTwo, supportedTypeListCast, DmlGraphSupport::Supported)},
{REG_INFO_VER( 15, CastLike, typeNameListTwo, supportedTypeListCast, DmlGraphSupport::Supported)},
{REG_INFO_VER( 19, CastLike, typeNameListTwo, supportedTypeListCast, DmlGraphSupport::Supported)},
{REG_INFO_VER( 21, CastLike, typeNameListTwo, supportedTypeListCast21, DmlGraphSupport::Supported)},
{REG_INFO( 7, MemcpyFromHost, typeNameListDefault, supportedTypeListAll)},
{REG_INFO( 7, MemcpyToHost, typeNameListDefault, supportedTypeListAll)},
{REG_INFO_VER( 7, TopK, typeNameListTopK, supportedTypeListTopK, DmlGraphSupport::Supported)},
Expand All @@ -1102,6 +1114,7 @@ constexpr static OperatorRegistrationInformation operatorRegistrationInformation
{REG_INFO( 7, Size, typeNameSize, supportedTypeListSize, DmlGraphSupport::NotSupported)},
{REG_INFO( 13, Size, typeNameSize, supportedTypeListSize, DmlGraphSupport::NotSupported)},
{REG_INFO( 19, Size, typeNameSize, supportedTypeListSize, DmlGraphSupport::NotSupported)},
{REG_INFO( 21, Size, typeNameSize, supportedTypeListSize, DmlGraphSupport::NotSupported)},
{REG_INFO_DYNAMIC_OUTPUTS( 9, NonZero, typeNameListDefault, supportedTypeListNonZero, DmlGraphSupport::NotSupported)},
{REG_INFO_DYNAMIC_OUTPUTS(13, NonZero, typeNameListDefault, supportedTypeListNonZero, DmlGraphSupport::NotSupported)},

Expand Down
3 changes: 3 additions & 0 deletions onnxruntime/core/providers/dml/OperatorAuthorHelper/OperatorHelper.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1673,6 +1673,7 @@ using ShapeInferenceHelper_Flatten7 = FlattenHelper;
using ShapeInferenceHelper_Flatten9 = FlattenHelper;
using ShapeInferenceHelper_Flatten11 = FlattenHelper;
using ShapeInferenceHelper_Flatten13 = FlattenHelper;
using ShapeInferenceHelper_Flatten21 = FlattenHelper;
using ShapeInferenceHelper_Split7 = VersionedOpsetHelper<SplitHelper, 7>;
using ShapeInferenceHelper_Split11 = VersionedOpsetHelper<SplitHelper, 11>;
using ShapeInferenceHelper_Split13 = VersionedOpsetHelper<SplitHelper, 13>;
Expand All @@ -1689,6 +1690,7 @@ using ShapeInferenceHelper_Pad11 = VersionedOpsetHelper<PaddingHelper, 11>;
using ShapeInferenceHelper_Pad13 = VersionedOpsetHelper<PaddingHelper, 13>;
using ShapeInferenceHelper_Pad18 = VersionedOpsetHelper<PaddingHelper, 18>;
using ShapeInferenceHelper_Pad19 = VersionedOpsetHelper<PaddingHelper, 19>;
using ShapeInferenceHelper_Pad21 = VersionedOpsetHelper<PaddingHelper, 21>;

using ShapeInferenceHelper_SpaceToDepth = SpaceToDepthHelper;
using ShapeInferenceHelper_DepthToSpace = DepthToSpaceHelper;
Expand Down Expand Up @@ -1865,6 +1867,7 @@ using ShapeInferenceHelper_Range = RangeHelper;

using ShapeInferenceHelper_CastLike15 = GetOutputShapeAsInputShapeHelper;
using ShapeInferenceHelper_CastLike19 = GetOutputShapeAsInputShapeHelper;
using ShapeInferenceHelper_CastLike21 = GetOutputShapeAsInputShapeHelper;

using ShapeInferenceHelper_DmlFusedConv = ConvHelper;
using ShapeInferenceHelper_DmlFusedConvTranspose = ConvTransposeHelper;
Expand Down
6 changes: 6 additions & 0 deletions onnxruntime/core/providers/dml/OperatorAuthorHelper/OperatorVersions.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -446,6 +446,12 @@
static const int sc_sinceVer_Reshape = 21;
static const int sc_sinceVer_Cast = 21;
static const int sc_sinceVer_Shape = 21;
static const int sc_sinceVer_Size = 21;

Check warning

Code scanning / PREfast

The const variable 'OperatorHelper::OnnxOperatorSet21::sc_sinceVer_Size' can be computed at compile-time. Consider using constexpr (con.5). Warning

The const variable 'OperatorHelper::OnnxOperatorSet21::sc_sinceVer_Size' can be computed at compile-time. Consider using constexpr (con.5).
static const int sc_sinceVer_CastLike = 21;

Check warning

Code scanning / PREfast

The const variable 'OperatorHelper::OnnxOperatorSet21::sc_sinceVer_CastLike' can be computed at compile-time. Consider using constexpr (con.5). Warning

The const variable 'OperatorHelper::OnnxOperatorSet21::sc_sinceVer_CastLike' can be computed at compile-time. Consider using constexpr (con.5).
static const int sc_sinceVer_ConstantOfShape = 21;

Check warning

Code scanning / PREfast

The const variable 'OperatorHelper::OnnxOperatorSet21::sc_sinceVer_ConstantOfShape' can be computed at compile-time. Consider using constexpr (con.5). Warning

The const variable 'OperatorHelper::OnnxOperatorSet21::sc_sinceVer_ConstantOfShape' can be computed at compile-time. Consider using constexpr (con.5).
static const int sc_sinceVer_Flatten = 21;

Check warning

Code scanning / PREfast

The const variable 'OperatorHelper::OnnxOperatorSet21::sc_sinceVer_Flatten' can be computed at compile-time. Consider using constexpr (con.5). Warning

The const variable 'OperatorHelper::OnnxOperatorSet21::sc_sinceVer_Flatten' can be computed at compile-time. Consider using constexpr (con.5).
static const int sc_sinceVer_Pad = 21;

Check warning

Code scanning / PREfast

The const variable 'OperatorHelper::OnnxOperatorSet21::sc_sinceVer_Pad' can be computed at compile-time. Consider using constexpr (con.5). Warning

The const variable 'OperatorHelper::OnnxOperatorSet21::sc_sinceVer_Pad' can be computed at compile-time. Consider using constexpr (con.5).
static const int sc_sinceVer_Transpose = 21;

Check warning

Code scanning / PREfast

The const variable 'OperatorHelper::OnnxOperatorSet21::sc_sinceVer_Transpose' can be computed at compile-time. Consider using constexpr (con.5). Warning

The const variable 'OperatorHelper::OnnxOperatorSet21::sc_sinceVer_Transpose' can be computed at compile-time. Consider using constexpr (con.5).
}

namespace MsftOperatorSet1
Expand Down
Loading