cuda_opset.md

NVIDIA GPU plugin operation set specification

This specification document based on opset12 operation set supported in OpenVINO. Examples of operation instances are expressed as IR V10 xml snippets. Such IR is generated by the Model Optimizer. The semantics match corresponding nGraph operation classes declared in namespace opset12.

NOTE: NVIDIA GPU plugin doesn't support operations with dynamic shape.

Table of Contents

Layers	NVIDIA GPU plugin
Abs	Supported
Acos	Not Supported
Acosh	Not Supported
Add	Supported*
Asin	Not Supported
Asinh	Not Supported
Assign	Not Supported
Atan	Not Supported
Atanh	Not Supported
AvgPool	Supported
BatchNormInference	Not Supported
BatchToSpace	Not Supported
BinaryConvolution	Not Supported
Broadcast	Not Supported
Bucketize	Not Supported
CTCGreedyDecoder	Not Supported
CTCGreedyDecoderSeqLen	Not Supported
CTCLoss	Not Supported
Ceiling	Not Supported
Clamp	Supported
Concat	Supported
Constant	Supported
Convert	Supported
ConvertLike	Not Supported
Convolution	Supported
ConvolutionBackpropData	Supported*
Cos	Supported
Cosh	Supported
CumSum	Not Supported
DeformableConvolution	Not Supported
DeformablePSROIPooling	Not Supported
DepthToSpace	Not Supported
DetectionOutput	Supported
Divide	Supported
Elu	Supported
EmbeddingBagOffsetsSum	Not Supported
EmbeddingBagPackedSum	Not Supported
EmbeddingSegmentsSum	Not Supported
Equal	Supported
Erf	Not Supported
Exp	Supported
ExtractImagePatches	Not Supported
FakeQuantize	Supported
Floor	Supported
FloorMod	Supported
Gather	Supported*
GatherElements	Not Supported
GatherND_5	Not Supported
GatherTree	Not Supported
Gelu	Supported
Greater	Supported
GreaterEqual	Supported
GRN	Not Supported
GroupConvolution	Supported
GroupConvolutionBackpropData	Supported
GRUCell	Supported*
GRUSequence	Supported*
HardSigmoid	Not Supported
HSigmoid	Not Supported
HSwish	Supported
Interpolate	Supported*
Less	Supported
LessEqual	Supported
Log	Supported
LogicalAnd	Not Supported
LogicalNot	Supported
LogicalOr	Not Supported
LogicalXor	Not Supported
LogSoftmax	Not Supported
Loop	Not Supported
LRN	Not Supported
LSTMCell	Supported*
LSTMSequence	Supported*
MatMul	Supported
MaxPool	Supported
Maximum	Supported
Minimum	Supported
Mish	Supported
Mod	Supported
MVN	Supported
Multiply	Supported*
Negative	Not Supported
NonMaxSuppression	Not Supported
NonZero	Not Supported
NormalizeL2	Not Supported
NotEqual	Supported
OneHot	Not Supported
Pad	Supported*
Parameter	Supported
Power	Supported
PReLU	Supported
PriorBoxClustered	Not Supported
PriorBox	Not Supported
Proposal	Not Supported
PSROIPooling	Not Supported
Range	Supported*
ReLU	Supported
ReadValue	Not Supported
ReduceL1	Not Supported
ReduceL2	Not Supported
ReduceLogicalAnd	Not Supported
ReduceLogicalOr	Not Supported
ReduceMax	Supported
ReduceMean	Supported
ReduceMin	Supported
ReduceProd	Supported
ReduceSum	Supported
RegionYolo	Not Supported
ReorgYolo	Not Supported
Reshape	Supported
Result	Supported
ReverseSequence	Not Supported
RNNCell	Not Supported
RNNSequence	Not Supported
ROIAlign	Not Supported
ROIPooling	Not Supported
Round	Supported
ScatterElementsUpdate	Not Supported
ScatterNDUpdate	Supported
ScatterUpdate	Not Supported
Select	Supported
Selu	Not Supported
ShapeOf	Not Supported
ShuffleChannels	Not Supported
Sigmoid	Supported
Sign	Not Supported
Sin	Supported
Sinh	Supported
SoftMax	Supported
SoftPlus	Not Supported
SpaceToBatch	Not Supported
SpaceToDepth	Not Supported
Split	Supported*
Sqrt	Supported
SquaredDifference	Supported
Squeeze	Supported
StridedSlice	Supported
Subtract	Supported
Swish	Supported
Tan	Not Supported
Tanh	Supported
TensorIterator	Supported
Tile	Not Supported
TopK	Supported
Transpose	Supported
Unsqueeze	Supported
VariadicSplit	Supported

*- support is limited to the specific parameters. Refer to "Known Layers Limitation" section.

Known Layers Limitations

• 'Split' layer is not optimized
• 'LSTMCell'
  • supported OpenVino version: default only (LSTMCell v4)
  • supported activations: default only ({"sigmoid", "tanh", "tanh"});
  • supported activation alphas: default only (1.0);
  • supported activation betas: default only (0.0);
  • supported clipping mode: default only (no clipping: 0.0 or infinity clip value);
  • shape limitations: no combination of input_size == 1 and hidden_size == 1 simultaneously
  • cuDNN v8.1 crash observed, see https://forums.developer.nvidia.com/t/cudnn-crash-in-v8-1-x/194346
• 'GRUCell'
  • supported OpenVino version: default only (GRUCell v3)
  • supported activations: default only ({"sigmoid", "tanh"});
  • supported activation alphas: default only (1.0);
  • supported activation betas: default only (0.0);
  • supported clipping mode: default only (no clipping: 0.0 or infinity clip value);
  • shape limitations: no combination of input_size == 1 and hidden_size == 1 simultaneously
  • cuDNN v8.1 crash observed, see https://forums.developer.nvidia.com/t/cudnn-crash-in-v8-1-x/194346
• 'ConvolutionBackpropData'
  • Asymmetric paddings are not supported yet
• 'Interpolate'
  • Supported modes: nearest, cubic, linear;
  • Only default zero values are supported for pads_begin and pads_end attributes;
  • True value of antialias attribute is supported in linear mode only;
  • Only 4D data tensors are supported in nearest mode;
• 'LSTMSequence'
  • All limitations of LSTMCell
  • "REVERSE" direction is not supported
• 'GRUSequence'
  • All limitations of GRUCell
  • "REVERSE" and "BIDIRECTIONAL" directions are not supported
• 'Add'
  • supported auto_broadcast types are: NONE and NUMPY;
• 'Multiply'
  • supported auto_broadcast types are: NONE and NUMPY;
• 'Gather'
  • supported range of values for indices tensor is [0, data[axis] - 1], if a value is out of this range, CUDA runtime will crash during one of the subsequent operations.
• 'Interpolate'
  • nearest, linear, cubic values of mode are supported only.
• 'Pad'
  • constant value of pad_mode is supported only.