Skip to content

Commit

Permalink
[QNN EP] Support Conv + Clip/Relu fusion (#21537)
Browse files Browse the repository at this point in the history
### Description
- Supports quantized Conv + Activation on the HTP backend:
- Translates `DQs -> Conv -> Relu/Clip -> Q` into a single QNN Conv
operator if the Relu (or Clip) are redundant.



### Motivation and Context
Expands support for QDQ models created with tools that do not wrap Relu
or Clip with QDQ nodes.

This PR introduces the `IQnnNodeGroup` class. In the same way that a
`NodeUnit` represents a collection of `Nodes`, a `IQnnNodeGroup` can
represent one or more `NodeUnits` that are translated into a QNN
operator. QNN EP parses the ONNX graph to create a list of
`IQnnNodeGroup` objects, each representing a single `NodeUnit` or a
fusion of multiple `NodeUnits`.
  • Loading branch information
adrianlizarraga authored Aug 2, 2024
1 parent b87e8ed commit 0e708de
Show file tree
Hide file tree
Showing 19 changed files with 1,670 additions and 485 deletions.
15 changes: 15 additions & 0 deletions onnxruntime/core/framework/node_unit.cc
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
#if !defined(ORT_MINIMAL_BUILD) || defined(ORT_EXTENDED_MINIMAL_BUILD)

#include "node_unit.h"
#include <utility>
#include "core/graph/graph_viewer.h"

namespace onnxruntime {
Expand Down Expand Up @@ -272,6 +273,20 @@ NodeUnit::NodeUnit(const GraphViewer& graph_viewer, const QDQ::NodeGroup& node_g
}
}

NodeUnit::NodeUnit(gsl::span<const Node* const> dq_nodes, const Node& target_node,
gsl::span<const Node* const> q_nodes, Type unit_type,
gsl::span<const NodeUnitIODef> inputs, gsl::span<const NodeUnitIODef> outputs,
size_t input_edge_count, Node::EdgeSet output_edges)
: dq_nodes_(dq_nodes.begin(), dq_nodes.end()),
target_node_(target_node),
q_nodes_(q_nodes.begin(), q_nodes.end()),
type_(unit_type),
inputs_(inputs.begin(), inputs.end()),
outputs_(outputs.begin(), outputs.end()),
input_edge_count_(input_edge_count),
output_edges_(std::move(output_edges)) {
}

const std::string& NodeUnit::Domain() const noexcept { return target_node_.Domain(); }
const std::string& NodeUnit::OpType() const noexcept { return target_node_.OpType(); }
const std::string& NodeUnit::Name() const noexcept { return target_node_.Name(); }
Expand Down
4 changes: 4 additions & 0 deletions onnxruntime/core/framework/node_unit.h
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,10 @@ class NodeUnit {
public:
explicit NodeUnit(const Node& node);
explicit NodeUnit(const GraphViewer& graph_viewer, const QDQ::NodeGroup& node_group);
NodeUnit(gsl::span<const Node* const> dq_nodes, const Node& target_node,
gsl::span<const Node* const> q_nodes, Type unit_type,
gsl::span<const NodeUnitIODef> inputs, gsl::span<const NodeUnitIODef> outputs,
size_t input_edge_count, Node::EdgeSet output_edges);

Type UnitType() const noexcept { return type_; }

Expand Down
294 changes: 0 additions & 294 deletions onnxruntime/core/providers/qnn/builder/qnn_fusions.cc

This file was deleted.

38 changes: 0 additions & 38 deletions onnxruntime/core/providers/qnn/builder/qnn_fusions.h

This file was deleted.

Loading

0 comments on commit 0e708de

Please sign in to comment.