Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reuse existing output node when replacing operator with a fusion #401

Merged
merged 3 commits into from
Nov 9, 2024
Merged

Reuse existing output node when replacing operator with a fusion #401

merged 3 commits into from
Nov 9, 2024

Commits on Nov 9, 2024

  1. Reuse existing output node when replacing operator with a fusion 

    When replacing a subgraph with a fused operator, reuse the output value node
from the subgraph instead of creating a new one. This preserves metadata such as
the name and shape associated with that value node. Also it simplifies the code
by removing the need to replace all references to the previous output node with
the new one.

This improves the runtime of the Whisper example using the whisper-base model by
~25% by fixing an issue where fused Transpose + MatMul operations did not get
used.
    @robertknight
    robertknight committed Nov 9, 2024
    Configuration menu
    Copy the full SHA
    809ba69 View commit details
    Browse the repository at this point in the history

  2. Correct outdated comment for GraphOptimizer::optimize

    @robertknight
    robertknight committed Nov 9, 2024
    Configuration menu
    Copy the full SHA
    6a3abb4 View commit details
    Browse the repository at this point in the history

  3. Make Graph::add_node private again 

    Exposing `add_node` as a public method creates a hazard as it allows callers to
add nodes to the graph without post-processing steps that other methods do (eg.
`add_op`). Make this method private again and add a more specific
`add_constant_node` alternative to handle the single use case for it outside the
graph module.
    @robertknight
    robertknight committed Nov 9, 2024
    Configuration menu
    Copy the full SHA
    4937d83 View commit details
    Browse the repository at this point in the history