Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Training] The gradient builder has not been registered: ConvTranspose for node #17710

Closed
elephantpanda opened this issue Sep 26, 2023 · 5 comments
Labels
stale issues that have not been addressed in a while; categorized by a bot training issues related to ONNX Runtime training; typically submitted using template

Comments

@elephantpanda
Copy link

elephantpanda commented Sep 26, 2023

Describe the issue

When trying to create the training artifacts for a model containing a ConvTranspose2d node I get this error.

Using: https://onnxruntime.ai/docs/api/python/on_device_training/training_artifacts.html

Basically I just want it to create a training ONNX containing a ConvTransposeGrad node the same way it creates a ConvGrad node for a Conv2D node.

But I get the error: "The gradient builder has not been registered: ConvTranspose for node"

Anyway to work round this?

P.S. Is there a list of supported/unsupported operators for training artifacts?

(Possibly this may be implemented in 1.16 but when I do pip install onnxruntime.training it installs version 1.15.1)

To reproduce

Try to create training artifacts with a model containing a ConvTranspose2D node.

Urgency

No response

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.15.1

PyTorch Version

1.13.1+cpu

Execution Provider

Default CPU

Execution Provider Library Version

CPU

@elephantpanda elephantpanda added the training issues related to ONNX Runtime training; typically submitted using template label Sep 26, 2023
@baijumeswani
Copy link
Contributor

ConvTransposeGrad is available for CUDA EP in ORT 1.16 (PR for reference #17201).

The ConvTransposeGrad kernel has not been implemented for CPU EP yet.

@elephantpanda
Copy link
Author

ConvTransposeGrad is available for CUDA EP in ORT 1.16 (PR for reference #17201).

The ConvTransposeGrad kernel has not been implemented for CPU EP yet.

Thanks how do I get it? When I do "pip install onnxruntime.training" it gives me version 1.15.1

@baijumeswani
Copy link
Contributor

python -m pip install cerberus flatbuffers h5py numpy>=1.16.6 onnx packaging protobuf sympy setuptools>=41.4.0
pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT/pypi/simple/ onnxruntime-training

^ on linux with cuda 11.8 support.

Refer to https://onnxruntime.ai/index.html#getStartedTable to get other installation commands.

@github-actions
Copy link
Contributor

This issue has been automatically marked as stale due to inactivity and will be closed in 7 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

@github-actions github-actions bot added the stale issues that have not been addressed in a while; categorized by a bot label Oct 28, 2023
Copy link
Contributor

github-actions bot commented Nov 4, 2023

This issue has been automatically closed due to inactivity. Please reactivate if further support is needed.

@github-actions github-actions bot closed this as completed Nov 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale issues that have not been addressed in a while; categorized by a bot training issues related to ONNX Runtime training; typically submitted using template
Projects
None yet
Development

No branches or pull requests

2 participants