How to Infer Data in Video Memory

[KunMengcode]

Describe the issue

I used FFmpeg to pull the RTSP video stream, which uses hard decoding.
Now I will store the decoded YVU data in the Video memory (later I will process it into RGB), and I want to directly use my ONNX model to infer the RGB data in the Video memory.
I don't quite understand how to operate.

Platform

Windows

OS Version

10

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.13

ONNX Runtime API

C++

Architecture

X64

Execution Provider

TensorRT

[RyanUnderhill]

If you're using C++ you want to do something like this (incomplete, but should convey the idea). You want the memory info to be cuda, and you want to create a tensor using the existing pointers (vs having Onnxruntime allocate it):

// Create Ort::MemoryInfo instance for GPU memory
Ort::MemoryInfo memory_info = Ort::MemoryInfo::Create("Cuda", OrtAllocatorType::OrtDeviceAllocator, 0, OrtMemType::OrtMemTypeDefault);

// Allocate memory on the GPU
float* input_data = (Your Cuda GPU allocated memory)

// Create tensor with pre-allocated GPU memory
// Note: this doesn't allocate memory, it just wraps the user-provided buffer in an OrtValue
Ort::Value input_tensor = Ort::Value::CreateTensor<float>(memory_info, input_data, input_tensor_size, input_shape.data(), input_shape.size());

[KunMengcode]

Thank you for your answer,I think I roughly understand what you mean, and I will try it out in the coming period. And refine the example you provided to end the issue.
Based on your prompt, I have found other previous proposals that also illustrate this issue.
#11528
And code examples
https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/test/shared_lib/test_inference.cc
(Download Find "USE_CUDA..." keywords)