Add an option in OpenVINOProviderOptions to support the queue-based overload for creating ClContext

[fireyoshiqc]

Description

Currently, the OpenVINO EP only provides a way to share an OpenCL context (for IO Buffering) through a context pointer, given in its provider options (either the struct or string-map based API).

This is problematic when wanting to use a specific OpenCL queue instead, as there is no way to do so through the current API, even though OpenVINO itself provides an overload for it.

This PR addresses this issue by adding an option to the OpenVINO EP provider options that enables the use of that second overload. This makes it possible to explicitly share OpenCL command queues when using OpenVINO EP.

Motivation and Context

As described in issue #19697.

[fireyoshiqc]

@microsoft-github-policy-service agree

[jywu-msft]

+@sfatimar , @preetha-intel

[jywu-msft]

/azp run Linux OpenVINO CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[preetha-intel]

The struct is frozen and its our legacy API. Kindly upgrade to ORT ProviderOptions Map structure

[preetha-intel]

@fireyoshiqc Can you explain on the use case behind OpenCL queue option and its impact over using OpenCL context.

[fireyoshiqc]

@fireyoshiqc Can you explain on the use case behind OpenCL queue option and its impact over using OpenCL context.

@preetha-intel
Sure thing. In the project I'm working on (camera image processing pipeline), we are using the OpenVINO API v2.0 for sharing our existing OpenCL queue (used by OpenCV and previously set up on the Intel iGPU) when creating a ClContext for inference on the GPU:

// OpenVINO headers:
    /**
     * @brief Constructs context object from user-supplied OpenCL context handle
     * @param core A reference to OpenVINO Runtime Core object
     * @param queue An OpenCL queue to be used to create shared remote context. Queue will be reused inside the plugin.
     * @note Only latency mode is supported for such context sharing case.
     */
    ClContext(Core& core, cl_command_queue queue) {
        cl_context ctx;
        auto res = clGetCommandQueueInfo(queue, CL_QUEUE_CONTEXT, sizeof(cl_context), &ctx, nullptr);
        OPENVINO_ASSERT(res == CL_SUCCESS, "Can't get context from given opencl queue");
        AnyMap context_params = {{ov::intel_gpu::context_type.name(), ov::intel_gpu::ContextType::OCL},
                                 {ov::intel_gpu::ocl_context.name(), static_cast<gpu_handle_param>(ctx)},
                                 {ov::intel_gpu::ocl_queue.name(), static_cast<gpu_handle_param>(queue)}};
        *this = core.create_context(device_name, context_params).as<ClContext>();

// In our code:
	if( inferOnGPU )
	{
		auto openCvOpenClQueue{ cv::ocl::OpenCLExecutionContext::getCurrent().getQueue() };

		// create the context from the current command queue so that OpenVINO can infer Async with correct scheduling.
		context = std::make_shared<ov::intel_gpu::ocl::ClContext>( core, static_cast<cl_command_queue>( openCvOpenClQueue.ptr() ) );
	}

We wanted to migrate to using ONNX Runtime with the OpenVINO EP to standardize our API across different backends. However, this function isn't "available" to use through the Provider options because only a context can be supplied, not a queue.
Using this constructor overload allows us to ensure that async inference is correctly scheduled after previous GPU processing operations, and the results can be safely used by subsequent GPU operations without explicitly waiting (.wait()) on the inference request. We're using multiple queues (to reduce image copy overhead, for example), so it's important for us to ensure that the right queue is used.