RunAsync C# API crashes without any error

[vishnu-ms]

Describe the issue

I'm running Onnx runtime 1.16.3 in a .NET Core web application.
I'm able to run the model and get outputs when using session.Run() but want to be able to run the model async and evaluate any possible performance benefits.
However, the application crashes without any exception when just RunAsync calls, but returns a result when session.Run(options, inputValues, session.OutputNames) is called before "session.RunAsync".

This occurs both in a Linux container app and in local Docker container test (Linux using default CPU as ORT provider)

Can someone help resolve this issue?

To reproduce

Here's the C# code, can share ONNX file if needed

public async Task<double> RunInference(InferenceSession session, List<float> inputs)
{
    using RunOptions options = new RunOptions();
    using var inputOrt = OrtValue.CreateTensorValueFromMemory<float>(OrtMemoryInfo.DefaultInstance, inputs.ToArray(), [1, 30]);

    var outputOrtKey = OrtValue.CreateAllocatedTensorValue(OrtAllocator.DefaultInstance, TensorElementType.Int64, [1]);
    var outputOrtValue = OrtValue.CreateAllocatedTensorValue(OrtAllocator.DefaultInstance, TensorElementType.Float, [1]);

    using var ortMap = OrtValue.CreateMap(ref outputOrtKey, ref outputOrtValue);

    var inputOrtValues = new List<OrtValue> { inputOrt };

    var outputOrtValues = new List<OrtValue>
    {
        OrtValue.CreateAllocatedTensorValue(OrtAllocator.DefaultInstance, TensorElementType.Int64, [1]),
        OrtValue.CreateSequence(new List<OrtValue> { ortMap })
    };

    var inputValues = new Dictionary<string, OrtValue>
    {
        { session.InputNames[0], inputOrt }
    };

    // Works when line below is uncommented
    // var temp = session.Run(options, inputValues, session.OutputNames);
    var task = session.RunAsync(options, session.InputNames, inputOrtValues, session.OutputNames, outputOrtValues);

    var outputs2 = await task;

    var lastOutput = outputs2.Last();

    using var firstMap = lastOutput.GetValue(0, OrtAllocator.DefaultInstance);

    var inferenceMap = new Dictionary<long, double>();

    firstMap.ProcessMap((k, v) =>
    {
        var keySpan = k.GetTensorDataAsSpan<long>().ToArray();
        var valuesSpan = v.GetTensorDataAsSpan<float>().ToArray();
        var entryCount = (int)k.GetTensorTypeAndShape().ElementCount;

        for (int i = 0; i < entryCount; i++)
        {
            inferenceMap.Add(keySpan[i], valuesSpan[i]);
        }

    }, OrtAllocator.DefaultInstance);

    return inferenceMap.Aggregate((x, y) => x.Value > y.Value ? x : y).Key;
}

Urgency

Not able to proceed

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.16.3

Execution Provider

Other / Unknown

[pranavsharma]

@RandySheriff can you take a look?

[vishnu-ms]

UPDATE: When running session.Run and session.RunAsync back to back in same block of code, the web app crashes exactly in 2 requests:

@RandySheriffH @pranavsharma can you pls help with this?

[pranavsharma]

Just spoke to @vishnu-ms. While there might be issues with RunAsync C# API (which is why I'll just keep this open), Vishnu's use case doesn't require using RunAsync. I explained to him on the call and he'll report back.

[GeorgeS2019]

Add

[fs-eire]

removed 'platform:web' tag, which is used for client-end (JS) app, not for server app.

[mario-dg]

Any updates on this?