OnnxRuntime C# bindings and multi gpu memory allocation.

[BernhardGlueck]

Describe the issue

System: 2x RTX 4090, fully working ( pytorch, etc )

I am trying to implement multi gpu inference ( by splitting my inputs up etc )
When i create my memory allocator for Ort Tensors like this:

 var memoryInfo = new OrtMemoryInfo(OrtMemoryInfo.allocatorCUDA_PINNED,
               OrtAllocatorType.DeviceAllocator,
                deviceId,
                OrtMemType.CpuInput | OrtMemType.CpuOutput);

  var ortAllocator = new OrtAllocator(session, memoryInfo);

The session was of course created on the correct device as well:

  var cudaOptions = new OrtCUDAProviderOptions()
        {
        };

        cudaOptions.UpdateOptions(new Dictionary<string, string>()
        {
            { "device_id", deviceId.ToString() },
            { "enable_cuda_graph", "1" },
            { "cudnn_conv_use_max_workspace", "1" }
        });

        sessionOptions.AppendExecutionProvider_CUDA(cudaOptions);

This works fine for my first card ( deviceId = 0 ) but for deviceId = 1 i get an error:
[ErrorCode:InvalidArgument] No requested allocator available

Could this be an issue or is there some incantantion to make this work ?

To reproduce

´´´
var memoryInfo = new OrtMemoryInfo(OrtMemoryInfo.allocatorCUDA_PINNED,
OrtAllocatorType.DeviceAllocator,
deviceId,
OrtMemType.CpuInput | OrtMemType.CpuOutput);

var ortAllocator = new OrtAllocator(session, memoryInfo);
´´´

Urgency

No response

Platform

Windows

OS Version

Fedora 41

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.21.0

ONNX Runtime API

C#

Architecture

X64

Execution Provider

CUDA

Execution Provider Library Version

Cuda 12.8

[yuslepukhin]

We do not support multiple devices, and the second device allocator is simply not instantiated.