Support requests larger than the default gRPC message length limit of 4 MB

[berndverst]

The Dot Net SDK does not support gRPC payloads beyond the default 4 MB.

Allow configuring the max send and receive message length for gRPC

You have to set these gRPC client options. The value provided is in bytes with a default of 4 MB.

grpc.max_send_message_length
grpc.max_receive_message_length

For reference see:
dapr/python-sdk#375
dapr/python-sdk#458

EDIT:
Actors still appear to be using HTTP. Please verify that the request size limit can be configured for Actors also.

[berndverst]

This issue is impacting a container apps user for example: microsoft/azure-container-apps#411

[berndverst]

As an aside, this also need to be implemented for actors. Essentially it needs to be added for every gRPC client in the DotNet SDK!

[kendallroden]

Thanks for creating this bernd!

[yash-nisar]

/assign

[yash-nisar]

I was not able to reproduce the issue with the dotnet-sdk. I was able to successfully send a >4MB payload through a remoting client.

for (int i = 0; i < 120000; i++)
{
    testData.Add(i, i);
}

and I used the following code to measure the total payload size:

var payloadSize = request.Headers.ToString().Length + (request.Content?.Headers.ContentLength ?? 0); 
Console.WriteLine(payloadSize/(1024.0 * 1024.0));

[yash-nisar]

Also, the default maximum allowed size of any request body is 28.6 MB according to https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.server.kestrel.core.kestrelserverlimits.maxrequestbodysize?view=aspnetcore-7.0.

[yash-nisar]

@dale-personal Moving forward the conversation here because don't wanna notify too many people on the container apps issue.

Can you make sure the max-request-size is reflected to 32 on the container apps end at the UI under dapr settings ?

[yash-nisar]

@dale-personal Is this code that you can share publicly via GitHub? I would love to test myself and see if I can get some insight back to you and it would be much more convenient to replicate.

[dale-personal]

@yash-nisar Let me re-test this. It looks like there have been some updates / additional settings since i first reported it here:
microsoft/azure-container-apps#411

[yash-nisar]

Sure @dale-personal, let us know how it goes so we can act on this issue.

[dale-personal]

@yash-nisar I have confirmed that this is still an issue. I have verified that the HTTP Max Request Size is set to 32 MB in the portal. The key here is that you need to send the request via the .net actor proxy

to reproduce this, create an IActor interface with the following method:
Task<byte[]> Foo(byte[] data);

Implementation:

public class MyActor : Actor, IActor
{
public async Task<byte[]> Foo(byte[] data)
{
return await Task.FromResult(data);
}
}

deploy the actor service to an azure container app with dapr enabled.
then invoke the actor method from the client with data larger than 4MB:

var proxy = ActorProxy.Create(new ActorId("foo"), nameof(IActor))
var data = new byte[5000000];
_ = proxy.Foo(data); // larger than 4MB fails

[hsinghdeol-wpay]

Looks like there is no way to override limit during DurableTask registration #1163

[kvo181]

Just to confirm that with the gprc options and daprHTTPMaxRequestSize set, using the latest release 1.13.1, it is still impossible to publish a message to an Azure Service Bus Queue that exceeds 4MB in size. Any updates on how and when this will be solved?

Status(StatusCode="Internal", Detail="error when publish to topic ... in pubsub ...: the message is too large")
at Dapr.Client.DaprClientGrpc.MakePublishRequest(String pubsubName, String topicName, ByteString content, Dictionary`2 metadata, String dataContentType, CancellationToken cancellationToken)

Is a workaround available?