Buffer chunked requests

[artonge]

Clients like xDavv5 on Android, or Cyberduck, use chunked requests.
When FastCGI or FPM is used with apache, requests arrive to Nextcloud without any content.
This leads to the creation of empty files.
The following directive will force the problematic requests to be buffered before being forwarded to Nextcloud.
This way, the "Transfer-Encoding" header is removed, the "Content-Length" header is set, and the request content is proxied to Nextcloud.
Here are more information about the issue:

• https://docs.cyberduck.io/mountainduck/issues/fastcgi/
• https://docs.nextcloud.com/server/latest/admin_manual/issues/general_troubleshooting.html#troubleshooting-webdav

Tested for a few weeks by a customer without any issue.

[szaimen]

I guess I can remove this from aio then again? :)
https://github.com/nextcloud/all-in-one/blob/b17561d41e575c7efd45fac1a6fb410be79bbe37/Containers/apache/nextcloud.conf#L42

[artonge]

/backport to stable28

[artonge]

/backport to stable27

[artonge]

/backport to stable26

[artonge]

I guess I can remove this from aio then again? :)

I guess yes. :) The current PR is also narrower, so only changes the behavior for problematic requests.

[SystemKeeper]

Would this also need to be added to https://github.com/nextcloud/server/blob/master/lib/private/Setup.php#L506 ?
Ah misunderstood the code, sorry.

[joshtrichards]

I like the targeted approach! 👍

How does mod_php avoid this? Does it just buffer every transaction?

If I'm understanding this correctly:

• this applies to fpm only
• this could have implications for resource usage on proxies (but since this is narrowly targeted and these transactions failed before I guess it doesn't matter too much?)

Related issues:

• Chunked transfers lead to 0 byte files #7995
• Files getting set to 0 in data folder #3056

If we proceed with this:

• We should probably document it somewhere, particularly if there's a chance for resource usage trade-offs and/or if mod_php might be preferable in some environments over using this solution
• Probably some work required for parity within the Nginx example config we provide in the Admin Manual since we explicitly disable the fastcgi_request_buffering behavior there
• Might be some other touch spots in the docs - e.g. we reference the Sabre URL about 0-byte files currently in the Admin Manual (https://docs.nextcloud.com/server/latest/admin_manual/issues/general_troubleshooting.html#troubleshooting-webdav)

[artonge]

How does mod_php avoid this? Does it just buffer every transaction?

From what I understand, as it is closer to Apache, the communication is better, and mod_php is able to give the file content to Nextcloud. But I am not an expert at all, feel free to correct me :).

• this applies to fpm only

And FastCGI, no ?

• this could have implications for resource usage on proxies (but since this is narrowly targeted and these transactions failed before I guess it doesn't matter too much?)

It does, but as you said, this only targets specific requests, which are currently failing, so better than nothing I guess.

We should probably document it somewhere, particularly if there's a chance for resource usage trade-offs and/or if mod_php might be preferable in some environments over using this solution

Is it possible to add a condition to exclude mod_php ?

Probably some work required for parity within the Nginx example config we provide in the Admin Manual since we explicitly disable the fastcgi_request_buffering behavior there

Indeed, the issue also occurs with Nginx.

Might be some other touch spots in the docs - e.g. we reference the Sabre URL about 0-byte files currently in the Admin Manual (https://docs.nextcloud.com/server/latest/admin_manual/issues/general_troubleshooting.html#troubleshooting-webdav)

Better to keep it, no?

[come-nc]

🐘 First time I hear about those things

[nursoda]

The PR doesn't fix the issue for me on 28.0.3RC1. Do I also need to tweak my NGINX Nextcloud config to make it work?