Release limiters when response is returned

[johnhferland]

fixes #1192

Before this PR

Concurrency permits are only released when the response body is closed.

After this PR

==COMMIT_MSG==
Concurrency permits are now released when the response is received.
==COMMIT_MSG==

Possible downsides?

• It's now technically possible to have more concurrent connections than the maximum number of permits
• This increases the rate at which limits will increase, which may increase load on servers from clients. I expected this will be particularly noticeable for reading larger objects.

[changelog-app]

Generate changelog in changelog/@unreleased

Type

• Feature
• Improvement
• Fix
• Break
• Deprecation
• Manual task
• Migration

Description

Concurrency permits are now released when the response is received.

Check the box to generate changelog(s)

• Generate changelog entry

[jkozlowski]

I think this was discussed in the past and in general would be kinda against the principle of concurrency limiting: the fact that the body is being streamed usually means that the worker thread might have been released, but streaming the body still usually consumes resources (especially if it's just a blocking read that goes from some external resource and writes to the response body).

I think the reason why you are not observing any 429 in general in that situation is that the internal product that I expect you're referring too has request/sec rate limiting implemented, not concurrency limiting. I suspect releasing more of requests here would eventually lead to 503s from the product.

[stale]

This PR has been automatically marked as stale because it has not been touched in the last 14 days. If you'd like to keep it open, please leave a comment or add the 'long-lived' label, otherwise it'll be closed in 7 days.

[iamdanfox]

On your original issue (this is artifacts right?) doesn't c-j-r's limit still increase, so this should only affect a cold startup?

[stale]

This PR has been automatically marked as stale because it has not been touched in the last 14 days. If you'd like to keep it open, please leave a comment or add the 'long-lived' label, otherwise it'll be closed in 7 days.