Expected behavior of Response.aclose?

[james-mchugh]

Hello everyone!

I am using the httpx.AsyncClient to send a request to an endpoint that responds with an infinite stream of data. Specifically, I am using it to proxy a request from a client to the API with the streaming response, like so:

CLI Client --> Proxy --> Streaming API

I have followed the docs here to return the streaming response from the proxy, resulting in code that looks similar to (modified for brevity)

client = httpx.AsyncClient(base_url=API_URL)

def proxy(request):
    request = client.build_request(
        "GET",
        request.path,
    )
    resp = await client.send(request, stream=True)
    return fastapi.responses.StreamingResponse(
        resp.aiter_lines(),
        background=starlette.background.BackgroundTask(resp.aclose),
    )

As instructed in the link above:

When using this "manual streaming mode", it is your duty as a developer to make sure that Response.aclose() is called eventually. Failing to do so would leave connections open, most likely resulting in resource leaks down the line.

We have a BackgroundTask to ensure the response is closed. However, this does not appear to actually close any connections. After running the Proxy and Streaming API through debuggers, I can see that the proxy calls resp.aclose as expected, but the Streaming API continues to attempt to stream the response back, ignorant of the resp.aclose call.

On closer inspection with strace, I saw that the Proxy was not making only making a single close call to close the connection with the Client, but it was not called to close the connection with the Streaming API. Furthermore, tcpdump does not show any Fin TCP packets sent between the Proxy and Streaming API, so the Streaming API is never notified that it should stop streaming the response. It seems that the Response.aclose method is just doing some client-side housekeeping.

I'm assuming that this is due to connection pooling done by the httpx.Client and HTTP 1.1 persistent connections. I was able to resolve the issue by rewriting some of the proxy code to close the client instead of the response, which got me the behavior I was looking for.

My question is, is this right? Or am I just doing something wrong? Given the way the documentation explains how to handle streaming responses when proxying requests, it seems like the server would always be left in a state where it streaming responses back to a client that isn't acting on them. Does the documentation have to be updated to avoid this situation? I felt a bit misled that it says "Failing to do so [call Response.aclose()] would leave connections open", which seemed to imply that calling Response.aclose would close the connection. But maybe it is actually referring to the client being told that the response is closed so it can eventually clean up the connection if no other requests are made.

[ehsandeep]

@james-mchugh you possibly looking for a discussion on https://github.com/encode/httpx

[james-mchugh]

Oh that's embarrassing. Sorry about that. I'll close this and move it there. Thank you for the heads up

[olearycrew]

No worries! it happens all the time.