[BUG] Using AsyncHttpConnection changes exceptions raised by client

[pudo]

What is the bug?

We're trying to use AWS signer auth in conjunction with the opensearch-py library in an async app. In order to do so, we first added a AWSV4SignerAsyncAuth to http_auth and then realized this breaks (see #698) if you don't also configure the http_connection to be AsyncHttpConnection. When doing that, however, HTTP errors raised by the server (e.g. for an index that already exists) seem to all be translated to different exceptions than with the standard connection type (some with HTTP error 500), instead of relaying the error code issued by the server.

e.g.:
App error 500: Could not create index: RequestError(400, '{"error":{"root_cause":[{"type":"resource_already_exists_exception","reason":"index [yente-test-329d51df12bf4b06abfdce8a4efc8e34-entities-parteispenden-011030704-100/01fcWlpPSgOGEIBDpbapyQ] already exists","index":"yente-test-329d51df12bf4b06abfdce8a4efc8e34-entities-parteispenden-011030704-100","index_uuid":"01fcWlpPSgOGEIBDpbapyQ"}],"type":"resource_already_exists_exception","reason":"index [yente-test-329d51df12bf4b06abfdce8a4efc8e34-entities-parteispenden-011030704-100/01fcWlpPSgOGEIBDpbapyQ] already exists","index":"yente-test-329d51df12bf4b06abfdce8a4efc8e34-entities-parteispenden-011030704-100","index_uuid":"01fcWlpPSgOGEIBDpbapyQ"},"status":400}') client_ip=testclient trace_context={'traceparent': '00-304c5f65545a3fc3ddb0efde68af1c8f-2630808bbfbde601-00', 'tracestate': 'yente=2630808bbfbde601'}

In this test log: https://github.com/opensanctions/yente/actions/runs/11456748840/job/31875554942

Is AsyncHttpConnection not supposed to raise TransportError any more?

How can one reproduce the bug?

• Make an AsyncOpenSearch with http_auth=AWSV4SignerAsyncAuth(...) and connection_class=AsyncHttpConnection.
• Call await self.client.indices.create(index=index, body=body) and catch TransportError in case it already exists
• Error isn't caught.

Full code is open source: https://github.com/opensanctions/yente/blob/main/yente/provider/opensearch.py

What is the expected behavior?

The exceptions raised by AsyncHttpConnection should be compatible with those raised by other connection classes.

What is your host/environment?

Ubuntu in Docker

Do you have any additional context?

Is there perhaps a way to just get AWSV4SignerAsyncAuth to work on the standard connection without completely refactoring the app?

Our issue: opensanctions/yente#542

[dblock]

Thanks for reporting this. I think "The exceptions raised by AsyncHttpConnection should be compatible with those raised by other connection classes." makes sense. Try fixing this here instead of refactoring your entire app?