Redirect not reaching final URL

[loic-bellinger]

I am encountering an issue where aiohttp follows redirects, but the final page is never reached. This happens when querying certain URLs that result in a chain of redirects.

For example, when querying http://ma-demarche-fse.fr, the redirect process begins, but aiohttp does not successfully reach the final page https://ma-demarche-fse.fr/si_fse/servlet/login.html.

I’ve included a minimal reproducible example below to demonstrate these issues:

import aiohttp
import asyncio

async def fetch_url(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url, allow_redirects=True) as response:
            final_url = str(response.url)
            body = await response.text()

            print(f"Initial URL: {url}")
            print(f"Final URL: {final_url}")
            print(f"Body Length: {len(body)}\n")

if __name__ == "__main__":
    urls = [
        "http://ma-demarche-fse.fr",
        "https://cronos.anah.gouv.fr"
    ]
    
    asyncio.run(asyncio.gather(*(fetch_url(url) for url in urls)))

Expected Behavior:

For http://ma-demarche-fse.fr, aiohttp should successfully follow redirects and reach the final page https://ma-demarche-fse.fr/si_fse/servlet/login.html.
For https://cronos.anah.gouv.fr, aiohttp should follow the redirect chain and reach the final page https://acces.anah.fr/cas/login?service=https%3A%2F%2Fcronos.anah.gouv.fr%2Fctrl%2Flogin.

Actual Behavior:

The final URLs are never reached for both http://ma-demarche-fse.fr and https://cronos.anah.gouv.fr, despite aiohttp following the initial redirects.

Environment:

aiohttp version: 3.10.10
Python version: 3.12.7

Steps to Reproduce:

Run the provided code snippet.

Observe that the final URLs are not reached for both http://ma-demarche-fse.fr and https://cronos.anah.gouv.fr.

Question:

Is there something specific about the redirection process for these domains that might prevent reaching the final pages? Is aiohttp missing part of the redirection chain for these URLs?

Please let me know if you need any additional information or if there are any known workarounds for this issue.

[Dreamsorcerer]

Try requote_redirect_url=False:
https://docs.aiohttp.org/en/stable/client_reference.html#aiohttp.ClientSession

[loic-bellinger]

After debugging, turns out that these pages redirect to pages that contain HTTP-EQUIV meta tag .... (<meta http-equiv="Refresh" content="0; URL=url">). I'll need to implement the following logic myself.