ERR_FR_TOO_MANY_REDIRECTS using Axios but not in browser, using curl or python-requests

[ojoched]

Describe the issue

I'm trying to fetch the following url using Axios on NodeJS: https://chromewebstore.google.com/detail/bleabhffpgoeopcfeapggnbdlhklbajk

When doing so with Axios, I'm getting ERR_FR_TOO_MANY_REDIRECTS. When looking at the error carefully, to me it seems like it's possibly because of faulty encoding of the redirected URL, as eventually it should result to https://chromewebstore.google.com/detail/%E5%85%AD%E7%9C%81%E5%8F%91%E6%96%87/bleabhffpgoeopcfeapggnbdlhklbajk (with Chinese chars if entered in the browser).

I tried many many things, including using Axios interceptors to reencode the URL, but nothing works.

Surprisingly, everything works when using curl (curl -L https://chromewebstore.google.com/detail/bleabhffpgoeopcfeapggnbdlhklbajk) or python requests (requests.get(' https://chromewebstore.google.com/detail/bleabhffpgoeopcfeapggnbdlhklbajk').

Here's the code I used, where I tried to encode the redirect URL to try and somehow resolve it (obviously the simpler version doesn't work as well)

Example Code

const axiosInstance = axios.create({
        maxRedirects: 5,
        responseType: 'arraybuffer'
    });

    try {
        const response = await axiosInstance.get('https://chromewebstore.google.com/detail/bleabhffpgoeopcfeapggnbdlhklbajk', {
            headers: {
                'User-Agent': userAgent,
                'Content-Type': 'text/html; charset=UTF-8',
                'Accept-Language': 'en-US,en;q=0.5'
            }
        });
        return response.data;
    } catch (error) {
        if (error.response && error.response.status === 302) {
            const location = error.response.headers.location;
            console.log('Original redirect URL:', location);

            // Split the URL into its components
            const urlParts = location.split('/');

            // Find the index of the "detail" part
            const detailIndex = urlParts.findIndex(part => part === 'detail');

            if (detailIndex !== -1 && detailIndex + 1 < urlParts.length) {
                // Encode the part after "detail"
                urlParts[detailIndex + 1] = encodeURIComponent(urlParts[detailIndex + 1]);

                // Join the URL parts back together
                const modifiedLocation = urlParts.join('/');
                console.log('Modified redirect URL:', modifiedLocation);

                // Send a new request with the modified URL
                const response = await axiosInstance.get(modifiedLocation, {
                    headers: {
                        'User-Agent': userAgent,
                        'Accept-Charset': 'UTF-8',
                        'Content-Type': 'text/html; charset=UTF-8'
                    }
                });
                return response.data;
            } else {
                throw new Error('Invalid URL format');
            }
        } else {
            throw error;
        }
    }
}

Expected behavior

Successful redirect is expected without entering an infinite loop (the infinite loop imo is caused by faulty encoding of the redirect)

Axios Version

^0.21.1

Adapter Version

No response

Browser

No response

Browser Version

No response

Node.js Version

21.7.3

OS

Linux

Additional Library Versions

No response

Additional context/Screenshots

No response

[ojoched]

Reproduces with node-fetch.
Does NOT reproduce with vanilla fetch.

[ojoched]

Related to: nodejs/undici#2971
Updating node resolved the issue.