Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(fetch): properly redirect non-ascii location header url #2971

Merged
merged 8 commits into from
Mar 21, 2024
Merged

fix(fetch): properly redirect non-ascii location header url #2971

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
e07ade3
fix(fetch): properly redirect non-ascii location header url
Xvezda Mar 18, 2024
48fd3c1
chore: fix typo
Xvezda Mar 18, 2024
01ceae8
test: use simpler code
Xvezda Mar 19, 2024
6d5f770
chore: clarify what the code does
Xvezda Mar 19, 2024
ea34bc2
chore: add comment
Xvezda Mar 19, 2024
efccdde
chore: normalize location url only if it contains invalid character
Xvezda Mar 19, 2024
45f2e48
chore: apply suggestion
Xvezda Mar 19, 2024
3519ca1
chore: remove redundant condition check
Xvezda Mar 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 4 additions & 2 deletions lib/web/fetch/util.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ function responseLocationURL (response, requestFragment) {
// Some websites respond location header in binary form without encoding them
Xvezda marked this conversation as resolved.
Show resolved Hide resolved
// and major browsers redirect them to correctly UTF-8 encoded addresses.
// Here, we handle that behavior in the same way.
location = new URL(decodeBinaryStringToUtf8(location), responseURL(response))
location = new URL(normalizeBinaryStringToUtf8(location), responseURL(response))
Xvezda marked this conversation as resolved.
Show resolved Hide resolved
}

// 4. If location is a URL whose fragment is null, then set location’s
Expand All @@ -61,10 +61,12 @@ function responseLocationURL (response, requestFragment) {
}

/**
* If string contains non-ASCII characters, assumes it's UTF-8 encoded and decodes it.
* Since UTF-8 is a superset of ASCII, this will work for ASCII strings as well.
* @param {string} value
* @returns {string}
*/
function decodeBinaryStringToUtf8 (value) {
function normalizeBinaryStringToUtf8 (value) {
return Buffer.from(value, 'binary').toString('utf8')
}
Copy link
Member

@lemire lemire Mar 19, 2024 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

  let b = Buffer.from(value, 'binary');
  if(Buffer.isAscii(b)) { return value; } // skip new string alloc.
  return b.toString('utf8'); // must alloc a new string

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to call Buffer.from(). Buffer.isAscii(b) is a static function.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anonrig We get at the core of the issue here. The value is a string not a buffer. At some point it was wrongly interpreted as latin1 (maybe in http) and so we must now convert it to a buffer and then back to a string.

I think that this function should receive a Buffer and then the Buffer could be interpreted as UTF-8 and we'd be done.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this is far beyond the scope of this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The irony is that we seem to try to purposefully (in http) prevent people from sending UTF-8 content as UTF-8... and then we go back and say "no, it is really UTF-8".

Much technical debt.


Expand Down