Pages route suddenly fails with Headers Overflow Error.

[haensl]

Verify canary release

• I verified that the issue exists in the latest Next.js canary release

Provide environment information

    Operating System:
      Platform: darwin
      Arch: arm64
      Version: Darwin Kernel Version 22.5.0: Thu Jun  8 22:22:19 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8103
    Binaries:
      Node: 18.16.1
      npm: 9.7.2
      Yarn: N/A
      pnpm: N/A
    Relevant Packages:
      next: 13.4.19
      eslint-config-next: 13.4.19
      react: 18.2.0
      react-dom: 18.2.0
      typescript: 5.2.2
    Next.js Config:
      output: N/A

Which area(s) of Next.js are affected? (leave empty if unsure)

App Router

Link to the code that reproduces this issue or a replay of the bug

See below.

To Reproduce

This is the route code, it splits traffic by iOS and other operating systems:

import { getSelectorsByUserAgent } from 'react-device-detect';
import { extUrl } from '../support/targets/ios';

const redirectHome = {
  redirect: {
    destination: '/',
    permanent: false
  },
  props: {}
};

const redirectExt = {
  redirect: {
    destination: extUrl,
    permanent: false
  },
  props: {}
};

export const getServerSideProps = async (ctx) => {
  const userAgent = ctx.req.headers['user-agent'] || '';
  const { isIOS } = getSelectorsByUserAgent(userAgent);

  const requestUrl = new URL(ctx.req.url, `https://${ctx.req.headers.host}`);

  const redirect = (() => {
    if (isIOS) {
      const _redirect = {
        ...redirectExt
      };
      
      return _redirect;
    } else {
      const _redirect = {
        ...redirectHome
      };

      return _redirect;
    }
  })();

  return redirect;
};

const Page = () => (
  <></>
);


export default Page;

Describe the Bug

We maintain an app that still uses pages/ routing. In this app there is a redirecting route that, depending on client OS redirects to an external third party or the landingpage (see reproduction section). The implementation uses getServerSideProps() to return a redirect object, e.g.

{
  redirect: {
    destination: 'https://example.com',
    permanent: false
  },
  props: {}
}

This worked fine until recently, presumably the update to next 13.

Since then we're seeing the page crash at the redirect route with a 500. The server logs show a 'UND_ERR_HEADERS_OVERFLOW' error with the following stack:

TypeError: fetch failed\n    at Object.fetch (node:internal/deps/undici/undici:11576:11)\n    at async invokeRequest (/var/www/node_modules/next/dist/server/lib/server-ipc/invoke-request.js:17:12)\n    at async invokeRender (/var/www/node_modules/next/dist/server/lib/router-server.js:254:29)\n    at async handleRequest (/var/www/node_modules/next/dist/server/lib/router-server.js:447:24)\n    at async requestHandler (/var/www/node_modules/next/dist/server/lib/router-server.js:464:13)\n    at async Server.<anonymous> (/var/www/node_modules/next/dist/server/lib/start-server.js:117:13)
cause: HeadersOverflowError: Headers Overflow Error
at Parser.trackHeader (/var/www/node_modules/next/dist/compiled/undici/index.js:1:59276)
at Parser.onHeaderValue (/var/www/node_modules/next/dist/compiled/undici/index.js:1:59155)
at wasm_on_header_value (/var/www/node_modules/next/dist/compiled/undici/index.js:1:55720)
at wasm://wasm/00036ac6:wasm-function[49]:0xf91
at wasm://wasm/00036ac6:wasm-function[68]:0x7316
at wasm://wasm/00036ac6:wasm-function[67]:0x154d
at wasm://wasm/00036ac6:wasm-function[21]:0x552
at Parser.execute (/var/www/node_modules/next/dist/compiled/undici/index.js:1:57682)
at Parser.readMore (/var/www/node_modules/next/dist/compiled/undici/index.js:1:57424)
at Socket.onSocketReadable (/var/www/node_modules/next/dist/compiled/undici/index.js:1:62761) {
  code: 'UND_ERR_HEADERS_OVERFLOW'
}

Expected Behavior

The redirect route should work without throwing an exception.

Which browser are you using? (if relevant)

No response

How are you deploying your application? (if relevant)

Google Cloud

[github-actions]

We cannot recreate the issue with the provided information. Please add a reproduction in order for us to be able to investigate.

Why was this issue marked with the please add a complete reproduction label?

To be able to investigate, we need access to a reproduction to identify what triggered the issue. We prefer a link to a public GitHub repository (template for App Router, template for Pages Router), but you can also use these templates: CodeSandbox: App Router or CodeSandbox: Pages Router.

To make sure the issue is resolved as quickly as possible, please make sure that the reproduction is as minimal as possible. This means that you should remove unnecessary code, files, and dependencies that do not contribute to the issue. Ensure your reproduction does not depend on secrets, 3rd party registries, private dependencies, or any other data that cannot be made public. Avoid a reproduction including a whole monorepo (unless relevant to the issue). The easier it is to reproduce the issue, the quicker we can help.

Please test your reproduction against the latest version of Next.js (next@canary) to make sure your issue has not already been fixed.

If you cannot create a clean reproduction, another way you can help the maintainers' job is to pinpoint the canary version of next that introduced the issue. Check out our releases, and try to find the first canary release that introduced the issue. This will help us narrow down the scope of the issue, and possibly point to the PR/code change that introduced it. You can install a specific version of next by running npm install next@<version>.

I added a link, why was it still marked?

Ensure the link is pointing to a codebase that is accessible (e.g. not a private repository). "example.com", "n/a", "will add later", etc. are not acceptable links -- we need to see a public codebase. See the above section for accepted links.

What happens if I don't provide a sufficient minimal reproduction?

Issues with the please add a complete reproduction label that receives no meaningful activity (e.g. new comments with a reproduction link) are automatically closed and locked after 30 days.

If your issue has not been resolved in that time and it has been closed/locked, please open a new issue with the required reproduction.

I did not open this issue, but it is relevant to me, what can I do to help?

Anyone experiencing the same issue is welcome to provide a minimal reproduction following the above steps. Furthermore, you can upvote the issue using the 👍 reaction on the topmost comment (please do not comment "I have the same issue" without reproduction steps). Then, we can sort issues by votes to prioritize.

I think my reproduction is good enough, why aren't you looking into it quicker?

We look into every Next.js issue and constantly monitor open issues for new comments.

However, sometimes we might miss one or two due to the popularity/high traffic of the repository. We apologize, and kindly ask you to refrain from tagging core maintainers, as that will usually not result in increased priority.

Upvoting issues to show your interest will help us prioritize and address them as quickly as possible. That said, every issue is important to us, and if an issue gets closed by accident, we encourage you to open a new one linking to the old issue and we will look into it.

Useful Resources

• How to Contribute to Open Source (Next.js)
• How to create a Minimal, Complete, and Verifiable example
• Reporting a Next.js bug
• Next.js Triaging issues

[adoprog]

I have the same error after upgrading to any version above 13.4.9 (tested latest and canary too).

There is a related issue here nodejs/node#47246 and it seems there is no way in NextJS to tweak this setting.

[adoprog]

It is not possible to reduce the headers size for us as they are large because of the JWT token / Auth0 session.

[balazsorban44]

Thanks for the context @adoprog. This sounds like a Node.js/undici bug then.

Note that if globalThis.fetch is available in the runtime environment (issue claims Node.js "18.16.1" and fetch has been exposed since 18.0.0), Next.js does nothing to patch that global value, so we can do nothing here, as we don't override the undici Agent in that case:

next.js/packages/next/src/server/node-polyfill-fetch.ts

Line 4 in ac56bca

if (typeof fetch === 'undefined' && typeof globalThis.fetch === 'undefined') {

Without anyone adding a reproduction though, it's also not clear how bumping a Next.js version affected this compared to previous versions. Instead of commenting "same issue", etc. please provide us with a minimal code/reproduction that we can use to reproduce the issue to verify it. Any unhelpful comment will be marked as off-topic.

Thanks!

cc @Ethan-Arrowood if you have more advice here, but I feel that this should be fixed upstream by undici respecting the flag as suggested here: nodejs/node#47246 (comment)

[balazsorban44]

I opened a PR to fix this upstream in Node.js/undici. You can follow along here: nodejs/undici#2234

I'm closing this as it's not a Next.js bug, but how Node.js+undici works currently. When that PR lands in Node.js, you'll be able to set the Node.js flag --max-http-header-size

[github-actions]

This closed issue has been automatically locked because it had no new activity for 2 weeks. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.