[NEXT-1157] deploying NextJS 13 appDir site to Digital Ocean App platform, Direct link opening doesn't work.

[dhruvilxcode]

Verify canary release

• I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
      Platform: darwin
      Arch: x64
      Version: Darwin Kernel Version 21.6.0: Wed Aug 10 14:25:27 PDT 2022; root:xnu-8020.141.5~2/RELEASE_X86_64
    Binaries:
      Node: 18.12.1
      npm: 9.6.0
      Yarn: 1.22.19
      pnpm: N/A
    Relevant packages:
      next: 13.3.4
      eslint-config-next: 13.3.4
      react: 18.2.0
      react-dom: 18.2.0

Which area(s) of Next.js are affected? (leave empty if unsure)

App directory (appDir: true), Routing (next/router, next/navigation, next/link)

Link to the code that reproduces this issue

github.com/dhruvilxcode/eshop

To Reproduce

Deploy the NextJS site to Digital Ocean App Platform or any host.
then access the any sub page such as Login page by accessing URL directly, not by Link component (because that works well)

Describe the Bug

When i access url let's say Login page, by this domain.com/login/ then it's not loading, instead it showing this (refer to linked image).

Expected Behavior

A page should render by accessing it's URL.

Which browser are you using? (if relevant)

Chrome

How are you deploying your application? (if relevant)

Digital Ocean App Platform

_NEXT-1157

[dhruvilxcode]

For now, i downgraded the next version to 13.2 and It worked well.

npm i next@13.2

[dhruvilxcode]

Looks like there is bug in newer version of next.

[HamAndRock]

Is there a CDN involved? Got the same weird RSC payload like you did when we started using CDN on a project:

#49140

[Fredkiss3]

One potential solution (for cloudfare) : #48569 (comment)

[conor909]

Same issue here deploying to firebase, I thought it was a firebase issue but it seems to be a next thing.

Upgrading to the latest canary version seemed to work yesterday, but the issue is back now somehow, so a CDN issue could make sense?

[dhruvilxcode]

@conor909 downgrade next version to 13.2, and check.
it might work for you as well, as it worked for me.

[timneutkens]

Can you share a deployed url? My guess based on the description is that your hosting is set up wrong and is not passing Vary to browsers, causing it to cache wrong.

[conor909]

I know my issue could be different, being on Firebase, but I've just deployed successfully using a local node 16 environment, when I use 18 it fails with the NextJS error Error: > Couldn't find a pages directory. Please create one under the project root

I'm using next: 13.3.5-canary.3

[dhruvilxcode]

@timneutkens not changed any settings on Digital Ocean app Platform. currently I downgraded next version to 13.2 and it works well on same.

sorry I can't push my website with the latest version. One thing is sure I setted up the server correctly.

[Livog]

I experienced the same issue with my Next.js 13 app when using Cloudflare in front of it. I realized that the problem is caused by the DigitalOcean App Platform's built-in Cloudflare, and since Cloudflare doesn't properly handle the Vary header by default it became clear that I have to switch. To resolve this, I had to move my deployment away from the DigitalOcean App Platform.

I chose a pre-installed easypanel droplet and set it up with continuous deployment using webhooks, and now my app is now building the same way as it did on the App Platform. Thankfully, my Dockerfile made the transition smooth. With Cloudflare now in front of my droplet, I can ensure that everything works as expected using my custom worker.

Ideally, Cloudflare would need to add an option to add cache rules that can consider response or request headers to then force Cloudflare to cache the response with a different cache key / cache URL.
Additionally, it would be helpful if DigitalOcean provided an option to disable their global CDN ( it's probablt possibly through an app.yaml configuration), but their documentation is too extensive to quickly find a solution.

For reference, here is my Cloudflare worker code. I'd appreciate any suggestions for improvements, bug fixes, or more efficient solutions:

/* eslint-disable import/no-anonymous-default-export */
const addHeadersToUrl = (headersObj: Headers, url: URL, headers: string[]): URL => {
  headers.forEach((header) => {
    if (headersObj.has(header)) url.searchParams.set(header.toLowerCase(), encodeURIComponent(`${headersObj.get(header)}`))
  })
  return url
}

function shouldCacheResponse(response: Response): boolean {
  const cacheControl = response.headers.get('Cache-Control')

  if (!cacheControl) {
    return true
  }

  const cacheDirectives = ['private', 'no-cache', 'no-store']
  return !cacheDirectives.some((directive) => cacheControl.includes(directive))
}

function addUrlParams(url: URL, params: Record<string, string>): URL {
  for (const [key, value] of Object.entries(params)) {
    url.searchParams.set(key, value)
  }
  return url
}

const hasRequestSomeHeaders = (request: Request, headers: string[]) => {
  const headersObj = request.headers
  return headers.some((header) => headersObj.has(header))
}

async function handleRequest(request: Request, ctx: any) {
  const originUrl = new URL(request.url)
  const cacheUrl = new URL(request.url)

  console.log('Request URL:', request.url)

  if (originUrl.pathname.startsWith('/api')) {
    // @ts-ignore
    return fetch(request, originUrl.pathname.startsWith('/api/auth') ? { cf: { cacheTtl: -1 } } : undefined)
  }

  if (hasRequestSomeHeaders(request, ['RSC', 'Next-Router-State-Tree', 'Next-Router-Prefetch'])) {
    addUrlParams(cacheUrl, { 'content-type': 'text/x-component' })
  }

  const cacheKey = new Request(cacheUrl, request)
  /* @ts-ignore */
  const cache = caches.default

  let response = await cache.match(cacheKey)
  console.log('Cache match:', response ? 'Found' : 'Not found', originUrl.toString(), cacheUrl.toString())

  if (!response) {
    response = await fetch(request)
    console.log('Cache URL before reading response:', cacheUrl.toString())
    const modifiedResponse = new Response(response.body, response)

    // Modify cacheUrl based on response 'Content-Type' header
    if (hasRequestSomeHeaders(request, ['RSC', 'Next-Router-State-Tree', 'Next-Router-Prefetch']) && response.headers.get('Content-Type') === 'text/x-component') {
      addHeadersToUrl(modifiedResponse.headers, cacheUrl, ['Content-Type'])
    }

    console.log('Cache URL after reading response:', cacheUrl.toString())

    const modifiedCacheKey = new Request(cacheUrl, request)

    if (shouldCacheResponse(modifiedResponse)) {
      // Cache the response
      console.log('Caching the response with modified cache key:', cacheUrl.toString())
      await cache.put(modifiedCacheKey, modifiedResponse.clone())
    }

    return modifiedResponse
  }

  return response
}

export default {
  async fetch(request: Request, ctx: any) {
    return handleRequest(request, ctx)
  }
}

[timneutkens]

Posted a reply here: #49140 (comment)

[dhruvilxcode]

👍 Thanks

[l-you]

Did #50970 fix it?

[timneutkens]

Should be fixed in #50970 indeed 👍

[roeean]

It's still not working in v13.4.10... I deployed my site to AWS ECS with AWS Cloudfront as CDN, and I still can't load the pages under App Router properly. Everyone keeps coming as text/x-component

[cduff]

It's still not working in v13.4.10... My site deployed to AWS ECS with AWS Cloudfront as CDN and I still can't load the pages that are under App Router properly, everyone keeps coming as x-document

@roeean I've got a similar setup working but you need a custom CloudFront cache policy like:

The q, w & url query string entries allow use of next/image.

[timneutkens]

@roeean please provide a reproduction in a new issue. We've added a query string parameter to the end of the request for the RSC payload so it's hard to say why you'd still see this without being able to investigate the application.