Starting at 13.4.13-canary.0 Internal Server Error due to connection refused #53171
Comments
joacub
changed the title
joacub
changed the title
|
Can you try adding |
|
im looking at the docs in how to pass that var to production server in standalone mode: CMD node apps/web/server.js do you know how? |
|
This is the second time I'm letting you know, please follow the bug report template when opening an issue. #52621 (comment) Please add a minimal reproduction so we can investigate. |
balazsorban44
added
the
please add a complete reproduction
|
We cannot recreate the issue with the provided information. Please add a reproduction in order for us to be able to investigate. Why was this issue marked with the
|
Please, allow me to explain the situation more clearly. I have encountered a recurring issue with the server not starting in production. The problem does not have any clear reproduction steps, and it happens consistently. It's important to note that the issue is not caused by anything on my end, as I've tested it on a completely empty Next.js project, and the problem persists. This problem was first noticed by another person as well, and we have been trying to pinpoint the specific code that triggers this error. Despite our efforts, the root cause remains elusive. While I understand that you may want more information from me, I assure you that we have thoroughly investigated the situation. I would appreciate any guidance or assistance you can provide to resolve this matter. As it is affecting the production environment, I'm eager to find a solution as soon as possible. Please let me know if you need any further details or if there's anything specific you would like me to do to help with the investigation and resolution of this issue. Thank you for your support |
|
@joacub |
Thank you so much I didn’t thought they were using that by another config, thank you so much |
|
if i do that nothin works, maybe becosue is in docker i will when have time investigate how to solve it thanks |
|
|
i guess this is the cause of this ===> this is frustrating, every single day is a new issue, this is critical... this is only happening in production when doing a NextResponse.rewrite |
|
Please read #53171 (comment) and #53171 (comment) |
|
Error finally detected! As mentioned by @balazsorban44, it's an IPv6 handling error. If you force the use of 0.0.0.0 in Docker environments because you can't probably use 127.0.0.1, it will cause issues when you invoke the rewrite like this: return NextResponse.rewrite(
new URL(
url,
request.url,
),
getResponseInit(),
);The next call will invoke the middleware again with the new request, which is not intended. To avoid this, modify it to: return NextResponse.rewrite(
// this is to try to not called the middleware twice remove when next handle this better
new URL(
url,
request.url
.replace('https://localhost', 'https://0.0.0.0')
.replace('http://localhost', 'http://0.0.0.0'),
),
getResponseInit(),
);The middleware is not called twice because Next.js detects it as the same request and simply re-routes it. |
|
Same problem here. It started with I know a small example would help, but like I said, I can't narrow it down yet. Still trying to reproduce it locally. Anyway, here are the logs from |
|
I had the same issues with As well as @charnog I do not have any rewrites in the app. Here is the error from my container (sorry I no longer have it in plaintext)
|
|
We face the same issue beginning from 13.4.13-canary.0 - I tried to quickly reproduce with https://github.com/vercel/next.js/blob/canary/examples/with-docker/README.md and app dir. But it works every time. We have quite a fancy setup (turborepo, multiple apps, multiple middlewares) but as stated here in the issue it seems to also happen without any of that. Settings the hostname like in the example: https://github.com/vercel/next.js/blob/canary/examples/with-docker/Dockerfile#L60 didn't work for me Only thing I can share from the error docker container is the line which throws the error:
From https://github.com/vercel/next.js/blob/canary/packages/next/src/server/lib/start-server.ts#L332 Looks a lot like this change c11bce5#diff-0ff576bd02e76f96680accff48ecbe4126a7f6bc4eafa547b6d44dee49bab770R246 from @shuding & @ijjk |
It will work in a normal setup, I also reproduce myself in a my side and works every time, to reproduce this error you need to have the setup in a docker container mapping the port from for example 3035:3000 and have Nginx proxy going to http://container-name:3035 and the error will happen every single time. There are many other errors when you have that setup like the env vars now are not going to the node, seems like nextjs is wiping all env vars since that versión and no env vars are going to the node server |
|
@florianliebig, try setting it to |
|
@charnog yeah they really thought replacing |
Yeah the way to fix this is setting hostname as 0.0.0.0 but then other things fails as next seems to be wiping All env vars now |
Btw great work with next-pwa working really nice. |
💀 |
Any rewrite made at any point will lead to the same behavior, triggering the re-execution of all processes and consequently altering the entire outcome of the subsequent code execution. This rewrite is perceived as a new request and should be handled differently to ensure the desired results. |
btw that error you are facing there is not caused by the rewirte, the rewrite is another issue related to this versions becouse you have been force to indicated a HOSTNAME to properly make next working behind proxies and docker, so you need now to indicate in your setup a HOSTNAME 0.0.0.0 so then nextjs is hearing all incoming networks |
|
For self-hosted apps that use Below is the /** @type {import('next').NextConfig} */
const nextConfig = {
output: 'standalone',
}
module.exports = nextConfigFor reproduction do the following:
In such an environment, the following 2 scenarios fail: 1. Internationalization middleware always redirects to
|
|
Moreover, from what I tested it seems that @joacub is right and that this problem was already introduced in 13.4.13-canary.0, maybe via #53004. Nevertheless, for the reproduction scenarios I described, I would recommend the Stable Version |
|
It should be noted that:
So in k8s, the server will never listen on 0.0.0.0 after updating to 13.4.13. |
|
i have a similar issue |
leerob
added
linear: next
we use k8s as well, however I was able to reproduce the problem with HOSTNAME locally using only Docker. When HOSTNAME was set in Dockerfile the issue was resolved locally and in k8s cluster |
|
Thank you for providing the minimal reproduction – I've marked this one to be dug into further. In the meantime, please remain on |
### What? This PR makes it easier to use Next.js with IPv6 hostnames such as `::1` and `::`. ### How? It does so by removing rewrites from `localhost` to `127.0.0.1` introduced in #52492. It also fixes the issue where Next.js tries to fetch something like `http://::1:3000` when `--hostname` is `::1` as it is not a valid URL (browsers' `URL` class throws an error when constructed with such hosts). It also fixes `NextURL` so that it doesn't accept `http://::1:3000` but refuse `http://[::1]:3000`. It also changes `next/src/server/lib/setup-server-worker.ts` so that it uses the server's `address` method to retrieve the host instead of our provided `opts.hostname`, ensuring that no matter what `opts.hostname` is we will always get the correct one. ### Note I've verified that `next dev`, `next start` and `node .next/standalone/server.js` work with IPv6 hostnames (such as `::` and `::1`), IPv4 hostnames (such as `127.0.0.1`, `0.0.0.0`) and `localhost` - and with any of these hostnames fetching to `localhost` also works. Server Actions and middleware have no problems as well. This also removes `.next/standalone/server.js`'s logging as we now use `start-server`'s logging to avoid duplicates. `start-server`'s logging has also been updated to report the actual hostname.   The above pictures also demonstrate using Server Actions with Next.js after this PR.  Fixes #53171 Fixes #49578 Closes NEXT-1510 Co-authored-by: Tim Neutkens <6324199+timneutkens@users.noreply.github.com> Co-authored-by: Zack Tanner <1939140+ztanner@users.noreply.github.com>
|
This has been fixed ✅ |
|
no it's not fixed. At least not for me. Kubernete pods are assigned default HOSTNAME however when in 13.4.19 app somehow listens an ip address instead of the hostname and many restarts after, liveness probe detects it's somehow alive, next (node) begins to give ssl issues. does anybody know any workaround for this? |
|
@XEngine that doesn't look like an URL, nor an IP to me, which makes me wonder if that environment variable was even meant to be used like this. Can you check what did Next.js log back in older versions? I mean, 803bbe5 fixed the standalone mode not getting |
|
Ye. Before it was starting like this: and it was working until 13.4.12. After that version whole thing begins to fail. Our environment still the same, the hostname you see is the same hostname format. I have tried to run my dockerfile with 0.0.0.0, 127.0.0.1, [::1], localhost it started with by not sending a custom HOSTNAME env. but received those ssl issues I don't know. |
|
@XEngine if it broke in 13.4.12 then perhaps it's not really related to my fix and this issue. Maybe I'll investigate into it later if possible :) That IP being shown is definitely due to my PR though, but you shouldn't worry about that. |
Sorry for my mistake on version definition. Correction : Working fine until 13.4.13. |
|
@XEngine yeah if it's 13.4.13 then it will be a lot more painful 💀 |
another fix for that is to add a postbuild script that rewrites a bit server.js by deleting the process.env.HOSTNAME conditon. |
|
Hello, Same issue on our side. App is running fine locally outside docker container. Everything did work on kubernetes env for next.js 13.4.12. When using 13.4.13, readiness probe or liveness probe cannot be reached on kubernetes pods with following error Please note as well that our application use a env var named "HOSTNAME" and that it does conflict with the process.env.HOSTNAME used inside the generated next It is not fixed on v13.4.20-canary.9. Please reopen this issue |
|
@izi-p does adding |
|
@DuCanhGH We need to override it because our application use a runtime variable named "HOSTNAME" and conflicts with the one server.js try to use. It seems to work by replacing hostname with 0.0.0.0 inside server.js |
|
Hi! |
|
@izi-p just asking, does |
|
I've been losing my mind as well with too much time sunk into this... I'm on Next version: I constantly get these errors in my logs; I imagine they're from K8's probes for liveness and readiness; even though the container launches just fine, but later fails with 500 errors it seems for those probes. I used to utilize a basic health check page, then I tried utilizing an API route that returned a simple 200 OK for the body. It seems odd to me that the ErrorsI also tried setting OdditiesI just forwarded a tunnel to my container and the root page loaded up quickly, which is what I was using for probes previously, but this time when I tried to hit my new It seems that these requests are majorly lagging now or failing. But if I access the app from the domain on the internet everything is loading up quickly and smoothly oddly... 😔 The first request on the tunnel to a page goes through quickly, but then any subsequent request lags horribly or times out 😬 Utilizing curl on the container via |
|
This closed issue has been automatically locked because it had no new activity for 2 weeks. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you. |
and others
Verify canary release
Provide environment information
Which area(s) of Next.js are affected? (leave empty if unsure)
No response
Link to the code that reproduces this issue or a replay of the bug
no response
To Reproduce
just update to this version and get up a production server
Describe the Bug
TypeError: fetch failed
at Object.fetch (node:internal/deps/undici/undici:11576:11)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async invokeRequest (/app/node_modules/.pnpm/next@13.4.13-canary.0_@babel+core@7.22.5_react-dom@18.2.0_react@18.2.0_sass@1.64.0/node_modules/next/dist/server/lib/server-ipc/invoke-request.js:34:23)
at async requestHandler (/app/node_modules/.pnpm/next@13.4.13-canary.0_@babel+core@7.22.5_react-dom@18.2.0_react@18.2.0_sass@1.64.0/node_modules/next/dist/server/lib/start-server.js:329:35)
at async Server. (/app/node_modules/.pnpm/next@13.4.13-canary.0_@babel+core@7.22.5_react-dom@18.2.0_react@18.2.0_sass@1.64.0/node_modules/next/dist/server/lib/start-server.js:148:13) {
cause: Error: connect ECONNREFUSED 127.0.0.1:41367
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1592:16) {
errno: -111,
code: 'ECONNREFUSED',
syscall: 'connect',
address: '127.0.0.1',
port: 41367
}
}
Expected Behavior
Server Working
Which browser are you using? (if relevant)
No response
How are you deploying your application? (if relevant)
No response
NEXT-1510
The text was updated successfully, but these errors were encountered: