-
Notifications
You must be signed in to change notification settings - Fork 26k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[NEXT-1143] Dev mode slow compilation #48748
Comments
same here, and in docker env is even worse, seems like is processing same files over and over without caching them. |
Same for me also dev env ,navigating to different pages via link component is pretty slow |
+1 its same here, hitting the page first time seems fine but routing via links gets stuck |
last canary version has a better cold build times improvements, still slow like 2-5 seconds (in docker env) waiting but much better the version im talking about is 13.3.2-canary.6 |
Hey, @jeengbe there have been some patch updates (13.3.1 -> 13.3.4) did it improve for you? |
Hi @denu5, unfortunately, I can't report any real performance changes since I opened this issue. You might want to check out the above linked issue in the TypeScript repo though - might be related. |
As @jeengbe mention there is no performance improvement, there is also a lot of I/O I don’t know why, one request gets pretty much like 1gb-2gb of io. And it is very slow. |
Unfortunately, I can't confirm this for my case |
That’s pretty good, in my case there is a lot of I/O, maybe is because I’m using material-ui but I think is too much even though. |
Possibly, it would align with what your trace shows: #48407 (comment) |
I see that slow route changes in dev mode are showing a '[Fast Refresh] rebuilding' message in the browser console. Sometimes it performs a full page reload when changing routes even if no files have been edited. |
entry next-app-loader
spans?)entry next-app-loader
spans?)
its slowing down the development..! |
Having the same issue here, in the Docker environment it's come to a point where it's almost unusable, and sometimes I even have to do a hard reload, after waiting too long for navigation. This is the case both with component from next/navigation, as with the router.push (useRouter hook imported from next/navigation). We're using Next.js 13.4.2. |
same here, it is almost not usable in dokcer enviorements, but also outside docker is very slow, something is not working nice. this is painfully slow. |
Yeah same for me. I used to remote develop inside our k8s cluster but dev --turbo is super slow inside a container and causes my health check endpoint to sigkill it regularly. The whole app router is super slow when containerized in Dev mode. It works perfectly fine when I run both on my local machine and connect it via reverse proxy. This way it's faster than the old setup (which was not significantly faster before) and takes advantage of preloading pages via next/link. I see inconsistencies in caching too where it's a mix of instant navigation or long builds (around 3.5k modules for some things) around 2-10 sec. Also there is this weird thing happening that a page compiles just fine and then later it grinds to a halt being stuck in waiting for compiling forever until the pod is crashed. |
I love next, but this is a complete show stopper. Sometimes it takes 10+ seconds outside of docker for me on a Mac M2 to navigate one page. This is insane. |
Yep even more I get sometimes 50 seconds in a simple page, that’s because is also building other things related to that in pralllel I guess. not even outside docker, i just make a test to work outside docker and timing is exactly the same no difference…. Is getting slower and slower |
Btw webpack lazy building cold is faster than turbopack 🙂 by far |
Yes! I'm surprised this is not more prevalent as an issue atm; unless turbo will somehow fix all of this in 13.5 and they're waiting to address it. What configs do you have for the faster webpack builds? I've tried quite a bit and can't lower my built time by much. I need a temporary fix for this ASAP :( |
A month later no updates on this? Makes development on appDir absolutely impossible. @timneutkens ? Linked a bunch of related issues on this: |
I confirm that next app dir on dev mode and dynamic routing are very very slow on docker now |
Changes in the past weekI've been investigating this over the past week. Made a bunch of changes, some make a small impact, some make a large impact. Here's a list:
You can try them using Help InvestigateIn order to help me investigate this I'll ideally need an application that can be run, if you can't provide that (I understand if you can't) please provide the If possible follow these steps which would give me the best picture to investigate:
Known application-side slowdownsTo collect things I've seen before that cause slow compilation as this is often the root cause:
This and other slowdown reports are currently the top priority for our team. We'll continue optimizing Next.js with webpack where possible. |
entry next-app-loader
spans?)
Changed the initial post in this issue to reflect my reply above in order to ensure people see it as the first thing when opening the issue. I'm going to close the duplicate issues reporting similar slowdowns in favor of this one. I'll need help from you all to ensure this thread doesn't spiral in "It is slow" comments that are not actionable when e.g. without traces / reproduction / further information. Thank you 🙏 |
That's expected, it holds much more data than the |
Hi, Thanks ;) |
@AlaaCherif in your case it's a few libraries, some can still be optimized more in Turbopack, but just so you're aware:
@Chai1310b When you edit the file in your case you're importing On the initial compile there is a weird amount of time spent on sealing the build, around 24 seconds, did you customize the Next.js configuration (i.e. added webpack plugins)? |
Turbopack for development is now available as release candidate in Next.js 14.2: https://nextjs.org/blog/next-14-2 |
Thanks again for the help Update: I uploaded a new trace.log to the drive after the changes done as well as another |
This works great 'next dev --turbo' for package.json dev script 🚀 |
using the latest nextjs version. We are moving from pages to app directory. Pages in the app directory are fast. But pages in the pages directory are extremely slow. It takes like 7-10 minutes to get a response. Any idea why? I already tried to use a node profiler but could not see what the bottleneck is. Many file operations and string concat calls. We are already using turbopack. |
@millsoft please see the initial post of this issue, it outlines exactly how you can provide information that is critical for us to do any type of investigation if you don't want to share the codebase: #48748 (comment). |
@birukbelay checking in, are you able to provide the traces, happy to take a look if you can share them. Thanks! |
I'm having similar issues: Following this guide: #48748 (comment) Trace: https://gist.github.com/MarcusHSmith/dba462d850fdb69217027ab481edd3b2 Next config: https://gist.github.com/MarcusHSmith/68ad4b992164d5f4c1e3aaeb53a6e807
Any ideas how to improve our speed? |
@MarcusHSmith seems you've added postcss customization and it's taking forever to run that. |
For Tailwind CSS users: I've been able to find a pretty large slowdown with Tailwind CSS in general when Worth having a look at this thread: https://twitter.com/timneutkens/status/1783851267237781574. |
Hey @timneutkens Is turbo-trace-viewer.vercel.app a Vercel-internal? I can't find how to upload a trace file Thanks in advance |
I had the same problem with nextjs appdir, compilation in devmode took up to 5-6 seconds. |
@AhmedBaset correct, it's an internal tool we've been building that we'll be sharing at a later point, it's not fully ready yet, but useful for us 🙂 |
It helped me. Deeply appreciate. Thank you 🫡
|
Incredible. This really solved my problem of slow development, build and deployment. My Next.js app is small, but building and compiling took about 20~40 minutes to complete. I already used the @timneutkens Thanks for your post 😸 🚀. |
It would be cool if there was a |
@yasincandev this issue is specifically for discussing slowdowns in compilation, not for bundle sizes, feel free to open a discussion if you need help. Keep in mind that sharing a screenshot with numbers and the mention of 3 packages is not enough for people to be able to help you get an answer, you'll have to share more, i.e. code. |
## What? Implements support for running the Turbopack trace server, which is the websocket server that powers https://turbo-trace-viewer.vercel.app/ when using `NEXT_TURBOPACK_TRACING=1 NEXT_TURBOPACK_TRACE_SERVER=1`. Currently you have to manually run the server through the Turbo repository which in practice means that only people working on Turbopack are able to run it. With the bindings implemented anyone should be able to run the trace server. Note that the traces that come out of Turbopack are very low level, they're meant for optimizing Turbopack like finding slowdowns / large memory usage / optimizing performance. However, it's useful for people that want to peek into why their application is slow to compile. I.e. we've used https://turbo-trace-viewer.vercel.app to investigate reports in #48748. This PR adds support for `trace.log` by default, so if you add `NEXT_TURBOPACK_TRACING=1 NEXT_TURBOPACK_TRACE_SERVER=1` it will automatically select the `trace.log` for the current instance of Next.js. You can only have one trace server running at the same time. ### `next internal` In order to support running the trace server standalone, which is useful for investigating trace files other people have shared, I've added a new subcommand `internal` that is not covered by semver / use at your own risk. It's meant for internal tools that are useful to be bound to the version of Next.js, the turbo-trace-server is a great example of that as it has an internal binary format for storing data that needs to match the trace.log file. If you want to take a look at `.next/trace` instead the new `next internal` subcommand can be used for that: ```sh # Replace [path] with a path to a file. next internal turbo-trace-server [path] ``` For example: ```sh next internal turbo-trace-server ~/Downloads/trace ``` Currently the trace server does not support loading multiple files, just hasn't been implemented yet. Once we can load two or more files we can load both `.next/trace` and `trace.log` when `NEXT_TURBOPACK_TRACE_SERVER=1` and support multiple paths passed to `next internal turbo-trace-server`. ### Turbopack upgrade PR includes a Turbopack upgrade: * vercel/turbo#8073 <!-- OJ Kwon - feat(webpack-loaders): support dummy span interface --> * vercel/turbo#8083 <!-- OJ Kwon - fix(webpack): print resource, project path when relative calc fails --> * vercel/turbo#8094 <!-- Tim Neutkens - Implement bindings for Turbopack trace server --> <!-- Thanks for opening a PR! Your contribution is much appreciated. To make sure your PR is handled as smoothly as possible we request that you follow the checklist sections below. Choose the right checklist for the change(s) that you're making: ## For Contributors ### Improving Documentation - Run `pnpm prettier-fix` to fix formatting issues before opening the PR. - Read the Docs Contribution Guide to ensure your contribution follows the docs guidelines: https://nextjs.org/docs/community/contribution-guide ### Adding or Updating Examples - The "examples guidelines" are followed from our contributing doc https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md - Make sure the linting passes by running `pnpm build && pnpm lint`. See https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md ### Fixing a bug - Related issues linked using `fixes #number` - Tests added. See: https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs - Errors have a helpful link attached, see https://github.com/vercel/next.js/blob/canary/contributing.md ### Adding a feature - Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. (A discussion must be opened, see https://github.com/vercel/next.js/discussions/new?category=ideas) - Related issues/discussions are linked using `fixes #number` - e2e tests added (https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs) - Documentation added - Telemetry added. In case of a feature if it's used or not. - Errors have a helpful link attached, see https://github.com/vercel/next.js/blob/canary/contributing.md ## For Maintainers - Minimal description (aim for explaining to someone not on the team to understand the PR) - When linking to a Slack thread, you might want to share details of the conclusion - Link both the Linear (Fixes NEXT-xxx) and the GitHub issues - Add review comments if necessary to explain to the reviewer the logic behind a change ### What? ### Why? ### How? Closes NEXT- Fixes # --> Closes NEXT-3328
Hey @timneutkens , trace: https://gist.github.com/jaskome/18f97ed79117b022a38558196ff0851d Thanks in advance |
hi @timneutkens trace: https://gist.github.com/msrks/2dc9cfef6ec3f42db3e318b3cae31f81 thanks in advance |
Hey everyone, as a bunch of people have asked about the trace viewer I've worked on publishing the tooling we use so that you can have a look yourself using Just so that everyone is on the same page this tooling is intentionally very low level, it's the tools we use to investigate slowdowns. Specifically you can run these files:
You can run it using this command on
It will link to https://turbo-trace-viewer.vercel.app/ where you can view the trace. Few notes on navigation:
Only one trace file can be viewed at a time currently. The trace viewer is still a work in progress, but already quite useful to us. |
EDIT: great work on the community support! Old commentis there a similar tool for the webpack compiler?When I run the command I get this error:
I didn't expect this to work but it would be nice if webpack users also had a similar way of viewing traces. Thanks in advance! |
It's not a good error message. You need to provide the path to the trace
file relative to the working directory, see Tim's comment.
…Sent from my mobile device
On Fri, 24 May 2024, 2:26 am Louis Giët, ***@***.***> wrote:
@timneutkens <https://github.com/timneutkens> is there a similar tool for
the webpack compiler?
When I run the command I get this error:
$ pnpx ***@***.*** internal turbo-trace-server
Packages: +29
+++++++++++++++++++++++++++++
Progress: resolved 55, reused 30, downloaded 0, added 29, done
⚠ Turbopack trace server started. View trace at https://turbo-trace-viewer.vercel.app/
***@***.******@***.******@***.***/node_modules/next/dist/build/swc/index.js:911 ***@***.******@***.******@***.***/node_modules/next/dist/build/swc/index.js:911>
(customBindings ?? bindings).startTurbopackTraceServer(traceFilePath);
^
Error: Failed to convert JavaScript value `Undefined` into rust type `String`
at Object.startTurbopackTraceServer ***@***.******@***.******@***.***/node_modules/next/dist/build/swc/index.js:911:50)
at Module.startTurboTraceServerCli ***@***.******@***.******@***.***/node_modules/next/dist/cli/internal/turbo-trace-server.js:14:20) {
code: 'StringExpected'
}
Node.js v18.19.0
I didn't expect this to work but it would be nice if webpack users also
had a similar way of viewing traces.
Thanks in advance!
—
Reply to this email directly, view it on GitHub
<#48748 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A3SBII5MKCRVG6SL5J4HVLLZDX4A5AVCNFSM6AAAAAAXIQHFM6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRXGI3DCNBSGI>
.
You are receiving this because you commented.Message ID: <vercel/next.
***@***.***>
|
@timneutkens here is a gist with all the files mentioned in this comment Gist: https://gist.github.com/tiriana/0e2dd12da8473d33ad6fcaeff182fac0 First file change after It's much better with --turbo for me (refresh times below 1-2 sec) |
Changes in the past week
I've been investigating this over the past week. Made a bunch of changes, some make a small impact, some make a large impact. Here's a list:
pages
andapp
and you're only working onapp
it will no longer compile the runtime forpages
. Note: this shifts the compilation of the runtime to when you first open a pageYou can try them using
npm install next@canary
.Help Investigate
In order to help me investigate this I'll ideally need an application that can be run, if you can't provide that (I understand if you can't) please provide the
.next/trace
file.If possible follow these steps which would give me the best picture to investigate:
npm install next@canary
(use the package manager you're using) -- We want to make sure you're using the very latest version of Next.js which includes the fixes mentioned earlier.rm -rf .next
NEXT_CPU_PROF=1
andNEXT_TURBOPACK_TRACING=1
(regardless of if you're using Turbopack, it only affects when you do use Turbopack) environment variable. E.g.:NEXT_TURBOPACK_TRACING=1 NEXT_CPU_PROF=1 npm run dev
NEXT_TURBOPACK_TRACING=1 NEXT_CPU_PROF=1 yarn dev
NEXT_TURBOPACK_TRACING=1 NEXT_CPU_PROF=1 pnpm dev
ctrl+c
).next/trace
file to https://gist.github.com -- Please don't run trace-to-tree yourself, as I use some other tools (e.g. Jaeger) that require the actual trace file..next/trace.log
as well, if it's too large for GitHub gists you can upload it to Google Drive or Dropbox and share it through that.next.config.js
(if you have one) to https://gist.github.comKnown application-side slowdowns
To collect things I've seen before that cause slow compilation as this is often the root cause:
react-icons
, material icons, etc. Most of these libraries publish barrel files with a lot of re-exports. E.g. material-ui/icons ships 5500 module re-exports, which causes all of them to be compiled. You have to addmodularizeImports
to reduce it, here's an example: long compile times locally - along with "JavaScript heap out of memory" since upgrade to NextJS 13 #45529 (comment)content
setting that tries to read too many files (e.g. files not relevant for the application)This and other slowdown reports are currently the top priority for our team. We'll continue optimizing Next.js with webpack where possible.
The Turbopack team is currently working on getting all Next.js integration tests passing when using Turbopack as we continue working towards stability of Turbopack.
Original post
Verify canary release
Provide environment information
Which area(s) of Next.js are affected? (leave empty if unsure)
No response
Link to the code that reproduces this issue
https://github.com/DigitalerSchulhof/digitaler-schulhof
To Reproduce
Note that I have been unable to replicate this issue in a demo repository.
Describe the Bug
The issue is that Next.js is generally slow in dev mode. Navigating to new pages takes several seconds:
The only somewhat reasonable time would be 600ms for the API route
/api/schulhof/auth/login/route
, even though that is still quite a lot slower than what it should be given its size.It also doesn't look right to compile ~1500 modules for each page, as most of them should be cached. The pages are not very different.
Even an empty API route takes several hundreds of ms. The following example contains solely type exports:
I am not exactly sure how to read trace trees, but what stands out is that there are (over multiple runs) several
entry next-app-loader
that take 2+ seconds to complete:Find both dev and build traces here: https://gist.github.com/jeengbe/46220a09846de6535c188e78fb6da03e
Note that I have modified
trace-to-tree.mjs
to include event times for all events.It also seems unusual that none of the modules have child traces.
Expected Behavior
Initial load and navigating should be substantially faster.
Which browser are you using? (if relevant)
No response
How are you deploying your application? (if relevant)
No response
From SyncLinear.com | NEXT-1143
The text was updated successfully, but these errors were encountered: