Rust WebAssembly module in an ES module wrapper from wasm-pack fails to load in Next.js #29362
Comments
|
The But In the latest canary build, v11.1.3-canary.32, But enabling that |
|
As per #29485, I'm actually not sure this is |
|
I believe it is #22697 that's causing this conflict. On a production build with webpack 5 the output configuration is as follows: It seems as though |
|
@gthb Lord forgive me, but I've created a workaround: // next.config.js
module.exports = {
webpack(config, { isServer, dev }) {
// Enable webassembly
config.experiments = { asyncWebAssembly: true };
// In prod mode and in the server bundle (the place where this "chunks" bug
// appears), use the client static directory for the same .wasm bundle
config.output.webassemblyModuleFilename =
isServer && !dev ? "../static/wasm/[id].wasm" : "static/wasm/[id].wasm";
// Ensure the filename for the .wasm bundle is the same on both the client
// and the server (as in any other mode the ID's won't match)
config.optimization.moduleIds = "named";
return config;
},
};Then to ensure the (function () {
import("wasm/add.wasm");
// Import which .wasm files you need here
});This won't download the file onto the client, just ensure that it's in the bundle This is obviously far from an ideal solution, because it still generates the server bundle at |
|
The above workaround breaks when using I've created a more reliable workaround in the form of an embedded webpack plugin. This behaviour is pretty close to how Next.js internally handles the // next.config.js
module.exports = {
webpack(config, { isServer, dev }) {
config.experiments = {
asyncWebAssembly: true,
layers: true,
};
if (!dev && isServer) {
config.output.webassemblyModuleFilename = "chunks/[id].wasm";
config.plugins.push(new WasmChunksFixPlugin());
}
return config;
},
};
class WasmChunksFixPlugin {
apply(compiler) {
compiler.hooks.thisCompilation.tap("WasmChunksFixPlugin", (compilation) => {
compilation.hooks.processAssets.tap(
{ name: "WasmChunksFixPlugin" },
(assets) =>
Object.entries(assets).forEach(([pathname, source]) => {
if (!pathname.match(/\.wasm$/)) return;
compilation.deleteAsset(pathname);
const name = pathname.split("/")[1];
const info = compilation.assetsInfo.get(pathname);
compilation.emitAsset(name, source, info);
})
);
});
}
} |
Oh man! 4 hours later I bump into this thread. I had packed a battle snake solver with However this seems to work only with version 11, on Next 12, it breaks during import, I guess I gotta read further. Thanks for the snippet! @sam3d ! |
|
@icyJoseph is it breaking when you try and deploy it on Vercel? The snippet is working for me in Next 12 for both dev and prod build locally. However on Vercel it seems like output file tracing doesn't detect the .wasm file. I believe the place this is happening in @vercel/nft is here: |
Ah I spoke too soon, when I deployed it on vercel it fails on, with Next 11. And locally with next 12, it all falls apart. Here's how I import the wasm file, on export default async function handler(req, res) {
if (req.method !== "GET") return res.status(404).json({ error: "No" });
try {
const wasm = await import("../../../snake/pkg/logic");
return res.status(200).json(wasm.get_info());
} catch (e) {
console.error(e);
return res.status(500).json({ message: "Internal Server Error" });
}
}
I think for now I'll just solve these with TypeScript. |
|
@icyJoseph What happens if you remove the WasmPackPlugin? I don't think they're interoperable |
|
@sam3d I had to take a moment to re-do the entire thing, my code had changed quite a lot from the fork. Now I have it all running on TypeScript, and a sub route https://github.com/icyJoseph/next-battlesnake/tree/with-wasm -> set at the correct branch I did as you suggested, and now things work fine locally, with Next 12, but still, when deploying to Vercel, upon reaching for the 2021-11-23T12:54:00.429Z acc7efcd-6066-45df-8452-37d1feb32982 ERROR [Error: ENOENT: no such file or directory, open '/var/task/.next/server/chunks/817.wasm'] {
errno: -2,
code: 'ENOENT',
syscall: 'open',
path: '/var/task/.next/server/chunks/817.wasm'
}EditLooking at https://github.com/vercel/nft/blob/e89f3ca8cfa41838d0c4d7c16255eba7f561609f/src/node-file-trace.ts#L40-L42, as you suggest, seems very telling, have you had a chance to try and debug it or make PR? if (path.endsWith('.js') || path.endsWith('.cjs') || path.endsWith('.mjs') || path.endsWith('.node') || job.ts && (path.endsWith('.ts') || path.endsWith('.tsx'))) {
return job.emitDependency(path);
} |
|
@icyJoseph I filed this issue about a week ago (https://github.com/vercel/nft/issues/247) but I'm afraid I haven't yet the chance to work on a reproduction or PR for it (as I don't have the Next.js dev stack configured locally). My assumption is that a simple addition of |
|
(p.s. you don't need the following by the way, I posted my patch hastily and forgot to remove this part from my own Next.js config:) config.module.rules.push({
test: /\.svg$/,
use: ["@svgr/webpack"],
}); |
|
@icyJoseph (yet another) temporary fix could be something like this: #14807 except specifying the chunks directory with |
|
@sam3d Ah I gave it a go, but I am not sure how am I to point to the wasm file, if its hashed, and placed inside .next, because that's the one that can't be found. I guess I'll skip the WASM version of this for now. |
|
Hi has there been any progress on a fix for the deployment on Vercel? I have the same exact issue as @icyJoseph |
|
This worked for me. ` module.exports = nextConfig module.exports = { // generate wasm module in ".next/server" for ssr & ssg return config |
|
🚨 This comment is partially out of date – see below I managed to get Here is my /** @type {import("next").NextConfig} */
export default {
webpack: (webpackConfig, { isServer }) => {
// WASM imports are not supported by default. Workaround inspired by:
// https://github.com/vercel/next.js/issues/29362#issuecomment-1149903338
// https://github.com/vercel/next.js/issues/32612#issuecomment-1082704675
return {
...webpackConfig,
experiments: {
asyncWebAssembly: true,
layers: true,
},
optimization: {
...webpackConfig.optimization,
moduleIds: "named",
},
output: {
...webpackConfig.output,
webassemblyModuleFilename: isServer
? "./../static/wasm/[modulehash].wasm"
: "static/wasm/[modulehash].wasm",
},
};
},
};WASM modules load fine in an SSR page, but an API route returns Both Next page and API handler works correctly in Repo with reproduction: hasharchives/wasm-ts-esm-in-node-jest-and-nextjs |
|
Having similar issues as above ^. Anyone any idea? |
|
What happens when you use the workaround in #29362 (comment)? |
|
Oh wow thanks for the pointer @sam3d – I have missed that comment somehow. I just tried your workaround in hasharchives/wasm-ts-esm-in-node-jest-and-nextjs and it did it’s job! 🎉 Awesome stuff! |
- Fixes `with-webassembly` example unable to build with the current `next.config.js`. #29362 (comment) - Converted example to TypeScript ```js // Before config.output.webassemblyModuleFilename = 'static/wasm/[modulehash].wasm' // After config.output.webassemblyModuleFilename = isServer && !dev ? '../static/wasm/[modulehash].wasm' : 'static/wasm/[modulehash].wasm' ``` ``` > Build error occurred Error: Export encountered errors on following paths: / at /Users/max/dev/next.js/examples/with-webassembly/node_modules/next/dist/export/index.js:408:19 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async Span.traceAsyncFn (/Users/max/dev/next.js/examples/with-webassembly/node_modules/next/dist/trace/trace.js:79:20) at async /Users/max/dev/next.js/examples/with-webassembly/node_modules/next/dist/build/index.js:1342:21 at async Span.traceAsyncFn (/Users/max/dev/next.js/examples/with-webassembly/node_modules/next/dist/trace/trace.js:79:20) at async /Users/max/dev/next.js/examples/with-webassembly/node_modules/next/dist/build/index.js:1202:17 at async Span.traceAsyncFn (/Users/max/dev/next.js/examples/with-webassembly/node_modules/next/dist/trace/trace.js:79:20) at async Object.build [as default] (/Users/max/dev/next.js/examples/with-webassembly/node_modules/next/dist/build/index.js:65:29) ELIFECYCLE Command failed with exit code 1. ``` ## Documentation / Examples - [X] Make sure the linting passes by running `pnpm build && pnpm lint` - [X] The "examples guidelines" are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md)
github-actions
bot
added
the
linear: next
|
I was able to get a WASM module compiled from Rust with wasm-pack working both locally and on Vercel, except in a non-edge API route on Vercel.
When deployed to Vercel, the logs show the following error when trying to call a WASM module from a non-edge API route: I'm assuming this is expected because the docs state that WebAssembly is only supported in Edge Functions and Edge Middleware. But has anyone been able to get WASM working on a non-edge route on Vercel? The example at https://github.com/hasharchives/wasm-ts-esm-in-node-jest-and-nextjs/blob/main/web-app/pages/api/wasm-package-answer.ts looked like it was successfully calling a WASM module from a non-edge API route, but perhaps that was only working locally. |
|
I am using Nextjs13 with wasm. This thread has been a real help. 🤗 Though I would prefer a more official fix. |
|
I'm facing a similar issue trying to use node-webpmux, which is compiled with Emscripten, and I'm so glad I found this thread. Unfortunately the workaround above by @sam3d is not working for me. For context, I am trying to use this library in an API route, but I get the following runtime error: I see that the library is trying to load If it's helpful, I'm using NextJS |
I'm running into the same issue but with both Next App API routes and Server Actions. When API routes or Server Actions import WASM, it builds fine using the fix above, but when deployed to Vercel the WASM file doesn't exist and leads to "ENOENT: no such file or directory" upon server code execution. |
|
@calclavia I was able to get WASM imports to work for regular API routes and on the frontend with the config below. Note that I had to enable Node 20 in the runtime settings (in Node 20 WASM support no longer requires a flag). const CopyPlugin = require("copy-webpack-plugin");
const nextConfig = {
webpack: (config, { isServer, dev }) => {
config.experiments = {
layers: true,
asyncWebAssembly: true
};
if (!dev && isServer) {
webassemblyModuleFilename = "./../server/chunks/[modulehash].wasm";
const patterns = [];
const destinations = [
"../static/wasm/[name][ext]", // -> .next/static/wasm
"./static/wasm/[name][ext]", // -> .next/server/static/wasm
"." // -> .next/server/chunks (for some reason this is necessary)
];
for (const dest of destinations) {
patterns.push({
context: ".next/server/chunks",
from: ".",
to: dest,
filter: (resourcePath) => resourcePath.endsWith(".wasm"),
noErrorOnMissing: true
});
}
config.plugins.push(new CopyPlugin({ patterns }));
}
return config;
}
}; |
What version of Next.js are you using?
11.1.2
What version of Node.js are you using?
14.17.6
What browser are you using?
Firefox, Chrome
What operating system are you using?
macOS
How are you deploying your application?
Not yet deployed
Describe the Bug
A simple Rust WebAssembly module packaged with its glue code into an ES module with wasm-pack (patched as in rustwasm/wasm-pack#1061) loads and works just fine under webpack, as illustrated in webpack/webpack#14313, but fails to import under Next.js. This is apparently because Next.js generates the
.wasmgenerated at one path but then tries to load it from a different path.Expected Behavior
I expected the
npm run devto successfully run the application and render a page with the greeting from inside the WebAssembly module.I expected
npm run buildto successfully build a production distribution that would do the same.To Reproduce
This is demonstrated in https://github.com/gthb/try-to-use-wasm-in-next.js/ — the README there recounts my circuitous path of trying to get this to work, but the current state serves to illustrate the problem I'm reporting here.
In that repo, first setup:
Then try
yarn run dev— it fails like this:Then try
yarn run build— it fails like this:Note the path to the
.wasmchunk. The actual generated.wasmchunks are:$ find .next -iname '*.wasm' .next/server/chunks/static/wasm/1905d306caca373cb9a6.wasm .next/static/wasm/b9a4f4672eb576798896.wasmSo apparently the failure is that Next.js generates the
.wasmchunk on the server side with an extrachunks/path element (which presumably should not be there) and then looks for it at a path without that path element and fails to find it.I'm guessing that the same problem is the cause of the
npm run devfailure (i.e. the module imports as empty).The text was updated successfully, but these errors were encountered: