Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new output option, workerPublicPath #16671

Merged
merged 32 commits into from Mar 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
895df2a
Worker public path changes
thomastay Oct 21, 2022
c188270
Merge branch 'webpack:main' into main
thomastay Jan 25, 2023
0df564b
Make workerPublicPath optional and update docs
thomastay Jan 25, 2023
7a1ccea
change json schema description
thomastay Jan 25, 2023
2e64c44
add into output schema
thomastay Jan 25, 2023
13be78b
lint fixes
thomastay Jan 25, 2023
674dd74
serialize and update hash per options
thomastay Jan 26, 2023
8676f74
Add statsCase - the different asset hashes show that worker public pa…
thomastay Jan 26, 2023
1eb549b
remove TODO comment
thomastay Jan 26, 2023
5c03c4f
accidentally added a dummy file, removed
thomastay Jan 26, 2023
292dd48
Add in a config case as well
thomastay Jan 26, 2023
ee2c87b
lint
thomastay Jan 26, 2023
e5fd6a4
update snapshots
thomastay Jan 26, 2023
7ef871b
lint fix (somehow by version of Prettier adds this but on linux it do…
thomastay Jan 27, 2023
0636863
Merge branch 'webpack:main' into main
thomastay Feb 9, 2023
0d91c55
Fix CI build failures due to extranous double quotes and await worker…
thomastay Feb 9, 2023
9e8fe6e
fix Defaults snapshot
thomastay Feb 9, 2023
d33f32d
Updated validation snapshot
thomastay Feb 10, 2023
2a39a0a
No cache for stat test
thomastay Feb 10, 2023
9fa1415
snapshot is cached
thomastay Feb 10, 2023
ea07777
stats test is flaky, revert to just one
thomastay Feb 10, 2023
47e1440
read/write options as a whole (follow ConsumeSharedModule)
thomastay Feb 10, 2023
6e21e6e
Update schemas/WebpackOptions.json
TheLarkInn Mar 3, 2023
954d98a
special lint fix
thomastay Mar 15, 2023
9e3f5bb
update snapshots and more special lint fix
thomastay Mar 16, 2023
4b4c6a1
update snapshots according to CI
thomastay Mar 16, 2023
2822ed3
weird case where snapshots have extra spaces
thomastay Mar 16, 2023
460ed46
asset hash
thomastay Mar 16, 2023
628125f
revert weird space issues
thomastay Mar 22, 2023
2cad865
change hash
thomastay Mar 22, 2023
f33c2b0
Merge branch 'webpack:main' into main
thomastay Mar 22, 2023
8c6a1a4
revert weird space issues part 2
thomastay Mar 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
12 changes: 12 additions & 0 deletions declarations/WebpackOptions.d.ts
Expand Up @@ -569,6 +569,10 @@ export type UniqueName = string;
* The filename of WebAssembly modules as relative path inside the 'output.path' directory.
*/
export type WebassemblyModuleFilename = string;
/**
* Worker public path. Much like the public path, this sets the location where the worker script file is intended to be found. If not set, webpack will use the publicPath. Don't set this option unless your worker scripts are located at a different path from your other script files.
*/
export type WorkerPublicPath = string;
/**
* The number of parallel processed modules in the compilation.
*/
Expand Down Expand Up @@ -2164,6 +2168,10 @@ export interface Output {
* The method of loading chunks (methods included by default are 'jsonp' (web), 'import' (ESM), 'importScripts' (WebWorker), 'require' (sync node.js), 'async-node' (async node.js), but others might be added by plugins).
*/
workerChunkLoading?: ChunkLoading;
/**
* Worker public path. Much like the public path, this sets the location where the worker script file is intended to be found. If not set, webpack will use the publicPath. Don't set this option unless your worker scripts are located at a different path from your other script files.
*/
workerPublicPath?: WorkerPublicPath;
/**
* The method of loading WebAssembly Modules (methods included by default are 'fetch' (web/WebWorker), 'async-node' (node.js), but others might be added by plugins).
*/
Expand Down Expand Up @@ -3349,6 +3357,10 @@ export interface OutputNormalized {
* The method of loading chunks (methods included by default are 'jsonp' (web), 'import' (ESM), 'importScripts' (WebWorker), 'require' (sync node.js), 'async-node' (async node.js), but others might be added by plugins).
*/
workerChunkLoading?: ChunkLoading;
/**
* Worker public path. Much like the public path, this sets the location where the worker script file is intended to be found. If not set, webpack will use the publicPath. Don't set this option unless your worker scripts are located at a different path from your other script files.
*/
workerPublicPath?: WorkerPublicPath;
/**
* The method of loading WebAssembly Modules (methods included by default are 'fetch' (web/WebWorker), 'async-node' (node.js), but others might be added by plugins).
*/
Expand Down
3 changes: 2 additions & 1 deletion lib/WebpackOptionsApply.js
Expand Up @@ -391,7 +391,8 @@ class WebpackOptionsApply extends OptionsApply {
new WorkerPlugin(
options.output.workerChunkLoading,
options.output.workerWasmLoading,
options.output.module
options.output.module,
options.output.workerPublicPath
).apply(compiler);

new DefaultStatsFactoryPlugin().apply(compiler);
Expand Down
1 change: 1 addition & 0 deletions lib/config/defaults.js
Expand Up @@ -916,6 +916,7 @@ const applyOutputDefaults = (
? "auto"
: ""
);
D(output, "workerPublicPath", "");
D(output, "chunkLoadTimeout", 120000);
D(output, "hashFunction", futureDefaults ? "xxhash64" : "md4");
D(output, "hashDigest", "hex");
Expand Down
1 change: 1 addition & 0 deletions lib/config/normalization.js
Expand Up @@ -369,6 +369,7 @@ const getNormalizedWebpackOptions = config => {
uniqueName: output.uniqueName,
wasmLoading: output.wasmLoading,
webassemblyModuleFilename: output.webassemblyModuleFilename,
workerPublicPath: output.workerPublicPath,
workerChunkLoading: output.workerChunkLoading,
workerWasmLoading: output.workerWasmLoading
};
Expand Down
39 changes: 37 additions & 2 deletions lib/dependencies/WorkerDependency.js
Expand Up @@ -25,10 +25,16 @@ class WorkerDependency extends ModuleDependency {
/**
* @param {string} request request
* @param {[number, number]} range range
* @param {Object} workerDependencyOptions options
* @param {string} workerDependencyOptions.publicPath public path for the worker
*/
constructor(request, range) {
constructor(request, range, workerDependencyOptions) {
super(request);
this.range = range;
// If options are updated, don't forget to update the hash and serialization functions
this.options = workerDependencyOptions;
TheLarkInn marked this conversation as resolved.
Show resolved Hide resolved
/** Cache the hash */
this._hashUpdate = undefined;
}

/**
Expand All @@ -48,6 +54,31 @@ class WorkerDependency extends ModuleDependency {
get category() {
return "worker";
}

/**
* Update the hash
* @param {Hash} hash hash to be updated
* @param {UpdateHashContext} context context
* @returns {void}
*/
updateHash(hash, context) {
if (this._hashUpdate === undefined) {
this._hashUpdate = JSON.stringify(this.options);
}
hash.update(this._hashUpdate);
}

serialize(context) {
const { write } = context;
write(this.options);
super.serialize(context);
}

deserialize(context) {
const { read } = context;
this.options = read();
super.deserialize(context);
}
}

WorkerDependency.Template = class WorkerDependencyTemplate extends (
Expand All @@ -69,6 +100,10 @@ WorkerDependency.Template = class WorkerDependencyTemplate extends (
chunkGraph.getBlockChunkGroup(block)
);
const chunk = entrypoint.getEntrypointChunk();
// We use the workerPublicPath option if provided, else we fallback to the RuntimeGlobal publicPath
const workerImportBaseUrl = dep.options.publicPath
? `"${dep.options.publicPath}"`
: RuntimeGlobals.publicPath;

runtimeRequirements.add(RuntimeGlobals.publicPath);
runtimeRequirements.add(RuntimeGlobals.baseURI);
Expand All @@ -77,7 +112,7 @@ WorkerDependency.Template = class WorkerDependencyTemplate extends (
source.replace(
dep.range[0],
dep.range[1] - 1,
`/* worker import */ ${RuntimeGlobals.publicPath} + ${
`/* worker import */ ${workerImportBaseUrl} + ${
RuntimeGlobals.getChunkScriptFilename
}(${JSON.stringify(chunk.id)}), ${RuntimeGlobals.baseURI}`
);
Expand Down
7 changes: 5 additions & 2 deletions lib/dependencies/WorkerPlugin.js
Expand Up @@ -48,10 +48,11 @@ const DEFAULT_SYNTAX = [
const workerIndexMap = new WeakMap();

class WorkerPlugin {
constructor(chunkLoading, wasmLoading, module) {
constructor(chunkLoading, wasmLoading, module, workerPublicPath) {
this._chunkLoading = chunkLoading;
this._wasmLoading = wasmLoading;
this._module = module;
this._workerPublicPath = workerPublicPath;
}
/**
* Apply the plugin
Expand Down Expand Up @@ -298,7 +299,9 @@ class WorkerPlugin {
}
});
block.loc = expr.loc;
const dep = new WorkerDependency(url.string, range);
const dep = new WorkerDependency(url.string, range, {
publicPath: this._workerPublicPath
});
dep.loc = expr.loc;
block.addDependency(dep);
parser.state.module.addBlock(block);
Expand Down
2 changes: 1 addition & 1 deletion schemas/WebpackOptions.check.js

Large diffs are not rendered by default.

10 changes: 10 additions & 0 deletions schemas/WebpackOptions.json
Expand Up @@ -3241,6 +3241,9 @@
"workerChunkLoading": {
"$ref": "#/definitions/ChunkLoading"
},
"workerPublicPath": {
"$ref": "#/definitions/WorkerPublicPath"
},
"workerWasmLoading": {
"$ref": "#/definitions/WasmLoading"
}
Expand Down Expand Up @@ -3397,6 +3400,9 @@
"workerChunkLoading": {
"$ref": "#/definitions/ChunkLoading"
},
"workerPublicPath": {
"$ref": "#/definitions/WorkerPublicPath"
},
"workerWasmLoading": {
"$ref": "#/definitions/WasmLoading"
}
Expand Down Expand Up @@ -5271,6 +5277,10 @@
}
},
"required": ["apply"]
},
"WorkerPublicPath": {
"description": "Worker public path. Much like the public path, this sets the location where the worker script file is intended to be found. If not set, webpack will use the publicPath. Don't set this option unless your worker scripts are located at a different path from your other script files.",
"type": "string"
}
},
"title": "WebpackOptions",
Expand Down
10 changes: 7 additions & 3 deletions test/Defaults.unittest.js
Expand Up @@ -360,6 +360,7 @@ describe("snapshots", () => {
"wasmLoading": "fetch",
"webassemblyModuleFilename": "[hash].module.wasm",
"workerChunkLoading": "import-scripts",
"workerPublicPath": "",
"workerWasmLoading": "fetch",
},
"parallelism": 100,
Expand Down Expand Up @@ -1303,8 +1304,9 @@ describe("snapshots", () => {
+ "wasmLoading": "async-node",
@@ ... @@
- "workerChunkLoading": "import-scripts",
- "workerWasmLoading": "fetch",
+ "workerChunkLoading": "require",
@@ ... @@
- "workerWasmLoading": "fetch",
+ "workerWasmLoading": "async-node",
@@ ... @@
- "aliasFields": Array [
Expand Down Expand Up @@ -1447,8 +1449,9 @@ describe("snapshots", () => {
+ "wasmLoading": "async-node",
@@ ... @@
- "workerChunkLoading": "import-scripts",
- "workerWasmLoading": "fetch",
+ "workerChunkLoading": "require",
@@ ... @@
- "workerWasmLoading": "fetch",
+ "workerWasmLoading": "async-node",
@@ ... @@
- "aliasFields": Array [
Expand Down Expand Up @@ -1573,8 +1576,9 @@ describe("snapshots", () => {
+ "wasmLoading": "async-node",
@@ ... @@
- "workerChunkLoading": "import-scripts",
- "workerWasmLoading": "fetch",
+ "workerChunkLoading": "require",
@@ ... @@
- "workerWasmLoading": "fetch",
+ "workerWasmLoading": "async-node",
@@ ... @@
- "aliasFields": Array [
Expand Down
2 changes: 1 addition & 1 deletion test/Validation.test.js
Expand Up @@ -498,7 +498,7 @@ describe("Validation", () => {
expect(msg).toMatchInlineSnapshot(`
"Invalid configuration object. Webpack has been initialized using a configuration object that does not match the API schema.
- configuration.output has an unknown property 'ecmaVersion'. These properties are valid:
object { assetModuleFilename?, asyncChunks?, auxiliaryComment?, charset?, chunkFilename?, chunkFormat?, chunkLoadTimeout?, chunkLoading?, chunkLoadingGlobal?, clean?, compareBeforeEmit?, crossOriginLoading?, cssChunkFilename?, cssFilename?, devtoolFallbackModuleFilenameTemplate?, devtoolModuleFilenameTemplate?, devtoolNamespace?, enabledChunkLoadingTypes?, enabledLibraryTypes?, enabledWasmLoadingTypes?, environment?, filename?, globalObject?, hashDigest?, hashDigestLength?, hashFunction?, hashSalt?, hotUpdateChunkFilename?, hotUpdateGlobal?, hotUpdateMainFilename?, iife?, importFunctionName?, importMetaName?, library?, libraryExport?, libraryTarget?, module?, path?, pathinfo?, publicPath?, scriptType?, sourceMapFilename?, sourcePrefix?, strictModuleErrorHandling?, strictModuleExceptionHandling?, trustedTypes?, umdNamedDefine?, uniqueName?, wasmLoading?, webassemblyModuleFilename?, workerChunkLoading?, workerWasmLoading? }
object { assetModuleFilename?, asyncChunks?, auxiliaryComment?, charset?, chunkFilename?, chunkFormat?, chunkLoadTimeout?, chunkLoading?, chunkLoadingGlobal?, clean?, compareBeforeEmit?, crossOriginLoading?, cssChunkFilename?, cssFilename?, devtoolFallbackModuleFilenameTemplate?, devtoolModuleFilenameTemplate?, devtoolNamespace?, enabledChunkLoadingTypes?, enabledLibraryTypes?, enabledWasmLoadingTypes?, environment?, filename?, globalObject?, hashDigest?, hashDigestLength?, hashFunction?, hashSalt?, hotUpdateChunkFilename?, hotUpdateGlobal?, hotUpdateMainFilename?, iife?, importFunctionName?, importMetaName?, library?, libraryExport?, libraryTarget?, module?, path?, pathinfo?, publicPath?, scriptType?, sourceMapFilename?, sourcePrefix?, strictModuleErrorHandling?, strictModuleExceptionHandling?, trustedTypes?, umdNamedDefine?, uniqueName?, wasmLoading?, webassemblyModuleFilename?, workerChunkLoading?, workerPublicPath?, workerWasmLoading? }
-> Options affecting the output of the compilation. \`output\` options tell webpack how to write the compiled files to disk.
Did you mean output.environment (output.ecmaVersion was a temporary configuration option during webpack 5 beta)?"
`)
Expand Down
13 changes: 13 additions & 0 deletions test/__snapshots__/Cli.basictest.js.snap
Expand Up @@ -6683,6 +6683,19 @@ Object {
"multiple": false,
"simpleType": "string",
},
"output-worker-public-path": Object {
"configs": Array [
Object {
"description": "Worker public path. Much like the public path, this sets the location where the worker script file is intended to be found. If not set, webpack will use the publicPath. Don't set this option unless your worker scripts are located at a different path from your other script files.",
"multiple": false,
"path": "output.workerPublicPath",
"type": "string",
},
],
"description": "Worker public path. Much like the public path, this sets the location where the worker script file is intended to be found. If not set, webpack will use the publicPath. Don't set this option unless your worker scripts are located at a different path from your other script files.",
"multiple": false,
"simpleType": "string",
},
"output-worker-wasm-loading": Object {
"configs": Array [
Object {
Expand Down
10 changes: 10 additions & 0 deletions test/__snapshots__/StatsTestCases.basictest.js.snap
Expand Up @@ -4675,3 +4675,13 @@ cacheable modules 2.31 KiB (javascript) 1.37 KiB (webassembly)
./node_modules/env.js 34 bytes [built] [code generated]
webpack x.x.x compiled successfully in X ms"
`;

exports[`StatsTestCases should print correct stats for worker-public-path 1`] = `
"asset main-27d65b836727f9226214.js 3.51 KiB [emitted] [immutable] (name: main)
asset 442-579eebb6602aecc20b13.js 219 bytes [emitted] [immutable]
runtime modules 1.75 KiB 5 modules
cacheable modules 250 bytes
./index.js 115 bytes [built] [code generated]
./worker.js 135 bytes [built] [code generated]
webpack x.x.x compiled successfully in X ms"
`;
14 changes: 14 additions & 0 deletions test/configCases/output/worker-public-path/index.js
@@ -0,0 +1,14 @@
import { Worker } from "worker_threads";

it("should define public path", async () => {
const worker = new Worker(new URL("./worker.js", import.meta.url), {
type: "module"
});
worker.postMessage("ok");

var fs = require("fs"),
path = require("path");
var source = fs.readFileSync(path.join(__dirname, "main.js"), "utf-8");
expect(source).toMatch("workerPublicPath2");
await worker.terminate()
});
5 changes: 5 additions & 0 deletions test/configCases/output/worker-public-path/test.config.js
@@ -0,0 +1,5 @@
module.exports = {
findBundle: function () {
return ["./main.js"];
}
};
5 changes: 5 additions & 0 deletions test/configCases/output/worker-public-path/test.filter.js
@@ -0,0 +1,5 @@
var supportsWorker = require("../../../helpers/supportsWorker");

module.exports = function (config) {
return supportsWorker();
};
13 changes: 13 additions & 0 deletions test/configCases/output/worker-public-path/webpack.config.js
@@ -0,0 +1,13 @@
/** @type {import("../../../../").Configuration} */
module.exports = {
mode: "none",
target: "node",
node: {
__dirname: false,
__filename: false
},
output: {
filename: "[name].js",
workerPublicPath: "/workerPublicPath2/"
}
};
6 changes: 6 additions & 0 deletions test/configCases/output/worker-public-path/worker.js
@@ -0,0 +1,6 @@
function upper(str) {
return str.toUpperCase();
}
onmessage = async event => {
postMessage(`data: ${upper(event.data)}, thanks`);
};
4 changes: 4 additions & 0 deletions test/statsCases/worker-public-path/index.js
@@ -0,0 +1,4 @@
const worker = new Worker(new URL("./worker.js", import.meta.url), {
type: "module"
});
worker.postMessage("ok");
8 changes: 8 additions & 0 deletions test/statsCases/worker-public-path/webpack.config.js
@@ -0,0 +1,8 @@
/** @type {import("../../../types").Configuration} */
module.exports = {
mode: "production",
entry: "./index.js",
output: {
filename: "[name]-[contenthash].js"
}
};
6 changes: 6 additions & 0 deletions test/statsCases/worker-public-path/worker.js
@@ -0,0 +1,6 @@
function upper(str) {
return str.toUpperCase();
}
onmessage = async event => {
postMessage(`data: ${upper(event.data)}, thanks`);
};
10 changes: 10 additions & 0 deletions types.d.ts
Expand Up @@ -8680,6 +8680,11 @@ declare interface Output {
*/
workerChunkLoading?: string | false;

/**
* Worker public path. Much like the public path, this sets the location where the worker script file is intended to be found. If not set, webpack will use the publicPath. Don't set this option unless your worker scripts are located at a different path from your other script files.
*/
workerPublicPath?: string;

/**
* The method of loading WebAssembly Modules (methods included by default are 'fetch' (web/WebWorker), 'async-node' (node.js), but others might be added by plugins).
*/
Expand Down Expand Up @@ -8974,6 +8979,11 @@ declare interface OutputNormalized {
*/
workerChunkLoading?: string | false;

/**
* Worker public path. Much like the public path, this sets the location where the worker script file is intended to be found. If not set, webpack will use the publicPath. Don't set this option unless your worker scripts are located at a different path from your other script files.
*/
workerPublicPath?: string;

/**
* The method of loading WebAssembly Modules (methods included by default are 'fetch' (web/WebWorker), 'async-node' (node.js), but others might be added by plugins).
*/
Expand Down