Add option to SourceMapDevToolPlugin to exclude webpack:// protocol

[steinybot]

Feature request

What is the expected behavior?

When using devtool: 'nosources-source-map' I expect all entries in the sources array of the source map to be relative paths and not contain the webpack:// protocol prefix.

Personally I would rather there never be a webpack:// protocol prefix but that could be because I'm not aware of the rationale for having it in the first place.

As a last resort I would at least like the workaround to be documented. Currently the https://webpack.js.org/configuration/devtool/ docs just says:

shortcut	explanation
nosources-* addition	source code is not included in SourceMap. This can be useful when the original files should be referenced (further config options needed).

There is no mention of what further config options are needed. It should also be added as an example to https://webpack.js.org/plugins/source-map-dev-tool-plugin/.

What is motivation or use case for adding/changing the behavior?

I want to generate a source map file like:

{
  "version": 3,
  "sources": [
    "index.ts"
  ],
  "names": [
    "Error"
  ],
  "mappings": "mBAAA,MAAM,IAAIA,MAAM,U",
  "file": "bundle.js",
  "sourceRoot": "src"
}

I can then very easily deploy this to dev, test and prod and maintain full source mappings for error messages. In dev and test I can also deploy the src folder and have the sources available. Then in prod I don't deploy the src folder.

How should this be implemented in your opinion?

There is a basic workaround:

moduleFilenameTemplate: '[namespace]/[resourcePath]'

However this doesn't take into account what the public path would be or the sourcePath.

The full workaround looks something like:

const path = require('path');

const CopyPlugin = require("copy-webpack-plugin");
const HtmlWebpackPlugin = require('html-webpack-plugin');
const { SourceMapDevToolPlugin } = require('webpack');

const sourceDir = 'src';
const distDir = path.join(__dirname, 'dist');
const distSourceDir = 'src'

module.exports = {
  mode: 'production',
  entry: './src/index.ts',
  // Disable the devtool and specify the options to the SourceMapDevToolPlugin manually.
  devtool: false,
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        use: 'ts-loader',
        exclude: /node_modules/,
      },
    ],
  },
  resolve: {
    extensions: [ '.tsx', '.ts', '.js' ],
  },
  output: {
    filename: 'bundle.js',
    path: path.resolve(__dirname, 'dist'),
  },
  plugins: [
    new CopyPlugin({
      patterns: [
        { from: sourceDir, to: path.join(distDir, distSourceDir) },
      ],
    }),
    new HtmlWebpackPlugin(),
    new SourceMapDevToolPlugin({
      // Specify the filename so that the source map is not inlined.
      filename: '[file].map',
      // Prevent source content from being included in the source map.
      noSources: true,
      // This sets the sourceRoot but now we have to make sure to relativize everything to this.
      sourceRoot: distSourceDir,
      moduleFilenameTemplate: params => {
        const normalizedResourcePath = path.normalize(params.resourcePath);
        if (normalizedResourcePath.startsWith(sourceDir)) {
          return path.relative(sourceDir, normalizedResourcePath);
        } else {
          // Default.
          return 'webpack://' + params.namespace + '/' + params.resourcePath;
        }
      }
    })
  ],
  devServer: {
    contentBase: distDir
  },
};

I don't get what the namespace is. I also don't get why there is a fallback template.

One solution would be to add an option:

rebase: [{ from: sourceDir, to: distSourceDir }]

Are you willing to work on this yourself?
yes

[webpack-bot]

For maintainers only:

• webpack-4
• webpack-5
• bug
• critical-bug
• enhancement
• documentation
• performance
• dependencies
• question

[alexander-akait]

Personally I would rather there never be a webpack:// protocol prefix but that could be because I'm not aware of the rationale for having it in the first place.

Because you have multiple source maps on the page and it creates collisions between them.

As a last resort I would at least like the workaround to be documented.

PR welcome in docs site. A lot of time was spent to design this and to make it work, do not think that some things are made to interfere.

You have moduleFilenameTemplate and fallbackModuleFilenameTemplate options to implement what you want.

If you real need something not standard you can write own plugin to do it.

But I don't recommend to do it.

[alexander-akait]

Also sources should be URL by spec, not paths

[steinybot]

I'm sorry but I'm not following. Could you please help me understand?

Because you have multiple source maps on the page and it creates collisions between them.

Do you mean that there can be a collision between two different params.resourcePath? It's not clear to me what params.resourcePath or where it comes from. If it were either absolute or relative to a single directory then collisions couldn't occur.

A lot of time was spent to design this and to make it work, do not think that some things are made to interfere.

I don't doubt that there was a lot of time put into the design but it's not clear to me how to use it. Someone must have added the nosources- addition for a reason. There must be a way to get the browser to load it. What is meant by "further config options needed"? Does that mean by overriding moduleFilenameTemplate and fallbackModuleFilenameTemplate?

---

Also sources should be URL by spec, not paths

The spec says:

If the sources are not absolute URLs after prepending of the “sourceRoot”, the sources are resolved relative to the SourceMap (like resolving script src in a html document).

[alexander-akait]

On page you can have source maps from other files, for example from CDN or from other files, so we need to use prefix to avoid collisions

If the sources are not absolute URLs after prepending of the “sourceRoot”, the sources are resolved relative to the SourceMap (like resolving script src in a html document).

But webpack:// is absolute URL

[steinybot]

I do not believe that this is true, you do not need a prefix to avoid collisions.

On page you can have source maps from other files, for example from CDN or from other files, so we need to use prefix to avoid collisions

Uniqueness of source files is exactly the same problem as the uniqueness of any other resource that is loaded on the page. We already have everything we need to disambiguate them without needing a new protocol. In fact there is no ambiguity in the first place.

Consider we have a site on https://example.com. A relative resource foo/bar.js is resolved as https://example.com/foo/bar.js. There can be no other foo/bar.js resource. If there is then it has to have a different path such as foo2/bar.js or be on a different host and be an absolute URL https://mycdn.com/foo/bar.js.

Are you able to provide a concrete example of where not having webpack:// would be ambiguous and how making it absolute fixes it?

[alexander-akait]

I do not believe that this is true, you do not need a prefix to avoid collisions.

Why? We have a lot of issues about this, just use search...

[steinybot]

Why?

If there are 2 source maps a.js.map and b.js.map that both have the source file foo/bar.js then just putting webpack://foo/bar.js doesn't solve the collision. The collision has to be solved at a different level such as having those sources in different locations, for example a/foo/bar.js and b/foo/bar.js. This is exactly the same as any other resource.

We have a lot of issues about this, just use search...

Can you please give me a clue what to search for or provide a link to the search results?

I originally said:

Personally I would rather there never be a webpack:// protocol prefix but that could be because I'm not aware of the rationale for having it in the first place.

I've already said that I don't know what it is there and I'm trying to understand why it is. I'm sensing a lot of hostility on this issue.

[alexander-akait]

@steinybot

If there are 2 source maps a.js.map and b.js.map that both have the source file foo/bar.js then just putting webpack://foo/bar.js doesn't solve the collision. The collision has to be solved at a different level such as having those sources in different locations, for example a/foo/bar.js and b/foo/bar.js. This is exactly the same as any other resource.

Simple case, you have HTML:

<script src="https://cdn.com/library.js"></script>

Now you use the same library (import library from 'library';) and generate source maps (and you don't have webpack:// prefix as you want). Now you have multiple library.js in dev tools (sources), same for popular packages like lodash/underscore/etc.

But with webpack:// prefix you will have original source maps from https://cdn.com/library.js and from library in your code

You can partial look at #5767.

There are more complex cases. If you really want, I can describe them.

[steinybot]

Thank you for the explanation.

I agree with you that the scenario that you describe is a case where there is a conflict, however if you look at the description in #5767, it is a conflict with a file that does have the webpack:// prefix. That is a proof by counter-example that the webpack:// prefix does not solve conflicts.

If you look at the solution that issue (easiest way is to look at the diff for the docs https://github.com/webpack/webpack.js.org/pull/1653/files) it is to add a unique namespace prefix. This is the same as what I described in the workaround in my very first post with moduleFilenameTemplate: '[namespace]/[resourcePath]'. It is also the same as the example above with a/foo/bar.js and b/foo/bar.js.

This shows that the solution to conflicts is to make the paths unique by adding a unique prefix and not by adding the webpack://. With a unique namespace, it makes no different whether there is a webpack:// prefix or not, they are still unique. I feel that there is likely another reason for the webpack:// prefix that is not to solve conflicts but to solve some other problem that I am unaware of. Let's not get too hung up on the webpack:// prefix part of the issue, even without that there is a more difficult problem to solve. I brought that up more out of curiosity. Perhaps the issue title needs to be changed.

I have to remove the webpack:// prefix and make the paths relative in order to make the source map relocatable onto other domains. I can do this easily enough by setting moduleFilenameTemplate. As I mentioned, this simple workaround does not cover all cases as it ignores the public path / sourceRoot . My proposal is to add some functionality that will allow us to easily modify the resourcePath and make it relative to the source directory.

[alexander-akait]

I think you want to this #3603, not only for Node.js (i.e. target: 'node'), am i right?