How should we support language features between projects?

[sndrs]

these are minutes from/the result of a discussion in the meeting...

For example Object.fromEntries:

• currently has 85% browser support
• can be polyfilled by core-js@3
• can be polyfilled by pollyfill.io
• not supported by core-js@2
• will never be transpiled by TypeScript

This means any instance of Object.fromEntries in a package's source code will go straight through to the browser. The only way to guarantee it will then run is if the consuming application manages polyfilling it (if necessary).

Possible solutions

1. @guardian modules must only ship ES5

• universal support near-enough guaranteed
• consumer applications cannot benefit from more efficient bundling
• package source code stays running old JS

2. Define a guardian JS environment

An example spec might be language features that do not need polyfilling in 90% of browsers on the Guardian.

For example, @guardian/js-env might look like:

// the response is normally empty
export default 'https://polyfill.io/v3/polyfill.js?features=a,b,c,d,etc';

• would be a rolling spec, would need to be maintained and versioned
• packages would need to declare a peer dependency on the spec (e.g. @guardian/js-env@123)
• consuming apps would need to ensure they met it (e.g. <script src="${require('@guardian/js-env@123')}" />)
• difficult to trickle changes out

3. Stop trusting that NPM packages ship ES5

• not transpiling node_modules presents a risk – you assume the published code will run in your target env
• if we assume that all of NPM may need transpiling and polyfilling, then that will include our packages by default
• consuming apps need to ensure their final bundles will work
  • will need to stay up to date with feature handling

Option 3 seemed the most popular, simply because it's hard to counter that we basically do not know what we install from NPM (not really). By acknowledging this, we can make our applications more resilient and solve the support question in one go.

Any thoughts on this? @SiAdcock I know you've put a lot of thought into this for source.

[SiAdcock]

I'm not sure if this qualifies as an Option 4, but there has been some discussion in the Source issues (guardian/source#305). We suggested shipping heavily transpiled bundles for legacy browsers (using the main property in package.json) and lightly transpiled bundles for modern browsers (the module property in package.json). We then cut the mustard on <script type="module"> - the module/nomodule pattern

EDIT: nope! Doesn't qualify, as we're talking about langauge features here, not syntax. Can we conditionally polyfill different features for different builds?

However I do like Option 3. This solves the problem for all dependencies, not just @guardian projects. What would be the downsides to this approach?

• slower build times in CI
• slow recompilation in development?
• anything else?

[sndrs]

slow recompilation in development?

would we be safe to leave node_modules out in development? i wonder how much slower it is. i'll do some tests.

@sindresorhus makes the point that babel caches well, so it may be that the hit is only taken when you have a cold cache.

Can we conditionally polyfill different features for different builds?

would we end up needing to maintain a set of standards that way? it wouldn't solve the stray esnext npm module problem, but is it something that would be useful nonetheless?

anything else?

one thing that i wondered about is communicating and enforcing the requirement (of option 3). it's not widespread practice, even if not unprecedented, and generally it's been safe so far to not worry about node_modules. if we agree to explicitly make them a problem, how do we manage consumers who don't anticipate that?

[sndrs]

Running make compile-javascript in frontend:

node_modules	cache	time
❌	✅	1:07
✅	❌	1:23
✅	✅	1:12

[sndrs]

ha ha! @SiAdcock that discussion in source runs a very similar tack :)

[JamieB-gu]

I think we're starting to see the value of discussing things on GitHub issues and discussion boards more these days - if we'd had that discussion ^ in person or on chat it would have been lost.

[sndrs]

...valuable side effect of working remotely – you're always working remotely from future developers/you

[sndrs]

if there is some appetite for this, should it be our first addition to the recommendations?

any thoughts how could we go about making sure everyone who needed to know about it does? and make sure people are prepared for it?

[JamieB-gu]

should it be our first addition to the recommendations

Sounds good. Where do they live? GitHub repo? Similar to the managers docs? PR in changes?

[SiAdcock]

I think Mariot is keen for us to revive these client side recommendations

[sndrs]

guardian/recommendations#38

[sndrs]

that is now merged. but it did throw up a new question around which module types a package should provide.

how does this sound?

packages can provide either or all of the following:

*.js

• commonjs module
• targets browserslist's maintained node versions with babel-preset-env

*.mjs

• es module
• targets ES2020

package is for the web (i.e. will go through a build pipeline)

{
    "module": "index.mjs",
}

package is for node (i.e. will be used untranspiled)

{
    "main": "index.js", 
}

package is for either

{
    "main": "index.js", 
    "module": "index.mjs",
}

[SiAdcock]

Thanks for moving forward with this @sndrs, I'm totally on board.

The only part I'd question is the mjs file extension for ES Modules. I understand that this file extension is required by NodeJS to distinguish between CommonJS and ES Modules, but it's not part of the ES Modules spec.

Therefore I wanted to raise the question whether we should peg ourselves to what is essentially a workaround to a platform-specific problem? Browsers, for example, don't require modules to have this extension.

Also are there any tradeoffs in using mjs? MDN suggests that TypeScript can't support mjs files although I can't find a source for this info. Maybe there are other tools and frameworks that would have difficulty with it.

[sndrs]

yeah agreed, i think mjs is a diversion here

how about this?

packages can provide any/all of the following:

• pkg.main
  • commonjs module
  • targets browserslist's maintained node versions with babel-preset-env
• pkg.module
  • es module
  • targets ES2020

package is for the web (i.e. will go through a build pipeline)

{
    "module": "path/to/my_esm_index.js",
}

package is for node (i.e. will be used untranspiled)

{
    "main": "path/to/my_cjs_index.js", 
}

package is for either

{
    "main": "path/to/my_cjs_index.js", 
    "module": "path/to/my_esm_index.js",
}

[AWare]

Why is maintained node versions our web target?

[sndrs]

web target would be es2020, with the expectation that you would use module when bundling for the web. main would be intended for use in node, untouched. that's the idea at least...

[AWare]

oh, so common is for node! totally misread that.

[sndrs]

full on read here facebook/react#11503 (comment) from webpack author!

also been looking at @pika/pack – it takes a very similar tack to this discussion (but with much fuller options).

feel like a common package that we can use to build @guardian packages would be useful here. capture best practices in @pika/pack-style @guardian/pack builder.

maybe it could pretty much just wrap @pika/pack with some defaults for now?

here's a test of using @pika/pack. the pkg directly is what it builds with a very simple config