TypeScript and ES6+ support using esbuild. #3738
Conversation
With the introduction of the "enhanced" compatibility mode, the test source code is transformed using esbuild instead of Babel. Source files with the extension ".ts" are loaded by esbuild's TypeScript loader, which results in partial TypeScript support. This esbuild removes exactly the type information, but does not provide type safety. Source files other than ".ts" are loaded by esbuild's JavaScript loader, which results in the support of a more modern JavaScript dialect than goja.
…rors. Running the tc39 tests also in "enhanced" (esbuild) compatibility mode. Previously, the test wrote the JSON format file to be saved to the standard output, which had to be manually cleaned from the other test output. This solution is difficult to apply in the case of one or more compatibility modes, so it has been changed to the usual golden file pattern. The reference breaking_test_errors-extended.json and breaking_test_errors-enhanced.json files can be updated using the go test -update command.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #3738 +/- ##
==========================================
+ Coverage 70.84% 70.90% +0.05%
==========================================
Files 289 290 +1
Lines 21156 21191 +35
==========================================
+ Hits 14988 15025 +37
+ Misses 5211 5209 -2
Partials 957 957
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
szkiba
requested review from
codebien and
olegbespalov
and removed request for
a team
There was a problem hiding this comment.
Source files with the extension ".ts" are loaded by esbuild's TypeScript loader, which results in partial TypeScript support. This esbuild removes exactly the type information, but does not provide type safety.
@szkiba can you expand on this topic? Isn't the main advantage of having TypeScript? Does it mean we are delegating it to another tool and k6 makes the assumption to receive a valid script?
With the introduction of the "enhanced" compatibility mode
Can you explain the reason behind the enhanced term, please? Isn't better for clarity to use something related to typescript or esbuild?
Last point, I see we are not running the js/tc39 suite. I expect it is the first requirement for being able to merge this pull request. Can you please make the changes for running it with the new mode? It is probably the fastest way to catch any potential issue without manually checking all the bits.
|
Hi @codebien , thank you for your review.
Sorry if I was unclear: we do not delegate the loading to a tool, but to the esbuild library. Within the esbuild library, there are so-called loaders for each type of content. The quoted sentence indicates that files with the .ts extension will be treated as TypeScript files, in the case of other file extensions, the input will be assumed to be JavaScript.
The name of the mode is not
I think the tc39 test is running, I modified the tc39_test.go file: |
If this is the case, what is missing for attempting a Babel replacement? So, instead of creating a new mode, we do an expansion of the extended mode.
Oh, thanks. I missed it. |
That's exactly what I think, esbuild will replace babel (one day). I created a new compatibility mode so that esbuild can be completely disabled (and disabled by default). With such a major, potential incompatibility change, I think this is important. Later, esbuild can take over the role of babel and then the two compatibility modes (extended and enhanced) can mean the same thing. |
|
If this is the case, should we go with something like |
How about |
|
@szkiba I tried running a tests in grafana-api-test library but I get this error: |
This is a consequence of the fact that esbuild does not run as a bundler, but transforms per module. The module resolving (require function) is implemented by the k6 runtime. Specifying the file extension is mandatory in this implementation. |
But the convention is not to include the extension, right? |
js/compiler/enhanced.go
Outdated
| opts := api.TransformOptions{ | ||
| Sourcefile: filename, | ||
| Loader: api.LoaderJS, | ||
| Target: api.ES2017, |
There was a problem hiding this comment.
This makes esbuild unneedingly transform a whole bunch of code that goja supports.
In practice from the list of features on https://esbuild.github.io/content-types/#javascript
The only ones not supported are async generators, async iteration and ??=. And in my test the async generator and async iteration doesn't work with the example from mdn.
This also adds a bunch of esnext stuff which I haven't tested, but will argue are probably even less desired by even our less average users given that they aren't released yet.
If you will likely to enable any of those it will be nice to also enable their tc39 tests respectfully. I would recommend that this is done after the initial release of this instead of now.
So my strong recommendation is this to be set to ESNext - which in practice disables esbuild doing any transformation for features and leave just the TypeScript + commonJS transofmration. Some tc39 tests get slightly different results witht that.
You can use the Supported field to set which transformation you want to add. Again I will recommend doing this after this is merged, maybe even after we have had some feedback.
There was a problem hiding this comment.
Done, I changed the target to ESNext (and then adjusted the golden file of the tc39 test)
| @@ -29,10 +29,10 @@ | |||
| "test/built-ins/RegExp/unicode_restricted_octal_escape.js-strict:true": "test/built-ins/RegExp/unicode_restricted_octal_escape.js: Test262Error: RegExp(\"\\1\", \"u\"): Expected a SyntaxError to be thrown but no exception was thrown at all <at omitted>", | |||
| "test/built-ins/RegExp/unicode_restricted_quantifiable_assertion.js-strict:true": "test/built-ins/RegExp/unicode_restricted_quantifiable_assertion.js: Test262Error: RegExp(\"(?=.)*\", \"u\"): Expected a SyntaxError to be thrown but no exception was thrown at all <at omitted>", | |||
| "test/built-ins/TypedArrayConstructors/BigUint64Array/is-a-constructor.js-strict:true": "test/built-ins/TypedArrayConstructors/BigUint64Array/is-a-constructor.js: ReferenceError: BigUint64Array is not defined <at omitted>", | |||
| "test/language/comments/hashbang/function-constructor.js-strict:true": "test/language/comments/hashbang/function-constructor.js: SyntaxError: test/language/comments/hashbang/function-constructor.js: Unexpected token (17:47)\n 15 | const AsyncFunction = (async function (){}).constructor;\n 16 | const GeneratorFunction = (function *(){}).constructor;\n> 17 | const AsyncGeneratorFunction = (async function *(){}).constructor;\n | ^\n 18 | for (const ctor of [\n 19 | Function,\n 20 | AsyncFunction, <at omitted>", | |||
| "test/language/comments/hashbang/function-constructor.js-strict:true": "test/language/comments/hashbang/function-constructor.js: SyntaxError: Async generators are not supported yet <at omitted>", | |||
There was a problem hiding this comment.
I guess this is due to some change in tc39_test.go - most likely because this is the error from babel before and the newer one is from goja.
I don't think this matters in this case - just mentioning it for others.
This hasn't been the convetion with ESM ... at all.
Browsers have always done this.
This was also one of Ryan Dahl regrets with node. The video is from the middle of a bunch of regrets aroudn how node does modules and resolution and such. |
|
I would argue #3738 (comment) is also a fairly good idea. But otherwise I don't see anything else |
Sure, but you suggested to do it after the merge (or after releasing and get some feedback)
Thank you, I'll rename the enhanced mode to experimental-enhanced |
|
The recommended after merging and feedback was about enabling back some of the transformation, not moving to ESNext |
Oh, I see, sorry |
…ransformation for features and leave just the TypeScript + commonJS transformation.
On master, the k6 binary size is 39612416 bytes, on branch with esbuild the k6 binary size is 43503616 bytes (x86 linux). |
What?
With the introduction of the "enhanced" compatibility mode, the test source code is transformed using esbuild instead of Babel.
Why?
Source files with the extension ".ts" are loaded by esbuild's TypeScript loader, which results in partial TypeScript support. This esbuild removes exactly the type information, but does not provide type safety.
Source files other than ".ts" are loaded by esbuild's JavaScript loader, which results in the support of a more modern JavaScript dialect than goja.
Checklist
make lint) and all checks pass.make tests) and all tests pass.Related PR(s)/Issue(s)
#3703
Closes #3703