Idea: “dev-serverless” JS

[Jarred-Sumner]

I wonder if ESBuild WASM could enable “dev-serverless” JS

Specifically the flow would be:

• Go to special (remote) webpage
• Drag and drop code folder into page with index.html
• Using the new Filesystem API, contents of page are replaced with index.html, but esbuild runs in web worker and live bundles app / assets.
• NPM imports replaced with Skypack/JSPM/whatever
• URL changes to match directory name and the directory handle is persisted to IndexedDB. Next time you refresh it reloads the folder and bundles again (remembers)
• Now you have an in-browser “dev-server” without installing any software or running Node.js and with modern JS tooling
• Unlike CodeSandbox/etc this lets you use your own local and comfortable development environment/IDE.
• maybe could do deploys this way too. With a literal one-click in-app deploy button
• with a lot of work, could probably make it Next.js compatible via intercepting fetch requests in service workers & magic

if the bundler is fast enough to run inside the webpage and works with the local dev environment, why would people still use and install a CLI for this?

The 2nd order effects would be interesting too.

Will the next generation of frontend engineers know how to use terminals? If you only use a terminal to run “git pull”, “git clone”, “git commit” and “git push” then a git GUI makes more sense.

If you’re not running npm install, will web apps still have a package.json?

[nettybun]

This is discussed in #797. You're welcome to try implementing all of these features around esbuild but they will be outside of the scope of esbuild itself.

[Jarred-Sumner]

Yeah, I'm not suggesting this is something in-scope for esbuild.

It's not the same idea though – there wouldn't be an in-browser editor or REPL. You would edit locally in Visual Studio Code, or Vim or whatever you use normally. It would sync via the Filesystem API to bundle & run in the page, without the developer downloading any local software. Your "dev server" becomes the same webpage you use to load the webpage locally.

[nettybun]

Right ok but can this issue be closed then if its not in esbuild's scope?

[Jarred-Sumner]

it works better than expected, the 69ms here bundles a create-react-app directly from the local filesystem in a service worker on page load (though it is rewriting imports to use jspm instead of loading all the node_modules)

[zaydek]

@Jarred-Sumner If you want to keep posting updates here, I’d be interested to see FWIW.

[Jarred-Sumner]

Heres a demo of what it actually looks like right now. The demo app is https://github.com/ahfarmer/calculator but very slightly modified, I added a <script> pointing to the index.js file in the index.html and removed the github-ribbon-css import. Then it uses https://github.com/jarred-sumner/htmlbuild to configure esbuild entry points.

Screen.Recording.2021-03-10.at.10.48.37.PM.mov

Notice how after selecting public/index.html, it shows you the entry points you used. It also detects when the file is not found and shows a warning.

[zaydek]

Ah very interesting. I saw htmlbuild recently as well. Very cool work you got here. I assume this isn’t the project you’re working on (mentioned in your Tw bio) because this seems like a new idea for you?

Have you hooked up any fast refresh functionality or planning to?

[Jarred-Sumner]

Yeah that’s different I’ve only looked at it a little. Open questions: 1. How do you implement it without a full JS ast? From reading the guide facebook/react#16604 (comment), I’m not sure if there’s a great way. https://github.com/facebook/react/blob/master/packages/react-refresh/src/ReactFreshBabelPlugin.js Looking through the code for the Babel plugin, this does a lot of work. Much more than most Babel plugins I’ve seen. It seems to look specifically for built in hook usages, as well as the various ways you can define a React component. A runtime-only version probably lacks the correct persistent IDs. I wonder if there’s a way to do it via source maps. Like if you diff the source maps on bundle updates, you probably can figure out which exports had changes if you parse it again. Then you just say, every hot reloadable thing is a module boundary. 2. How would you tell esbuild to compile a specific file within the bundle without recompiling the entire bundle? The equivalent to webpack’s hot module loader. Or is there a different way of framing that issue? I think the userland of this — notifying clients of updates, is quite easy (service worker postMessage) 3. How do you implement file watchers? This ones probably the simplest — just poll the File object returned by the Filesystem Access API and keep a key of probably the file path + last modified + size and when the last two change you say the file changed and you just check and recheck repeatedly, possibly in a SharedWorker or maybe not

…

On Thu, Mar 11, 2021 at 1:49 PM Zaydek ***@***.***> wrote: Ah very interesting. I saw htmlbuild recently as well. Very cool work you got here. I assume this isn’t the project you’re working on (mentioned in your Tw bio) because this seems like a new idea for you? Have you hooked up any fast refresh functionality or planning to? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#925 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFNGS2DOPEPQ5RP5JFIBRDTDE3G7ANCNFSM4YWVTYNQ> .

[zaydek]

Looking through the code for the Babel plugin, this does a lot of work.
Much more than most Babel plugins I’ve seen. It seems to look specifically
for built in hook usages, as well as the various ways you can define a
React component. A runtime-only version probably lacks the correct
persistent IDs. I wonder if there’s a way to do it via source maps. Like if
you diff the source maps on bundle updates, you probably can figure out
which exports had changes if you parse it again. Then you just say, every
hot reloadable thing is a module boundary.

Oh wow. I know that hot module replacement is this cool thing abstractly but I’ve never understood what the heck it actually means until now. I didn’t realize how much work HMR is doing. I don’t even know if it’s worth it at that point since it tends to be buggy (in my experience). Honestly I’m pretty happy with fast reload in general. But I’m grateful I now have a sense of how HMR actually works because the idea has always escaped me previously. Tagging @nojvek since this is relevant to his interests.

How would you tell esbuild to compile a specific file within the bundle
without recompiling the entire bundle? The equivalent to webpack’s hot
module loader. Or is there a different way of framing that issue? I think
the userland of this — notifying clients of updates, is quite easy (service
worker postMessage)

Out of curiosity, does incremental: true not solve for this problem space more generally?

How do you implement file watchers? This ones probably the simplest —
just poll the File object returned by the Filesystem Access API and keep a
key of probably the file path + last modified + size and when the last two
change you say the file changed and you just check and recheck repeatedly,
possibly in a SharedWorker or maybe not

I’ve found file watchers to be pretty fun and simple to implement. I noted in one of Evan’s old projects he pretty much did what you’re talking about: https://unpkg.com/browse/fast-watch@1.0.0/watch. This is my MVP implementation Go: https://github.com/zaydek/esbuild-watch-demo/blob/master/watch.go. Neither of these use some of the advanced random polling techniques Evan is using on this project but they’re still good for validation.

I don’t know enough about Worker threads in JS to comment more.

[Jarred-Sumner]

Oh wow. I know that hot module replacement is this cool thing abstractly but I’ve never understood what the heck it actually means until now. I didn’t realize how much work HMR is doing. I don’t even know if it’s worth it at that point since it tends to be buggy (in my experience). Honestly I’m pretty happy with fast reload in general. But I’m grateful I now have a sense of how HMR actually works because the idea has always escaped me previously. Tagging [preventing double tag] since this is relevant to his interests.
Out of curiosity, does incremental: true not solve for this problem space more generally?

It does, but for HMR/fast refresh, you don't want to reload all the code for the page. Instead, you want to only update "changed" code. While typing this comment on GitHub, if GitHub's bundle reloaded, my work in progress comment text would be gone, which would be frustrating if I was only changing the "Comment" button color to blue.

Currently, esbuild returns the entire bundle. One thing I could try without asking for any of @evanw's time is look through metafile at the inputs and outputs and automatically generate a new entrypoint for each file importing the file that changed. If I could map it to specific functions, then I could just return new entry points for javascript code that "imports" the function call in a plugin.

import {Foo} from 'changed-component'; 

$RefreshReg["changed-component.Foo"] = Foo;

export {$RefreshReg["changed-component.Foo"] as Foo}

esbuild would bundle that into a single file.

Then the server would postMessage to the client "new code 4 u" and the client would do something like:

try {
  for (let id of updatedComponentID) {
     $RefreshRegWillChange(id);
  }
  await import(newCodeURL)
  for (let id of updatedComponentID) {
     $RefreshRegDidChange(id);
  }
} catch(exception) {
  for (let id of updatedComponentID) {
     $RefreshRegRevert(id);
  }
  ErrorPage.render(exception);
}

I’ve found file watchers to be pretty fun and simple to implement. I noted in one of Evan’s old projects he pretty much did what you’re talking about: unpkg.com/browse/fast-watch@1.0.0/watch. This is my MVP implementation Go: https://github.com/zaydek/esbuild-watch-demo/blob/master/watch.go. Neither of these use some of the advanced random polling techniques Evan is using on this project but they’re still good for validation.

thanks for the links

[Jarred-Sumner]

I got it to automatically work with most create-react-app's last night, meaning its just drag + drop create-react-app into browser, click yes, and you have a dev server running. On subsequent runs, it loads automatically when you go to the subdomain (and updates on refresh)

[zaydek]

Is that a real domain?

[Jarred-Sumner]

I have it deployed under a real domain with wildcard subdomains but for local development I use “*.example.d”. It’s just mkcert with the dns resolver configured so that I can test the service worker stuff

…

On Fri, Mar 12, 2021 at 2:33 PM Zaydek ***@***.***> wrote: Is that a real domain? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#925 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFNGSYLFY75VXJ2B4PLAPTTDKJDLANCNFSM4YWVTYNQ> .