RFC: node_compat next steps

[threepointone]

We've landed preliminary support for node.js compatibility in 2.0, powered by @esbuild-plugins/node-modules-polyfill/@esbuild-plugins/node-globals-polyfill. We can also auto detect usage of node built-ins, and suggest to the user to add node_compat = true to their wrangler.toml. This was a good start. Follow up items.

• The plugins we're using don't provide much in terms of customisation or control. We should replace the plugin with one we own "in house", so to speak. This will let us have control on which polyfills we enable, and how they're configured (eg: this will solve 🐛 BUG: Cannot use node builtin polyfill "crypto" #1047)
• The polyfills used by those packages (https://www.npmjs.com/package/rollup-plugin-node-polyfills) are old and not maintained, we should replace them with ones that are maintained better (options Project Not Maintained ionic-team/rollup-plugin-node-polyfills#28)
• That is still temporary tho, we should also start planning for using our own polyfills that fit well in the Workers environment. As an example, the community polyfills for http are built on XMLHTTPRequest, which isn't available in Workers. We'd need to rebuild it with fetch (if that's feasible)
• A long term plan might be to also provide runtime support for node compatibility; i.e., for the Workers runtime itself to provide require-able/import-able polyfills for node built-ins and globals.
• There's also an immediate opportunity to provide "anti" polyfills for fetch polyfills as described in feat: automatically replace fetch polyfills (isomorphic-fetch, node-fetch, unidici etc) with shims #1011.

cc @jasnell who I bet would be interested.

[evanderkoogh]

I would have to dive into both crypto libraries again, but I am reasonably sure that it will be extremely hard to polyfill the nodejs crypto library with the WebCrypto one, because the NodeJS one is synchronous and the WebCrypto one asynchronous if I remember things correctly..

And I am also pretty sure that providing runtime support for polyfills is going to be a dependency nightmare.. Where we are instantly inheriting all the backwards compatibility challenges in Node.

[threepointone]

There are inherent tradeoffs, and we're aware of them, but a best effort approach is better than not having anything at all for people who want to use libraries in the ecosystem. The consequence of not having a blessed solution is that people will try anyway.

[penalosa]

[petebacondarwin]

More

[JacobMGEvans]

Yo

[3x071c]

Hey 👋 Thanks for all the fantastic work on the project to all you guys. When can we expect a halfway working node crypto polyfill, if at all? I wished there was no need for it, but the node ecosystem is what it is...

[jasnell]

Crypto is in progress. No specific timeline other than soon.

[Zerebokep]

Could someone explain the serious tradeoffs?