Question: Support for overriding chunk loading

[wilcoxmd]

Hi! Great work on this!

Is it possible yet to support overriding how chunks are loaded? I was taking a look at this package and trying to work out whether we could support Node.js environments. Doing so may require some additional customization in order to add headers or perform other operations to do server-side chunk loading. From what I've gathered, it looks like the webpack/node federation plugins allow you to override globalThis.webpackChunkLoad to accomplish this.

[zhangHongEn]

esmodule does not need to override webpackChunkLoad

[zhangHongEn]

But the full support of node has not yet been completed

[zhangHongEn]

You can run it in node, including overriding webpackChunkLoad. I haven't studied whether it will cause memory leaks.

[zhangHongEn]

Once this PR is merged it will work on node, but I haven't verified stability yet. Not in full state yet.
module-federation/core#2934

[wilcoxmd]

Thanks for the quick response @zhangHongEn!

esmodule does not need to override webpackChunkLoad

Could you explain this more? I'm not sure what it would look like to fetch from a private S3 bucket or other server without being able to customize how the modules are loaded (e.g. using S3 sdk or fetching with custom headers)

[ScriptedAlchemy]

Node federation works is by patching webpacks runtime, the webpack runtime has a concept of chunk handlers. webpack_require.f = {}

In webpack for loading node.js modules we use "async-node" target, which makes requrie() === fs.readFile(vm.runInThisContext)

We then patch the handler so webpack_require.f.readFileVm = handleHttpChunksToo == fetch().then(vm.runInThisContext)

Outside of webpack/rspack/turbopack - there is no concept other bundlers provide of a chunk handler, instead they just delegate this to the platforms native import() - so in Vite, there is no chunk loading to override, at least not within the bundlers runtime. Since it does not augment chunk loading.

Node federation is basically middleware on how the bundler IO, we replace what webpack thinks its filesystem read method is, and all webpack cares about is that a string returns an object that evaluates to the data structure of a chunk.

[ScriptedAlchemy]

the chunk handlers is also how federaiton works in general. shared code is require.f.consumes, and remote imports are require.f.remotes, when a chunk is loaded, or about to be, handlers are called before install. So like main, we pass through .f.remotes('main', promises); f.consumes('main', promises), f.readFileVm('main',promises);

This ensures that all the things 'main' needs are loaded before main is installed or evaluated, since MF is JIT - you cannot add the async requirements someone needs as they are loaded, we have to hold up evaluating main till all its requirements are ready. which is all done via chunk handlers. Thus in node, we .f.consumes will call remoteEntry .f.readFileVm, which is node federations runtime that teaches remote how to load its chunks from network origin to provide them back to whoever is needing them