Module Federation Redesign

[ScriptedAlchemy]

note on backward compatibility: the existing interface will remain compatible. Moreover we will support a webpack plugin with parity to rspack and may implement the new federation design as a webpack plugin to start

Ive posted a blog regarding the broader future: https://medium.com/@scriptedalchemy/the-future-of-module-federation-665dd33947cb and more depth

Plugin api:

What should the plugin give us?

remotes already accepts an array like:

{myremote: ["global@url", "promise new Promise()", "global@otherurl"]}

If the first item fails, or throws error, it will loop over the array of options till one resolves.

remotes

• more options
• typescript field that connects/syncs types
• built in middleware, allowing dynamic glue code (picking multiple envs and so on)
• fallback strategy per remote (some normalization pattern needed)
  • local: fall back to file
  • buildHttp: pull remoteEntry and chunks associated with it, back them up in consumer build
  • npm: fall back to npm version of module
  • support layers like entrypoints and loaders

{
        name: '@zack-dev/product',
        // exposed modules
        exposes: {
            '.': './src/logo.tsx',
            './routes/product': {
                   import: './src/routes/$.tsx',
                   layer: 'server'
           },
            './routes/product.loader': './src/routes/$.loader.ts',
        },
        startup: require.resolve('./somefile'), // code that ships with the remoteEntry and runs at startup, middleware runs on consumer side, startup runs on the provider 
        middleware: require.resolve('./src/remote-middleware.js'),
        remotes: {
            normal: 'normal@http://remoteEntry.js',
            home: {
                name: '@zack-dev/home',
                shareScope: 'default',
                external: 'home@http://',
                types: 'http://' | function() {} | './relative-path', //can be url, function that gets url or map, or relative path to local types
                fallback: {
                    local: require.resolve('./src/home-fallback.js'), //if remote offline, select a local file to bundle in as backup view.
                    buildHttp: true, // download remote over http at compile time, fetch all the remotes chunks too and back them up in current build, each rebuild would fetch new copy of remote
                    npm: '@somePackage/home' || config.home.name, // lazy load npm package or library as fallback. 
                },
            },
        },
        // Set shared dependencies
        shared: {
            'my-vue': {
                // can be referenced by import "my-vue"
                import: 'vue', // the "vue" package will be used a provided and fallback module
                import: 'React@https://someCDN.com/react-umd', // allow async externals passthrough for shared. 
                shareKey: 'shared-vue', // under this name the shared module will be placed in the share scope
                shareScope: 'default', // share scope with this name will be used
                singleton: true, // only a single version of the shared module is allowed
                strictVersion: true, // don't use shared version when version isn't valid. Singleton or modules without fallback will throw, otherwise fallback is used
                version: '1.2.3', // the version of the shared module
                requiredVersion: '^1.0.0', // the required version of the shared module
                preferOrigin: ['host', 'self', 'specificremote'], // allow multiple origin preferences to be set
                runtime?: {
                    host: {
                        eager: true,
                    },
                    remote: {
                        eager: false,
                    },
                },
            },
        },
    }

Shared

Share scope should allow "quantum modules" concept - allowing us to make a dependnecy eager/lazy per runtime instead of globally.
This would allow for react to be eager:true in a host, but eager: false in a remote runtime.

Share scope runtime api today:

{
    "react-dom": {
        "18.2.0": {
            "from": "home",
            "eager": false,
            "loaded": 1
        },
      "18.1.0": {
            "from": "shop",
            "eager": false,
            "loaded": 0
        }
    },
    "react": {
        "18.2.0": {
            "from": "checkout",
            "eager": false,
            "loaded": 1
        }
    },
    "styled-jsx/style": {
        "5.1.1": {
            "from": "v1691033992291",
            "eager": false,
            "loaded": 1
        }
    },
    "use-async-resource": {
        "2.2.2": {
            "from": "v1691034151816",
            "eager": false,
            "loaded": 1
        }
    }

If duplicate shared modules offered, keep them in array and "pick" top one by default

{
    "react-dom": {
        "18.2.0": [{
            "from": "home",
            "eager": false,
            "loaded": 1
        },
        {
            "from": "shop",
            "eager": false,
            "loaded": 0
        }]
    },
}

Doing so allows us to avoid overwriting share scope, we can then "pick" what version is chosen by changing the runtime modules slightly, similar to how @2heal1 has suggested for lifecycle middleware.

I have actually done this before, but as a hack - by making share_scope a Proxy(), through that i was able to choose what "react" actually returns to a consumer during negotiations.

Middleware/runtimeplugins:

• what should this look like?
• historically ive used the middleware require.resolve convention to create a module. that can be programmatically controlled.
• what information should be passed to middleware?

//TODO: what should ctx contain?
module.exports = async (ctx)=> {
  return {get,init}
}

Node
Node apis / plugins need enhancements.

We node federation should support filesystem calls that are relative / local volume on network as well. Like EFS from aws.

This offers alternatives to my traditional remote code execution and a better pattern than offering a hidden chunk load global patch.

globalThis.__remote_scope__ needs something better within the node plugin.

We need to register the known paths to remoteEntry in memory, so that when node needs to fetch chunk, it knows what the origin path is. @2heal1

Runtime API

Currently returns {get,init} api, but this is not enough for modern demands. Federation interface needs additional root interfaces.

chunkMap - reflection of chunks needed mapped to a exposed module, represents both css and js chunks as well as section for shared module chunks that it depends on as well. Useful for SSR manifests to remove waterfall
info - anything that cant/shouldnt go into share scope. Ideally things like tracking who is using what and where, things we would want chrome devtools to read, info about whats in use, and so on.
get - like typical federation get - however, there should be the option to decorate it.

Additional metadata about what module is being fetched would be very helpful, for example if its a react component or util function.

const response = await get('./header');

response = {
default: ()=>{},
meta: {
 type: 'jsx'
}
}

 window.remote  = {
  get: async function (id) {
    try {
    return await __webpack_require__.e(someID).then((id)=>__webpack_require__(id));
    } catch (e) {
      return await onGetError(e, id);
    }
  },
  init: async function (initScope, initToken, ctx) {
    try {
      await onBeofreInit(initScope, initToken, ctx);
      await this.init(initScope, initToken, ctx);
      await onAfterInit(Object.freeze(initScope), initToken, ctx);
    } catch (e) {
      await onInitError(e, initScope, initToken, ctx);
    }
  },
 setPubilcPath: function(path){}, // yes?
 info: {
    shareInfo: {'react': {prefers: ['host']},
    exposeInfo: {
     './header': function(){return __webpack_require__(345);}
   }
  },
  chunkMap: {
    integrity: "sha384",
    "home_header": {
      moduleId: 345, //to help with something like loadable/component
      chunks: ["/chunks/home_header.js", "/css/chunks/home_header.css"],
      requires: [
        {name: 'app2/App'}, {name: 'app3/Button'},  // dependent remote
        {name: 'react', info: [0, 18, 0, 2]},//[strictVersion,major,minor,patch]
        {name: 'lodash', info: [0, 4, 17, 3], chunks: ['/chunks/lodash.js']}
      ],
    },
  }
}

Are there any other aspects we would want to have runtime control? from both global and independent remote interfaces?

__webpack_require__.federation for example:

federation.getAllConnectedRemotes = ()=>{}
federation.getAllSharedModulesInUse = ()=>{}
federation.uninstallRemote = ()=>{}
federation.useFallbackof(remote)
Global Runtime Api?
Should there be a global runtime api for things like

federation.getAllConnectedRemotes
federation.getAllSharedModules
federation.registerRemote
federation.import

@web-infra-dev has put a lot work into "VMOK" which is similar to medusa. The runtime api / module info provided to users is very comprehensive.

Dynamic, pluggable api:

@2heal1 has already shared some api, this is great start

export const RuntimePlugin = (config)=>{
   return {
// this should pass arguments "ctx" - what should ctx provide?
       beforeLoadRemoteEntry(ctx){
              // can programmatically return remoteEntry url
       },
       loadRemoteEntry(){},
       beforeInit(){
           // before remote entry / shared init
       },
// this should pass arguments "ctx" - what should ctx provide?
       init(ctx){
         // init remote entry and shared
       },
// this should pass arguments "ctx" - what should ctx provide?
       loadRemote(ctx){
           // call remoteEntry.get('remote module')
       },
// this should pass arguments "ctx" - what should ctx provide?
       errorLoadRemote(ctx){
          // error catch 
       },
   }
}

WIP

Compile Time Features

• Support partial import paths: import('myremote/${someVar}')
• Support AWS FE discovery service/adjust medusa/VMOK to return the spec. https://github.com/awslabs/frontend-discovery

[2heal1]

I think runtimePlugins can combine with delegate module ,and may like this structure:

export const RuntimePlugin = (config)=>{
   return {
       beforeLoadRemoteEntry(){
              // can programmatically return remoteEntry url
       },
       loadRemoteEntry(){},
       beforeInit(){
           // before remote entry / shared init
       },
       init(){
         // init remote entry and shared
       },
       loadRemote(){
           // call remoteEntry.get('remote module')
       },
       errorLoadRemote(){
          // error catch 
       },
   }
}

But don't sure whether we can programmatically control shared loading process.

[ScriptedAlchemy]

Yes in fact we can control shared negotiation process!! This is a weakness in federation and an area we can improve upon. Let me update the description and add my ideas about shared

[ScriptedAlchemy]

Update: this is what would become "middleware" and replace delegate module design.

[pavandv]

We should also support dynamic remotes as in a remote URL should/can be able to fetch from a MFE Discovery Service api at runtime.

[ScriptedAlchemy]

Agreed. Runtime and dynamic api needs to be more robust and accessible.

[stacey-guild]

We're building this exact thing using https://github.com/module-federation/utilities/tree/main/runtime/importRemote. It's been challenging to say the least.

Here's an example I made (from my personal account) staceyshk/module-federation-examples#1

[AlexAegis]

This is very important to avoid compiling a bundle for each deployment. I was able to hack it into webpacks module federation using a compile time bundle mutation suggested by @ScriptedAlchemy (Thank you!) https://gist.github.com/ScriptedAlchemy/60d0c49ce049184f6ce3e86ca351fdca

At my job we're using importmaps to locate federated bundles, so the information of what's the remote, is already present in the importmap.

[ScriptedAlchemy]

I dont even remember writing MutateBundle plugin LOL!!! Well im glad it works for you.

Okay @stacey-guild can you elaborate on this, maybe not as a comment to this thread but a new comment on the discussion? Please be thorough about what you need, what you've built to try and achieve this, and what you would like (can be psudo code)

Bytedance has an incredible runtime api internally, so alot of things ive wanted from my ecosystem - theyve got or have something close to an ideal state. We need to understand what you want so we can factor this into the approach / shed light on areas we may not think about, but will end up wishing we had :P

[stacey-guild]

Most definitely, I'll write something more thorough after work today. Thanks for being open to feedback, I really appreciate it!

[yzydeveloper]

Can it be made as easy to use as esm's importmap? Just build the commonjs module and do the mapping of the module to the remote paths via something like require-map .

[ScriptedAlchemy]

Present an example. Remember this needs to work for servers and other areas as well. Open to ideas but show some pseudo code so we can understand what part you're referring to

[lucamezzalira]

shared: {
            'my-vue': {
                // can be referenced by import "my-vue"
                import: 'vue', // the "vue" package will be used a provided and fallback module
                shareKey: 'shared-vue', // under this name the shared module will be placed in the share scope
                shareScope: 'default', // share scope with this name will be used
                singleton: true, // only a single version of the shared module is allowed
                strictVersion: true, // don't use shared version when version isn't valid. Singleton or modules without fallback will throw, otherwise fallback is used
                version: '1.2.3', // the version of the shared module
                requiredVersion: '^1.0.0', // the required version of the shared module
                preferOrigin: ['host', 'self', 'specificremote'], // allow multiple origin preferences to be set
                runtime: {
                    host: {
                        eager: true,
                    },
                    remote: {
                        eager: false,
                    },
                },
            },

I think this is the settings for the host right?
If so:

• why does it need to know about the remote eager configuration? can you derive it at runtime?
• can you elaborate the use cases for preferOrigin?
• what do you think to call shareKey and shareScope just scope and key considering we are configuring the shared object?

[ScriptedAlchemy]

1. Remote eager Configuration
   The eager configuration embeds a dependency into the Webpack runtime. It's often required only in stubborn hosts, not in remotes, which are asynchronous by nature. Setting a dependency as eager in the host can be helpful in cases like Next.js middleware, where we cannot load extra chunks and need a single chunk with all dependencies. However, forcing the remote entry to follow the same restrictions can be counterproductive.

So, we're aiming to give Webpack instructions on a per-runtime basis - whether to embed the module into the runtime or lazy load it. In the absence of a runtime setting, a global eager: true would be treated as it is today. Please note that eager impacts the chunk graph and module locations during build-time chunk creation, which is why it cannot be altered at runtime.

2. preferOrigin Use Cases
   preferOrigin is designed to control the source preference for shared modules. Without this control, a shared module (like react) can be sourced from any application in your federation, causing potential issues like Hot Module Replacement (HMR) failure in dev mode or cache thrashing.

Key use cases are:

• Development Mode: Ensuring that the shared module (e.g., react) is sourced from a dev build rather than a production build to enable HMR.

• Cache Management and Consistency: By setting a preference, you can ensure that the shared modules are sourced more consistently from a specific application as long as semver rules are satisfied.

• Vending Remotes: This feature allows teams to create "dll" style remotes with an ideal set of dependencies. It helps in achieving more consistent operations and testing upgrades via platforms like Medusa.

• Hotfixing and Self-healing: With the help of preferOrigin, teams can hotfix the entire module graph in real-time. This opens up opportunities for self-healing, self-optimizing, and self-forming "application mesh".

3. shareKey and shareScope Naming
   While the names "key" and "scope" might seem more intuitive within the "shared" context, changing these could result in a breaking change across the ecosystem. If such a change is proposed for Webpack, it's likely to be postponed until the next major release (v6). We could consider introducing new keys with deprecation warnings, but this would require the Webpack core team's consensus given the potential impact on existing builds.

[ranshamay]

my 2 cents -

1. retry/fallback mechanism in case of failed remotes fetching is missing, i think it should be configured in the plugin config and allow users to control it.
2. robust logging across runtime operations

[RussellCanfield]

I do like the schema for discoverability, it feels like the container entry as Zack mentioned is pretty limited with a get/init interface once it's loaded. The additional metadata that we could capture may be useful longterm for consumers

[ScriptedAlchemy]

Retry / fallback should be covered under fallback key on remote, allowing you to bundle a npm package as a backup or use buildHttp to download the remoteEntry at compile time, and backup the remote and its chunks into the consumers graph in the event remote is offline.

[ScriptedAlchemy]

Additional metadata plug perhaps additional function methods beyond {get,init} what would they be?

[ScriptedAlchemy]

Yes, i have plans to support/pivot to @lucamezzalira JSON schema, this will allow others to also auto-deploy or scaffold infra with ease or even via medusa service which could produce the schema for both build and aws on demand.

Schema may have limitations as its json, so if we support functions or other non standard extras - might need to be optional to use as JS and so on

[RussellCanfield]

I think having the ability to run arbitrary javascript before mounting the container would be useful - not necessarily tied to the container entry API. For the container entry API, I think at least exposing the module map could be nice. Maybe the arbitrary javascript could live as an extension to the init method? So allow a remote to run code before/after the standard init.

[stiaanj]

Looking forward to rspack support 😁

[ScriptedAlchemy]

Redesign is largely for implementing it once and only once into rspack, so might as well design while we are at it lol

[yuanyuanlife]

Looking forward to to a simpler interface design for sharing files across Angular projects, something like this:

{
     exposes: [],
     imports: [],
}

Ideally, it could even be configured to be compatible with Angular's Dependency Injection (DI).

With this design, I could directly import the exposed entities as if the files were locally available in my projects.

Just to clarify, the exposes and imports arrays in this configuration object are placeholders. In a real use case, we would replace them with strings representing the paths to the modules we want to expose or import.

Such a simple interface saves a lot of time when reusing my code and my teammates' code from other Angular projects. Moreover, this approach would eliminate the need for me to grapple with the complex concepts underlying Module Federation.

[ScriptedAlchemy]

I usually make a folder called exposed then have a util recursively map over the folder and expose anything in there.

Id probably refer to the aws schema set out and try to adhere to the spec as agreed upon.

[RussellCanfield]

My suggestions:

Middleware/Runtime API

• I think having a middleware both on the consumer (host) and provider (remote) could offer the most flexibility. The provider middleware idea is something I've been toying around with to put a sort of "encrypted remote" feature in where I can put a small proxy around the IFFE in the container entry (middleware in the future), not saying it needs to work exactly like this but on the consumer side I could pass it encryption keys or something similar for it to validate. Seemed useful to execute code before I expose a module map, etc. and felt like a nice lifecycle type hook.

• I agree that exposing more than get/init such as including the chunk map would be beneficial as well, allowing a remote to possibly execute code before it's bootstrapped could allow you to extend that even further.

• I spoke a little with @valorkin @zackarychapple about this idea as well, but I love the middleware idea in general. My thoughts were that the "MF Runtime" would be what you'd want exposed in the middleware. Existing remotes/shared scope, the remote info (json schema file?) along with pretty standard lifecycle methods for a remote container:

---

I do like the shared scope idea, I feel like this could be another great place to introduce hooks into the process. It would be very useful at the very least for logging to know how the shared scopes are being negotiated and determined. I also agree that the consumer should not try to specify how a remote handles eager shared modules as shown below.

runtime: { host: { eager: true, }, remote: { eager: false, }, },

---

Does this fit with something like a MF/core concept at all? @ScriptedAlchemy

[ScriptedAlchemy]

Okay so you want middleware on both host consumer and provider side. I can see value in this, because you could set up env for remote to run in

what about “startup” key or something similar to middleware

[RussellCanfield]

I think that would work, middleware definitely offers flexibility to inject code. I guess the question comes down to, is there ever a case where you'd want to expose custom code from a remote before it's initialized and it's exposed modules are available. But maybe that could still be accomplished with middleware on the remote side as well as on the host side. Let it hook into those lifecycle events

[tysonrm]

Support federated isomorphic client, so we can write something like this:

import Client from '@aegis/core'
import { IModel, IAdapter, IDataSource } from './component-model'

const url = 'https://raw.githubusercontent.com/module-federation/aegis-app/master/dist/remoteEntry.js'
const client = new Client(url)

const order = await client.import<IModel>('models/order')
const fedex = await client.import<IAdapter>('adapters/fedex')
const store = await client.import<IDataSource>('datasources/orderdb')
order
  .addAdapter('ship', fedex)
  .addDataSource('orderdb', store)
  .createInstance()
  .list({ status: 'APPROVED' })
  .then(list => list.forEach(order => order.ship()))

[ScriptedAlchemy]

Yes, you want runtime api to be able to dynamically attach containers to system like webpack would do at compile time/runtime

[tysonrm]

Thanks for your help, Zack. The above code now runs. For the redesign effort, we might want to eliminate the task of exporting the runtime and reexporting the utilities. Note that the runtime wasnt needed for remote import, i.e.:

  const __webpack_require__ = require('./runtime.js')

Nor did i need

import('fake')

all that's needed is

export { injectScript, getModule } from './main.js'

import { injectScript, getModule } from '../exported-utils'

Of course the utils need webpack_require.l which they have access to in main.js.

[tysonrm]

...and maybe something like this:

new CentralFederationConfigPlugin ({
    awsOrderContainer: {
       module: 'models',
       exportName: 'orderService',
       url: 'https://my-bucket.s3-us-west-2.amazonaws.com/remoteEntry.js',
   },
   cloudflareOrderContainer: {
       module: 'adapters',
       exportName: 'orderCacheDurableObject',
       url: 'https://cloudflare.com/my-cdn/remoteEntry.js',
  }
})

[ilteoood]

I would also add the ability to completely exclude libraries from the bundle, to speed up its compilation time, and give the user the possibility to choose the CDN that will be used to download that dependency.

I'm already doing this with import maps, would be awesome to have a simple interface to it with module federation!

[ScriptedAlchemy]

I believe that's possible. We could treat some as async externals and allow "import" to accept url, this may be problematic for commonjs consumers though (server side)

But I can also use BuildHttp which will download the url and bundle it into the app just as if it were file on disk

[tysonrm]

const nodeExternals = require('webpack-node-externals')

[ScriptedAlchemy]

@ilteoood do you have suggestion for how config would look?

[ilteoood]

@tysonrm that plugin is different than what I'm proposing. It allows you just to not include some dependencies. On top of that, I'm also suggesting to allow the user to define where to pick that dependency at runtime: your cdn, a public cdn or something else.

@ScriptedAlchemy do you think that reusing the same configuration structure of import maps is bad? IMO it allows users to not deal with something too complex that can also be reused.

[manuelc2209]

@ScriptedAlchemy I'm not sure if my edge case enters on this scenario but i had a rough time dealing with multiple service workers from different mfes to a single host. Is it viable we enhance this a bit more? Besides that, had a good experience using module fed 🚀.

[manuelc2209]

I dont have a public example of it yet, but lets imagine the following scenario and you probably can already predict the direction im heading with it.

Host -> Offline capability
Specific MicroFrontend -> Cached also for offline usage.
Other MicroFrontends -> Online
(Each, Host & MicroFEs should be responsible for their own SW)

Ignoring some of the headaches of PWAs, if they're well built with small bundle size, its quite tricky to achieve it, at least, based on the lack of examples out there and full support, i feel like it should be something more friendly to setup and it took me a couple of sleepless nights to achieve, hehe.

What is your opinion on this @ScriptedAlchemy, can we improve it overall?

[ScriptedAlchemy]

If you were able to create a proof of concept. It would help. I did have an idea where module graph could have bindings to other modules as serialize, this would let function args be passed between threads or local and the cloud

[manuelc2209]

Will do, as soon as i have something up, i'll tag you on it!

Also, side question, not adding an dependency to one of those MicroFE's in the (Shared section) of the ModuleFederationPlugin (lke e.g. a Component library that i want tree shaken, seems viable ? )

I know that the purpose of the Plugin is to allow to reutilize those same deps. But i don't want to bundle each and everyone of them if one is a PWA and i have bundle size constaints. Is there any negative side effects of not adding it to the shared deps on a specific use case?

Thanks a lot @ScriptedAlchemy!

[ScriptedAlchemy]

no, only issue will be if you need a singleton. But i usually suggest against blanket sharing all node modules and ask the userbase to pick and choose what makes sense to share

[manuelc2209]

I had that doubt based on the shared examples of MF, it's common for them to share the pckg.json deps. That's why i was wondering.

[jheyming]

Any way we can support cache-busting features out of the box for remoteEntry.js?

[staceyshk]

Hello @RussellCanfield,

importRemote is using a query parameter of t=<timeInMs> as the cache buster which as mentioned above does not work in all cases... It depends on both the browser you're using and the set up of your CDN.

We ran into this additional param while running end to end tests where our tests expected to intercept a certain url that didn't have an appended timestamp. I do wish that was optional, we handle our cache busting using versioning.

I've never tried delegate modules, I'll definitely have to investigate.

[RussellCanfield]

Definitely, I still deploy to unique urls within my CDN to prevent any cache issues. Then you can safely leverage cache control headers

[staceyshk]

Thanks for the great discussion @RussellCanfield, I love all this caching talk. A lot of developers don't consider it until it becomes a problem.

[jheyming]

@staceyshk yeah caching is important, that's why I started this thread. Seems like there still is no one-size fits all solution so feel free to ignore my initial question.

[alex-vukov]

@staceyshk Maybe it's a bit late but the cache busting query parameter in importRemote is actually optional and can be turned off by putting bustRemoteEntryCache: false in the object which you pass to the function.

[2heal1]

Will this redesign change the webpack implementation? Or it will implement by external webpack plugin?

Although we can add JavaScript's plugin in rspack rather than rust code, performance will not good .

[ScriptedAlchemy]

I believe that we should start his as a forked plugin, like VMOK is. we can look to add back to webpack in future but the process will take a very long time if we have to wait for webpack. So i suggested we ship as a standalone plugin, the JS plugin would let us iterate and decide if we are happy with new api - then we would write it in rust for RSpack.

I think JS is cheaper and faster to test ideas on while rust and rspack will take a little longer, so proving the api in JS seems like a easy way to go and would ensure support for webpack still.
what do you think?

[2heal1]

Gotcha, I agree with it . By the way , will current(or previous) mf implementations into rspack ?

Or we can use JS plugin until it becomes complete enough , then we start to writing it in rust.

[ScriptedAlchemy]

Js plugin till complete enough. We can check what apis we need in rspack to support js plugin. That might offer nearer term support and still supports webpack ecosystem. I can look at what is missing that rspack needs for federation plugin to work as js api

[zhoushaw]

I was wondering if we could use loader to implement some of mf's original capabilities

[ScriptedAlchemy]

@zhoushaw Ive done this long ago in webpack 4 - i think we can pull off something like this till rspack team is ready

[staceyshk]

Reposting for tracking through my personal account - aka I'm @stacey-guild as well.

Hello @ScriptedAlchemy! Took me a minute to get bandwidth, here's what we're doing now plus a few suggestions for improvements. I feel like a lot of these have been mentioned already which is awesome!

We’re building a host platform where many different teams can plug in remote pages and add homepage widgets. The remotes span many teams across the company and require their own development/deployment cycles. Our host had become a deployment bottleneck for every time someone wanted to add a new remote, even just in staging because of review cycles and general "too many moving parts, not enough context" syndrome.

As a result, we are retrieving our remote list from an api endpoint where each team can add their remote location and important details like “title, permissions, etc” to include for context in a database. This allows us dynamic locations and a changing number of remotes without redeploying our host as well as having different sets of remotes in dev/staging/prod. We are using the importRemote utility to load the pages at runtime.

Example return object from our api:

RegistryApiObject {
  url: 'https://test-remote.whereisit.com/remoteEntry.js', 
  scope: 'nameInPlugin',
  module: 'exposedComponentName',
  path: '/test-remote’, // Relative to the base url, where it should live in the host added as a route
  name: ‘Test Remote’, // Title for rendering in side menu
  permissions: ['test-permission'], // Who can see it, compared with our user token
}

Generic example here of loading a remote list and adding remotes to routes - https://github.com/staceyshk/module-federation-examples/pull/1/files#diff-831816579a766d2738d58fcb01416a8e11606b16f17bd5ee022f2a5b43295dc5R29

The benefit of this is we're using the user token permissions to decide what the user isn't allowed to see before it even gets to the host. For security reasons checking permissions within your client-side javascript isn't the best idea if you can avoid it (we have security on the backend as well, it's always "when you get hacked, not if").

For our use case some remotes expose 2 components from the same location, one that is a page and one that is a widget used on the homepage which share functionality. Right now importRemote requires you to specify the module so we end up calling the function twice; once on the homepage load for the widget and once when we load the corresponding page.

importRemote(url, scope, module) // returns one component to load

Ideally importRemote becomes a core feature where you can call importRemote(url, nameInPlugin) and it would return an array of component objects such as:

RemoteComponentObject {
  exposedComponentName: string, // Naming conventions are helpful
  component: Component, // To be lazy loaded
  type: string, // Programmer defined type like - “homepageWidget” or something like that, or maybe options or something?
}

Alternatively, we could also solve this with middleware prefetching as you’ve got in the queue above and some browser caching or with server-side prefetching and rendering (that’s a big lift for us though, all of our apps are client-side only at the moment).

Thanks a lot for taking the time to read our use case! Let me know if you have any follow up questions.

[ScriptedAlchemy]

@2heal1 @zhoushaw this looks like similar themes of vmok

[zhoushaw]

I have a few suggestions：

1. You can use assets instead of chunks, and distinguish between synchronous and asynchronous chunks, which can help us better distinguish between exposed synchronous and asynchronous resources. For example:

{
    "exposes": [
        {
            "moduleName": "Button",
            "assets": {
                "js": {
                    "sync": [
                        "static/js/async/__vmok_expose_Button.eb1083b5.js"
                    ],
                    "async": [
                        "static/js/async/page.df6d5d28.js"
                    ]
                },
                "css": {
                    "sync": [],
                    "async": []
                }
            }
        }
    ],
    "shared": [] // Same data structure
}

2. The Runtime Api adds init, which can inject shared and plugins at runtime, which can help consumers completely separate from the use of build tools. All loadRemote and loadShare implementations can be implemented on this sdk, and build tools can directly reference this plugin，such as:

import { init } from '@module-federation/utilities/runtime';

init({
  remotes: [
        {
            name: "@demo/app3",
            alias: "app3",
            entry: "http:://xxx/remoteEntry.js"
        }
    ],
   shared: {
        react: {
            version: "17.0.0",
            scope: "default",
            lib: ()=> React,
            shareConfig: {
                singleton: true,
                requiredVersion: "^17.0.0"
            }
        },
    },
   plugins: [],
})

Load remote modules, support the same reference method aliases and names as import, so that the runtime api also supports type hints
importRemote: (id: string | { url: string, scope: string, module: string })=> Module

[zhoushaw]

My idea is to put mf loading modules, dependency reuse, and other build-independent logic into the runtime api and sdk for maintenance as much as possible, and reference the runtime sdk in the build tool, so that the sdk can also be completely separated build tools run

[zhoushaw]

@ScriptedAlchemy I don't understand federation.getAllConnectedRemotes and federation.getAllSharedModules api very well, can you provide the type and usage instructions of this api

[ScriptedAlchemy]

I think the portable logic you speak of, would be best served by creating a library from one of the remoteEntryies already, or enhancing a the api interface further so that it can serve this logic via runtime sdk. recreating the whole logic wouldnt make sense, but we can make it so that a remoteEntry could have the runtime hooks needed, then it can serve as a runtime api even if npm packaged as a library?

[zhoushaw]

Yes, I just want to put the main logic in the sdk, not directly in the build runtime, so that it can be used completely away from the build tool, which is more important to help the project use in the enterprise

[ScriptedAlchemy]

My idea can https://github.com/module-federation/module-federation-examples/blob/master/dynamic-remotes-node/pure-server.js

So reason a remoteEntry with better api cannot serve as the library itself. We want the webpack runtime, not the webpack build.

This would allow other codebases to be managed by "webpack reuntime" regardless of what they are

[virtoolswebplayer]

a few suggestions

Offer clearer error notifications, such as when micro-module B relies on a module that is not exported in micro-module A, display an error message within module A to indicate the source of the reference.

container-entry:669 Uncaught (in promise) Error: Module "./@chanjet/chanjet-h5-biz-components/lib/voucher-number-input/util" does not exist in container.
while loading "./@chanjet/chanjet-h5-biz-components/lib/voucher-number-input/util" from 73804

better to

"./@chanjet/chanjet-h5-biz-components/lib/voucher-number-input/util" does not exist in app-A source of the reference from app-B

[ScriptedAlchemy]

Will see, but these are webpack issues

[ScriptedAlchemy]

start by sharing nothing, then only share singleton packages that you must, like react and react dom.

[insaneinside]

My team has found that it's very difficult to introduce token-based authentication (e.g. Bearer authentication) for remote module assets, in the context of the importRemote library. Authentication of requests for code assets can be important in certain environments.

• We can shim in authentication for remoteEntry.js by modifying importRemote to instead fetch the script and execute it with a <script> tag, with appropriate headers passed to fetch.
  • However, subsequent chunks will still be loaded using the Webpack-native __webpack_require__.l. It does not appear to be trivially possible to simply overwrite __webpack_require__.l to add authentication; I assume this is because Webpack maintains an internal reference to the function.
  • This method also appears to break Webpack's automatic publicPath feature, which assumes that it's loaded via a <script src="..."> tag.
• Using service workers to inject authentication headers into requests is a possibility but has scaling issues (service workers' same-origin restriction means that a library-based loader would have to configure a service worker for each host).

Our conclusion is that while injecting authentication is likely possible via workarounds, doing so is overly burdensome and it would be better to have first-class support for authenticated remote modules.

[AlexAegis]

Have you tried hacking the feature in yourself? https://gist.github.com/ScriptedAlchemy/60d0c49ce049184f6ce3e86ca351fdca

I had a similar problem, in the sense that I wanted to do something that's not supported: Resolving the publicPath at runtime so I don't have to compile a new bundle for every place we're deploying to.

I'd give a shot trying to mutate how webpack loads something, similar to this, maybe you could add your Headers in like that. For us this little hack will work just fine until it's actually supported.

[RussellCanfield]

You could put the url retrieval behind an authenticated service, such as launch darkly. Then you use a token to authenticate and retrieve the url. Does it need to be on content retrieval itself?

But you could also extend the remoteEntry file if needed as mentioned above.

[zhangHongEn]

Is chunkMap used for preloading for performance optimization? In addition to including shared modules, it should be better to include other remote references, right?

chunkMap: {
    integrity: "sha384",
    "home_header": {
      moduleId: 345, //to help with something like loadable/component
      chunks: ["/chunks/home_header.js", "/css/chunks/home_header.css"],
      requires: [
        {name: 'app2/App'}, {name: 'app3/Button'},  // dependent remote
        {name: 'react', info: [0, 18, 0, 2]},//[strictVersion,major,minor,patch]
        {name: 'lodash', info: [0, 4, 17, 3], chunks: ['/chunks/lodash.js']}
      ],
    },
  }

[ScriptedAlchemy]

yes i agree.

[ilteoood]

I think that in order to enhance the DX, we should also create a "database" of known dependencies that needs to be singleton to work properly, e.g.: react and react-dom.
These dependencies, then, will be automatically shared as singleton so the user doesn't need to do it by himself.

[ilteoood]

Additionally, versions should be automatically picked from node_modules or package.json, so the user doesn't need to deal with it as well.

[dcesiel]

As long as there is some sort of opt out mechanism. We have an architecture with multiple react roots on a page. This allows us to have multiple react versions in different scopes, but we share other dependencies that aren't usually singletons between the applications.

[ScriptedAlchemy]

Additionally, versions should be automatically picked from node_modules or package.json, so the user doesn't need to deal with it as well.

It should do this already. I rarely specify a version manually unless I'm pinning it or something specific

[jamime]

First of all, thank you!

One thing that's not clear to me right now is when to use a remote vs shared.

For example, I can publish my remote-entry.js to npm and reference it in shared as a package, or I can publish it to a CDN and reference it as a remote.

The benefit of a remote is that I don't need to redeploy my host to update a remote.

I watched this video on module federation versioning which shows the concept of loading a semver remote. I took that a step further and loaded esm modules in a promise-based dynamic remote.

// mf-app-one/webpack.config.js
const fetchUselessLib = (resolve) => {
  import('http://localhost:3000/npm/@local/jamime-usless-lib/^3.0.0/dist/browser/remote-entry.js').then(module => {
    resolve({
      get: (request) => module.get(request),
      init: (scope) => {
        try {
          return module.init(scope);
        } catch (e) {
          console.log("localJamimeUslessLib has already been loaded");
        }
      },
    });
  })
};

const remotes = {
  '@local/jamime-usless-lib': `promise new Promise(${fetchUselessLib.toString()})`,
}

// mf-app-two/webpack.config.js
const fetchUselessLib = (resolve) => {
  import('http://localhost:3000/npm/@local/jamime-usless-lib/3.0.1/dist/browser/remote-entry.js').then(module => {
    resolve({
      get: (request) => module.get(request),
      init: (scope) => {
        try {
          return module.init(scope);
        } catch (e) {
          console.log("localJamimeUslessLib has already been loaded");
        }
      },
    });
  })
};

const remotes = {
  '@local/jamime-usless-lib': `promise new Promise(${fetchUselessLib.toString()})`,
}

// jamime-useless-lib/webpack.config.js
plugins: [
    new webpack.container.ModuleFederationPlugin({
      name,
      filename: "remote-entry.js",
      library: { type: "module" },
      exposes: {
        "./version": "./src/version.js",
      },
      shared: {
        ...pkg.dependencies,
      },
    }),
  ],

  output: {
    uniqueName: `${name}-${pkg.version}}`,
  },

Using promise-based remotes allows me to use multiple different versions of the same package by defining a uniquename for the output. If I was to export the library as a var it would race as the names would be the same. If I was to change the name based on the version, the mfe wouldn't know the name of the global to use.

I currently lack the ability to re-use the remote - we don't have any information about the resolved version and it's not added to the shared cache.

I could replicate the shared behavior and store the remote on the window. However, if we introduced the version into the build (I'm currently exposing a version const in my package) then we could manage it in the shared module cache like any other module.

If we had this functionality, could we then look to merge the concepts of shared modules and remotes? Is there another difference that I'm missing?

For context - this is the problem I'm trying to solve. Remote lib team wants to be able to publish changes to a CDN and for consumers to automatically upgrade if it meets their semver criteria.

When using different major versions it works well

When the CDN resolves to the same version it works well

When using different sem ver ranges that could use the same module, we end up with duplication.

Update: The diagrams were updated to better depict the issue. Code examples were added for the remote lib.

[ScriptedAlchemy]

Okay so versioned remotes essentially. Yeah, i believe internally at bytedance we have remotes that respect versions similar to shared does. Shared always has a version thanks to npm, but remotes are harder to bake in the same thing since they need to be deployed and are not artifacts distributed in compile time to choose from.

So while version registration or module cache map of registered version can exist, ultimately to do it well - we need to improve federation so its a bit more intelligent on the {get} similar to {init}

I am introducing middleware and startup apis which should allow for robust enhancement and controls. promise new promise works, but its a bit of a hack and lack of compile and require() support in the promise template reduces its maintainability or possible complexity.

[jamime]

My thinking was that if shared had the ability to load a remote-entry.js from a CDN and the file had the version information for de-duplication then it's possible we wouldn't need the concept of remotes.

[dcesiel]

It would be helpful to be able to hook into runtime shared resolution to know what dependency version and chunk a federated module actually resolved to from the share scope. This would allow us to create better runtime inspection tooling. I noticed this runtime hook in your blog post: loadShare: new AsyncHook<[VmokHost, string, ShareInfos]>(),

Do you have an intention of including this resolution information there or making that ShareInfos context object extensible?

[ScriptedAlchemy]

yes this is something Byte needs as well, should be shipped and we will likely have devtools for chrome too so you dont have to build your own <3

[dcesiel]

Maybe I missed it. Is this shipped publicly?

[zackarychapple]

@dcesiel this will be something that will be part of Zephyr Cloud for sure.

[infoxicator]

@ScriptedAlchemy congrats on the release of MF 1.5! 🎉

One thing I could not find there was this proposal for shared:

preferOrigin: ['host', 'self', 'specificremote'], // allow multiple origin preferences to be set

Is that something that is still to come or is there another way of achieving the same?