Multiple import maps

[yoavweiss]

こんにちは TAG-さん!

I'm requesting a TAG review of multple import maps (née "dynamic import maps").

Import maps currently have to load before any ES module and there can only be a single import map per document. That makes them fragile and potentially slow to use in real-life scenarios: Any module that loads before them breaks the entire app, and in apps with many modules the become a large blocking resource, as the entire map for all possible modules needs to load first.

This proposal is to enable multiple import maps per document, by merging them in a consistent and deterministic way.

• Explainer¹ (minimally containing user needs and example code): url
• Specification URL: spec url
• Tests: Not yet
• User research: N/A
• Security and Privacy self-review²: url
• GitHub repo: url
• Primary contacts (and their relationship to the specification):
  • Yoav Weiss (@yoavweiss), Shopify
• Organization(s)/project(s) driving the specification: Shopify
• Key pieces of existing multi-stakeholder (e.g. developers, implementers, civil society) support, review or discussion of this specification
• Key pieces of multi-implementer support:
  • Chromium comments: Intent to Prototype
  • Mozilla comments: Multiple import maps mozilla/standards-positions#1058
  • WebKit comments: Multiple Import Maps WebKit/standards-positions#381
• External status/issue trackers for this specification (publicly visible, e.g. Chrome Status): https://chromestatus.com/feature/5121916248260608

Further details:

• I have reviewed the TAG's Web Platform Design Principles
• Relevant time constraints or deadlines: Hoping to ship by EoY
• The group where the work on this specification is currently being done: HTML
• The group where standardization of this work is intended to be done (if current group is a community group or other incubation venue): HTML
• Major unresolved issues with or opposition to this specification: None AFAIK
• This work is being funded by: Shopify

[LeaVerou]

This is very exciting stuff! 😍 ¹

As a very initial comment, could you please add usage examples to the explainer (or a move it to actual, separate explainer) that show what design is actually being proposed? We simply don't have the bandwidth to be wading through pull request diffs to hunt this information down, and I really wouldn't want something like this to fall off our radar because of this.

Footnotes

1. A comment to the problem being solved; I have not seen the proposed design because looking for it exceeded the time I had to allocate… ↩

[yoavweiss]

Thanks!!

I added a usage examples section to the explainer.

We simply don't have the bandwidth to be wading through pull request diffs to hunt this information down

Yup, totally understandable! Let me know if the above helps, or if you'd like me to pull the explainer to its own doc. (The current form felt more "discoverable", but happy to move it if that helps)

[martinthomson]

@yoavweiss, at first glance, this looks great. We had a few questions:

• As we understand it, the requirement to ensure that a mapping cannot change is hard to meet. It looks like a resolved mapping has the effect of fixing that mapping so that new additions to the map don't change it. How do you avoid nondeterminism? We see that you require the mapping to be stable while loading a module graph, but what about between different graphs? We're thinking about top-level async module loading here and we're not clear on how this might interact with that.

• If a mapping is used, but does not resolve successfully (perhaps because no mapping exists) is that a state that needs to persist? Why is that safe?

• The discussion in Sharing styles with Declarative Shadow DOM w3c/tpac2024-breakouts#31 (comment) seemed to indicate that server-rendered custom elements would want to add inlined CSS modules to the module map as they're sent to the client. Does that need support from this proposal, and if so does this proposal support it? cc/ @justinfagnani @rniwa

[justinfagnani]

Does that need support from this proposal, and if so does this proposal support it

I don't know if it needs support from this proposal directly, but I do think that the edge cases we talked about wrt inlined style modules that populate the module map are the same and should be handled consistently - probably by referring to common steps?

The main edge cases are race conditions between an import from a module in the module graph, and the handling of an inline module (or dynamic import map here), ie a fetch of a module is in-flight while an inline entry for the module is discovered. Assuming this proposal specifies what happens in those cases, it'll be a big help for the inline modules proposal.

[yoavweiss]

• We see that you require the mapping to be stable while loading a module graph, but what about between different graphs?

Apologies! That part has changed as part of the PR review, and I failed to update the relevant part in the explainer.
After lengthy discussions, @hiroshige-g convinced me that a single global tree is better.

I agree that we can't get rid of non-determinism here entirely. As you say, async top-level modules where the import map is defined later may resolve their dependencies either before or after the map is processed. The same can be true for dynamic imports.

At the same time, I wouldn't expect this to be a problem in practice:

• Non-determinism already exists, and it doesn't seem to be a problem in practice (e.g. a dynamic import can invalidate an import map if it currently loads before it)
• I'd expect developers to define the import maps relevant to their modules and load them when needed. I'd wouldn't expect these maps to have a lot of conflicts in practice.
  • A real-life scenario we're seeing is that different parts of an app are handled by different teams/entities, each managing its own modules and their mapping. In this case I'd expect zero overlap.
  • Another real-life scenario this would enable is a large app that dynamically loads parts as they are needed, and injects the relevant mapping before the dynamic import is called. In these apps, any overlap between the maps is likely to be duplicate rules, and hence ignoring them won't change resolution.

• If a mapping is used, but does not resolve successfully (perhaps because no mapping exists) is that a state that needs to persist? Why is that safe?

Can you expand on this? I'm not following..

• The discussion in Sharing styles with Declarative Shadow DOM w3c/tpac2024-breakouts#31 (comment) seemed to indicate that server-rendered custom elements would want to add inlined CSS modules to the module map as they're sent to the client. Does that need support from this proposal, and if so does this proposal support it? cc/ @justinfagnani @rniwa

Injecting inline modules into the module map seems largely orthogonal to import maps. I played around with it a bit (as a prototype for some interesting loading experiments that I need to get back to), and I think the main question there is "how do we determine the specifier?". Once we've answered that, I don't know that it makes sense to use import maps to translate that specifier to a URL, where there is no such URL in practice.

I don't know if it needs support from this proposal directly, but I do think that the edge cases we talked about wrt inlined style modules that populate the module map are the same and should be handled consistently - probably by referring to common steps?

The main edge cases are race conditions between an import from a module in the module graph, and the handling of an inline module (or dynamic import map here), ie a fetch of a module is in-flight while an inline entry for the module is discovered. Assuming this proposal specifies what happens in those cases, it'll be a big help for the inline modules proposal.

While there are some similarities, relying on a specifier to be in the module map for resolution and defining specifier mapping in the import map seem like different problems, at least at first glance. For the latter, I think we can safely ignore rules which overlap with past import maps or past resolved modules. For the former, we'd need to decide what we do when the required modules are not (yet?) there at resolution time.

[martinthomson]

OK, with respect to non-determinism, it would be good to have some text on that in the specification. Or even some way to manage it (can a script that performs dynamic loading test whether a particular import map has been seen?).

The spec PR seems to be a little out of sync with what you just said (it's written in the usual impenetrable language, so I'm not 100% here) with what you say. It says something about the map being fixed and then propagated throughout each top-level module load.

If a mapping is used, but does not resolve successfully (perhaps because no mapping exists) is that a state that needs to persist? Why is that safe?

I think that you indirectly answered this one with your non-determinism answer. My question, expanded:

It appears as though there is a requirement that the map, once queried, cannot change. Does that apply only in cases where the query results in a successful resolution, or does it apply in other cases:

1. An attempt is made to load "foo" and "foo" doesn't exist in the map. Does that mean that a future addition to the map cannot add a mapping for "foo"?
2. An attempt is made to load "foo" and "foo" exists in the map, but the resolution fails (an HTTP 404 error on the indicated resource, say). Does that make "foo" available for overriding in a future addition to the map?

These are not particularly obvious to me from the explanatory material, though I'll admit to not having gone through all of your updated examples.

[yoavweiss]

The spec PR seems to be a little out of sync with what you just said (it's written in the usual impenetrable language, so I'm not 100% here) with what you say. It says something about the map being fixed and then propagated throughout each top-level module load.

Good catch. I removed that processing model, but left in an inaccurate note. I'll fix that.

If a mapping is used, but does not resolve successfully (perhaps because no mapping exists) is that a state that needs to persist? Why is that safe?

I think that you indirectly answered this one with your non-determinism answer. My question, expanded:

It appears as though there is a requirement that the map, once queried, cannot change. Does that apply only in cases where the query results in a successful resolution, or does it apply in other cases:

1. An attempt is made to load "foo" and "foo" doesn't exist in the map. Does that mean that a future addition to the map cannot add a mapping for "foo"?
2. An attempt is made to load "foo" and "foo" exists in the map, but the resolution fails (an HTTP 404 error on the indicated resource, say). Does that make "foo" available for overriding in a future addition to the map?

These are not particularly obvious to me from the explanatory material, though I'll admit to not having gone through all of your updated examples.

Good points! The addition of a module to the resolved module set happens when the module is resolved, but before any import map rules are applied.
So you're right, and failed resolution attempts would end up preventing future rules from being set for these specifiers.

That doesn't seem like what we'd want here. I'll change the addition to only happen after a successful import map resolution (if any).

At the same time, for 404s, I don't think we should change future resolution for them, as they were properly resolved.

[martinthomson]

Glad to be of assistance. Do you have any thoughts on providing assistance with managing any non-determinism? I missed a step above, but the idea is that if a script performing a dynamic load depends on the presence of a specific import map, how would it guarantee that the map has been seen and integrated?

It seems like your resolution here would ensure that a missing mapping only gets committed once it is resolved (whether it fails or not), but that isn't ideal because a missing mapping would then permanently enter that state if the dynamic load acted before the mapping was in place. "Don't do that" is a perfectly fine response if there are obvious ways to manage that.

[yoavweiss]

Glad to be of assistance. Do you have any thoughts on providing assistance with managing any non-determinism? I missed a step above, but the idea is that if a script performing a dynamic load depends on the presence of a specific import map, how would it guarantee that the map has been seen and integrated?

Given that import maps are loaded synchronously (inline) that doesn't seem hard to manage from a developer perspective, as long as they control both pieces of code. But this is something we can add in the future as a function of developer demand.

It seems like your resolution here would ensure that a missing mapping only gets committed once it is resolved (whether it fails or not)

My intention is to make it so that a failed resolution (missing mapping) doesn't get added to the "resolved module set", so that it can be added in the future. I'll push a commit to that effect shortly.

[yoavweiss]

My intention is to make it so that a failed resolution (missing mapping) doesn't get added to the "resolved module set", so that it can be added in the future. I'll push a commit to that effect shortly.

whatwg/html@dd0ff09 handled that logic. I'd appreciate a review to see that we're aligned on this point.

[martinthomson]

I don't know if I understand well enough to review. As far as I can see, you commit something if result OR asURL are non-null. That asURL bit seems off, but I wasn't able to confirm. (Funny how GitHub stops working properly for like 110k of a file, even if the diff is modest in size, almost like it is a bad idea to have files that large).

[yoavweiss]

That asURL bit seems off, but I wasn't able to confirm

asURL is null in case that the URL is not a URL-like specifier (essentially, if we fail to parse the specifier as a URL)

[jyasskin]

Thank you for the discussion. We're happy to see this proceed through the HTML review process.

[yoavweiss]

Thanks for reviewing!! :)