Garbage collection

[kriskowal]

We need to take the garbage out of mop’s build products.

For example, for every HTML, JavaScript, and JSON file, Mop creates a new script-injection ".load.js" file. We don’t really have a reliable indicator of whether one of these files is intended to be used as a module or as a resource. An HTML file might be loaded directly in an iframe, or a JSON file might be XHR’ed. A script might just be a script. For this reason, up to this point, I elected to retain both files.

However, we could mandate that the user make an explicit "resources" block in their package.json, or enable garbage collection when they do so. The "resources" block would be an array of glob patterns identifying files that are not modules. If something is a resource, we won’t create the script injection form. If it is not a resource, we would remove the plain file and retain only the script injection file.

Then there’s bundling. A package can have multiple entry points in various HTML files. We can only remove the non-bundled script-injection files if they are missing from the bundles, including preload bundles, for every entry point.

We can’t remove a script-injection module just because it is not included in any bundle because of require.async and the lazy property in a serialization. These permit a file to be loaded even if they are missing from all bundles. We could add a flag to package.json that disables lazy loading, allowing us to remove all these files from bundles.

[Stuk]

"We can only remove the non-bundled script-injection files if they are missing from the bundles, including preload bundles, for every entry point."

Should this be included in? The way I read it we would only remove script-injection files if they are included in the bundle.

Or by "missing from" do you mean, not referenced?

[kriskowal]

This is a bit easier to express with symbolic logic.

file should be removed from build products
if file is of type ".load.js" and
and
     bundle includes file
     for every entry point
     for every bundle in the union of
         the bootstrapping script for the entry point
         every shard of every phase of preloading for that entry point

[Stuk]

Okay, to reword it as I understand: a .load.js file can be removed if for every case where the file is required it is available in a bundle from every entry point.

[kriskowal]

Right. And the opposite: the .load.js file must be retained if there’s any chance that it will be brought in by require.async, which can happen in two ways (at present): it could be in an explicit require.async in the code, or it could be done by the deserializer if the serialization says "lazy": true for a component.

[kriskowal]

This is byzantine. My plan is for the v2 architecture of mop/mr to address this by eliminating the need to generate .load.js scripts and remove any .js file that’s not expressly referred to in HTML as a script. All modules will be statically bundled for every entry point, and we will no longer expose require.async to users.

[cdebost]

@tejaede @johnnykahalawai Check out the comment above. I've seen discussion of the v2 architecture scattered around both the mr and mop repositories, a lot of which seems highly relevant to the discussions we're having now regarding bundling.

Unfortunately the v2 branch of mr was never merged and is now extremely out of date compared to master, but I have a feeling there are some great ideas in that branch that would be worth exploration.