Add support for Webpack 5 and drop previous versions #390

vhiairrassary · 2021-01-09T08:56:08Z

Add support for Webpack 5 and drop previous versions (removing up to Webpack 4 was discussed in Webpack 5 Support #350, it allows to focus time on Webpack 5 only, especially as webpack-dev-server invocation changed as well as the output format for errors and warnings)
Size from assets are displayed in a human format (similar to webpack, like 345,12 Kib instead of 345127)
Remove Built at log line from stats, as in Webpack 5 (sbt is already outputting a similar line after each task).
As webpack-dev-server invocation changed, the server port is not part of the command line but it is injected in the generated webpack configuration (might be a breaking for some users).
Remove --bail flag when invoking webpack as it does not return a JSON but print the error in stderr directly (parsing is failing because of that)
Bump net.sourceforge.htmlunit in test to support JS from Webpack 5. Bump some npm dependencies in test to support Webpack 5.
Remove Uglify from test as code is now automatically minified in production mode, and this plugin is not compatible with Webpack 5.

Currently I am able to use this version to compile a small application using webpack 5 (including webpack-dev-server).

Remaining tasks for future 🚧:

Fix test;
Errors & Warnings are not correctly parsed;
Include related assets;
Write unit tests for config. generation (code has been moved in side-effect free functions)
Update documentation
- https://scalacenter.github.io/scalajs-bundler/cookbook.html#how-to-use-webpack-4

Links

Official migration documentation (from 4 to 5).

Raw Webpack outputs

Webpack 5 (development)

asset myapp-fastopt.js 1.34 MiB [emitted] (name: myapp-fastopt)
asset index.html 1.72 KiB [emitted]
asset icon-32.svg 180 bytes [emitted] [from: assets/icon-32.svg]
./myapp-fastopt.js 1.31 MiB [built] [code generated]
webpack 5.12.0 compiled successfully in 544 ms

Webpack 5 (production)

asset myapp-opt-fe44374c39438c893626.js 258 KiB [emitted] [immutable] [minimized] (name: myapp-opt)
asset index.html 1.63 KiB [emitted]
asset icon-32-51fa7e7e665ad3b294fe37c44f732d62.svg 180 bytes [emitted] [immutable] [from: assets/icon-32.svg]
./myapp-opt.js 260 KiB [built] [code generated]
webpack 5.12.0 compiled successfully in 3547 ms

Webpack 4 (development)

Hash: 3d2baa6a7539e076a4f3
Version: webpack 4.44.2
Time: 482ms
Built at: 01/09/2021 9:50:03 PM
            Asset       Size          Chunks             Chunk Names
      icon-32.svg  180 bytes                  [emitted]
       index.html   1.72 KiB                  [emitted]
myapp-fastopt.js   1.34 MiB  myapp-fastopt  [emitted]  myapp-fastopt
Entrypoint myapp-fastopt = myapp-fastopt.js
[0] multi ./myapp-fastopt.js 28 bytes {myapp-fastopt} [built]
[./myapp-fastopt.js] 1.31 MiB {myapp-fastopt} [built]
Child HtmlWebpackCompiler:
          Asset       Size  Chunks             Chunk Names
    icon-32.svg  180 bytes          [emitted]
     + 1 hidden asset
    Entrypoint HtmlWebpackPlugin_0 = __child-HtmlWebpackPlugin_0
    [./assets/icon-32.svg] 55 bytes {HtmlWebpackPlugin_0} [built]
    [./node_modules/html-webpack-plugin/lib/loader.js!./assets/index.ejs] 2.04 KiB {HtmlWebpackPlugin_0} [built]

Webpack 4 (production)

Hash: d5a693b21ce4f40515c4
Version: webpack 4.44.2
Time: 391ms
Built at: 01/09/2021 9:50:50 AM
                                       Asset       Size  Chunks                         Chunk Names
icon-32-51fa7e7e665ad3b294fe37c44f732d62.svg  180 bytes          [emitted] [immutable]
                                  index.html   1.76 KiB          [emitted]
          myapp-opt-350577d29ed7f637efda.js    259 KiB       0  [emitted] [immutable]  myapp-opt
Entrypoint myapp-opt = myapp-opt-350577d29ed7f637efda.js
[0] multi ./myapp-opt.js 28 bytes {0} [built]
[1] ./myapp-opt.js 260 KiB {0} [built]
Child HtmlWebpackCompiler:
                                           Asset       Size  Chunks                         Chunk Names
    icon-32-51fa7e7e665ad3b294fe37c44f732d62.svg  180 bytes          [emitted] [immutable]
     + 1 hidden asset
    Entrypoint HtmlWebpackPlugin_0 = __child-HtmlWebpackPlugin_0
    [0] ./assets/icon-32.svg 88 bytes {0} [built]
    [1] ./node_modules/html-webpack-plugin/lib/loader.js!./assets/index.ejs 2.04 KiB {0} [built]

Raw warnings

Webpack 5

WARNING in ./app-fastopt.js
Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map from '/Users/lihaoyi/Github/scalatags/scalatags/src/scalatags/text/Builder.scala' file: Error: ENOENT: no such file or directory, open '/Users/lihaoyi/Github/scalatags/scalatags/src/scalatags/text/Builder.scala'

WARNING in ./app-fastopt.js
Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map from '/Users/lihaoyi/Github/sourcecode/sourcecode/src/sourcecode/SourceContext.scala' file: Error: ENOENT: no such file or directory, open '/Users/lihaoyi/Github/sourcecode/sourcecode/src/sourcecode/SourceContext.scala'

WARNING in ./app-fastopt.js
Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map: 'https://raw.githubusercontent.com/scala-js/scala-js-dom/v1.1.0/src/main/scala/org/scalajs/dom/package.scala' URL is not supported

WARNING in ./app-fastopt.js
Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map: 'https://raw.githubusercontent.com/scala-js/scala-js/v1.4.0/javalanglib/src/main/scala/java/lang/Boolean.scala' URL is not supported

Webpack 5

WARNING in ./app-fastopt.js
Module Warning (from ./node_modules/source-map-loader/index.js):
(Emitted value instead of an instance of Error) Cannot find source file '../../../../../../../../../lihaoyi/Github/scalatags/scalatags/src/scalatags/text/Builder.scala': Error: Can't resolve '../../../../../../../../../lihaoyi/Github/scalatags/scalatags/src/scalatags/text/Builder.scala' in '/Users/vhiairrassary/Code/test/test/app/target/scala-2.13/scalajs-bundler/main'
 @ multi ./app-fastopt.js app-fastopt[0]

WARNING in ./app-fastopt.js
Module Warning (from ./node_modules/source-map-loader/index.js):
(Emitted value instead of an instance of Error) Cannot find source file '../../../../../../../../../lihaoyi/Github/sourcecode/sourcecode/src/sourcecode/SourceContext.scala': Error: Can't resolve '../../../../../../../../../lihaoyi/Github/sourcecode/sourcecode/src/sourcecode/SourceContext.scala' in '/Users/vhiairrassary/Code/test/test/app/target/scala-2.13/scalajs-bundler/main'
 @ multi ./app-fastopt.js app-fastopt[0]

WARNING in ./app-fastopt.js
Module Warning (from ./node_modules/source-map-loader/index.js):
(Emitted value instead of an instance of Error) Cannot find source file 'https://raw.githubusercontent.com/scala-js/scala-js-dom/v1.1.0/src/main/scala/org/scalajs/dom/package.scala': Error: Can't resolve './https://raw.githubusercontent.com/scala-js/scala-js-dom/v1.1.0/src/main/scala/org/scalajs/dom/package.scala' in '/Users/vhiairrassary/Code/test/test/app/target/scala-2.13/scalajs-bundler/main'
 @ multi ./app-fastopt.js app-fastopt[0]

WARNING in ./app-fastopt.js
Module Warning (from ./node_modules/source-map-loader/index.js):
(Emitted value instead of an instance of Error) Cannot find source file 'https://raw.githubusercontent.com/scala-js/scala-js/v1.4.0/javalanglib/src/main/scala/java/lang/Boolean.scala': Error: Can't resolve './https://raw.githubusercontent.com/scala-js/scala-js/v1.4.0/javalanglib/src/main/scala/java/lang/Boolean.scala' in '/Users/vhiairrassary/Code/test/test/app/target/scala-2.13/scalajs-bundler/main'
 @ multi ./app-fastopt.js app-fastopt[0]

vhiairrassary · 2021-01-18T09:00:08Z

The PR now contains changes to parse warning and errors (I removed the --bail option as it prevents the plugin to parse the errors: when using it then there is no JSON available on stdout, the only error is directly written to stderr by Webpack).

vhiairrassary · 2021-01-18T17:01:51Z

sbt-scalajs-bundler/src/sbt-test/sbt-scalajs-bundler/webpack-assets/test

@@ -1,12 +1,12 @@
 # the server should properly start
 > fastOptJS::webpack
-# 3 assets and 2 for loader and entry
-> html index.html 5
+# 1 assets and 3 for library, loader and app


Stats differentiate between files and auxiliaryFiles now
Source)

i.e. map files are now included in returned assets anymore (breaking change ⚠️). Same for associated production test below

vhiairrassary · 2021-01-18T17:03:28Z

sbt-scalajs-bundler/src/sbt-test/sbt-scalajs-bundler/webpack-assets/prod.config.js

  entry: {
    app: [path.resolve(resourcesDir, "./entry.js")]
  },
  module: {
    rules: [
      {
        test: /\.css$/,
-        use: ["style-loader", MiniCssExtractPlugin.loader, "css-loader"]
+        use: [MiniCssExtractPlugin.loader, "css-loader"]


style-loader should not be included as html-webpack-plugin will automatically handle the CSS files

vhiairrassary · 2021-01-18T17:04:26Z

sbt-scalajs-bundler/src/sbt-test/sbt-scalajs-bundler/webpack-assets/build.sbt

@@ -51,10 +47,14 @@ InputKey[Unit]("html") := {
  assert(files.length == assetsCount)
  // Check all files are present
  assert(files.map(_.data.exists).forall(_ == true))
+  // There is only one app file
+  assert(files.count(_.metadata.get(BundlerFileTypeAttr) == Some(BundlerFileType.Application)) == 1)


Not sure to understand how this test worked before, as from my understanding LibraryOnly mode produces three files: library, loader & app.

vhiairrassary · 2021-01-18T17:05:33Z

sbt-scalajs-bundler/src/sbt-test/sbt-scalajs-bundler/sharedconfig/prod.webpack.config.js

-    })
-  ]
-});
+module.exports = merge(generatedConfig, commonConfig);


Uglify is not needed anymore (code is automatically minified in production mode, and this plugin is not compatible with Webpack 5)

Do you need terser?

From https://webpack.js.org/plugins/terser-webpack-plugin/:

if you are using webpack v5 or above you do not need to install this plugin. Webpack v5 comes with the latest terser-webpack-plugin out of the box

julienrf · 2021-02-10T12:44:56Z

Thanks a lot for opening this PR @vhiairrassary!

Is anyone watching this repository interested in helping?

cquiroz · 2021-02-10T19:11:24Z

I could review it as I did webpack 4 but I,m a bit stretched on time

cquiroz

This looks great, sorry it took so long to review
It maybe nice to release this as a RC to let other people test it (i could test it on my projects) before a public release
as it contains breaking changes

cquiroz · 2021-02-24T23:11:23Z

manual/src/ornate/cookbook.md

@@ -287,7 +287,7 @@ webpackDevServerExtraArgs := Seq("--inline")
 `webpack` is then called with the following arguments:

 ~~~
--bail --config <configfile>
+--config <configfile>


Why is that we don't want to bail on errors?

Never mind I read what is no the description

cquiroz · 2021-02-24T23:14:20Z

sbt-scalajs-bundler/src/main/scala/scalajsbundler/Stats.scala

@@ -91,17 +110,28 @@ object Stats {
  implicit val assetsReads: Reads[Asset] = (
    (JsPath \ "name").read[String] and
    (JsPath \ "size").read[Long] and
-    (JsPath \ "emitted").readNullable[Boolean] and
-    (JsPath \\ "chunkNames").read[List[String]]


I suppose this is a breaking change in the stats output? If so it would be hard to simultaneously support webpack 4 and 5

Yes it is. To reduce the complexity this PR only supports webpack 5 format (it was discussed as part of #350)

cquiroz · 2021-02-24T23:15:57Z

sbt-scalajs-bundler/src/main/scala/scalajsbundler/WebpackDevServer.scala

    * @param extraArgs - additional arguments for webpack-dev-server.
    * @param logger - a logger to use for output
    */
  def start(
    workDir: File,
    configPath: File,
-    port: Int,


Is it expected to configure the port via webpack config?

Yes as the invocation method the the dev-server is different with webpack 5 (I mentioned it as a possible breaking change in the issue description)

cquiroz · 2021-02-24T23:17:22Z

sbt-scalajs-bundler/src/main/scala/scalajsbundler/sbtplugin/ScalaJSBundlerPlugin.scala


-    version in startWebpackDevServer := "3.4.1",
+    version in startWebpackDevServer := "3.11.1",

    version in installJsdom := "9.9.0",


We may as well update jsdom

cquiroz · 2021-02-24T23:18:18Z

sbt-scalajs-bundler/src/sbt-test/sbt-scalajs-bundler/sharedconfig/prod.webpack.config.js

-    })
-  ]
-});
+module.exports = merge(generatedConfig, commonConfig);


Do you need terser?

cquiroz · 2021-02-24T23:19:37Z

sbt-scalajs-bundler/src/sbt-test/sbt-scalajs-bundler/webpack-assets/build.sbt

+  // There is only one library file
+  assert(files.count(_.metadata.get(BundlerFileTypeAttr) == Some(BundlerFileType.Loader)) == 1)
+  // And 1 asset (HTML file)
+  assert(files.count(_.metadata.get(BundlerFileTypeAttr) == Some(BundlerFileType.Asset)) == 1)


This is odd, I'd expect to see the css as an asset

It was near 2 months ago, but if I remember correctly the stats for these files has been moved to stats.relatedAssets (https://webpack.js.org/blog/2020-10-10-webpack-5-release/)

cquiroz · 2021-02-26T11:54:48Z

@vhiairrassary Do you think this needs more work? There are a few checkboxes on the description as "pending"

vhiairrassary · 2021-03-04T15:49:10Z

@cquiroz Thank for the review. I just rebased the PR and I think it is good enough for now. If maintainers agree to merge (it means we drop support for webpack < 5, on master at least) we can:

Deploy a RC to get feedbacks from users (with a major version bump due to breaking changes);
Create new PR to:
- Update documentation
- Decide how to handle related assets

Toughts? On my side I don't have the permission to publish, can you do it please?

cquiroz · 2021-03-04T15:53:03Z

I think we should do an RC for testers, it would be great to have a sort of migration guide but I don’t know how feasible is that considering how different webpack configurations are

I can produce releases but not update the documentation site

vhiairrassary · 2021-03-06T11:16:35Z

I rebased the PR to verify tests are still ✅ with GitHub Actions. Can I ask you to produce a release for this PR, without merging it, so interested users can test it? It would allow to get feedbacks, find bugs, help us to write a migration guide, etc.

Also, if we merge it it will require a bump of the major version due to the breaking changes. It might be a good idea to tackle the migration from fastOptJS to fastLinkJS.

cquiroz · 2021-03-08T01:31:42Z

I'm not sure I can make a release only for test, I could do 0.21.0-RC1 though. I'm not sure if that will trigger scala-steward updates

vhiairrassary · 2021-03-08T09:35:40Z

sbt-scalajs-bundler/src/sbt-test/sbt-scalajs-bundler/webpack-assets-cookbook/build.sbt

@@ -52,7 +52,6 @@ TaskKey[Unit]("checkArchive") := {
    "assets/webpack-assets-opt-bundle.js",
    "assets/webpack-assets-opt-loader.js",
    "assets/webpack-assets-opt-library.js",
-    "assets/webpack-assets-opt-library.js.map",


.map files are now in related assets, which are not extracted (yet)

cquiroz · 2021-03-08T11:16:58Z

It seems this is ready for testing

viktor-podzigun · 2022-10-14T09:34:56Z

Since it was released in v0.21.0 already, could we have ChangeLog page updated with the notes, please?

vhiairrassary mentioned this pull request Jan 9, 2021

Webpack 5 Support #350

Open

vhiairrassary commented Jan 18, 2021

View reviewed changes

vhiairrassary force-pushed the vhiairrassary/webpack5 branch from 1281e3a to 8465163 Compare January 18, 2021 17:38

cquiroz reviewed Feb 24, 2021

View reviewed changes

vhiairrassary force-pushed the vhiairrassary/webpack5 branch 2 times, most recently from 6ac53a7 to bfc2e15 Compare March 4, 2021 14:01

vhiairrassary marked this pull request as ready for review March 4, 2021 15:43

vhiairrassary force-pushed the vhiairrassary/webpack5 branch from bfc2e15 to a13b9be Compare March 6, 2021 10:32

vhiairrassary added 3 commits March 8, 2021 07:57

Add support for Webpack 5 and drop previous versions

a1c272b

Parse errors and warnings

d5a154f

Fix tests

5f8ee62

vhiairrassary force-pushed the vhiairrassary/webpack5 branch from a13b9be to 5f8ee62 Compare March 8, 2021 07:57

Fix new tests

4811d07

vhiairrassary commented Mar 8, 2021

View reviewed changes

cquiroz merged commit 5c36f19 into scalacenter:master Mar 8, 2021

vhiairrassary deleted the vhiairrassary/webpack5 branch August 31, 2022 14:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for Webpack 5 and drop previous versions #390

Add support for Webpack 5 and drop previous versions #390

vhiairrassary commented Jan 9, 2021 •

edited

Loading

vhiairrassary commented Jan 18, 2021

vhiairrassary Jan 18, 2021

vhiairrassary Jan 18, 2021

vhiairrassary Jan 18, 2021

vhiairrassary Jan 18, 2021

cquiroz Feb 24, 2021

vhiairrassary Feb 25, 2021

julienrf commented Feb 10, 2021

cquiroz commented Feb 10, 2021

cquiroz left a comment

cquiroz Feb 24, 2021

cquiroz Feb 24, 2021

cquiroz Feb 24, 2021

vhiairrassary Feb 25, 2021 •

edited

Loading

cquiroz Feb 24, 2021

vhiairrassary Feb 25, 2021

cquiroz Feb 24, 2021

cquiroz Feb 24, 2021

cquiroz Feb 24, 2021

vhiairrassary Feb 25, 2021

cquiroz commented Feb 26, 2021

vhiairrassary commented Mar 4, 2021

cquiroz commented Mar 4, 2021

vhiairrassary commented Mar 6, 2021

cquiroz commented Mar 8, 2021

vhiairrassary Mar 8, 2021

cquiroz commented Mar 8, 2021

viktor-podzigun commented Oct 14, 2022

Add support for Webpack 5 and drop previous versions #390

Add support for Webpack 5 and drop previous versions #390

Conversation

vhiairrassary commented Jan 9, 2021 • edited Loading

Links

Raw Webpack outputs

Raw warnings

vhiairrassary commented Jan 18, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

julienrf commented Feb 10, 2021

cquiroz commented Feb 10, 2021

cquiroz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vhiairrassary Feb 25, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cquiroz commented Feb 26, 2021

vhiairrassary commented Mar 4, 2021

cquiroz commented Mar 4, 2021

vhiairrassary commented Mar 6, 2021

cquiroz commented Mar 8, 2021

Choose a reason for hiding this comment

cquiroz commented Mar 8, 2021

viktor-podzigun commented Oct 14, 2022

vhiairrassary commented Jan 9, 2021 •

edited

Loading

vhiairrassary Feb 25, 2021 •

edited

Loading