Monorepo structure #2300

gchoqueux · 2024-03-21T14:50:56Z

Description

Refactoring iTowns to monorepo structure.

The goal is to split itowns down into feature packages.
As an example, I've started with the geographic functionalities. (@itowns/geographic)

For the moment Debug and Widget modules are private.

Motivation and Context

Split code in packages for clearly structured code.
Simplifies development and facilitates contributions.
Increases the scope of users who only want to use a few functions.
This structure makes it necessary to make functions independent

Code movement

The classes Coordinates, Crs, Ellipsoid, Extent, OrientationUtils files are moved to ./packages/geographic/src.
The proj4 dependency moves to geographic package.

The unit tests moves to packages/xxx/test.
the code utils/debug moves packages/Debug
the code src/Utils/Gui moves packages/Widget

dependencies

This proposal from monorepo doesn't use monorepo package (by example Lerna).
New dev dependency is Concurrently, it's used to watch script in parallel
Itowns dependencies (three, proj4...) are moved to the packages that use them

node scripts

The scripts uses the options --workspaces or -ws , to call the corresponding script in each sub packages.

gchoqueux · 2024-03-27T17:54:40Z

@Desplandis I resolve some issues

resolve module with exportsFields option
remove lerna with --workspace cli option

Desplandis · 2024-03-27T20:00:17Z

@gchoqueux Nice! I will take a quick look of your changes and write a todo list of things that should absolutely be tested before reviewing more thoroughly this PR. Expect it for tomorrow or next tuesday!

@jailln @mgermerie @ftoromanoff @AnthonyGlt I think we'll need your inputs on this change! ;)

This should be a good start to fix #2197, #1930, #2201 (list not exhaustive, feel free to complete). Shall we create a meta-issue to aggregate all those issues?

Desplandis · 2024-11-05T16:15:45Z

Hi, I'm currently reviewing your PR. I didn't had time to answer to the proposal. I'll submit both review and answer to the proposal tomorrow morning (TLDR; I agree with the package-splitting of this PR and have a few comments for future package-splitting).

gchoqueux · 2024-11-05T16:45:46Z

for the moment I haven't completely correlated the proposal with the PR.

For example:
in PR : @itowns/main -> @itowns/main in proposal
in PR : @itowns/debug -> @itowns/plugins/debug in proposal

Desplandis

Hi! This is first batch of my review of this PR. Unfortunately I'm kinda blocked on testing the distribution since the compiled code for the itowns package is incorrect (especially with imports to relative modules).

Moreover, there is a lot of points that I would like you to address. I'll resume the review once those points have been resolved.

Since this PR is massive, @jailln there may be some things that eluded me, do not hesitate to complement this review. =)

Monorepo_Roadmap.md

.gitignore

Gruntfile.cjs

package-lock.json

packages/Main/.eslintignore

Desplandis · 2024-11-07T12:32:41Z

packages/Main/package.json

+    "test-with-coverage_lcov": "c8 -n src --reporter=lcov cross-env npm run test-unit",
+    "watch": "cross-env BABEL_DISABLE_CACHE=1 babel --watch src --out-dir lib",
+    "postinstall": "cross-env NO_UPDATE_NOTIFIER=true node ./scripts/prepare.mjs && node ./scripts/replace.config.mjs",
+    "prepublishOnly": "npx copyfiles -u 1 \"../../examples/**/*\" ./examples/ && npx copyfiles -u 1 \"../../docs/**/*\" ./docs/ && npx copyfiles -u 1 \"../../dist/**/*\" ./dist/ ",


Or just not distribute examples and docs like most other packages.

to be decided

Desplandis · 2024-11-07T12:34:46Z

packages/Widget/.babelrc

+            "alias": {
+                "@itowns/geodesy": "../Geodesy/src/Main.js",
+                "itowns": "../Main/src/Main.js"
+            }


Once again this leads to incorrect compilation of path.

Desplandis · 2024-11-07T12:35:12Z

packages/Widget/.babelrc

+        ["babel-plugin-inline-import", {
+            "extensions": [
+                ".json",
+                ".geojson",
+                ".glsl",
+                ".css"
+            ]
+        }],


Not necessary since we do not imports from file with those extensions.

Desplandis · 2024-11-07T12:36:01Z

packages/Widget/.babelrc

+        ["minify-replace", {
+            "replacements": [{
+                "identifierName": "__DEBUG__",
+                "replacement": {
+                    "type": "booleanLiteral",
+                    "value": false
+                }
+            }]
+        }],
+        ["minify-dead-code-elimination"]


Same here, unused.

gchoqueux · 2024-11-13T17:30:49Z

Hi! This is first batch of my review of this PR. Unfortunately I'm kinda blocked on testing the distribution since the compiled code for the itowns package is incorrect (especially with imports to relative modules).

Moreover, there is a lot of points that I would like you to address. I'll resume the review once those points have been resolved.

Since this PR is massive, @jailln there may be some things that eluded me, do not hesitate to complement this review. =)

@Desplandis

I've finished answering/fixing all your comments.
I've also corrected the transpiling for the tests.

jailln

Not much to add to @Desplandis review.

I just think we should explain this new structure in the README and add sub package installation possibilities.

Also, how does it work with releasing and versioning? does subpackage all have the same version or can we (should we?) release them independently ?

jailln · 2024-11-26T16:27:21Z

packages/Geographic/README.md

@@ -0,0 +1,51 @@
+## iTowns Geographic


Just a thought: we may update the doc template to be structured per subpackage and export this page as the welcome page of the subpackage. WDYT @gchoqueux @Desplandis?

yes, I think it would be recommended for future implementations to structure documentation by sub-module.

jailln · 2024-11-26T16:29:22Z

packages/Geographic/README.md

+* [Extent](http://www.itowns-project.org/itowns/docs/#api/Geographic/Extent) : Extent is geographical bounding rectangle defined by 4 limits: west, east, south and north.
+* [Crs](http://www.itowns-project.org/itowns/docs/#api/Geographic/CRS) : This module provides basic methods to manipulate a CRS (as a string). Visiting [epsg.io](https://epsg.io/) to more coordinate system worldwide. Use **Proj4js** definition in epsg.io.
+* [OrientationUtils](http://www.itowns-project.org/itowns/docs/#api/Geographic/OrientationUtils) : it provides methods to compute the quaternion that models a rotation defined with various conventions, including between different CRS.
+* Ellipsoid : In geodesy, a reference ellipsoid is a mathematically defined surface that approximates the geoid.


I'm not sure we should add all classes here as they already each have their own doc? WDYT @Desplandis? If we keep this list, we should add all classes though

I think it's a good idea to introduce the module's features and point to the detailed documentation.

If we keep this list, we should add all classes though

Except for the Main sub-module, which has yet to be divided.

package.json

gchoqueux · 2024-11-28T12:38:40Z

I just think we should explain this new structure in the README and add sub package installation possibilities.

Yes, I'll document

Also, how does it work with releasing and versioning? does subpackage all have the same version or can we (should we?) release them independently ?

In a monorepo, it's more complicated to manage a different version for each module, but if we can find a simple and robust solution we can implement it later.

gchoqueux added enhancement 🚀 help wanted 📢 chore 🚚 refactor 🏭 architecture 🏬 labels Mar 21, 2024

gchoqueux force-pushed the subpackages branch 8 times, most recently from 9431214 to e23cfef Compare March 27, 2024 17:31

gchoqueux requested a review from Desplandis March 27, 2024 17:54

gchoqueux force-pushed the subpackages branch 5 times, most recently from e5ec2ea to b2308ff Compare April 3, 2024 12:02

gchoqueux marked this pull request as ready for review April 3, 2024 13:54

gchoqueux force-pushed the subpackages branch 6 times, most recently from ea0116d to 962c99d Compare April 9, 2024 14:56

Desplandis requested review from jailln and ftoromanoff April 15, 2024 14:16

Desplandis requested changes Nov 7, 2024

View reviewed changes

gchoqueux force-pushed the subpackages branch 3 times, most recently from d578888 to 1bd17ac Compare November 7, 2024 17:51

gchoqueux mentioned this pull request Nov 12, 2024

Obsolete prepare script #2462

Open

gchoqueux force-pushed the subpackages branch 12 times, most recently from f0ddd49 to 61e950f Compare November 13, 2024 17:22

gchoqueux force-pushed the subpackages branch from 61e950f to af45df1 Compare November 15, 2024 13:43

jailln reviewed Nov 26, 2024

View reviewed changes

gchoqueux force-pushed the subpackages branch 2 times, most recently from ac0a1ad to 451da03 Compare November 28, 2024 12:16

gchoqueux force-pushed the subpackages branch 2 times, most recently from 6f3ada2 to d4616ba Compare November 28, 2024 14:34

gchoqueux added 2 commits November 28, 2024 15:34

chore(architecture): monorepo structure

5066e79

fix(test): increase time out

b1d0ba1

gchoqueux force-pushed the subpackages branch from d4616ba to b1d0ba1 Compare November 28, 2024 14:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Monorepo structure #2300

Monorepo structure #2300

gchoqueux commented Mar 21, 2024 •

edited

Loading

gchoqueux commented Mar 27, 2024 •

edited

Loading

Desplandis commented Mar 27, 2024 •

edited

Loading

Desplandis commented Nov 5, 2024

gchoqueux commented Nov 5, 2024

Desplandis left a comment •

edited

Loading

Desplandis Nov 7, 2024

gchoqueux Nov 13, 2024

Desplandis Nov 7, 2024

gchoqueux Nov 28, 2024

Desplandis Nov 7, 2024

gchoqueux Nov 13, 2024

Desplandis Nov 7, 2024

gchoqueux Nov 13, 2024

gchoqueux commented Nov 13, 2024

jailln left a comment

jailln Nov 26, 2024

gchoqueux Nov 28, 2024

jailln Nov 26, 2024

gchoqueux Nov 28, 2024

gchoqueux commented Nov 28, 2024

Monorepo structure #2300

Are you sure you want to change the base?

Monorepo structure #2300

Conversation

gchoqueux commented Mar 21, 2024 • edited Loading

Description

Motivation and Context

Code movement

dependencies

node scripts

gchoqueux commented Mar 27, 2024 • edited Loading

Desplandis commented Mar 27, 2024 • edited Loading

Desplandis commented Nov 5, 2024

gchoqueux commented Nov 5, 2024

Desplandis left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gchoqueux commented Nov 13, 2024

jailln left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gchoqueux commented Nov 28, 2024

gchoqueux commented Mar 21, 2024 •

edited

Loading

gchoqueux commented Mar 27, 2024 •

edited

Loading

Desplandis commented Mar 27, 2024 •

edited

Loading

Desplandis left a comment •

edited

Loading