From 7f04b31a8d533c997b9bb685b76f2dfc6d08e264 Mon Sep 17 00:00:00 2001 From: Iacopo Ciao Date: Thu, 12 Dec 2024 14:52:53 +0100 Subject: [PATCH] docs(site): added documentations --- doc/pages/_meta.json | 1 + doc/pages/index.mdx | 157 ----------------------- doc/pages/standards.mdx | 28 ++++ doc/pages/standards/application-port.mdx | 10 ++ doc/pages/standards/configuration.mdx | 13 ++ doc/pages/standards/monorepo.mdx | 8 ++ doc/pages/standards/package-commands.mdx | 23 ++++ doc/pages/standards/package-manager.mdx | 27 ++++ doc/pages/standards/workflow-git.mdx | 95 ++++++++++++++ 9 files changed, 205 insertions(+), 157 deletions(-) create mode 100644 doc/pages/standards.mdx create mode 100644 doc/pages/standards/application-port.mdx create mode 100644 doc/pages/standards/configuration.mdx create mode 100644 doc/pages/standards/monorepo.mdx create mode 100644 doc/pages/standards/package-commands.mdx create mode 100644 doc/pages/standards/package-manager.mdx create mode 100644 doc/pages/standards/workflow-git.mdx diff --git a/doc/pages/_meta.json b/doc/pages/_meta.json index af063c9..90f05d9 100644 --- a/doc/pages/_meta.json +++ b/doc/pages/_meta.json @@ -1,6 +1,7 @@ { "index": "Introduction", "manifesto": "Manifesto", + "standards": "Standards", "installation": "Installation", "get-started": "Get Started", "commands": "Commands", diff --git a/doc/pages/index.mdx b/doc/pages/index.mdx index 04e8ddd..80c54a5 100644 --- a/doc/pages/index.mdx +++ b/doc/pages/index.mdx @@ -44,160 +44,3 @@ Application examples: frontend in Angular or NextJS backend with NestJS or Aws f - **Library**: Utilities that can be used by applications in the monorepo. the DTO library shared between frontend and backend plugin engine. - **Project**: The monorepo and all the applications and libraries it contains - **Scaffold**: The bare monorepo structure without projects and libraries. - - -## Big Picture - -![Big Picture Scheme](/pages/index/big-picture.png) - -The main idea is to create a cohesive and easy-to-use structure for developers, favoring centralization rather than scattering resources. -The standardized commands in the package -(`dev`, `build`, `start`, `test`, `e2e`, `lint`, `release`) are not just predefined scripts; they represent the only scripts necessary -for any application. We should add new scripts only in cases of extreme necessity. -If something requires more complex management, it needs to be analyzed and integrated into an npm (or pub) script. - -For example, for integrating [Cypress](https://www.npmjs.com/package/cypress), instead of creating new scripts in the package, we developed -[@devmy/cypress-runner](https://www.npmjs.com/package/@devmy/cypress-runner) so that it can be used by everyone without -adding complexity to the package commands, thus maintaining a high developer experience. -The same principle applies to the [dotenv Vault](https://www.dotenv.org/docs/security/env-vault) file in the project root -with prefix filtering: it is designed to be shared with all applications, centralizing configurations in a single point -without duplicating them in [Gitlab](https://www.gitlab.com) environment variables or other pipeline systems. - -Regarding configurations, instead of replicating information in [Gitlab CI/CD Variables](https://docs.gitlab.com/ee/ci/variables), we created -[@devmy/dotenv2shell](https://www.npmjs.com/package/@devmy/dotenv2shell) to load [Dotenv Vault](https://www.dotenv.org/docs/security/env-vault) environments into the execution -shell of GitLab pipelines. This approach enhances the developer experience and promotes centralization of configurations. - -### Package Management with PNPM and Corepack - -For dependency management, `devmy cli` uses PNPM and Corepack, providing several advantages over NVM: - -- **Automated Version Management**: Corepack automatically manages package manager versions, eliminating the need for manual configurations. -- **Optimized Performance**: PNPM offers faster installations and more efficient disk space usage, reducing package duplication. -- **Standardized Commands**: Simplifies project management with standardized commands. - -#### Goodbye NVM -With the adoption of Corepack and PNPM, there is no longer a need to use NVM to manage Node.js versions, further simplifying the setup process and improving workflow efficiency. - -### Monorepo NX - -Each Devmy project is configured as an NX monorepo, allowing centralized management of all related applications and leveraging NX's caching mechanisms for builds. - -#### Benefits of the NX Monorepo - -- **Centralized Management**: Manage all applications in a single repository. -- **Build Cache**: Uses NX's cache to optimize builds, reducing compilation times. - -### Standardized Package Commands - -The package manager is standardized with the following commands: - -- `dev`: Launches the monorepo (or single application) in development mode, activating hot-reload. NX builds the libraries that the single project depends on. -- `build`: Executes the production build of the entire monorepo (or single application). NX builds the libraries that the single project depends on. -- `start`: Starts the applications compiled with the production build (or single application). NX runs the build for the single project and all its dependencies. -- `lint`: Performs lint operations on the project or the monorepo. -- `lint:fix`: Performs lint fixes on the project. -- `test`: Runs tests on the project or the monorepo. -- `test:headless`: Runs tests in headless mode, useful for pipelines. -- `e2e`: Runs end-to-end tests. -- `e2e:headless`: Runs end-to-end tests in headless mode, useful for pipelines. - -#### Design Pattern Composite - -By adopting the same interface for the package in sub-projects through NX, we implement the Composite Design Pattern. This pattern allows us to treat individual objects and compositions of objects uniformly through a common interface. - -##### Benefits of the Composite Design Pattern - -- **Uniform Access**: Simplifies the development and maintenance process by reducing dependency on specific implementation details. -- **Modularity and Reusability**: Promotes modularity and code reusability by allowing individual objects and compositions to be treated the same way. -- **Agnostic Pipelines**: Allows pipelines to be written in a way that is completely agnostic of the project they are working on. - -### Standardized ports -Regarding the ports for launching various applications, we have standardized them to centralize and unify the setup. This approach eliminates the frustration of remembering the specific port for the frontend application built with a particular framework in a given project. The standardized ports are: - -| target | port | localhost | -|----------|------|----------------| -| Frontend | 4000 | localhost:4000 | -| Backend | 3000 | localhost:3000 | - -This standardization ensures a cohesive development environment and simplifies the process for developers. - -### Configuration - -We use [Dotenv Vault](https://www.dotenv.org/docs/security/env-vault) created at the root of the monorepo. -Each application within the monorepo has access only to the parameters relevant to it, utilizing prefix filtering -tools such as [Vite env-and-mode](https://vitejs.dev/guide/env-and-mode#env-files), [Webpack with @dotenv-run/webpack](https://www.npmjs.com/package/@dotenv-run/webpack), -[Jest with @dotenv-run/jest](https://www.npmjs.com/package/@dotenv-run/jest), [Angular with @ngx-env/builder](https://www.npmjs.com/package/@ngx-env/builder) and -[dotenv-run](https://dotenv.run). - -#### Benefits of Centralized Configuration - -- **Single Management Point**: Centralized management of all configurations for all applications. -- **Enhanced Security**: Reduces the risk of sharing sensitive information between different applications due to prefix filtering. - -### Workflow and Pipelines - -GitHub Flow is a simple and flexible Git workflow ideal for software projects that require frequent releases. -To standardize our pipelines and speedup our daily workflow, we have adopted GitHub Flow, managing the execution of -validation pipelines in both the `main` branch and the development branches. -In the `main` branch, we also handle version generation through [semantic release](https://github.com/semantic-release/semantic-release) and the deployment of individual -packages. Here’s how it works in detail: - -1. **Main Branch (`main`)**: The `main` branch represents the stable and releasable version of the project. -Everything in `main` should be fully tested and production-ready. This branch also handles automatic version generation -via [semantic release](https://github.com/semantic-release/semantic-release). - -2. **Creating a Feature Branch**: When starting work on a new feature, create a separate branch from `main`. -Give the branch a descriptive name that reflects the feature or change you’re working on and the ClickUP code -(e.g., `feat/DE-1234-new-feature`). - - ```bash - git checkout -b feat/DE-1234-new-feature -``` - -3. **Working on the Feature Branch**: Make your commits with [semver](https://semver.org) on this branch as you develop -the new feature. This allows you to work in isolation without affecting the `main` branch. - - ```bash - git add . - git commit -m "feat: add new feature" -``` - -4. **Pull Request (PR)**: When the feature is complete, open a Pull Request on GitHub or Gitlab to merge the feature -branch into the `main` branch. This process allows for code review, discussion of changes, and ensuring everything is in order -before integration. - -5. **Review and Approval**: Other developers review the Pull Request, provide feedback, and if everything is correct, -approve the PR. - -6. **Merging the Pull Request**: Once approved, the Pull Request is merged into the `main` branch. - -7. **Deployment**: After merging, the `main` branch contains the new feature and is ready for production. -The pipeline handles version generation through [semantic release](https://github.com/semantic-release/semantic-release) -and the deployment of packages. The deployment of individual packages is divided as follows: - - - **Deploy to Dev**: Automatic deployment on the `main` branch. - - **Deploy to Staging**: Manual deployment on the `main` branch. - - **Deploy to Production**: Manual deployment on the `main` branch. - -GitHub Flow encourages an iterative and collaborative development cycle, keeping the production code always in a releasable state and integrating an efficient and well-organized deployment system. - -#### Benefits of GitHub Flow - -- **Automatic Deployments**: Automates deployments to staging, improving development cycle efficiency. -- **Manual Control**: Allows manually triggered jobs for production deployment, ensuring greater control. - -#### Pipeline uses - -| job | stage | branch | -|-------------------|---------|----------------------------------| -| install | pre | all | -| build | build | all | -| lint | test | all | -| test | test | all | -| e2e | test | main, manually in other branches | -| release | release | main | -| deploy dev | deploy | main | -| deploy staging | deploy | manually in main | -| deploy production | deploy | manually in main | - - diff --git a/doc/pages/standards.mdx b/doc/pages/standards.mdx new file mode 100644 index 0000000..281f5db --- /dev/null +++ b/doc/pages/standards.mdx @@ -0,0 +1,28 @@ +# Standards + +In this section, all the standards and conventions adopted to standardize the applications developed in Devmy are summarized. +Implementing uniform standards is crucial for ensuring consistency, maintainability, and code quality across all projects. +These principles not only facilitate collaboration among team members but also ensure that applications are easily understandable and scalable over time. +Following these guidelines helps reduce the risk of errors and improve development efficiency, allowing teams to focus +on innovation and problem-solving rather than managing discrepancies in the code. + +## Big Picture + +![Big Picture Scheme](/pages/index/big-picture.png) + +The main idea is to create a cohesive and easy-to-use structure for developers, favoring centralization rather than scattering resources. +The standardized commands in the package +(`dev`, `build`, `start`, `test`, `e2e`, `lint`, `release`) are not just predefined scripts; they represent the only scripts necessary +for any application. We should add new scripts only in cases of extreme necessity. +If something requires more complex management, it needs to be analyzed and integrated into an npm (or pub) script. + +For example, for integrating [Cypress](https://www.npmjs.com/package/cypress), instead of creating new scripts in the package, we developed +[@devmy/cypress-runner](https://www.npmjs.com/package/@devmy/cypress-runner) so that it can be used by everyone without +adding complexity to the package commands, thus maintaining a high developer experience. +The same principle applies to the [dotenv Vault](https://www.dotenv.org/docs/security/env-vault) file in the project root +with prefix filtering: it is designed to be shared with all applications, centralizing configurations in a single point +without duplicating them in [Gitlab](https://www.gitlab.com) environment variables or other pipeline systems. + +Regarding configurations, instead of replicating information in [Gitlab CI/CD Variables](https://docs.gitlab.com/ee/ci/variables), we created +[@devmy/dotenv2shell](https://www.npmjs.com/package/@devmy/dotenv2shell) to load [Dotenv Vault](https://www.dotenv.org/docs/security/env-vault) environments into the execution +shell of GitLab pipelines. This approach enhances the developer experience and promotes centralization of configurations. diff --git a/doc/pages/standards/application-port.mdx b/doc/pages/standards/application-port.mdx new file mode 100644 index 0000000..21d907c --- /dev/null +++ b/doc/pages/standards/application-port.mdx @@ -0,0 +1,10 @@ +### Application Port + +Regarding the ports for launching various applications, we have standardized them to centralize and unify the setup. This approach eliminates the frustration of remembering the specific port for the frontend application built with a particular framework in a given project. The standardized ports are: + +| target | port | localhost | +|----------|------|----------------| +| Frontend | 4000 | localhost:4000 | +| Backend | 3000 | localhost:3000 | + +This standardization ensures a cohesive development environment and simplifies the process for developers. diff --git a/doc/pages/standards/configuration.mdx b/doc/pages/standards/configuration.mdx new file mode 100644 index 0000000..59ac389 --- /dev/null +++ b/doc/pages/standards/configuration.mdx @@ -0,0 +1,13 @@ +# Configuration + +We use [Dotenv Vault](https://www.dotenv.org/docs/security/env-vault) created at the root of the monorepo. +Each application within the monorepo has access only to the parameters relevant to it, utilizing prefix filtering +tools such as [Vite env-and-mode](https://vitejs.dev/guide/env-and-mode#env-files), [Webpack with @dotenv-run/webpack](https://www.npmjs.com/package/@dotenv-run/webpack), +[Jest with @dotenv-run/jest](https://www.npmjs.com/package/@dotenv-run/jest), [Angular with @ngx-env/builder](https://www.npmjs.com/package/@ngx-env/builder) and +[dotenv-run](https://dotenv.run). + +## Benefits of Centralized Configuration + +- **Single Management Point**: Centralized management of all configurations for all applications. +- **Enhanced Security**: Reduces the risk of sharing sensitive information between different applications due to prefix filtering. + diff --git a/doc/pages/standards/monorepo.mdx b/doc/pages/standards/monorepo.mdx new file mode 100644 index 0000000..f157ec6 --- /dev/null +++ b/doc/pages/standards/monorepo.mdx @@ -0,0 +1,8 @@ +# Monorepo - NX + +Each Devmy project is configured as an NX monorepo, allowing centralized management of all related applications and leveraging NX's caching mechanisms for builds. + +## Benefits of the NX Monorepo + +- **Centralized Management**: Manage all applications in a single repository. +- **Build Cache**: Uses NX cache to optimize builds, reducing compilation times. diff --git a/doc/pages/standards/package-commands.mdx b/doc/pages/standards/package-commands.mdx new file mode 100644 index 0000000..b14b4a4 --- /dev/null +++ b/doc/pages/standards/package-commands.mdx @@ -0,0 +1,23 @@ +# Standardized Package Commands + +The package manager is standardized with the following commands: + +- `dev`: Launches the monorepo (or single application) in development mode, activating hot-reload. NX builds the libraries that the single project depends on. +- `build`: Executes the production build of the entire monorepo (or single application). NX builds the libraries that the single project depends on. +- `start`: Starts the applications compiled with the production build (or single application). NX runs the build for the single project and all its dependencies. +- `lint`: Performs lint operations on the project or the monorepo. +- `lint:fix`: Performs lint fixes on the project. +- `test`: Runs tests on the project or the monorepo. +- `test:headless`: Runs tests in headless mode, useful for pipelines. +- `e2e`: Runs end-to-end tests. +- `e2e:headless`: Runs end-to-end tests in headless mode, useful for pipelines. + +## Design Pattern Composite + +By adopting the same interface for the package in sub-projects through NX, we implement the Composite Design Pattern. This pattern allows us to treat individual objects and compositions of objects uniformly through a common interface. + +## Benefits of the Composite Design Pattern + +- **Uniform Access**: Simplifies the development and maintenance process by reducing dependency on specific implementation details. +- **Modularity and Reusability**: Promotes modularity and code reusability by allowing individual objects and compositions to be treated the same way. +- **Agnostic Pipelines**: Allows pipelines to be written in a way that is completely agnostic of the project they are working on. diff --git a/doc/pages/standards/package-manager.mdx b/doc/pages/standards/package-manager.mdx new file mode 100644 index 0000000..eaf10d2 --- /dev/null +++ b/doc/pages/standards/package-manager.mdx @@ -0,0 +1,27 @@ +# Package Manager - Corepack + PNPM + +Choosing the right package manager is crucial for ensuring efficient dependency management in our project, facilitating the work of developers, and optimizing the overall performance of the application. After careful consideration, we have decided to adopt **Corepack** alongside **PNPM** for the following reasons: + +- **Automated Version Management**: +- **Corepack** simplifies the management of package manager versions, ensuring that developers always use the correct versions of PNPM (or other supported managers) without having to manually configure each environment. +- This automation helps avoid version conflicts and improves the reliability of builds. + +- **Optimized Performance**: +- **PNPM** is designed to be highly performant, reducing installation times for dependencies through the use of an advanced caching system. +- Compared to other package managers, PNPM employs a global store approach for dependencies, allowing packages to be shared across projects, minimizing the time and space required for installations. + +- **Efficient Disk Space Usage**: +- PNPM's strategy of linking dependencies rather than duplicating them significantly reduces disk space usage, making the installation process leaner and less burdensome on the system. + +- **Stringent Package Policy**: +- PNPM enforces a strict policy regarding which packages the application can access, enhancing security and ensuring that only declared dependencies are included in the project. +- This approach minimizes the risk of vulnerabilities and conflicts, improving the overall stability of the application. + +In summary, the combination of **Corepack** and **PNPM** allows us to optimize dependency management, improve performance, and ensure greater security in our development environment. These choices reflect our commitment to maintaining an effective and modern workflow in line with industry best practices. + +## Goodbye NVM +With the adoption of `Corepack` and `PNPM`, there is no longer a need to use `NVM` to manage Node versions, further simplifying the setup process and improving workflow efficiency. + +## Sources +- [Corepack](https://nodejs.org/api/corepack.html) +- [PNPM](https://pnpm.io/) diff --git a/doc/pages/standards/workflow-git.mdx b/doc/pages/standards/workflow-git.mdx new file mode 100644 index 0000000..6996e82 --- /dev/null +++ b/doc/pages/standards/workflow-git.mdx @@ -0,0 +1,95 @@ +# Workflow Git - GitHub Workflow + +Choosing the right Git workflow is crucial to enabling developers to work efficiently and securely, avoiding unnecessary time spent on cumbersome operations. The criteria we used for our decision are as follows: + +- **High cohesion with automatic versioning tools**, to reduce complexity in version management. +- **Streamlined workflow**, allowing developers to focus on development rather than administrative tasks. +- **Flexible release management**, for both planned and unplanned releases. +- **Effective maintenance of legacy versions**, still in use by clients or other teams. + +Our choice is **[GitHub Flow](https://docs.github.com/en/get-started/using-github/github-flow)**, following the **[Semantic Release Workflow configuration](https://semantic-release.gitbook.io/semantic-release/usage/workflow-configuration)**, similar to the workflow used by the [Angular Project](https://github.com/angular/angular): + +```mermaid +gitGraph + checkout main + commit + commit + commit tag: "v1.0.0" + branch 1.0.x order: 3 + commit id: "hotfix-1" tag: "v1.0.1" + commit id: "hotfix-2" tag: "v1.0.2" + checkout main + branch next + branch feat-1 + commit + commit + checkout next + merge feat-1 tag: "v1.1.0-next.1" + branch feat-2 + commit + commit + checkout next + merge feat-2 tag: "v1.1.0-next.2" + + checkout main + merge next tag: "v1.1.0" + commit + commit +``` + +**Branches** +- **`main`**: contains the latest version released to production. +- **`next`** (or `next-major`): represents the "in-development" version, which will be released as the next stable version. +- **`x.x.x`**: contains previous versions requiring fixes or additional feature integrations. + +When `next` is ready, it is merged into `main`, and the branch is deleted. For new developments, a new `next` branch is created from `main`. +When a release is performed on a branch (`main`, `next`, or maintenance `x.x.x`), Semantic Release analyzes the commits since the last release and determines the version number to increment. +After determining the version number, update the `package.json` file with the new version, generate a changelog with the relevant changes and create a Git tag with the new version. +Tags are automatically created based on the new version calculated by Semantic Release, following the semantic versioning convention (e.g. `v1.0.0`). + +## Feature and Fix Workflow + +Each feature or fix must be developed in a dedicated branch. Merging these branches into `main`, `next`, or `maintenance` branches is only allowed via a merge request. To ensure quality and consistency, the merge request will be accepted only if the following conditions are met: + +1. **Merge Request Template Compliance**: The merge request adheres to the predefined template, ensuring all necessary information is included. +2. **Self-Review**: The developer has reviewed their own code to catch obvious issues or omissions before submitting. +3. **Peer Review**: Another developer has reviewed and approved the changes, offering a second layer of verification. +4. **Pipeline Success**: All associated pipelines have been executed successfully. + +## Pipelines + +| Job | Trigger | Stage | +|--------------------|-------------------------------------------------------------|---------| +| **Install** | Always | Prepare | +| **Build** | Only in merge requests | Test | +| **Lint** | Only in merge requests | Test | +| **Test** | Only in merge requests | Test | +| **E2E** | Optional for `next`, required for `main` and `maintenance` | Test | +| **Release** | Only in `main`, `next` (or `next-major`), and `maintenance` | Release | +| **Deploy Staging** | Only in `next` | Deploy | +| **Deploy Prod** | Only in `main` and `maintenance` | Deploy | + +## FAQ + +### Why `next` and not `develop`? +`next` represents the future version of the software, is entirely managed by Semantic Release, and is linear. By containing only the changes for the new version and being deleted after development is complete, `next` avoids the back-merge issues that affect a `develop` branch. With `develop`, every time `main` receives a new version or a fix, constant updates would be required. + +### Can `develop` be used instead of `next`? +No, it is not advisable to use `develop` instead of `next`. The `next` branch is designed to clearly and linearly represent the upcoming version of the software, fully managed by Semantic Release and pipelines. Using develop would introduce complications, such as the need to continuously manage merges and changes coming from main, which can lead to conflicts and a more complex development cycle. next avoids these issues by keeping the focus solely on the changes needed for the new version and is deleted once development is complete, thereby simplifying the workflow. + +If a decision is made to introduce the `develop` branch anyway, it will be the team's responsibility to manage the numerous issues arising from this approach, which may prove to be outdated and impractical. + +### Can I make a hotfix directly in `main`? +No. When a fix is needed for a production version: +1. Create a `maintenance` branch for the version that needs maintenance. +2. Create a `fix` branch and apply the fix on this branch. +3. Merge your fix in `maintenance`. +4. If the bug affects subsequent versions, create corresponding `maintenance` branches for those versions and cherry-pick the fix. +5. If the bug is present in the latest version, cherry-pick it to `main`. +6. If the bug affects the development version, cherry-pick it to `next`. + +### Do I always need to create a `maintenance` branch? +No. If the project does not have legacy versions to maintain, it is not necessary to create `maintenance` branches. Fixes and improvements can be handled directly in `main` or `next`. + +### Why aren't tests also performed in the main branches? +The main branches (main, next, x.x.x) are always considered stable by convention and therefore do not require test pipelines. The only time a main branch might be at risk is during the merging of a new feature. Therefore, it is at this point that we need to pay special attention and take action.