From 2feb8ae9c258d59162092032b384d9dc79be7299 Mon Sep 17 00:00:00 2001 From: Michael Rapp Date: Mon, 18 Nov 2024 14:13:50 +0100 Subject: [PATCH 1/6] Move documentation of CI jobs into a separate chapter. --- doc/developer_guide/coding_standards.md | 67 ------------------- doc/developer_guide/continuous_integration.md | 66 ++++++++++++++++++ doc/index.md | 1 + 3 files changed, 67 insertions(+), 67 deletions(-) create mode 100644 doc/developer_guide/continuous_integration.md diff --git a/doc/developer_guide/coding_standards.md b/doc/developer_guide/coding_standards.md index 9dd1183bf..c040f936a 100644 --- a/doc/developer_guide/coding_standards.md +++ b/doc/developer_guide/coding_standards.md @@ -4,73 +4,6 @@ As it is common for Open Source projects, where everyone is invited to contribute, we rely on coding standards to ensure that new code works as expected, does not break existing functionality, and adheres to best practices we agreed on. These coding standards are described in the following. -(ci)= - -## Continuous Integration - -We make use of [GitHub Actions](https://docs.github.com/actions) as a [Continuous Integration](https://en.wikipedia.org/wiki/Continuous_integration) (CI) server for running predefined jobs, such as automated tests, in a controlled environment. Whenever certain parts of the project's repository have changed, relevant jobs are automatically executed. - -```{tip} -A track record of past runs can be found on GitHub in the [Actions](https://github.com/mrapp-ke/MLRL-Boomer/actions) tab. -``` - -The workflow definitions of individual CI jobs can be found in the directory `.github/workflows/`. Currently, the following jobs are used in the project: - -- `release.yml` defines a job for releasing a new version of the software developed by this project. The job can be triggered manually for one of the branches mentioned in the section {ref}`release-process`. It automatically updates the project's changelog and publishes a new release on GitHub. -- `publish.yml` is used for publishing pre-built packages on [PyPI](https://pypi.org/) (see {ref}`installation`). For this purpose, the project is built from source for each of the target platforms and architectures, using virtualization in some cases. The job is run automatically when a new release was published on [GitHub](https://github.com/mrapp-ke/MLRL-Boomer/releases). It does also increment the project's major version number and merge the release branch into its upstream branches (see {ref}`release-process`). -- `publish_development.yml` publishes development versions of packages on [Test-PyPI](https://test.pypi.org/) whenever changes to the project's source code have been pushed to the main branch. The packages built by each of these runs are also saved as [artifacts](https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts) and can be downloaded as zip archives. -- `test_publish.yml` ensures that the packages to be released for different architectures and Python versions can be built. The job is run for pull requests that modify relevant parts of the source code. -- `test_build.yml` builds the project for each of the supported target platforms, i.e., Linux, Windows, and macOS (see {ref}`compilation`). In the Linux environment, this job does also execute all available unit and integration tests (see {ref}`testing`). It is run for pull requests whenever relevant parts of the project's source code have been modified. -- `test_doc.yml` generates the latest documentation (see {ref}`documentation`) whenever relevant parts of the source code are affected by a pull request. -- `test_format.yml` ensures that all source files in the project adhere to our coding style guidelines (see {ref}`code-style`). This job is run automatically for pull requests whenever they include any changes affecting the relevant source files. -- `test_changelog.yml` ensures that all changelog files in the project adhere to the structure that is necessary to be processed automatically when publishing a new release. This job is run for pull requests if they modify one of the changelog files. -- `merge_feature.yml` and `merge_bugfix.yml` are used to merge changes that have been pushed to the feature or bugfix branch into downstream branches via pull requests (see {ref}`release-process`). -- `merge_release.yml` is used to merge all changes included in a new release published on [GitHub](https://github.com/mrapp-ke/MLRL-Boomer/releases) into upstream branches and update the version numbers of these branches. - -The project's build system allows to automatically check for outdated GitHub Actions used in the workflows mentioned above. These are reusable building blocks provided by third-party developers. The following command prints a list of all outdated Actions: - -````{tab} Linux - ```text - ./build check_github_actions - ``` -```` - -````{tab} macOS - ```text - ./build check_github_actions - ``` -```` - -````{tab} Windows - ``` - build.bat check_github_actions - ``` -```` - -Alternatively, the following command may be used to update the versions of outdated Actions automatically: - -````{tab} Linux - ```text - ./build update_github_actions - ``` -```` - -````{tab} macOS - ```text - ./build update_github_actions - ``` -```` - -````{tab} Windows - ``` - build.bat update_github_actions - ``` -```` - -```{note} -The above commands query the [GitHub API](https://docs.github.com/rest) for the latest version of relevant GitHub Actions. You can optionally specify an [API token](https://docs.github.com/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) to be used for these queries via the environment variable `GITHUB_TOKEN`. If no token is provided, repeated requests might fail due to GitHub's rate limit. -``` - (testing)= ## Testing the Code diff --git a/doc/developer_guide/continuous_integration.md b/doc/developer_guide/continuous_integration.md new file mode 100644 index 000000000..47f80aa4a --- /dev/null +++ b/doc/developer_guide/continuous_integration.md @@ -0,0 +1,66 @@ +(ci)= + +# Continuous Integration + +We make use of [GitHub Actions](https://docs.github.com/actions) as a [Continuous Integration](https://en.wikipedia.org/wiki/Continuous_integration) (CI) server for running predefined jobs, such as automated tests, in a controlled environment. Whenever certain parts of the project's repository have changed, relevant jobs are automatically executed. + +```{tip} +A track record of past runs can be found on GitHub in the [Actions](https://github.com/mrapp-ke/MLRL-Boomer/actions) tab. +``` + +The workflow definitions of individual CI jobs can be found in the directory `.github/workflows/`. Currently, the following jobs are used in the project: + +- `release.yml` defines a job for releasing a new version of the software developed by this project. The job can be triggered manually for one of the branches mentioned in the section {ref}`release-process`. It automatically updates the project's changelog and publishes a new release on GitHub. +- `publish.yml` is used for publishing pre-built packages on [PyPI](https://pypi.org/) (see {ref}`installation`). For this purpose, the project is built from source for each of the target platforms and architectures, using virtualization in some cases. The job is run automatically when a new release was published on [GitHub](https://github.com/mrapp-ke/MLRL-Boomer/releases). It does also increment the project's major version number and merge the release branch into its upstream branches (see {ref}`release-process`). +- `publish_development.yml` publishes development versions of packages on [Test-PyPI](https://test.pypi.org/) whenever changes to the project's source code have been pushed to the main branch. The packages built by each of these runs are also saved as [artifacts](https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts) and can be downloaded as zip archives. +- `test_publish.yml` ensures that the packages to be released for different architectures and Python versions can be built. The job is run for pull requests that modify relevant parts of the source code. +- `test_build.yml` builds the project for each of the supported target platforms, i.e., Linux, Windows, and macOS (see {ref}`compilation`). In the Linux environment, this job does also execute all available unit and integration tests (see {ref}`testing`). It is run for pull requests whenever relevant parts of the project's source code have been modified. +- `test_doc.yml` generates the latest documentation (see {ref}`documentation`) whenever relevant parts of the source code are affected by a pull request. +- `test_format.yml` ensures that all source files in the project adhere to our coding style guidelines (see {ref}`code-style`). This job is run automatically for pull requests whenever they include any changes affecting the relevant source files. +- `test_changelog.yml` ensures that all changelog files in the project adhere to the structure that is necessary to be processed automatically when publishing a new release. This job is run for pull requests if they modify one of the changelog files. +- `merge_feature.yml` and `merge_bugfix.yml` are used to merge changes that have been pushed to the feature or bugfix branch into downstream branches via pull requests (see {ref}`release-process`). +- `merge_release.yml` is used to merge all changes included in a new release published on [GitHub](https://github.com/mrapp-ke/MLRL-Boomer/releases) into upstream branches and update the version numbers of these branches. + +The project's build system allows to automatically check for outdated GitHub Actions used in the workflows mentioned above. These are reusable building blocks provided by third-party developers. The following command prints a list of all outdated Actions: + +````{tab} Linux + ```text + ./build check_github_actions + ``` +```` + +````{tab} macOS + ```text + ./build check_github_actions + ``` +```` + +````{tab} Windows + ``` + build.bat check_github_actions + ``` +```` + +Alternatively, the following command may be used to update the versions of outdated Actions automatically: + +````{tab} Linux + ```text + ./build update_github_actions + ``` +```` + +````{tab} macOS + ```text + ./build update_github_actions + ``` +```` + +````{tab} Windows + ``` + build.bat update_github_actions + ``` +```` + +```{note} +The above commands query the [GitHub API](https://docs.github.com/rest) for the latest version of relevant GitHub Actions. You can optionally specify an [API token](https://docs.github.com/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) to be used for these queries via the environment variable `GITHUB_TOKEN`. If no token is provided, repeated requests might fail due to GitHub's rate limit. +``` diff --git a/doc/index.md b/doc/index.md index 82793cdb2..ad780dddd 100644 --- a/doc/index.md +++ b/doc/index.md @@ -55,6 +55,7 @@ maxdepth: 2 developer_guide/project_structure developer_guide/compilation developer_guide/documentation +developer_guide/continuous_integration developer_guide/coding_standards developer_guide/api/python/index developer_guide/api/cpp/index From 9c9b2fe16a8b3ccd3f22ecb160938d197d4486b5 Mon Sep 17 00:00:00 2001 From: Michael Rapp Date: Mon, 18 Nov 2024 14:47:00 +0100 Subject: [PATCH 2/6] Add subsections to section "Release Process". --- doc/developer_guide/coding_standards.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/doc/developer_guide/coding_standards.md b/doc/developer_guide/coding_standards.md index c040f936a..4150879f5 100644 --- a/doc/developer_guide/coding_standards.md +++ b/doc/developer_guide/coding_standards.md @@ -156,10 +156,16 @@ To enable releasing new major, feature, or bugfix releases at any time, we maint We do not allow directly pushing to the above branches. Instead, all changes must be submitted via pull requests and require certain checks to pass. +### Downstream Merges + Once modifications to one of the branches have been merged, {ref}`Continuous Integration ` jobs are used to automatically update downstream branches via pull requests. If all checks for such pull requests are successful, they are merged automatically. If there are any merge conflicts, they must be resolved manually. Following this procedure, changes to the feature branch are merged into the main branch (see `merge_feature.yml`), whereas changes to the bugfix branch are first merged into the feature branch and then into the main branch (see `merge_bugfix.yml`). +### Triggering Releases + We use a {ref}`Continuous Integration ` job for triggering a new release, including the changes of one of the branches mentioned above (see `release.yml`). Depending on the release branch, the job automatically collects the corresponding changelog entries from the files `.changelog-main.md`, `.changelog-feature.md`, and `.changelog-bugfix.md` and updates the file `CHANGELOG.md` in the project's root directory accordingly. Afterward, it will publish the new release on GitHub, which will in turn trigger the publishing of pre-built packages (see `publish.yml`). +### Upstream Merges + Whenever a new release has been published, the release branch is merged into the upstream branches (see `merge_release.yml`), i.e., major releases result in the feature and bugfix branches being updated, whereas minor releases result in the bugfix branch being updated. The version of the release branch and the affected branches are updated accordingly. The file `.version` in the project's root directory specifies the version of each of these branches. Similarly, the file `.version-dev` keeps track of the version number used for development releases (see `publish_development.yml`). (dependencies)= From 0d66abdac8d72fe2a77363aa05c6df168e940c1c Mon Sep 17 00:00:00 2001 From: Michael Rapp Date: Mon, 18 Nov 2024 14:47:48 +0100 Subject: [PATCH 3/6] Add subsections to section "Dependencies". --- doc/developer_guide/coding_standards.md | 54 ++++++++++++++++++- doc/developer_guide/continuous_integration.md | 44 --------------- 2 files changed, 52 insertions(+), 46 deletions(-) diff --git a/doc/developer_guide/coding_standards.md b/doc/developer_guide/coding_standards.md index 4150879f5..1482369fb 100644 --- a/doc/developer_guide/coding_standards.md +++ b/doc/developer_guide/coding_standards.md @@ -174,9 +174,11 @@ Whenever a new release has been published, the release branch is merged into the Adding dependencies to a software project always comes at a cost. Maintainers need to continuously test their software as new versions of dependencies are released and major changes in their APIs may break existing functionality. For this reason, we try to keep the number of dependencies at a minimum. -That being said, we still rely on several dependencies for compiling our source code, generating the documentation, or running the algorithms provided by this project. When using pre-built packages from [PyPI](https://pypi.org/project/mlrl-boomer/), there is no need to care about these dependencies, as they are already included in the packages. When {ref}`building from source `, dependencies are automatically installed by the build system once they are needed, unless explicitly stated in the documentation. +That being said, we still rely on several dependencies for continuous integration, compiling our source code, generating the documentation, or running the algorithms provided by this project. When using pre-built packages from [PyPI](https://pypi.org/project/mlrl-boomer/), there is no need to care about these dependencies, as they are already included in the packages. When {ref}`building from source `, dependencies are automatically installed by the build system once they are needed, unless explicitly stated in the documentation. -The dependencies that are required by different aspects of the project, such as the build system, the Python code, or the C++ code, are defined in separate `requirements.txt` files. For dependencies that use [Semantic Versioning](https://semver.org/), we specify the earliest and latest version we support. For other dependencies, we demand for a specific version number. This strives to achieve a balance between flexibility for users and comfort for developers. On the one hand, supporting a range of versions provides more freedom to users, as our packages can more flexibly be used together with other ones, relying on the same dependencies. On the other hand, the project's maintainers must not manually update dependencies that have a minor release, while still requiring manual intervention for major updates. +### Python Dependencies + +Python dependencies that are required by different aspects of the project, such as the build system, the documentation, or our own Python code, are defined in separate `requirements.txt` files. For dependencies that use [Semantic Versioning](https://semver.org/), we specify the earliest and latest version we support. For other dependencies, we demand for a specific version number. This strives to achieve a balance between flexibility for users and comfort for developers. On the one hand, supporting a range of versions provides more freedom to users, as our packages can more flexibly be used together with other ones, relying on the same dependencies. On the other hand, the project's maintainers must not manually update dependencies that have a minor release, while still requiring manual intervention for major updates. To ease the life of developers, the following command provided by the project's build system may be used to check for outdated dependencies: @@ -197,3 +199,51 @@ To ease the life of developers, the following command provided by the project's build.bat check_dependencies ``` ```` + +### GitHub Actions + +Our {ref}`Continuous Integration ` (CI) jobs heavily rely on so-called [Actions](https://github.com/marketplace?type=actions), which are reusable building blocks provided by third-party developers. As with all dependencies, updates to these Actions may introduce breaking changes. To reduce the risk of updates breaking our CI jobs, we pin the Actions to a certain version. Usually, we only restrict the major version required by a job, rather than specifying a specific version. This allows minor updates, which are less likely to cause problems, to take effect without manual intervention. + +The project's build system allows to automatically check for outdated Actions used by the project's CI jobs. The following command prints a list of all outdated Actions: + +````{tab} Linux + ```text + ./build check_github_actions + ``` +```` + +````{tab} macOS + ```text + ./build check_github_actions + ``` +```` + +````{tab} Windows + ``` + build.bat check_github_actions + ``` +```` + +Alternatively, the following command may be used to update the versions of outdated Actions automatically: + +````{tab} Linux + ```text + ./build update_github_actions + ``` +```` + +````{tab} macOS + ```text + ./build update_github_actions + ``` +```` + +````{tab} Windows + ``` + build.bat update_github_actions + ``` +```` + +```{note} +The above commands query the [GitHub API](https://docs.github.com/rest) for the latest version of relevant GitHub Actions. You can optionally specify an [API token](https://docs.github.com/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) to be used for these queries via the environment variable `GITHUB_TOKEN`. If no token is provided, repeated requests might fail due to GitHub's rate limit. +``` diff --git a/doc/developer_guide/continuous_integration.md b/doc/developer_guide/continuous_integration.md index 47f80aa4a..87fcb24f3 100644 --- a/doc/developer_guide/continuous_integration.md +++ b/doc/developer_guide/continuous_integration.md @@ -20,47 +20,3 @@ The workflow definitions of individual CI jobs can be found in the directory `.g - `test_changelog.yml` ensures that all changelog files in the project adhere to the structure that is necessary to be processed automatically when publishing a new release. This job is run for pull requests if they modify one of the changelog files. - `merge_feature.yml` and `merge_bugfix.yml` are used to merge changes that have been pushed to the feature or bugfix branch into downstream branches via pull requests (see {ref}`release-process`). - `merge_release.yml` is used to merge all changes included in a new release published on [GitHub](https://github.com/mrapp-ke/MLRL-Boomer/releases) into upstream branches and update the version numbers of these branches. - -The project's build system allows to automatically check for outdated GitHub Actions used in the workflows mentioned above. These are reusable building blocks provided by third-party developers. The following command prints a list of all outdated Actions: - -````{tab} Linux - ```text - ./build check_github_actions - ``` -```` - -````{tab} macOS - ```text - ./build check_github_actions - ``` -```` - -````{tab} Windows - ``` - build.bat check_github_actions - ``` -```` - -Alternatively, the following command may be used to update the versions of outdated Actions automatically: - -````{tab} Linux - ```text - ./build update_github_actions - ``` -```` - -````{tab} macOS - ```text - ./build update_github_actions - ``` -```` - -````{tab} Windows - ``` - build.bat update_github_actions - ``` -```` - -```{note} -The above commands query the [GitHub API](https://docs.github.com/rest) for the latest version of relevant GitHub Actions. You can optionally specify an [API token](https://docs.github.com/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) to be used for these queries via the environment variable `GITHUB_TOKEN`. If no token is provided, repeated requests might fail due to GitHub's rate limit. -``` From f1228c1da1619acab64bb10766995de4db1d6534 Mon Sep 17 00:00:00 2001 From: Michael Rapp Date: Mon, 18 Nov 2024 14:52:22 +0100 Subject: [PATCH 4/6] Use spelling "Continuous Integration" instead of "continuous integration". --- .changelog-bugfix.md | 2 +- CHANGELOG.md | 18 +++++++++--------- doc/developer_guide/coding_standards.md | 2 +- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/.changelog-bugfix.md b/.changelog-bugfix.md index a3db40c09..48f5d4c02 100644 --- a/.changelog-bugfix.md +++ b/.changelog-bugfix.md @@ -1,5 +1,5 @@ # Quality-of-Life Improvements -- Releases are now automated via continuous integration, including the update of the project's changelog. +- Releases are now automated via Continuous Integration, including the update of the project's changelog. - The presentation of algorithmic parameters in the documentation has been improved. - Outdated GitHub Actions can now be printed via the build target `check_github_actions`. Alternatively, the build target `update_github_actions` may be used to update them automatically. diff --git a/CHANGELOG.md b/CHANGELOG.md index 33cc4e0c8..3556d51a2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -46,8 +46,8 @@ A bugfix release that fixes the following issues. A bugfix release that comes with the following changes. - If the sparse value of a feature matrix is provided to the Python API, it is now properly taken into account when converting into a dense matrix. -- The C++ code is now checked for common issues by applying `cpplint` via continuous integration. -- The styling of YAML files is now verified by applying `yamlfix` via continuous integration. +- The C++ code is now checked for common issues by applying `cpplint` via Continuous Integration. +- The styling of YAML files is now verified by applying `yamlfix` via Continuous Integration. ## Version 0.10.0 (May 5th, 2024) @@ -87,14 +87,14 @@ This release comes with several API changes. For an updated overview of the avai - A build option that allows disabling multi-threading support via OpenMP at compile-time has been added. - The groundwork for GPU support was laid. It can be disabled at compile-time via a build option. - Added support for unit testing the project's C++ code. Compilation of the tests can be disabled via a build option. -- The Python code is now checked for common issues by applying `pylint` via continuous integration. +- The Python code is now checked for common issues by applying `pylint` via Continuous Integration. - The Makefile has been replaced with wrapper scripts triggering a [SCons](https://scons.org/) build. -- Development versions of wheel packages are now regularly built via continuous integration, uploaded as artifacts, and published on [Test-PyPI](https://test.pypi.org/). +- Development versions of wheel packages are now regularly built via Continuous Integration, uploaded as artifacts, and published on [Test-PyPI](https://test.pypi.org/). - Continuous integration is now used to maintain separate branches for major, feature, and bugfix releases and keep them up-to-date. -- The runtime of continuous integration jobs has been optimized by running individual steps only if necessary, caching files across subsequent runs, and making use of parallelization. -- When tests are run via continuous integration, a summary of the test results is now added to merge requests and GitHub workflows. +- The runtime of Continuous Integration jobs has been optimized by running individual steps only if necessary, caching files across subsequent runs, and making use of parallelization. +- When tests are run via Continuous Integration, a summary of the test results is now added to merge requests and GitHub workflows. - Markdown files are now used for writing the documentation. -- A consistent style is now enforced for Markdown files by applying the tool `mdformat` via continuous integration. +- A consistent style is now enforced for Markdown files by applying the tool `mdformat` via Continuous Integration. - C++ 17 or newer is now required for compiling the project. ## Version 0.9.0 (Jul. 2nd, 2023) @@ -150,8 +150,8 @@ This release comes with several API changes. For an updated overview of the avai ### Quality-of-Life Improvements - Continuous integration is now used to test the most common functionalities of the BOOMER algorithm and the corresponding command line API. -- Successful generation of the documentation is now tested via continuous integration. -- Style definitions for Python and C++ code are now enforced by applying the tools `clang-format`, `yapf`, and `isort` via continuous integration. +- Successful generation of the documentation is now tested via Continuous Integration. +- Style definitions for Python and C++ code are now enforced by applying the tools `clang-format`, `yapf`, and `isort` via Continuous Integration. ## Version 0.8.2 (Apr. 11th, 2022) diff --git a/doc/developer_guide/coding_standards.md b/doc/developer_guide/coding_standards.md index 1482369fb..18d0a7e22 100644 --- a/doc/developer_guide/coding_standards.md +++ b/doc/developer_guide/coding_standards.md @@ -174,7 +174,7 @@ Whenever a new release has been published, the release branch is merged into the Adding dependencies to a software project always comes at a cost. Maintainers need to continuously test their software as new versions of dependencies are released and major changes in their APIs may break existing functionality. For this reason, we try to keep the number of dependencies at a minimum. -That being said, we still rely on several dependencies for continuous integration, compiling our source code, generating the documentation, or running the algorithms provided by this project. When using pre-built packages from [PyPI](https://pypi.org/project/mlrl-boomer/), there is no need to care about these dependencies, as they are already included in the packages. When {ref}`building from source `, dependencies are automatically installed by the build system once they are needed, unless explicitly stated in the documentation. +That being said, we still rely on several dependencies for Continuous Integration, compiling our source code, generating the documentation, or running the algorithms provided by this project. When using pre-built packages from [PyPI](https://pypi.org/project/mlrl-boomer/), there is no need to care about these dependencies, as they are already included in the packages. When {ref}`building from source `, dependencies are automatically installed by the build system once they are needed, unless explicitly stated in the documentation. ### Python Dependencies From a214e208f0a89b4cdba3b403b9bc5432d65313f7 Mon Sep 17 00:00:00 2001 From: Michael Rapp Date: Mon, 18 Nov 2024 15:01:31 +0100 Subject: [PATCH 5/6] Add sections to chapter "Continuous Integration". --- doc/developer_guide/coding_standards.md | 12 ++++++-- doc/developer_guide/continuous_integration.md | 30 ++++++++++++++----- 2 files changed, 31 insertions(+), 11 deletions(-) diff --git a/doc/developer_guide/coding_standards.md b/doc/developer_guide/coding_standards.md index 18d0a7e22..84203ebd0 100644 --- a/doc/developer_guide/coding_standards.md +++ b/doc/developer_guide/coding_standards.md @@ -156,17 +156,23 @@ To enable releasing new major, feature, or bugfix releases at any time, we maint We do not allow directly pushing to the above branches. Instead, all changes must be submitted via pull requests and require certain checks to pass. +(downstream-merges)= + ### Downstream Merges -Once modifications to one of the branches have been merged, {ref}`Continuous Integration ` jobs are used to automatically update downstream branches via pull requests. If all checks for such pull requests are successful, they are merged automatically. If there are any merge conflicts, they must be resolved manually. Following this procedure, changes to the feature branch are merged into the main branch (see `merge_feature.yml`), whereas changes to the bugfix branch are first merged into the feature branch and then into the main branch (see `merge_bugfix.yml`). +Once modifications to one of the branches have been merged, {ref}`Continuous Integration ` jobs are used to automatically update downstream branches via pull requests. If all checks for such pull requests are successful, they are merged automatically. If there are any merge conflicts, they must be resolved manually. Following this procedure, changes to the feature branch are merged into the main branch, whereas changes to the bugfix branch are first merged into the feature branch and then into the main branch (see description of `merge_feature.yml` and `merge_bugfix.yml` in {ref}`ci-releases`). + +(triggering-releases)= ### Triggering Releases -We use a {ref}`Continuous Integration ` job for triggering a new release, including the changes of one of the branches mentioned above (see `release.yml`). Depending on the release branch, the job automatically collects the corresponding changelog entries from the files `.changelog-main.md`, `.changelog-feature.md`, and `.changelog-bugfix.md` and updates the file `CHANGELOG.md` in the project's root directory accordingly. Afterward, it will publish the new release on GitHub, which will in turn trigger the publishing of pre-built packages (see `publish.yml`). +We use a {ref}`Continuous Integration ` job for triggering a new release, including the changes of one of the branches mentioned above (see description of `release.yml` in {ref}`ci-releases`). Depending on the release branch, the job automatically collects the corresponding changelog entries from the files `.changelog-main.md`, `.changelog-feature.md`, and `.changelog-bugfix.md` and updates the file `CHANGELOG.md` in the project's root directory accordingly. Afterward, it will publish the new release on GitHub, which will in turn trigger the publishing of pre-built packages (see description of `publish.yml` in {ref}`ci-publishing`). + +(upstream-merges)= ### Upstream Merges -Whenever a new release has been published, the release branch is merged into the upstream branches (see `merge_release.yml`), i.e., major releases result in the feature and bugfix branches being updated, whereas minor releases result in the bugfix branch being updated. The version of the release branch and the affected branches are updated accordingly. The file `.version` in the project's root directory specifies the version of each of these branches. Similarly, the file `.version-dev` keeps track of the version number used for development releases (see `publish_development.yml`). +Whenever a new release has been published, the release branch is merged into the upstream branches (see description of `merge_release.yml` in {ref}`ci-releases`), i.e., major releases result in the feature and bugfix branches being updated, whereas minor releases result in the bugfix branch being updated. The version of the release branch and the affected branches are updated accordingly. The file `.version` in the project's root directory specifies the version of each of these branches. Similarly, the file `.version-dev` keeps track of the version number used for development releases (see description of `publish_development.yml` in {ref}`ci-publishing`). (dependencies)= diff --git a/doc/developer_guide/continuous_integration.md b/doc/developer_guide/continuous_integration.md index 87fcb24f3..673de5c70 100644 --- a/doc/developer_guide/continuous_integration.md +++ b/doc/developer_guide/continuous_integration.md @@ -8,15 +8,29 @@ We make use of [GitHub Actions](https://docs.github.com/actions) as a [Continuou A track record of past runs can be found on GitHub in the [Actions](https://github.com/mrapp-ke/MLRL-Boomer/actions) tab. ``` -The workflow definitions of individual CI jobs can be found in the directory `.github/workflows/`. Currently, the following jobs are used in the project: +The workflow definitions of individual CI jobs can be found in YAML files located in the directory `.github/workflows/`. In the following, we discuss the different tasks dealt with by these workflows. + +(ci-testing)= + +## Quality Control -- `release.yml` defines a job for releasing a new version of the software developed by this project. The job can be triggered manually for one of the branches mentioned in the section {ref}`release-process`. It automatically updates the project's changelog and publishes a new release on GitHub. -- `publish.yml` is used for publishing pre-built packages on [PyPI](https://pypi.org/) (see {ref}`installation`). For this purpose, the project is built from source for each of the target platforms and architectures, using virtualization in some cases. The job is run automatically when a new release was published on [GitHub](https://github.com/mrapp-ke/MLRL-Boomer/releases). It does also increment the project's major version number and merge the release branch into its upstream branches (see {ref}`release-process`). -- `publish_development.yml` publishes development versions of packages on [Test-PyPI](https://test.pypi.org/) whenever changes to the project's source code have been pushed to the main branch. The packages built by each of these runs are also saved as [artifacts](https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts) and can be downloaded as zip archives. -- `test_publish.yml` ensures that the packages to be released for different architectures and Python versions can be built. The job is run for pull requests that modify relevant parts of the source code. - `test_build.yml` builds the project for each of the supported target platforms, i.e., Linux, Windows, and macOS (see {ref}`compilation`). In the Linux environment, this job does also execute all available unit and integration tests (see {ref}`testing`). It is run for pull requests whenever relevant parts of the project's source code have been modified. +- `test_changelog.yml` ensures that all changelog files in the project adhere to the structure that is necessary to be processed automatically when publishing a new release. This job is run for pull requests if they modify one of the changelog files. - `test_doc.yml` generates the latest documentation (see {ref}`documentation`) whenever relevant parts of the source code are affected by a pull request. - `test_format.yml` ensures that all source files in the project adhere to our coding style guidelines (see {ref}`code-style`). This job is run automatically for pull requests whenever they include any changes affecting the relevant source files. -- `test_changelog.yml` ensures that all changelog files in the project adhere to the structure that is necessary to be processed automatically when publishing a new release. This job is run for pull requests if they modify one of the changelog files. -- `merge_feature.yml` and `merge_bugfix.yml` are used to merge changes that have been pushed to the feature or bugfix branch into downstream branches via pull requests (see {ref}`release-process`). -- `merge_release.yml` is used to merge all changes included in a new release published on [GitHub](https://github.com/mrapp-ke/MLRL-Boomer/releases) into upstream branches and update the version numbers of these branches. +- `test_publish.yml` ensures that the packages to be released for different architectures and Python versions can be built. The job is run for pull requests that modify relevant parts of the source code. + +(ci-releases)= + +## Automated Releases + +- `merge_feature.yml` and `merge_bugfix.yml` are used to merge changes that have been pushed to the feature or bugfix branch into downstream branches via pull requests (see {ref}`downstream-merges`). +- `merge_release.yml` is used to merge all changes included in a new release published on [GitHub](https://github.com/mrapp-ke/MLRL-Boomer/releases) into upstream branches and update the version numbers of these branches (see {ref}`upstream-merges`). +- `release.yml` defines a job for releasing a new version of the software developed by this project (see {ref}`triggering-releases`). The job can be triggered manually for one of the branches mentioned in the section {ref}`release-process`. It automatically updates the project's changelog and publishes a new release on GitHub. + +(ci-publishing)= + +## Publishing Packages + +- `publish.yml` is used for publishing pre-built packages on [PyPI](https://pypi.org/) (see {ref}`installation`). For this purpose, the project is built from source for each of the target platforms and architectures, using virtualization in some cases. The job is run automatically when a new release was published on [GitHub](https://github.com/mrapp-ke/MLRL-Boomer/releases). It does also increment the project's major version number and merge the release branch into its upstream branches (see {ref}`release-process`). +- `publish_development.yml` publishes development versions of packages on [Test-PyPI](https://test.pypi.org/) whenever changes to the project's source code have been pushed to the main branch. The packages built by each of these runs are also saved as [artifacts](https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts) and can be downloaded as zip archives. From 11c29bb9102b3074e432e203496000b122d6be79 Mon Sep 17 00:00:00 2001 From: michael-rapp <6638695+michael-rapp@users.noreply.github.com> Date: Mon, 18 Nov 2024 14:26:37 +0000 Subject: [PATCH 6/6] [Bot] Merge bugfix into feature branch. --- .version | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.version b/.version index a8839f70d..d33c3a212 100644 --- a/.version +++ b/.version @@ -1 +1 @@ -0.11.2 \ No newline at end of file +0.12.0 \ No newline at end of file