[RELEASE] MRC v23.11 #414
Merged
[RELEASE] MRC v23.11 #414
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR contains the changes from running the `ci/release/update-version.sh` script for a new release Authors: - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: #357
Forward-merge branch-23.07 to branch-23.11
* Adopt patched pybind11 * Update versions of boost & libhwloc to work-around clang CI issue * A test for this bug exists in Morpheus, but is difficult to reproduce in MRC. A MRC based test coming soon. fixes #262 Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #364
This PR replaces the `copy_prs` functionality from the `ops-bot` with the new dedicated `copy-pr-bot` GitHub application. Thorough documentation for the new `copy-pr-bot` application can be viewed below. - https://docs.gha-runners.nvidia.com/apps/copy-pr-bot/ **Important**: `copy-pr-bot` enforces signed commits. If an organization member opens a PR that contains unsigned commits, it will be deemed untrusted and therefore require an `/ok to test` comment. See the GitHub docs [here](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification) for information on how to set up commit signing. Any time a PR is deemed untrusted, it will receive a comment that looks like this: rapidsai/ci-imgs#63 (comment). Every subsequent commit on an untrusted PR will require an additional `/ok to test` comment. Any existing PRs that have unsigned commits after this change is merged will require an `/ok to test` comment for each subsequent commit _or_ the PR can be rebased to include signed commits as mentioned in the docs below: https://docs.gha-runners.nvidia.com/cpr/contributors. This information is all included on the documentation page linked above. _I've skipped CI on this PR since it's not a change that is tested._ [skip ci]
Cherry picks some changes from #361 to allow other PRs to run CI since that PR is marked as `DO_NOT_MERGE` Authors: - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - David Gardner (https://github.com/dagardner-nv) - Christopher Harris (https://github.com/cwharris) URL: #377
…ironment (#375) Adds a new `dependencies.yaml` which is used in RAPIDS devcontainer utils to generate conda env yamls, `project.toml`'s, and `setup.py`s on-the-fly using https://github.com/rapidsai/dependency-file-generator. Eventually we can generate/verify our existing environment files this way, but for now this gets us up and running in a RAPIDS devcontainer. Also changes `prometheus-cpp::core` to a `PRIVATE`ly linked library, as it's not longer necessary to be `PUBLIC`, and otherwise breaks a combined MRC/Morpheus build. Also adds a `MRC_PYTHON_INPLACE_BUILD` cmake option, which enables pybind11 modules to be copied back to the source tree. MRC/Morpheus unified build environment is a WIP, but available here: https://github.com/cwharris/cyber-dev. Contributes to nv-morpheus/Morpheus#704 Authors: - Christopher Harris (https://github.com/cwharris) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #375
Replace usage of deprecated `make_node_full` with `make_node` in the Quick Start Guide. Also fix a typo. Contributes to nv-morpheus/Morpheus#782. Authors: - Christopher Harris (https://github.com/cwharris) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #376
fixes #379 ? Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Christopher Harris (https://github.com/cwharris) - Michael Demoret (https://github.com/mdemoret-nv) URL: #382
…-cap-add=sys_nice (#383) REME.md update Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Christopher Harris (https://github.com/cwharris) - Michael Demoret (https://github.com/mdemoret-nv) URL: #383
PR updates `rapidsai/ci` references to `rapidsai/ci-conda`. Authors: - Jake Awe (https://github.com/AyodeAwe) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: #396
* Based on fixes from @mdemoret-nv : https://github.com/mdemoret-nv/MRC/tree/mdd_control-plane-promises & #381 * Adopts updated versions of boost, clang & IWYU * Only run IWYU on files changed in PR * Use clang for checks stage * Allocate progress engine promises on the heap fixes #379 Authors: - David Gardner (https://github.com/dagardner-nv) - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) - Christopher Harris (https://github.com/cwharris) URL: #391
Adds support for unlimited pipe arguments. Authors: - Christopher Harris (https://github.com/cwharris) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #387
* Docker image uses `rapidsai/ci-conda` as the base per rapidsai/ops#2806 * Adds the ability to CI locally via `./ci/scripts/run_ci_local.sh build` & `./ci/scripts/run_ci_local.sh test` * Fix type-o in benchmark stage causing benchmarks to be executed twice & preventing results from being published. * Add `.tmp` and `.clangd` to `.gitignore` * Document need to set `DOCKER_BUILDKIT=1` for docker builds. fixes #393 fixes #397 Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Christopher Harris (https://github.com/cwharris) URL: #394
* PR replicates issue #362, and will trigger a pybind11 internal error using an un-patched version of pybind11 * Only run IWYU on files changed in PR Note: * This bug requires the code in question to be run in a thread created by C++. fixes #362 Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #365
* Prevents `check_active_connection` from mistakenly returning true for a holder where `init_owned_edge` has been called but neither the `init_connected_edge method` or the `add_connector` method have not been called. Relates to issue #360 Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #402
PR #391 updated the version of boost, but I forgot to update the version in the package's yaml. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Devin Robison (https://github.com/drobison00) - Michael Demoret (https://github.com/mdemoret-nv) URL: #404
Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) - Devin Robison (https://github.com/drobison00) URL: #405
… containers (#408) This PR is related to the Morpheus Sherlock work which uses coroutines. Much of this code was pulled from this branch https://github.com/ryanolson/srf/tree/runnable_next and was in a pretty good state. A few minor tweaks and changes have been added. Authors: - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - Devin Robison (https://github.com/drobison00) - Christopher Harris (https://github.com/cwharris) URL: #408
Moves pycoro from Morpheus to MRC and incorperates tests from nv-morpheus/Morpheus#1286 Closes nv-morpheus/Morpheus#1268 Authors: - Christopher Harris (https://github.com/cwharris) Approvers: - Devin Robison (https://github.com/drobison00) URL: #409
Moves the CoroutineRunnable from Morpheus' Sherlock feature branch to MRC and renames it to AsyncioRunnable as it is heavily dependent on asyncio. Adjustments were made such that the Scheduler would no longer own a task container and/or tasks, leaving the scheduler interface simpler. Instead, the runnable is responsible for the lifetime of the tasks it creates. This leaves the scheduler with a single responsibility. Much of the code could be moved to MRC proper from PyMRC, but it's not immediately obvious where the code should live or whether it would be reused, so keeping it colocated with the AsyncioRunnable makes the most sense for now, imo. Authors: - Christopher Harris (https://github.com/cwharris) Approvers: - Devin Robison (https://github.com/drobison00) URL: #411
Closes nv-morpheus/Morpheus#1339 Replaces the ClosableRingBuffer usage in AsyncioRunnable to instead use a traditional semaphore which seems to be more reliable for this use case. Authors: - Christopher Harris (https://github.com/cwharris) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: #412
|
/ok to test |
|
/ok to test |
mdemoret-nv
force-pushed
the
branch-23.11
branch
from
809346f
to
dbce7f9
Compare
|
/ok to test |
mdemoret-nv
force-pushed
the
branch-23.11
branch
from
29e1abc
to
de48cb1
Compare
|
/ok to test |
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #414 +/- ##
==========================================
+ Coverage 73.14% 74.03% +0.89%
==========================================
Files 382 394 +12
Lines 13403 13955 +552
Branches 1010 1053 +43
==========================================
+ Hits 9804 10332 +528
- Misses 3599 3623 +24
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 13 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
❄️ Code freeze for
branch-23.11andv23.11releaseWhat does this mean?
Only critical/hotfix level issues should be merged into
branch-23.11until release (merging of this PR).All other development PRs should be retargeted towards the next release branch:
branch-24.03.What is the purpose of this PR?
branch-23.11intomainfor the release