Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RELEASE] MRC v23.11 #414

Merged
merged 24 commits into from
Dec 1, 2023
Merged

[RELEASE] MRC v23.11 #414

merged 24 commits into from
Dec 1, 2023

Commits on Jul 14, 2023

  1. Configuration menu
    Copy the full SHA
    e6a320f View commit details
    Browse the repository at this point in the history
  2. Update Versions for v23.11.00 (#357)

    This PR contains the changes from running the `ci/release/update-version.sh` script for a new release
    
    Authors:
      - Michael Demoret (https://github.com/mdemoret-nv)
    
    Approvers:
      - David Gardner (https://github.com/dagardner-nv)
    
    URL: #357
    mdemoret-nv authored Jul 14, 2023
    Configuration menu
    Copy the full SHA
    7ada881 View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2023

  1. Merge pull request #358 from nv-morpheus/branch-23.07

    Forward-merge branch-23.07 to branch-23.11
    GPUtester authored Jul 20, 2023
    Configuration menu
    Copy the full SHA
    9b47924 View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2023

  1. Adopt patched pybind11 (#364)

    * Adopt patched pybind11
    * Update versions of boost & libhwloc to work-around clang CI issue
    * A test for this bug exists in Morpheus, but is difficult to reproduce in MRC. A MRC based test coming soon.
    
    fixes #262
    
    Authors:
      - David Gardner (https://github.com/dagardner-nv)
    
    Approvers:
      - Michael Demoret (https://github.com/mdemoret-nv)
    
    URL: #364
    dagardner-nv authored Aug 18, 2023
    Configuration menu
    Copy the full SHA
    9ae280a View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2023

  1. Use copy-pr-bot (#369)

    This PR replaces the `copy_prs` functionality from the `ops-bot` with the new dedicated `copy-pr-bot` GitHub application.
    
    Thorough documentation for the new `copy-pr-bot` application can be viewed below.
    
    - https://docs.gha-runners.nvidia.com/apps/copy-pr-bot/
    
    **Important**: `copy-pr-bot` enforces signed commits. If an organization member opens a PR that contains unsigned commits, it will be deemed untrusted and therefore require an `/ok to test` comment. See the GitHub docs [here](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification) for information on how to set up commit signing.
    
    Any time a PR is deemed untrusted, it will receive a comment that looks like this: rapidsai/ci-imgs#63 (comment).
    
    Every subsequent commit on an untrusted PR will require an additional `/ok to test` comment.
    
    Any existing PRs that have unsigned commits after this change is merged will require an `/ok to test` comment for each subsequent commit _or_ the PR can be rebased to include signed commits as mentioned in the docs below:
    https://docs.gha-runners.nvidia.com/cpr/contributors.
    
    This information is all included on the documentation page linked above.
    
    _I've skipped CI on this PR since it's not a change that is tested._
    
    [skip ci]
    ajschmidt8 authored Aug 29, 2023
    Configuration menu
    Copy the full SHA
    fa5e40c View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2023

  1. Fixing an issue with update-versions.sh which always blocked CI (#377)

    Cherry picks some changes from #361 to allow other PRs to run CI since that PR is marked as `DO_NOT_MERGE`
    
    Authors:
      - Michael Demoret (https://github.com/mdemoret-nv)
    
    Approvers:
      - David Gardner (https://github.com/dagardner-nv)
      - Christopher Harris (https://github.com/cwharris)
    
    URL: #377
    mdemoret-nv authored Sep 6, 2023
    Configuration menu
    Copy the full SHA
    a2a9829 View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2023

  1. Updates for MRC/Morpheus to build in the same RAPIDS devcontainer env…

    …ironment (#375)
    
    Adds a new `dependencies.yaml` which is used in RAPIDS devcontainer utils to generate conda env yamls, `project.toml`'s, and `setup.py`s on-the-fly using https://github.com/rapidsai/dependency-file-generator. Eventually we can generate/verify our existing environment files this way, but for now this gets us up and running in a RAPIDS devcontainer.
    
    Also changes `prometheus-cpp::core` to a `PRIVATE`ly linked library, as it's not longer necessary to be `PUBLIC`, and otherwise breaks a combined MRC/Morpheus build.
    
    Also adds a `MRC_PYTHON_INPLACE_BUILD` cmake option, which enables pybind11 modules to be copied back to the source tree.
    
    MRC/Morpheus unified build environment is a WIP, but available here: https://github.com/cwharris/cyber-dev.
    
    Contributes to nv-morpheus/Morpheus#704
    
    Authors:
      - Christopher Harris (https://github.com/cwharris)
    
    Approvers:
      - Michael Demoret (https://github.com/mdemoret-nv)
    
    URL: #375
    cwharris authored Sep 7, 2023
    Configuration menu
    Copy the full SHA
    79a27e2 View commit details
    Browse the repository at this point in the history
  2. Make Quick Start Guide not use make_node_full (#376)

    Replace usage of deprecated `make_node_full` with `make_node` in the Quick Start Guide. Also fix a typo.
    
    Contributes to nv-morpheus/Morpheus#782.
    
    Authors:
      - Christopher Harris (https://github.com/cwharris)
    
    Approvers:
      - Michael Demoret (https://github.com/mdemoret-nv)
    
    URL: #376
    cwharris authored Sep 7, 2023
    Configuration menu
    Copy the full SHA
    483194d View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2023

  1. Revert boost upgrade, and update clang to v16 (#382)

    fixes #379 ?
    
    Authors:
      - David Gardner (https://github.com/dagardner-nv)
    
    Approvers:
      - Christopher Harris (https://github.com/cwharris)
      - Michael Demoret (https://github.com/mdemoret-nv)
    
    URL: #382
    dagardner-nv authored Sep 11, 2023
    Configuration menu
    Copy the full SHA
    5747320 View commit details
    Browse the repository at this point in the history
  2. Add missing flags to docker command to mount the working dir and set …

    …-cap-add=sys_nice (#383)
    
    REME.md update
    
    Authors:
      - David Gardner (https://github.com/dagardner-nv)
    
    Approvers:
      - Christopher Harris (https://github.com/cwharris)
      - Michael Demoret (https://github.com/mdemoret-nv)
    
    URL: #383
    dagardner-nv authored Sep 11, 2023
    Configuration menu
    Copy the full SHA
    9153243 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2023

  1. update rapidsai/ci to rapidsai/ci-conda (#396)

    PR updates `rapidsai/ci` references to `rapidsai/ci-conda`.
    
    Authors:
      - Jake Awe (https://github.com/AyodeAwe)
    
    Approvers:
      - David Gardner (https://github.com/dagardner-nv)
    
    URL: #396
    AyodeAwe authored Sep 22, 2023
    Configuration menu
    Copy the full SHA
    18ffe6e View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2023

  1. Safe handling of control plane promises & fix CI (#391)

    * Based on fixes from @mdemoret-nv : https://github.com/mdemoret-nv/MRC/tree/mdd_control-plane-promises & #381
    * Adopts updated versions of boost, clang & IWYU
    * Only run IWYU on files changed in PR
    * Use clang for checks stage
    * Allocate progress engine promises on the heap
    
    fixes #379
    
    Authors:
      - David Gardner (https://github.com/dagardner-nv)
      - Michael Demoret (https://github.com/mdemoret-nv)
    
    Approvers:
      - Michael Demoret (https://github.com/mdemoret-nv)
      - Christopher Harris (https://github.com/cwharris)
    
    URL: #391
    dagardner-nv authored Sep 23, 2023
    Configuration menu
    Copy the full SHA
    40d20a3 View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2023

  1. Update ObservableProxy::pipe to support any number of operators (#387)

    Adds support for unlimited pipe arguments.
    
    Authors:
      - Christopher Harris (https://github.com/cwharris)
    
    Approvers:
      - Michael Demoret (https://github.com/mdemoret-nv)
    
    URL: #387
    cwharris authored Sep 25, 2023
    Configuration menu
    Copy the full SHA
    5a4a1a4 View commit details
    Browse the repository at this point in the history
  2. Add local CI scripts & rebase docker image (#394)

    * Docker image uses `rapidsai/ci-conda` as the base per rapidsai/ops#2806
    * Adds the ability to CI locally via `./ci/scripts/run_ci_local.sh build` & `./ci/scripts/run_ci_local.sh test`
    * Fix type-o in benchmark stage causing benchmarks to be executed twice & preventing results from being published.
    * Add `.tmp` and `.clangd` to `.gitignore`
    * Document need to set `DOCKER_BUILDKIT=1` for docker builds.
    
    
    fixes #393
    fixes #397
    
    Authors:
      - David Gardner (https://github.com/dagardner-nv)
    
    Approvers:
      - Christopher Harris (https://github.com/cwharris)
    
    URL: #394
    dagardner-nv authored Sep 25, 2023
    Configuration menu
    Copy the full SHA
    079e371 View commit details
    Browse the repository at this point in the history
  3. Add test for gc being invoked in a thread finalizer (#365)

    * PR replicates issue #362, and will trigger a pybind11 internal error using an un-patched version of pybind11
    * Only run IWYU on files changed in PR
    
    Note:
    * This bug requires the code in question to be run in a thread created by C++.
    
    fixes #362
    
    Authors:
      - David Gardner (https://github.com/dagardner-nv)
    
    Approvers:
      - Michael Demoret (https://github.com/mdemoret-nv)
    
    URL: #365
    dagardner-nv authored Sep 25, 2023
    Configuration menu
    Copy the full SHA
    8b20469 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2023

  1. Fix EdgeHolder from incorrectly reporting an active connection (#402)

    * Prevents `check_active_connection` from mistakenly returning true for a holder where `init_owned_edge` has been called but neither the `init_connected_edge method` or the `add_connector` method have not been called.
    
    Relates to issue #360
    
    Authors:
      - David Gardner (https://github.com/dagardner-nv)
    
    Approvers:
      - Michael Demoret (https://github.com/mdemoret-nv)
    
    URL: #402
    dagardner-nv authored Sep 28, 2023
    Configuration menu
    Copy the full SHA
    ba483b1 View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2023

  1. Update boost versions to match version used in dev env (#404)

    PR #391 updated the version of boost, but I forgot to update the version in the package's yaml.
    
    Authors:
      - David Gardner (https://github.com/dagardner-nv)
    
    Approvers:
      - Devin Robison (https://github.com/drobison00)
      - Michael Demoret (https://github.com/mdemoret-nv)
    
    URL: #404
    dagardner-nv authored Oct 9, 2023
    Configuration menu
    Copy the full SHA
    34bacd1 View commit details
    Browse the repository at this point in the history
  2. Fix libhwloc & stubgen versions to match dev yaml (#405)

    Authors:
      - David Gardner (https://github.com/dagardner-nv)
    
    Approvers:
      - Michael Demoret (https://github.com/mdemoret-nv)
      - Devin Robison (https://github.com/drobison00)
    
    URL: #405
    dagardner-nv authored Oct 9, 2023
    Configuration menu
    Copy the full SHA
    24865be View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2023

  1. Adding more coroutine components to support async generators and task…

    … containers (#408)
    
    This PR is related to the Morpheus Sherlock work which uses coroutines. Much of this code was pulled from this branch https://github.com/ryanolson/srf/tree/runnable_next and was in a pretty good state. A few minor tweaks and changes have been added.
    
    Authors:
      - Michael Demoret (https://github.com/mdemoret-nv)
    
    Approvers:
      - Devin Robison (https://github.com/drobison00)
      - Christopher Harris (https://github.com/cwharris)
    
    URL: #408
    mdemoret-nv authored Oct 20, 2023
    Configuration menu
    Copy the full SHA
    36460c9 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2023

  1. Move Pycoro from Morpheus to MRC (#409)

    Moves pycoro from Morpheus to MRC and incorperates tests from nv-morpheus/Morpheus#1286
    
    Closes nv-morpheus/Morpheus#1268
    
    Authors:
      - Christopher Harris (https://github.com/cwharris)
    
    Approvers:
      - Devin Robison (https://github.com/drobison00)
    
    URL: #409
    cwharris authored Oct 25, 2023
    Configuration menu
    Copy the full SHA
    1ebd4e2 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2023

  1. Add AsyncioRunnable (#411)

    Moves the CoroutineRunnable from Morpheus' Sherlock feature branch to MRC and renames it to AsyncioRunnable as it is heavily dependent on asyncio. Adjustments were made such that the Scheduler would no longer own a task container and/or tasks, leaving the scheduler interface simpler. Instead, the runnable is responsible for the lifetime of the tasks it creates. This leaves the scheduler with a single responsibility.
    
    Much of the code could be moved to MRC proper from PyMRC, but it's not immediately obvious where the code should live or whether it would be reused, so keeping it colocated with the AsyncioRunnable makes the most sense for now, imo.
    
    Authors:
      - Christopher Harris (https://github.com/cwharris)
    
    Approvers:
      - Devin Robison (https://github.com/drobison00)
    
    URL: #411
    cwharris authored Nov 2, 2023
    Configuration menu
    Copy the full SHA
    62e1834 View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2023

  1. Use a traditional semaphore in AsyncioRunnable (#412)

    Closes nv-morpheus/Morpheus#1339
    
    Replaces the ClosableRingBuffer usage in AsyncioRunnable to instead use a traditional semaphore which seems to be more reliable for this use case.
    
    Authors:
      - Christopher Harris (https://github.com/cwharris)
    
    Approvers:
      - David Gardner (https://github.com/dagardner-nv)
    
    URL: #412
    cwharris authored Nov 3, 2023
    Configuration menu
    Copy the full SHA
    8aa9216 View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2023

  1. 23.11 Cleanup

    mdemoret-nv committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    dbce7f9 View commit details
    Browse the repository at this point in the history
  2. Updating CHANGELOG

    mdemoret-nv committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    de48cb1 View commit details
    Browse the repository at this point in the history