…RC (#1981)

Depends on PR nv-morpheus/MRC#511 being completed, merged, and a new conda package uploaded before testing can begin.

Fixes a small issue where MRC updated a function to be private to avoid incorrect usage of it.

Authors:
  - Michael Demoret (https://github.com/mdemoret-nv)

Approvers:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)
  - Christopher Harris (https://github.com/cwharris)

URL: #1981

Closes #1973

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #1975

* Adds `docs/source/basics/cpu_only_mode.md`

Closes #1959 

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Yuchen Zhang (https://github.com/yczhang-nv)

URL: #1969

…tage` and `HttpClientSourceStage` (#1834)

* Add optional `message_type`, `task_type`, `task_payload` arguments to `HttpServerSourceStage` & `HttpClientSourceStage`.
* Add new `ConfigurableOutputSource` base class for single output source stages which support both `MessageMeta` and `ControlMessage` as output types.
* When `ControlMessage` output is selected, HTTP request headers will be added to the message's metadata.
* Define new request handler callback method `request_handler_fn_t` which receives the boost asio tcp endpoint along with the request object.
* `HttpEndpoint` now optionally receives a `request_handler_fn_t` method, and for compatibility the older `payload_parse_fn_t` method.
* Add an optional `include_headers` argument to `HttpEndpointInterfaceProxy::init`, which when `False` (the default) uses the existing `payload_parse_fn_t` method, and `request_handler_fn_t` when `True`. This changes the function signature, but not the behavior when the `include_headers` argument is omitted, so technically not an API breaking change.
* Move/rename `cm_task_t`-->`control_message_task_t` from `deserialize.hpp` to  be shared with other stages
* Don't use `accept_status` for live and ready endpoints, as these should always return 200 on success, while `accept_status` defaults to `201 CREATED` which isn't appropriate for a health check.

Closes #1811

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #1834

This change was made because of check-style failures when tests were refactored but is causing import failures.

Note: This commit also reverts the style fix and the local imports are mis-sorted. We may need to update the style checker to fix it post release.

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Yuchen Zhang (https://github.com/yczhang-nv)

URL: #1978

Docs update

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Yuchen Zhang (https://github.com/yczhang-nv)

URL: #1982

* Add missing `tensorrt-cu12` dependency to the `model-utils` environment.
* Update `onnx-to-trt` to function with TensorRT v10, removed usage of attributes/methods which were deprecated in v8.4 and removed in v10.
* Update documentation

Closes #1979

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Yuchen Zhang (https://github.com/yczhang-nv)
  - https://github.com/hsin-c

URL: #1984

Update `val-run-all.sh` to run cpp pipeline only and do not run in Python.

Closes #1985 

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Yuchen Zhang (https://github.com/yczhang-nv)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)

URL: #1986

…encies (#1974)

- Update `dependencies.yaml` and re-generate environment yaml's
- Avoids install of `pytorch` cpu packages which causes examples like DFP try to use.

Closes #1943 

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Eli Fajardo (https://github.com/efajardo-nv)
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #1974

The utilities submodule commit is referencing something that doesn't exist on `nv-morphues/utilities`. This PR fixes that.

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Christopher Harris (https://github.com/cwharris)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #1987

…ase container (#1976)

Move the data files needed for log-parsing from models to examples/data to avoid copying models to the release container

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)

URL: #1976

* Agents: Show an actual runnable command that can be copy/pasted, not just the command usage.
* Completion: Remove blank line in command block
* RAG: Remove section about fetching data, RAG doesn't need any LFS files.
* VDB Upload: Pipeline requires the `examples` dataset not the `datasets` dataset.
* VDB Upload: Document how to run Milvus

Closes #1991

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

URL: #1992

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Yuchen Zhang (https://github.com/yczhang-nv)

URL: #1998

- Dependencies for DFP container were recently moved from `examples/digital_fingerprinting/production/conda_env.yaml`  to `conda/environments/examples_cuda-125_arch-x86_64.yaml` but `python-logging-loki` was not included.
- Add `python-logging-loki` to `example-dfp-prod` in `dependencies.yaml`.
- Remove `conda_env.yaml` since it's no longer being used anywhere.

Closes #1917

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Eli Fajardo (https://github.com/efajardo-nv)
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

URL: #1977

* Fixes an issue where an `int` is being returned in the DFP rolling window stage, likely caused by the recent update to the version of pandas being used.
* Update file paths to reflect the move of the `dfp_duo_pipeline.py`, `dfp_azure_pipeline.py` & `dfp_integrated_training_batch_pipeline.py` scripts
* Document installing the missing dependencies needed for the DFP Grafana example

Closes #1994

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

URL: #1995

)

Closes #1934

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Christopher Harris (https://github.com/cwharris)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #2004

This is to account for the changes done during DFP refactoring

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #2003

* The deserialize stage no longer actually deserializes, but instead chunks incoming `MessageMeta` objects and converts to `ControlMessage`

Closes #2007

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Yuchen Zhang (https://github.com/yczhang-nv)

URL: #2009

Closes #1999

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Yuchen Zhang (https://github.com/yczhang-nv)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)

URL: #2008

…2006)

Closes #2002

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)
  - Christopher Harris (https://github.com/cwharris)

URL: #2006

…#2016)

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Yuchen Zhang (https://github.com/yczhang-nv)

URL: #2016

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Yuchen Zhang (https://github.com/yczhang-nv)

URL: #2017

…e correct rows for each batch (#2015)

* `DeserializeStage` now performs a copy of the rows needed for each batch, ensuring each output `ControlMessage` now has a unique `MessageMeta` and underlying `DataFrame`
* Works-around issue where calling `SlicedMessageMeta::get_mutable_info().checkout_obj()` returns the entire `DataFrame`
* This unfortunately means that `DeserializeStage` will require the GIL.
* Remove unused `make_output_message` method
* Avoid some redundant code in `SlicedMessageMeta` (unrelated but I was in this part of the code).

Closes #2002

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Christopher Harris (https://github.com/cwharris)

URL: #2015

Earlier the morpheus-pipeline stage was using the morpheus-runtime container as the base image. This change switches the base container to nvidia/cuda and installs the morpheus-dfp library.

There is a minor docs fixup in conda_packages.md that I pull in based on the review comments.

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)

URL: #1971

* Updates the version of the Triton Inference server container being used by Morpheus from `23.06` to `24.09`.
* Includes a re-trained `ransomw-model-short-rf` (PR #2023), the original model was not compatible with Triton 24.09
* Update the output dimensions of the `log-parsing-onnx` model per #1727

Closes #2000

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)
  - Tad ZeMicheal (https://github.com/tzemicheal)

Approvers:
  - https://github.com/raykallen
  - Christopher Harris (https://github.com/cwharris)
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #2001

Requires changes from MRC PR: nv-morpheus/MRC#502

Closes #1519

Authors:
  - Michael Demoret (https://github.com/mdemoret-nv)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)

URL: #1963

This bug is caused by the transition from `MultiMessage` to `ControlMessage`. 

`inference_stage.py::InferenceStage::_build_single` calls `_convert_one_response` in a loop for a batch, and the argument is passing is the same for the whole batch, but inside `_convert_one_response` we're grabbing the tensors and assigning starting at zero, so the tensors are overwriting each other and causes the issue. 

Added `batch_offset` variable to keep track where the next incoming tensor should start writing to the output message.

Closes #2019 


## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Yuchen Zhang (https://github.com/yczhang-nv)
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)
  - Christopher Harris (https://github.com/cwharris)

URL: #2024

* Use the index of the incoming DF as the source of the `doc` field not the first column of the sequence_ids
* Fix typeo in docstring for `TableInfo::num_indices` (unrelated)
* Remove declaration for unimplemtned/unused function in `python/morpheus/morpheus/_lib/include/morpheus/stages/deserialize.hpp` (unrelated)

Closes #2029


## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Yuchen Zhang (https://github.com/yczhang-nv)

URL: #2031

- Fixes benchmark tests that were broken from latest code updates
- Add `pynvml` dependency back to conda dev environment. It was removed when `nvtabular` was removed.
- Remove broken E2E autoencoder test since Starter DFP is being removed.

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Eli Fajardo (https://github.com/efajardo-nv)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)

URL: #1958

Closes #2030

This is needed to run root_cause_analysis in the release container.

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Christopher Harris (https://github.com/cwharris)

URL: #2032

See #2026

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Christopher Harris (https://github.com/cwharris)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)

URL: #2027

At some point these got out of sync where the CLI was using add-scores, but the Python API was using the filter stage, and some of the docs were using add-scores as well
 
## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

URL: #2037

Closes 

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - https://github.com/raykallen

URL: #2036

…rors (#2039)

Closes #2038

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

URL: #2039

This reverts commit 599e73f.