update readme

README.md

@@ -18,8 +18,7 @@ Both pipelines are inside a shared Model Control Plane model context - training
   * Search for an optimal model object architecture and tune its hyperparameters
   * Train the model object and evaluate its performance on the holdout set
   * Compare a recently trained model object with one promoted earlier
-  * If a recently trained model object performs better, then stage it as a new inference model object in the model registry
-  * On success of the current model object, then stage newly created Model Control Plane version as the one used for inference
+  * If a recently trained model object performs better, then stage it as a new inference model in the Model Control Plane
 * [CD] Batch Inference
   * Load the inference dataset and preprocess it reusing object fitted during training
   * Perform data drift analysis reusing training dataset of the inference Model Control Plane version as a reference
@@ -120,7 +119,50 @@ We also output `preprocess_pipeline` as an output artifact from `train_data_prep
 
 To ensure the high quality of ML models many ML Engineers go for automated hyperparameter tuning or even automated model architecture search. In this example, we are using prepared data from ETL to spin up a search of the best model parameters for different architectures in parallel.
 
-To create parallel processing of computationally expensive operations we use a loop over predefined potential architectures and respective parameters search grid and create one step for each candidate. After the steps are created we need to collect results (one best model per each search step) in a `hp_tuning_select_best_model` step to define the final winner and pass it to training. To ensure that collection goes smoothly and in full we use an `after` statement populated with all search steps names, so the selector job will wait for the completion of all searches.
+To create parallel processing of computationally expensive operations we use a loop over predefined potential architectures and respective parameters search grid and create one step for each candidate. Inside each hyperparameter tuning step instance, we run random search cross-validation to find the best parameters and after that evaluate the result using the metric of interest (accuracy_score in this example). We attach a computed metric to the output artifact as metadata to be used later in `hp_tuning_select_best_model`.
+<details>
+  <summary>Code snippet 💻</summary>
+
+```python
+from zenml import log_artifact_metadata
+
+score = accuracy_score(y_tst, y_pred)
+# log score along with output artifact as metadata
+log_artifact_metadata(
+    output_name="hp_result",
+    metric=float(score),
+)
+```
+</details>
+
+After the steps are executed we need to collect results (one best model per each search step) in a `hp_tuning_select_best_model` step to define the final winner and pass it to training. We use the Model Control Plane capabilities to pull correct artifacts from previous steps and fetch their metadata for final evaluation before actual training.
+<details>
+  <summary>Code snippet 💻</summary>
+
+```python
+from zenml import get_step_context
+
+model_version = get_step_context().model_config._get_model_version()
+
+best_model = None
+best_metric = -1
+# consume artifacts attached to current model version in Model Control Plane
+for full_artifact_name in model_version.artifact_object_ids:
+    # if artifacts comes from one of HP tuning steps
+    if full_artifact_name.endswith("hp_result"):
+        hp_output = model_version.artifacts[full_artifact_name]["1"]
+        model: ClassifierMixin = hp_output.load()
+        # fetch metadata we attached earlier
+        metric = float(hp_output.metadata["metric"].value)
+        if best_model is None or best_metric < metric:
+            best_model = model
+```
+</details>
+
+
+To ensure that collection goes smoothly and in full we use an `after` statement populated with all search steps names, so the selector job will wait for the completion of all searches.
+
+
 
 You can find more information about the current state of [Hyperparameter Tuning using ZenML in the documentation](https://docs.zenml.io/user-guide/advanced-guide/pipelining-features/hyper-parameter-tuning).
 
@@ -185,9 +227,9 @@ Once the model object is trained and evaluated on meeting basic quality standard
 
 In this example, we are implementing promotion based on metric comparison to decide on the spot and avoid more complex approaches like Champion/Challengers shadow deployments. In other projects, other promotion techniques and strategies can be used.
 
-To achieve this we would retrieve the model registry version from [Model Registry](https://docs.zenml.io/stacks-and-components/component-guide/model-registries): latest (the one we just trained) and current (the one having a proper tag). Next, we need to deploy both model objects using [Model Deployer](https://docs.zenml.io/stacks-and-components/component-guide/model-deployers) and run predictions on the testing set for both of them. Next, we select which one of the model registry versions has a better metric value and associate it with the inference tag.
+To achieve this we would retrieve the model version from the Model Control Plane: latest (the one we just trained) and current (the one having a proper tag). Next, we need to deploy both model objects using [Model Deployer](https://docs.zenml.io/stacks-and-components/component-guide/model-deployers) and run predictions on the testing set for both of them. Next, we select which one of the model registry versions has a better metric value. If the newly trained model is performing better we promote it to the inference stage in the Model Control Plane.
 
-As a last step we promote the current Model Control Plane version to the inference stage if a promotion decision was made. By doing so we ensure that the best Model Control Plane version would be used for inference later on and ensure seamless integration of relevant artifacts from training pipeline in the batch inference pipeline.
+By doing so we ensure that the best-performing version will be used for inference later on and ensure seamless integration of relevant artifacts from the training pipeline in the batch inference pipeline.
 
 ### [Continuous Deployment] Batch Inference
 The Batch Inference pipeline is designed to run with inference Model Control Plane version context. This ensures that we always infer only on quality-assured Model Control Plane version and provide seamless integration of required artifacts created during training of this Model Control Plane version.
@@ -208,7 +250,7 @@ model_config:
 
 The process of loading data is similar to training, even the same step function is used, but with the `is_inference` flag.
 
-But inference flow has an important difference - there is no need to fit preprocessing `Pipeline`, rather we need to reuse one fitted during training on the train set, to ensure that the model object gets the expected input. To do so we will use [ExternalArtifact](https://docs.zenml.io/user-guide/advanced-guide/pipelining-features/configure-steps-pipelines#pass-any-kind-of-data-to-your-steps) with lookup by `model_artifact_name` only to get the preprocessing pipeline fitted during the quality assurance training run. This is possible, since we configured batch inference pipeline to run inside a Model Control Plane version context.
+But inference flow has an important difference - there is no need to fit preprocessing sklearn `Pipeline`, rather we need to reuse one fitted during training on the train set, to ensure that the model object gets the expected input. To do so we will use [ExternalArtifact](https://docs.zenml.io/user-guide/advanced-guide/pipelining-features/configure-steps-pipelines#pass-any-kind-of-data-to-your-steps) with lookup by `model_artifact_name` only to get the preprocessing pipeline fitted during the quality-assured training run. This is possible since we configured the batch inference pipeline to run inside a Model Control Plane version context.
 <details>
   <summary>Code snippet 💻</summary>
 
@@ -230,9 +272,9 @@ df_inference = inference_data_preprocessor(
 
 [📂 Code folder](template/steps/%7B%25%20if%20data_quality_checks%20%25%7Ddata_quality%7B%25%20endif%20%25%7D)
 
-On the drift reporting stage we will use [standard step](https://docs.zenml.io/stacks-and-components/component-guide/data-validators/evidently#the-evidently-data-validator) `evidently_report_step` to build Evidently report to assess certain data quality metrics. `evidently_report_step` has a number of options, but for this example, we will build only `DataQualityPreset` metrics preset to get a number of NA values in reference and current datasets.
+In the drift reporting stage, we will use [standard step](https://docs.zenml.io/stacks-and-components/component-guide/data-validators/evidently#the-evidently-data-validator) `evidently_report_step` to build Evidently report to assess certain data quality metrics. `evidently_report_step` has a number of options, but for this example, we will build only `DataQualityPreset` metrics preset to get a number of NA values in reference and current datasets.
 
-We pass `dataset_trn` from training pipeline as a `reference_dataset` here. To do so we will use [ExternalArtifact](https://docs.zenml.io/user-guide/advanced-guide/pipelining-features/configure-steps-pipelines#pass-any-kind-of-data-to-your-steps) with lookup by `model_artifact_name` only to get the training dataset used during quality-assured training run. This is possible, since we configured batch inference pipeline to run inside a Model Control Plane version context.
+We pass `dataset_trn` from the training pipeline as a `reference_dataset` here. To do so we will use [ExternalArtifact](https://docs.zenml.io/user-guide/advanced-guide/pipelining-features/configure-steps-pipelines#pass-any-kind-of-data-to-your-steps) with lookup by `model_artifact_name` only to get the training dataset used during quality-assured training run. This is possible since we configured the batch inference pipeline to run inside a Model Control Plane version context.
 
 After the report is built we execute another quality gate using the `drift_quality_gate` step, which assesses if a significant drift in the NA count is observed. If so, execution is stopped with an exception.
 
@@ -242,11 +284,9 @@ You can follow [Data Validators docs](https://docs.zenml.io/stacks-and-component
 
 [📂 Code folder](template/steps/inference)
 
-As a last step concluding all work done so far, we will calculate predictions on the inference dataset and persist them in [Artifact Store](https://docs.zenml.io/stacks-and-components/component-guide/artifact-stores) for reuse.
-
-As we performed promotion as part of the training pipeline it is very easy to fetch the needed model registry version from [Model Registry](https://docs.zenml.io/stacks-and-components/component-guide/model-registries) and deploy it for inference with [Model Deployer](https://docs.zenml.io/stacks-and-components/component-guide/model-deployers).
+As a last step concluding all work done so far, we will calculate predictions on the inference dataset and persist them in [Artifact Store](https://docs.zenml.io/stacks-and-components/component-guide/artifact-stores) attached to the current inference model version of the Model Control Plane for reuse and observability.
 
-Once the model registry version is deployed the only thing left over is to call `.predict()` on the deployment service object and put those predictions as an output of the predictions step, so it is automatically stored in the [Artifact Store](https://docs.zenml.io/stacks-and-components/component-guide/artifact-stores) with zero effort. We additionally annotated the `predictions` output with `ArtifactConfig(overwrite=False)` to ensure that this artifact will get linked to a Model Control Plane version in a versioned fashion. This is required to deliver a comprehensive history to stakeholders since Batch Inference can be executed using the same Model Control Plane version multiple times.
+We will leverage metadata of `model` artifact linked to the inference model version of the Model Control Plane to create a deployment service and run `.predict()` to put those predictions as an output of the predictions step, so it is automatically stored in the [Artifact Store](https://docs.zenml.io/stacks-and-components/component-guide/artifact-stores) and linked to the Model Control Plane model version as a versioned artifact link with zero effort. This is achieved because we additionally annotated the `predictions` output with `ArtifactConfig(overwrite=False)`. This is required to deliver a comprehensive history to stakeholders since Batch Inference can be executed using the same Model Control Plane version multiple times.
 <details>
   <summary>Code snippet 💻</summary>