Skip to content

Commit

Permalink
add model control plane
Browse files Browse the repository at this point in the history
  • Loading branch information
@safoinme
safoinme committed Nov 9, 2023
1 parent 8ed95a7 commit 57f9b68
Show file tree
Hide file tree
Showing 26 changed files with 518 additions and 656 deletions.
177 changes: 140 additions & 37 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,69 +1,172 @@
# 💫 ZenML End-to-End NLP Training and Deployment Project Template
# 🌟 ZenML End-to-End NLP Training, Promotion, and Deployment Project Template

Welcome to the ZenML NLP project template, a comprehensive starting point for anyone looking to build and deploy NLP models using the ZenML framework. This template is designed to streamline the process of training, promoting, and deploying NLP models with a focus on reproducibility, scalability, and ease of use.

## 🚀 Quickstart

To get started with this template, you'll need to have ZenML installed along with the necessary extras for NLP:

```bash
pip install zenml[templates]
```

Once you have ZenML installed, you can initialize a new project using this template:

```bash
zenml init --template <short_name_of_template>
# example: zenml init --template template-nlp
```

Running the command above will result in input prompts being shown to you. If you would like to rely on default values for the ZenML project template - you can add --template-with-defaults to the same command, like this:

```bash
zenml init --template <short_name_of_template> --template-with-defaults
# example: zenml init --template template-nlp --template-with-defaults
```

This project template is designed to help you get started with training and deploying NLP models using the ZenML framework. It provides a comprehensive set of steps and pipelines to cover major use cases of NLP model development, including dataset loading, tokenization, model training, model registration, and deployment.

## 📃 Template Parameters

The template can be configured using the following parameters:

| Parameter | Description | Default |
|-----------|-------------|---------|
| Name | The name of the person/entity holding the copyright | ZenML GmbH |
| Email | The email of the person/entity holding the copyright | [email protected] |
| Project Name | Short name for your project | ZenML NLP project |
| Project Version | The version of your project | 0.0.1 |
| Project License | The license under which your project will be released | Apache Software License 2.0 |
| Technical product name | The technical name to prefix all tech assets (pipelines, models, etc.) | nlp_use_case |
| Target environment | The target environment for deployments/promotions | staging |
| Use metric-based promotion | Whether to compare metric of interest to make model version promotion | True |
| Notifications on failure | Whether to notify about pipeline failures | True |
| Notifications on success | Whether to notify about pipeline successes | False |
| ZenML Server URL | Optional URL of a remote ZenML server for support scripts | - |

## 🚀 Generate a ZenML Project

To generate a project from this template, make sure you have ZenML and its `templates` extras installed:
| Full Name | The name of the copyright holder | ZenML GmbH |
| Email | The email of the copyright holder | [email protected] |
| Product Name | The technical name for your NLP product | nlp_use_case |
| Target Environment | The target environment for deployments | staging |
| Accelerator | The accelerator to use for training | gpu |
| Deploy Locally | Whether to deploy the model locally | True |
| Deploy to HuggingFace | Whether to deploy to HuggingFace Hub | False |
| Deploy to SkyPilot | Whether to deploy to SkyPilot | False |
| Dataset | The dataset to use from HuggingFace Datasets | airline_reviews |
| Model | The model to use from HuggingFace Models | roberta-base |
| Cloud Provider | The cloud provider to use (AWS or GCP) | aws |
| Metric-Based Promotion | Whether to promote models based on metrics | True |
| Notifications on Failure | Whether to notify about pipeline failures | True |
| Notifications on Success | Whether to notify about pipeline successes | False |
| ZenML Server URL | URL of a remote ZenML server (optional) | |

After configuring your project, you can run the following commands to set up your environment and start the ZenML Dashboard:

```bash
pip install zenml[templates]
cd <your_project_directory>
make setup
make install-local-stack
zenml up
python run.py
```

Then, run the following command to generate the project:
For more details, check the `README.md` file in the generated project directory.

```bash
zenml init --template template-nlp
```
## 🛠️ Project Structure

This NLP project template includes three main pipelines:

### Training Pipeline

The training pipeline is designed to handle the end-to-end process of training an NLP model. It includes steps for data loading, tokenization, model training, and model registration. The pipeline is parameterized to allow for customization of the training process, such as sequence length, batch size, and learning rate.

### Promotion Pipeline

The promotion pipeline is responsible for promoting a model version to a specified stage. It can be configured to promote based on a metric comparison between the latest and currently promoted model versions or to simply promote the latest model version.

### Deployment Pipeline

The deployment pipeline handles the deployment of the model to various environments. It can be configured to deploy locally, to HuggingFace Hub, or to SkyPilot, depending on your requirements.

## 📚 Core Concepts

This template showcases several core concepts of ZenML:

- Designing and running ZenML pipelines for NLP tasks.
- Parameterizing steps for flexibility and reusability.
- Model promotion strategies based on metrics or latest versions.
- Deploying models to different environments, including local and cloud-based platforms.
- Best practices for implementing and running reproducible and reliable ML pipelines with ZenML.

## 📦 Additional Features

- Integration with HuggingFace Datasets and Models for easy access to pre-trained models and datasets.
- Support for different accelerators (GPU/CPU) to speed up training.
- Conditional deployment steps based on user configuration.
- Notification hooks for pipeline successes and failures.

## 🌐 Deployment Options

The template supports multiple deployment options, including local deployment for testing and cloud-based deployment for production use. You can choose to deploy to HuggingFace Hub for easy sharing and collaboration or to SkyPilot for managed deployment services.

## 📋 License

This project is licensed under the Apache License 2.0, ensuring that you have the freedom to use, modify, and distribute the software as you see fit.

### 🛠️ How this template is implemented

Let's dive into the implementation details and share some tips and best practices for using this template effectively.

### Training Pipeline

The training pipeline is the heart of the NLP project. It is responsible for preparing the data, training the model, and registering the model in the model registry. The pipeline is designed to ensure that only models that meet quality standards are promoted for inference.

The training pipeline is configured using the `{{product_name}}_training_pipeline` function, which includes steps for data loading, tokenization, model training, and model registration. The pipeline can be customized with parameters such as `lower_case`, `padding`, `max_seq_length`, and others to tailor the tokenization and training process to your specific NLP use case.

### Training Pipeline: Data and Tokenization

[📂 Code folder](template/steps/data_tokenization/)
<p align="center">
<img height=500 src="assets/nlp_data_tokenization.png">
</p>

The first stage of the training pipeline involves loading the dataset and preparing it for the model. The `data_loader` step fetches the dataset, which is then passed to the `tokenizer_loader` and `tokenization_step` to convert the raw text data into a format suitable for the NLP model.

Tokenization is a critical step in NLP pipelines, as it converts text into tokens that the model can understand. The tokenizer can be configured to handle case sensitivity, padding strategies, and sequence lengths, ensuring that the input data is consistent and optimized for training.

### Training Pipeline: Model Training

You will be prompted to provide values for the template parameters. If you want to use the default values, you can add the `--template-with-defaults` flag to the command.
[📂 Code folder](template/steps/model_training/)
<p align="center">
<img height=500 src="assets/nlp_model_training.png">
</p>

## 🧰 How this template is implemented
Once the data is tokenized, the `model_trainer` step takes over to train the NLP model. This step utilizes the tokenized dataset and the tokenizer itself to fine-tune the model on the specific task, such as sentiment analysis, text classification, or named entity recognition.

This template provides a set of pipelines and steps to cover the end-to-end process of training and deploying NLP models. Here is an overview of the main components:
The model training step can be configured with parameters like `train_batch_size`, `eval_batch_size`, `num_epochs`, `learning_rate`, and `weight_decay` to control the training process. After training, the model is evaluated, and if it meets the quality criteria, it is registered in the model registry with a unique name.

### Dataset Loading
### Promotion Pipeline

The template includes a step for loading the dataset from the HuggingFace Datasets library. You can choose from three available datasets: financial_news, airline_reviews, and imdb_reviews.
[📂 Code folder](template/steps/promotion/)
<p align="center">
<img height=500 src="assets/nlp_promotion.png">
</p>

### Tokenization
The promotion pipeline is responsible for promoting the best model to the chosen stage, such as Production or Staging. The pipeline can be configured to promote models based on metric comparison or simply promote the latest model version.

The tokenization step preprocesses the dataset by tokenizing the text data using the tokenizer provided by the HuggingFace Models library. You can choose from three available models: bert-base-uncased, roberta-base, and distilbert-base-cased.
The `{{product_name}}_promote_pipeline` function orchestrates the promotion process. If `metric_compare_promotion` is enabled, the pipeline retrieves the metrics for the latest and currently promoted model versions and compares them to decide which model to promote.

### Model Training
### Deployment Pipeline

The training pipeline consists of several steps, including model architecture search, hyperparameter tuning, model training, and model evaluation. The best model architecture and hyperparameters are selected based on the performance on the validation set. The trained model is then evaluated on the holdout set to assess its performance.
[📂 Code folder](template/steps/deployment/)
<p align="center">
<img height=500 src="assets/nlp_deployment.png">
</p>

### Model Registration and Promotion
The deployment pipeline handles the deployment of the model to various environments. It can be configured to deploy locally, to HuggingFace Hub, or to SkyPilot, depending on the project's needs.

After training, the best model version is registered in the ZenML Model Registry. The template provides an option to promote the model version based on a specified metric of interest. If metric-based promotion is enabled, the template compares the metric value of the new model version with the metric value of the current production model version and promotes the new version if it performs better.
The `{{product_name}}_deploy_pipeline` function manages the deployment process, which includes saving the model locally and deploying it to the chosen target. The pipeline can be customized with parameters like `labels`, `title`, `description`, `model_name_or_path`, and others to provide metadata for the deployed model.

### Batch Inference
### 📝 Configuration and Customization

The template includes a batch inference pipeline that loads the inference dataset, preprocesses it using the same tokenizer as during training, and runs predictions using the deployed model version. The predictions are stored as an artifact for future use.
The template can be configured using the provided YAML configuration file. You can specify the project name, version, license, target environment, dataset, model, cloud provider, and other options to tailor the template to your specific requirements.

### Deployment Options
### 📚 Further Reading

The template provides options to deploy the trained model locally or to the HuggingFace Hub. You can choose whether to deploy locally or to the HuggingFace Hub by setting the `deploy_locally` and `deploy_to_huggingface` parameters.
For more information on ZenML and how to use it for NLP projects, check out the [ZenML Documentation](https://docs.zenml.io/). You'll find guides on pipelining, stack management, and best practices for MLOps.

## Next Steps
Happy coding, and may your NLP models be ever accurate and insightful!

Once you have generated the project using this template, you can explore the generated code and customize it to fit your specific NLP use case. The README.md file in the generated project provides further instructions on how to set up and run the project.
## 📩 Contact

Happy coding with ZenML and NLP!
For any questions or support, please reach out to ZenML GmbH at [email protected].
Loading

0 comments on commit 57f9b68

Please sign in to comment.