diff --git a/colabs/huggingface/Huggingface_wandb.ipynb b/colabs/huggingface/Huggingface_wandb.ipynb index 4ff85657..a0d59a3c 100644 --- a/colabs/huggingface/Huggingface_wandb.ipynb +++ b/colabs/huggingface/Huggingface_wandb.ipynb @@ -1,7 +1,6 @@ { "cells": [ { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -10,18 +9,16 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# Hugging Face + W&B\n", "Visualize your [Hugging Face](https://github.com/huggingface/transformers) model's performance quickly with a seamless [W&B](https://wandb.ai/site) integration.\n", "\n", - "Compare hyperparameters, output metrics, and system stats like GPU utilization across your models. \n" + "Compare hyperparameters, output metrics, and system stats like GPU utilization across your models.\n" ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -31,7 +28,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -52,7 +48,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -63,7 +58,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -71,7 +65,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -104,7 +97,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -112,7 +104,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -131,7 +122,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -144,12 +134,11 @@ "metadata": {}, "outputs": [], "source": [ - "# Optional: log both gradients and parameters\n", - "%env WANDB_WATCH=all" + "# Optional: log model checkpoints as W&B Artifacts\n", + "%env WANDB_LOG_MODEL=\"checkpoint\"" ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -164,6 +153,7 @@ "outputs": [], "source": [ "%env WANDB_PROJECT=huggingface-demo\n", + "%env WANDB_LOG_MODEL=checkpoint\n", "%env TASK_NAME=MRPC\n", "\n", "!python run_glue.py \\\n", @@ -177,11 +167,13 @@ " --num_train_epochs 3 \\\n", " --output_dir /tmp/$TASK_NAME/ \\\n", " --overwrite_output_dir \\\n", - " --logging_steps 50" + " --logging_steps 50 \\\n", + " --save_steps 100 \\\n", + " --eval_steps 100 \\\n", + " --report_to \"wandb\"" ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -199,12 +191,37 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ + "## Model Artifacts\n", "\n", - "### 📈 Track key information effortlessly by default\n", + "See an example model checkpoint and its in the UI [here](https://wandb.ai/wandb/arttest/artifacts/model/iv3_trained/5334ab69740f9dda4fed/lineage?_gl=1*yyql5q*_ga*MTQxOTYyNzExOS4xNjg0NDYyNzk1*_ga_JH1SJHJQXJ*MTY5MjMwNzI2Mi4yNjkuMS4xNjkyMzA5NjM2LjM3LjAuMA..) These artifacts store all the metadata associated with this model, the W&B runs consuming it, and the whole lineage of upstream and downstream artifacts!\n", + "\n", + "[Here's](https://wandb.ai/parambharat/huggingface-demo/runs/szfbcsd7/artifacts?workspace=) an example of the artifacts generated from the above run\n", + "\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Model Registry\n", + "After logging a bunch of checkpoints across multiple runs during experimentation, now comes time to hand-off the best checkpoint to the next stage of the workflow (e.g. testing, deployment).\n", + "\n", + "The Model Registry is a central page that lives above individual W&B projects. It houses **Registered Models**, portfolios that store \"links\" to the valuable checkpoints living in individual W&B Projects.\n", + "\n", + "The model registry offers a centralized place to house the best checkpoints for all your model tasks. Any `model` artifact you log can be \"linked\" to a Registered Model. Pleaser refer to [this](https://docs.wandb.ai/guides/models) guide where you can find a [walkthrough](https://docs.wandb.ai/guides/models/walkthrough) of the Model Registry to learn more." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "## 📈 Track key information effortlessly by default\n", "Weights & Biases saves a new run for each experiment. Here's the information that gets saved by default:\n", "- **Hyperparameters**: Settings for your model are saved in Config\n", "- **Model Metrics**: Time series data of metrics streaming in are saved in Log\n", @@ -213,21 +230,19 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## 🤓 Learn more!\n", "- [Documentation](https://docs.wandb.com/huggingface): docs on the Weights & Biases and Hugging Face integration\n", "- [Videos](http://wandb.me/youtube): tutorials, interviews with practitioners, and more on our YouTube channel\n", - "- Contact: Message us at contact@wandb.com with questions " + "- Contact: Message us at contact@wandb.com with questions" ] } ], "metadata": { "accelerator": "GPU", "colab": { - "include_colab_link": true, "provenance": [], "toc_visible": true }, @@ -237,5 +252,5 @@ } }, "nbformat": 4, - "nbformat_minor": 4 + "nbformat_minor": 0 } diff --git a/colabs/huggingface/Optimize_Hugging_Face_models_with_Weights_&_Biases.ipynb b/colabs/huggingface/Optimize_Hugging_Face_models_with_Weights_&_Biases.ipynb index e950b3d4..58894e1e 100644 --- a/colabs/huggingface/Optimize_Hugging_Face_models_with_Weights_&_Biases.ipynb +++ b/colabs/huggingface/Optimize_Hugging_Face_models_with_Weights_&_Biases.ipynb @@ -143,7 +143,7 @@ "metadata": {}, "outputs": [], "source": [ - "%env WANDB_LOG_MODEL=true" + "%env WANDB_LOG_MODEL=\"checkpoint\"" ] }, { @@ -742,7 +742,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Once you're happy with a model, don't forget to [share it with the word](https://huggingface.co/transformers/model_sharing.html) on the Model Hub!" + "## Model Registry\n", + "After logging a bunch of checkpoints across multiple runs during experimentation, now comes time to hand-off the best checkpoint to the next stage of the workflow (e.g. testing, deployment).\n", + "\n", + "The Model Registry is a central page that lives above individual W&B projects. It houses **Registered Models**, portfolios that store \"links\" to the valuable checkpoints living in individual W&B Projects.\n", + "\n", + "The model registry offers a centralized place to house the best checkpoints for all your model tasks. Any `model` artifact you log can be \"linked\" to a Registered Model. Pleaser refer to [this](https://docs.wandb.ai/guides/models) guide where you can find a [walkthrough](https://docs.wandb.ai/guides/models/walkthrough) of the Model Registry to learn more." ] }, { @@ -761,7 +766,7 @@ "source": [ "## ❓ Questions about W&B\n", "\n", - "If you have any questions about using W&B to track your model performance and predictions, please reach out to the [slack community](http://bit.ly/wandb-forum)." + "If you have any questions about using W&B to track your model performance and predictions, please reach out to the [discord community](https://wandb.me/discord)." ] } ], diff --git a/colabs/huggingface/Visualize_your_Hugging_Face_data_with_Weights_&_Biases.ipynb b/colabs/huggingface/Visualize_your_Hugging_Face_data_with_Weights_&_Biases.ipynb index 6be6becf..495ab6ab 100644 --- a/colabs/huggingface/Visualize_your_Hugging_Face_data_with_Weights_&_Biases.ipynb +++ b/colabs/huggingface/Visualize_your_Hugging_Face_data_with_Weights_&_Biases.ipynb @@ -419,8 +419,8 @@ "metadata": {}, "outputs": [], "source": [ - "# automatically log model to W&B at the end\n", - "%env WANDB_LOG_MODEL=true" + "# automatically log model checkpoints to W&B\n", + "%env WANDB_LOG_MODEL=\"checkpoint\"" ] }, { @@ -537,6 +537,37 @@ "source": [ "wandb.finish()" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Model Registry\n", + "After logging a bunch of checkpoints across multiple runs during experimentation, now comes time to hand-off the best checkpoint to the next stage of the workflow (e.g. testing, deployment).\n", + "\n", + "The Model Registry is a central page that lives above individual W&B projects. It houses **Registered Models**, portfolios that store \"links\" to the valuable checkpoints living in individual W&B Projects.\n", + "\n", + "The model registry offers a centralized place to house the best checkpoints for all your model tasks. Any `model` artifact you log can be \"linked\" to a Registered Model. Pleaser refer to [this](https://docs.wandb.ai/guides/models) guide where you can find a [walkthrough](https://docs.wandb.ai/guides/models/walkthrough) of the Model Registry to learn more." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 📚 Resources\n", + "\n", + "* [Hugging Face and W&B integration documentation](https://docs.wandb.ai/integrations/huggingface) contains a few tips for taking most advantage of W&B\n", + "* [🤗 Transformers documentation](https://huggingface.co/transformers/) is extremely thorough and full of examples" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## ❓ Questions about W&B\n", + "\n", + "If you have any questions about using W&B to track your model performance and predictions, please reach out to the [discord community](https://wandb.me/discord)." + ] } ], "metadata": {