Skip to content

Commit

Permalink
doc updates to container runtime quickstarts (#1613)
Browse files Browse the repository at this point in the history
  • Loading branch information
sfc-gh-chammond authored Sep 30, 2024
1 parent 28b291d commit c2391da
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 15 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@ tags: Getting Started, Data Science
## Overview
Duration: 1

You can run Snowflake Notebooks on Snowpark Container Services through Container Runtime. Snowpark Container Services gives you a flexible container infrastructure that supports building and operationalizing a wide variety of workflows entirely within Snowflake. Container Runtime provides software and hardware options to support advanced data science and machine learning workloads on Snowpark Container Services. Compared to Virtual warehouses, Container Runtime provides a more flexible compute environment where you can install packages from multiple sources and select compute resources, including GPU machine types, while still running SQL queries on warehouses for optimal performance.
Snowflake Notebooks in the Container Runtime are a powerful IDE option for building ML workloads at scale. Container Runtime (Public Preview) gives you a flexible container infrastructure that supports building and operationalizing a wide variety of resource-intensive ML workflows entirely within Snowflake. Using Snowflake Notebooks in Container Runtime gives you access to distributed processing on both CPUs and GPUs, optimized data loading from Snowflake, automatic lineage capture and Model Registry integration. Container Runtime also provides flexibility to leverage a set of preinstalled packages or the ability to pip install any open-source package of choice.

This Quickstart will take you through the steps of running Snowflake Notebooks with Container Runtime. We will install packages, train a model using pre-installed packages, and view logs.
This introductory Quickstart will take you through the steps of running Snowflake Notebooks with Container Runtime. We will install packages, train a model using pre-installed packages, and view logs.

### Prerequisites
- Access to a Snowflake account with Accountadmin.
- Access to a Snowflake account with Accountadmin
- Access to run Notebooks in Snowflake
- Foundational knowledge of Data Science workflows

Expand All @@ -28,7 +28,7 @@ This Quickstart will take you through the steps of running Snowflake Notebooks w
- A [Snowflake](https://app.snowflake.com/) Account

### What You’ll Build
- A Snowflake Notebook that runs on Snowpark Container Services
- A Snowflake Notebook that runs on scalable CPUs or GPUs using any Python package of choice

<!-- ------------------------ -->
## Setup Your Account
Expand Down Expand Up @@ -141,13 +141,17 @@ Duration: 15
## Conclusion And Resources
Duration: 1

In conclusion, running Snowflake Notebooks on Snowpark Container Services through Container Runtime offers a robust and flexible infrastructure for managing advanced data science and machine learning workflows directly within Snowflake. With the ability to install external packages and choose optimal compute resources, including GPU machine types, Container Runtime provides a more versatile environment compared to Virtual Warehouses.
In conclusion, running Snowflake Notebooks Container Runtime offers a robust and flexible infrastructure for managing large-scale, advanced data science and machine learning workflows directly within Snowflake. With the ability to install external packages and choose optimal compute resources, including GPU machine types, Container Runtime provides a more versatile environment suited to the needs of data science and ML teams.

Ready to get started? Follow this Quickstart to begin running Snowflake Notebooks with Container Runtime, install essential packages, train your models, and monitor your logs effectively.
Ready for more? After you complete this quickstart, you can try [building an XGBoost model with GPUs in Snowflake Notebooks](https://quickstarts.snowflake.com/guide/train-an-xgboost-model-with-gpus-using-snowflake-notebooks/index.html#0).

### What You Learned
- The key features of Snowflake Notebooks with Container Runtime

### Related Quickstarts
- [Train an XGBoost model with GPUs in Snowflake Notebooks](https://quickstarts.snowflake.com/guide/train-an-xgboost-model-with-gpus-using-snowflake-notebooks/index.html#0)

### Related Resources
- [Snowflake ML Webpage](https://www.snowflake.com/en/data-cloud/snowflake-ml/)
- [Documentation](https://docs.snowflake.com/LIMITEDACCESS/snowsight-notebooks/ui-snowsight-notebooks-runtime)
- [YouTube Tutorials](https://www.youtube.com/playlist?list=PLavJpcg8cl1Efw8x_fBKmfA2AMwjUaeBI)
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,20 @@ tags: Getting Started, Data Science
## Overview
Duration: 1

In this quickstart, we'll explore how to harness the power of [Snowpark Container Services (SPCS)](https://docs.snowflake.com/en/LIMITEDACCESS/snowsight-notebooks/ui-snowsight-notebooks-runtime) using GPUs to execute a [notebook](https://docs.snowflake.com/en/user-guide/ui-snowsight/notebooks) directly within Snowflake. Specifically, we'll train an XGBoost model and walk through a workflow that involves inspecting GPU resources, loading data from a Snowflake table, and setting up that data for modeling. In the notebook, we will train two XGBoost models—one on CPUs and the other using a GPU cluster—and then compare their runtimes and results.
In this quickstart, we'll explore how to easily harness the power of containers to run ML workloads at scale using CPUs or GPUs from [Snowflake Notebooks](https://docs.snowflake.com/en/user-guide/ui-snowsight/notebooks) in the Container Runtime (Public Preview). Specifically, we'll train an XGBoost model and walk through a workflow that involves inspecting GPU resources, loading data from a Snowflake table, and setting up that data for modeling. In the notebook, we will train two XGBoost models—one on CPUs and the other using a GPU cluster—and then compare their runtimes and results.

This exercise will highlight the efficiency gains possible with GPUs, the flexibility SPCS offers by enabling notebook execution on containers, and the added advantage of integrating third-party Python libraries to tap into the broader open-source ecosystem.
This exercise will illustrate how Snowflake Notebooks lets you quickly tap into the CPU or GPU compute power you need to scalably build ML models using any open-source Python framework of choice.

### Prerequisites
- Access to a Snowflake account with Accountadmin.
- Access to run Notebooks in Snowflake
- Foundational knowledge of Data Science workflows
- For an intro to Snowflake Notebooks on Container Runtime, try this [quickstart](https://quickstarts.snowflake.com/guide/notebook-container-runtime/index.html#0) first

### What You Will Learn
- How to use SPCS to run notebook workloads that execute on containers, rather than virtual warehouses in Snowflake
- GPUs can greatly speed up model training jobs
- How to bring in third party python libraries to leverage great contirbutions to the OSS ecosystem
- Use Snowflake Notebooks with GPUs to speed up model training jobs with distributed processing
- Build using a set of pre-installed ML packages or pip install any of your favorite open-source package
- Run ML workloads at scale without any data movement

### What You’ll Need
- A [Snowflake](https://app.snowflake.com/) Account
Expand Down Expand Up @@ -145,15 +146,15 @@ Duration: 1

### Conclusion

In this quickstart, we demonstrated the power of Snowpark Container Services (SPCS) in leveraging GPUs to efficiently train an XGBoost model directly within a Snowflake notebook. By walking through the process of inspecting GPU resources, loading data from a Snowflake table, and setting up that data for modeling, we successfully trained and compared two XGBoost models—one using CPUs and the other on a GPU cluster. The results underscored the significant efficiency gains GPUs can offer, along with the flexibility SPCS provides by enabling containerized notebook execution and the ability to integrate third-party Python libraries seamlessly.
In this quickstart, we demonstrated how to useSnowflake Notebooks in the Container Runtime to efficiently train an XGBoost model with GPUs . By walking through the process of inspecting GPU resources, loading data from a Snowflake table, and setting up that data for modeling, we successfully trained and compared two XGBoost models—one using CPUs and the other on a GPU cluster. The results underscored the significant efficiency gains GPUs can offer, along with the flexibility Container Runtime provides by enabling containerized notebook execution and the ability to integrate third-party Python libraries seamlessly.

Ready to accelerate your ML workflows? Dive deeper into Snowpark Container Services and start leveraging GPUs for faster, more flexible model training!
Ready to accelerate your ML workflows? Dive deeper into Container Runtime and start leveraging GPUs for faster, more flexible model training!

### What You Learned
- How to use SPCS to run notebook workloads that execute on containers, rather than virtual warehouses in Snowflake
- How to use Container Runtime to run ML workloads directly from Snowflake Notebooks
- GPUs can greatly speed up model training jobs
- How to bring in third party python libraries to leverage great contirbutions to the OSS ecosystem

### Related Resources
- [Snowflake Notebooks](https://docs.snowflake.com/en/user-guide/ui-snowsight/notebooks)
- [Snowpark Services Container Runtime](https://docs.snowflake.com/en/LIMITEDACCESS/snowsight-notebooks/ui-snowsight-notebooks-runtime)
- [Container Runtime](https://docs.snowflake.com/en/LIMITEDACCESS/snowsight-notebooks/ui-snowsight-notebooks-runtime)

0 comments on commit c2391da

Please sign in to comment.