Skip to content

Latest commit

 

History

History
301 lines (205 loc) · 8.82 KB

google-cloud.md

File metadata and controls

301 lines (205 loc) · 8.82 KB
Raw

Google Cloud

Pegasus can be deployed to Google Cloud Run using containers. This feature is in beta and Celery is not yet supported.

To build for Google Cloud, choose the "google_cloud" option when installing Pegasus. Then follow the steps below to deploy your app.

Prerequisites

Pegasus deployments on Google Cloud Run loosely follow the Django on Cloud Run guide. However, instaed of running it in the Google Cloud Shell, we'll use the gcloud CLI on our development machine. This allows you to more easily work with your existing codebase.

1. Set up the gcloud sdk.

Though much of the setup can be completed in the cloud shell, you will need to install and configure the gcloud sdk on your development machine. Follow the google installation guide for your OS.

After you install the gcloud SDK you will also need to authenticate your account as described here. In particular, you will need to run:

gcloud auth login
gcloud init

2. Create and connect your project.

Follow the instructions in Step 2 of the Google guide to create a new project for your app and enable billing.

Once you've created your project you should be able to run:

gcloud projects list

On your development machine and see it.

Next run:

gcloud config set project <project_id>

To set the project as the default for future commands.

3. Enable the Cloud APIs

As per step 3 of the Google guide, run the following command to enable the services needed for your application.

gcloud services enable \
  run.googleapis.com \
  sql-component.googleapis.com \
  sqladmin.googleapis.com \
  compute.googleapis.com \
  cloudbuild.googleapis.com \
  secretmanager.googleapis.com \
  artifactregistry.googleapis.com

This command takes a while to finish, but should eventually output something like:

Operation "operations/acf.cc11852d-40af-47ad-9d59-477a12847c9e" finished successfully.

We will skip step 4 of Google's guide, since we already have a project and move on to step 5.

4. Create the backing services.

This section follows step 5 of Google's guide, with some minor changes.

First create a service account:

gcloud iam service-accounts create cloudrun-serviceaccount

Set up your environment variables:

Everything you will need is defined in /deploy/.env.google.example. It is recommended that you first copy this file to .env.google:

cp deploy/.env.google.example deploy/.env.google

Then update the values in it as needed---including setting any passwords and keys to random, autogenerated values. Once you have made all the modifications to this file, load it into your environment by running:

source deploy/.env.google

After running the above command you should be able to run the remaining commands in the same shell and the variables will be swapped in.

gcloud artifacts repositories create containers --repository-format docker --location $REGION

Create and configure the database:

Next, create the database instance. Note that in our .env.google file we are changing the instance name from the guide's mysinstance to something unique to your project, and storing that as an environment variable.

Note: for production you may want to use a higher value for --tier, however do be aware of the costs associated with doing this. More information here.

gcloud sql instances create $DATABASE_INSTANCE_NAME --project $PROJECT_ID --database-version POSTGRES_14 --tier db-f1-micro --region $REGION

This command takes a long time to run. Once it completes, create a database in the instance, again using a unique name instead of the guide's mydatabase:

gcloud sql databases create $DATABASE_NAME --instance $DATABASE_INSTANCE_NAME

Next, create the database user. Make sure you have set/changed DATABASE_PASSWORD in your deploy/.env.google file, (and run source deploy/.env.google again if necessary) then run:

gcloud sql users create ${DATABASE_USER} --instance ${DATABASE_INSTANCE_NAME} --password ${DATABASE_PASSWORD}

And finally, grant the service account permission to access the DB:

gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:${SERVICE_ACCOUNT} --role roles/cloudsql.client

Create and configure Redis:

If you are using tasks, caching, or websockets you will also need to set up Redis. To enable Redis support you first have to create a Redis instance.

gcloud redis instances create ${PROJECT_ID}-redis --size=1 --region=${REGION}

You might get prompted to enable the Redis API which you should say "yes" to. Then this will run for a long time.

After creating your Redis instance you will need to run source deploy/.env.google again to populate the necessary environment variables for the IP and network. You can also view these by running:

gcloud redis instances describe ${PROJECT_ID}-redis --region ${REGION}

Create and configure the storage bucket:

gcloud storage buckets create gs://${GS_BUCKET_NAME} --location ${REGION}

gcloud storage buckets add-iam-policy-binding gs://${GS_BUCKET_NAME} \
    --member serviceAccount:${SERVICE_ACCOUNT} \
    --role roles/storage.admin

Store configuration as a secret.

Once you have completed the above you should have everything you need in place to run your app. The final step before deploying is to save your configuration as a secret.

First create the .env.production file from .env.production.example:

cp .env.production.example .env.production

Then update the DATABASE_URL, GS_BUCKET_NAME, and REDIS_URL values in .env.production. You can find these values with the echo command, e.g.:

echo $DATABASE_URL

For Redis you will need to put in the value of REDIS_IP which you can get from:

echo $REDIS_IP

Next, save these values in google cloud:

gcloud secrets create application_settings --data-file .env.production

And allow the service account to access it:

gcloud secrets add-iam-policy-binding application_settings --member serviceAccount:${SERVICE_ACCOUNT} --role roles/secretmanager.secretAccessor

Create and deploy your docker containers

From here on out we'll stop using the guide, since Pegasus should handle everything else for you out of the box. Also, instead of using a Procfile we'll use our own Docker container.

First make sure you have loaded your environment:

set -o allexport && source deploy/.env.google && set +o allexport

Now you can build your container for Google Cloud by running the following command.

make gcp-build

This, and all other make commands are defined in the Makefile in your project. You can see what they are doing there.

Once you've built your container, enable docker pushes with:

gcloud auth configure-docker

And then you can push it with:

make gcp-push

And finally deploy it with:

make gcp-deploy

This should deploy your application to a new container. It should output the URL for your app, which is now online!

Future deploys can be done in the same manner. Or you can use the following command as a shortcut to run everything:

make gcp-full-deploy

Settings and Secrets

You can use Google Secret Manager to add additional settings and secrets by adding them to .env.production and uploading it to Secret Manager using:

gcloud secrets versions add application_settings --data-file .env.production

See settings_production.py for examples of using these secrets in your settings file.

Cookbooks

Automating Deployment to Cloud Run using GitHub Actions

If you would like to automate your Google Cloud deployment so it is deployed on every push to Github, you can refer to this community guide: Automating Deployment to Cloud Run using GitHub Actions