Deployment Guide: Google Cloud

[michaelbromley]

• Blog post: Serverless e-commerce: Vendure on Google Cloud Run

[michaelbromley]

The following is a discussion from Slack with @martijnvdbrug & @skid (who both have experience deploying Vendure to Google Cloud Run in production):

---

Yasser
Hello!
I’m trying to deploy to Google Cloud Run with little success - the containers are exiting with code 0 almost immediately. I have a few questions that will help me debug, hoping to get some answers to point me to the right direction:

Context:

• I have a docker-compose file which organizes Vendure into server and worker services.
• This results in building 2 images, one for each service.
• Running “docker-compose up” locally works fine - it runs a server, worker and a CloudSQL proxy services locally.
• I build and push the container images to the Google Cloud Container Registry, and there I find the 2 containers to deply to Cloud Run
• However, when I deploy the server and worker (each to a separate Cloud Run service), both fail almost immediately. The logs say “container called exit(0)” without any other details in the logs.

The questions I have are

• Should I deploy the 1) server and 2) worker services to 2 separate Cloud Run instances?
• I assume the server and worker will communicate through the Task Queue (I installed the Google Cloud Tasks plugin). Correct? So no need to be on the same network / same container.
• Cloud Run Logs don’t show detailed errors - I can’t see why the services can’t start. Is there a way around that?
• One of the possible issues that I may be facing is that I’m building locally on a Mac and pushing the built binary to a Linux machine - should I look more into that? possibly use Google Cloud Build instead of building locally?
• If you have a setup that works and can share your docker-compose yaml file for inspiration that’ll help :)

---

@martijnvdbrug:
Hi!

• Yes, I would suggest server and worker in separate cloud run instances OR start both worker and server in the same node process. Cloud Run can only listen to 1 port, so no multiple processes in the same container
• Worker communication is through task queue indeed. Just make sure your worker instance is publicly available (or atleast available for TaskQueue to call)
• Building on a mac shouldn't make a difference. I assume the docker image is still built upon a Linux base? This is the DockerFile I use: https://github.com/Pinelab-studio/shops/blob/master/vendure/Dockerfile
• All stdOut and stdError should go to Google's logging, make sure you select them in Google Clouds UI (see image). If it's not showing, there might be something wrong with your docker image
  

---

@skid

The way we deploy is that we eliminate the idea of a "worker" completely. Workers are just normal instances - if you dig in Vendure's codebase, you'll see that the only difference is that the bootstrapWorker code sets the server context to 'worker' and it doesn't listen to a socket. Since the task queue is handled via google cloud tasks using Martijn's plugin, so whenever you have a task it just pings a URL on a regular non-worker instance.
We don't use docker-compose either. For local dev we just run everything locally, and, as I said, the deployment consists of one single-process docker image.
Building the docker image on mac shouldn't make a difference.

When I think about it - I'm not sure how does vendure-plugin-google-cloud-tasks work with a Vendure worker at all, given that the worker doesn't listen to incoming connections.
@martijn?

---

@martijnvdbrug
Hmm, that's a very good point. I also use a normal Vendure instance as worker, because we need the NestJS controllers to listen for incoming requests indeed.
I deploy the same Docker image to server.run.app
and worker.run.app, but start the worker process in one of them based on an ENV var: https://github.com/Pinelab-studio/shops/blob/master/vendure/src/index.ts

---

Yasser
@martijn
where do you see the Cloud Run logging options? The options in the screenshot attached? I don’t see them anywhere (not in the main Cloud Run services page, service details page, logs page..etc)
Having said that - I think I found the main issue (thanks to
@martijn's Dockerfile!), I was missing the RUN command to start the server in the Dockerfile. It was in docker-compose.yml, but that wasn’t picked up via docker-compose for some reason.

@DiscoDancer your point about Worker not listening to requests added to my confusions yesterday as I was trying to deploy - exactly, if the worker instance doesn’t listen to requests, why deploy it as a standalone service? aren’t requests going from the main server instance to Google Cloud Task public endpoint?

Note that there are reported issues with docker builds from Macbook’s with the M1 chip reported, so the flag --platform linux/amd64 may be needed when running docker build commands.

---

@Yasser - Vendure was not designed with an essentially serverless deployment such as Cloud Run in mind. See this issue: #1799, though it takes very little effort to make it run on Cloud Run.
The worker is intended to communicate with the server process via some kind of a task queue. The default implementation is a database polling mechanism, and there's a message-queue implementation - but none of these ping the worker on a http endpoint.

Google cloud tasks, however, works by making a http request sending the task payload, so for this reason "workers" must have a http server running which, if they do, makes them exactly the same as the main process.
On cloud run, moreover, you shouldn't run two processes in parallel because Cloud Run by default throttles down the CPU almost to zero if its not busy serving a request, so behaviour is unpredictable. This can be disabled by using the "CPU is always allocated" - but that kind of defeats the purpose of serverless and makes it 50% more expensive - hence the issue that Martijn opened.
For a Cloud Run deployment I suggest that you don't start any workers - just run a regular server and use the cloud tasks plugin. Scaling, deployment, monitoring, restarting and all that is handled automatically for you by GCR. (edited)

[martijnvdbrug]

@yasserlens For connecting to cloud SQL instances from Cloud Run, you can also use sockets:https://cloud.google.com/sql/docs/mysql/connect-run

[martijnvdbrug]

Another addition from slack:

• You do need to explicitly start the worker with await app.get(JobQueueService).start(); to also have your main instance function as worker instance.
• We ping the server every minute between 6:00 and 23:00 to prevent cold starts, This is a lot less expensive then setting CPU always on in Cloud Run, because it will still throttle CPU after the ping request, and you are not billed for those after-request CPU cycles

[yasserlens]

Thanks Martijn!

Re: connecting to DB - I already pass the --add-cloudsql-instances flag with the DB "instance connection name" with the "gcloud run deploy" command and I can verify that the instance is indeed connected to the CloudSQL instance in the Console. I'm not 100% sure if the application is in fact talking to DB via that connection, or the IP host provided in vendure-config.ts. I need to verify that.

Re: worker: Thanks for clarifying, I see you have this tip in your blog post about deployment to GCR. You also mention it's better to run this as a separate instance, which I intend to do. How would the server and worker communicate as separate instances on Cloud Run though? Via Cloud Tasks?

Re: cold starts: Thanks for the tip! Our customers are spread around the world, so i guess I'll have to ping every minute of every hour of the day to achieve this. Depending on performance I may have to run an instance for each region (US/EU/AU).

[martijnvdbrug]

I already pass the --add-cloudsql-instances flag with the DB "instance connection name" with the "gcloud run deploy" command and I can verify that the instance is indeed connected to the CloudSQL instance in the Console. I'm not 100% sure if the application is in fact talking to DB via that connection, or the IP host provided in vendure-config.ts. I need to verify that.

You can check it by removing the IP and passwords from the config. It's probably using the socket already.

Thanks for clarifying, I see you have this tip in your blog post about deployment to GCR. You also mention it's better to run this as a separate instance, which I intend to do. How would the server and worker communicate as separate instances on Cloud Run though? Via Cloud Tasks?

Yes, explicitly via Cloud Tasks. (And they use the same DB)

Thanks for the tip! Our customers are spread around the world, so i guess I'll have to ping every minute of every hour of the day to achieve this. Depending on performance I may have to run an instance for each region (US/EU/AU).

I think (no data to prove this) it's more beneficial to have your Cloud Run instance in the same region as the DB, because there is far more communication between the Cloud Run and the DB, then there is communication between your Client and Cloud run.

Maybe you can use some framework on the storefront to make it more international proof? Or something with CDN's or Edge computing.. (maybe @michaelbromley has some suggestions on storefront frameworks for international usage

[yasserlens]

I already pass the --add-cloudsql-instances flag with the DB "instance connection name" with the "gcloud run deploy" command and I can verify that the instance is indeed connected to the CloudSQL instance in the Console. I'm not 100% sure if the application is in fact talking to DB via that connection, or the IP host provided in vendure-config.ts. I need to verify that.

You can check it by removing the IP and passwords from the config. It's probably using the socket already.

Thanks for clarifying, I see you have this tip in your blog post about deployment to GCR. You also mention it's better to run this as a separate instance, which I intend to do. How would the server and worker communicate as separate instances on Cloud Run though? Via Cloud Tasks?

Yes, explicitly via Cloud Tasks. (And they use the same DB)

Thanks for the tip! Our customers are spread around the world, so i guess I'll have to ping every minute of every hour of the day to achieve this. Depending on performance I may have to run an instance for each region (US/EU/AU).

I think (no data to prove this) it's more beneficial to have your Cloud Run instance in the same region as the DB, because there is far more communication between the Cloud Run and the DB, then there is communication between your Client and Cloud run.

Maybe you can use some framework on the storefront to make it more international proof? Or something with CDN's or Edge computing.. (maybe @michaelbromley has some suggestions on storefront frameworks for international usage

Thanks! Our storefront is SvelteKit based, with static pages like product listings and product detail pages pre-rendered and the rest either served via SSR or CSR. I read that the demo Remix storefront was deployed on the edge via cloudflare, so we plan to do the same thing but with Netlify Edge which has SvelteKit support.