Specific instructions to run a GCE cluster

NOTE: GCE VM lifecycle management has been deprecated in favor of Google Kubernetes Engine. Please refer to these instructions https://github.com/HSF/harvester/wiki/GKE-setup

1. Setup your PanDA queue in AGIS. You can take CERN-EXTENSION_GOOGLE_HARVESTER as example.
2. Setup the standard Harvester server following the general instructions in this Wiki
3. Install and configure the NGINX side service. The ports specified in the ngingx.conf template, need to be accessible from GCE.
4. Install the GCE python API following Google's documentation
5. Download the CernVM4 image for GCE and upload it to the image repository in GCE account. It's important to take version 4 (not version 3), since the contextualization mechanisms changed.
6. Create a service account under the "IAM & admin" tab in the GCE console. From our experience, the best Role is "Compute Admin", but roles evolve with time and you might find more suitable Roles. Download the service account json file and place it on the harvester server. Anyone with this json can start/stop VMs with your account, so keep the file safe.
7. Edit and place the user data file on the harvester machine
8. Be sure the GCE block in the harvester configuration is correct and all the files specified are in place.

[googlecloud]

# zone where you are booting up your VMs and storage, e.g. us-east1-b
zone = <the GCE zone your VMs will run on, e.g. us-east1-b>
# project defined in the google compute account, where the activity will be billed
project = <your GCE project ID>
# private service account json generated in the google cloud management console
service_account_file = /path/to/service_file.json
# file with the user data to send to CERN VM
user_data_file = /path/to/user_data.txt
# image to use
image = <URL to the CernVM4 image in your project>
# harvester frontend
harvester_frontend = your_server_name:nginx_listening_port

8. Add your queue configuration to the harvester queueconfig. One example for a cloud queue can be found here. Pay attention to the submitter parameters ram_per_core, disk_size and preemptible, which are optional and can be used to tune your VMs.

...
        "submitter": {
			"name": "GoogleSubmitter",
			"module": "pandaharvester.harvestersubmitter.cloud_google_submitter",
			"logBaseURL": "https://aipanda011.cern.ch:25443/cache/",
                        "ram_per_core": 4,
                        "disk_size": 100,
			"preemptible": true
		},
...