Releasing a Build

In general we release from the master branch. Release involves running build, tests, creating a docker container, and then publishing the docker container.

Optionally, but more than likely, the followup to release is to update our demo deployments.

Step 1 - Release Environment

In order to ensure we don't pick up any uncommitted changes, the first step is to clone the repository to a new location, and work within that clone.

git clone https://github.com/GoogleCloudPlatform/datalab.git release
cd release
source tools/initenv.sh

Step 2 - Creating a Build

cd $REPO_DIR/sources
gradle build

cd $REPO_DIR/docker/ipython
./build.sh latest

TODO:

1. Integrate the docker build as another gradle build target, so that builds for release just become gradle release?

Step 3 - Tests

Unit Tests

cd $REPO_DIR/sources/sdk/pygcp/tests
python main.py

Functional Tests

These tests create a derived docker container for local runs. Ensure that gcloud auth and the correct project are specified in gcloud configuration, as these will be packaged into the docker image built for local runs (and used for authorization and notebook storage).

The steps below start the docker container assuming boot2docker is up and running.

cd $REPO_DIR/sources/docker/ipythonlocal
./build.sh

cd $REPO_DIR/deploy/ipython
./local.sh

To connect to the server (using an ssh tunnel) running inside the docker container:

boot2docker ssh -L 8080:localhost:8080

and then browse to http://localhost:8080 to run through the UI and open/execute specific notebooks.

TODO:

1. Add tests for other areas of the product (eg. running an ipython notebook in functional test manner) as well as from within a docker container.
2. Automate some of this into something like gradle test?

Step 4 - Publishing

Switch to the cloud-datalab project. This contains the docker registry you want to publish to. You'll publish to the same tag as the one you picked when building ("latest" in the step above).

gcloud config set project cloud-datalab
cd $REPO_DIR/sources/docker/ipython
./publish.sh latest

If there are any sample/intro content changes, you'll want to publish those as well... note that these updates replace existing content without any versioning semantics (currently).

cd $REPO_DIR/content
./publish.sh

Step 5 - Final Verification

Switch to another project, the one that will contain the VM, and run the deployment script, and use the tag ("latest" from above).

gcloud config set project <some-test-project>
cd $REPO_DIR/deploy/ipython
./vm.sh latest <vm-name>

This should open up a browser navigating to the local port opened via an SSH tunnel to the resulting VM.