Gantry (Container Crane)

Gantry is a pipeline management tool using containers for all relevant steps. It supports a basic docker-compose subset allowing docker-compose like deployments with wharfer. If wharfer is not installed docker will be used directly.

Services and Steps

Services define docker containers which provide a continued service to other tasks in and outside of the pipeline. They directly resemble the service concept from docker-compose and once started run concurrently to the rest of the pipeline including anything depending on the service.

Steps on the other hand run to completion and only then are their dependents executed. Note however, that independent steps are executed concurrently with each other. Steps are often used for tasks that produce a result that is needed by their dependents such as a download, creation of a database index or training of a machine learning model.

The End-to-End tests of QLever example demonstrates the usage and interaction of both container types.

Installation

Download a prebuild Release

Binary releases are provided as github releases.

cd /tmp
rm gantry_$(uname -m).tar.bz2
wget https://github.com/ad-freiburg/gantry/releases/download/v0.5.0/gantry_$(uname -m).tar.bz2
tar -xavf gantry_$(uname -m).tar.bz2
sudo mv gantry_$(uname -m)/gantry /usr/local/bin/gantry

gantry --version

Arch Linux

Gantry is available in the AUR as gantry and gantry-git

From source

To install gantry into the users ~/go/bin path it is enough to just run

go install -ldflags="-X github.com/ad-freiburg/gantry.Version=$(git describe --always --long --dirty)" ./...

To install gantry globally copy it from ~/go/bin path or use

go build -ldflags="-X github.com/ad-freiburg/gantry.Version=$(git describe --always --long --dirty)" cmd/gantry/gantry.go
sudo mv gantry /usr/local/bin/

gantry --version

Using go get

Gantry is go getable through

go get github.com/ad-freiburg/gantry/cmd/gantry

This will result in a binary without a set version.

Building a (new) Release

To build a release version first make sure everything works. This includes checking the codestyle

gofmt -d .

and running the tests

go test ./...

The complete test coverage can be viewed with

go test -coverprofile=coverage.out ./... && go tool cover -html=coverage.out

When all is correct edit the Download a prebuild release section of this Readme so the download link points to the future version. Only after committing this final change tag the release

git tag -a vX.Y.Z -m <message>

Then build with -ldflags such that the version is added to the binary

go build -ldflags="-X github.com/ad-freiburg/gantry.Version=$(git describe --always --long --dirty)" cmd/gantry/gantry.go

Finally use the GitHub Releases mechanism to release a new version.