Merge pull request #59 from shizunge/readme

Add a webhook example

.github/workflows/on-pull-request.yml

@@ -4,6 +4,10 @@ on:
   pull_request:
     branches:
       - 'main'
+    paths:
+      - 'Dockerfile'
+      - 'src/*.sh'
+      - 'tests/*.sh'
   workflow_dispatch:
 
 jobs:

README.md

@@ -23,7 +23,7 @@ docker service create \
   shizunge/gantry
 ```
 
-Or with docker compose, see the [example](examples/README.md).
+The [examples folder](examples/README.md) contains example docker compose files, and more methods to launch *Gantry*, like [at a specific time](examples/cronjob) and [via webhook](examples/webhook).
 
 You can also run *Gantry* as a script directly on the host outside the container
 ```
@@ -36,14 +36,14 @@ You can also run *Gantry* as a script directly on the host outside the container
 
 You can configure the most behaviors of *Gantry* via environment variables.
 
-### Common ones
+### Common
 
 | Environment Variable  | Default |Description |
 |-----------------------|---------|------------|
 | GANTRY_LOG_LEVEL      | INFO | Control how many logs generated by *Gantry*. Valid values are `NONE`, `ERROR`, `WARN`, `INFO`, `DEBUG`. |
 | GANTRY_NODE_NAME      |      | Add node name to logs. If not set, *Gantry* will use the host name of the Docker Swarm's manager, which is read from either the Docker daemon socket of current node or `DOCKER_HOST`. |
-| GANTRY_POST_RUN_CMD   |      | Command(s) to `eval` after each updating iteration. |
-| GANTRY_PRE_RUN_CMD    |      | Command(s) to `eval` before each updating iteration. |
+| GANTRY_POST_RUN_CMD   |      | Command(s) to `eval` after each updating iteration. For [example](examples/prune-and-watchtower), you can use this to remove unused containers, networks and images and update standalone docker containers. |
+| GANTRY_PRE_RUN_CMD    |      | Command(s) to `eval` before each updating iteration. For [example](examples/prune-and-watchtower), you can use this to remove unused containers, networks and images and update standalone docker containers. |
 | GANTRY_SLEEP_SECONDS  | 0    | Interval between two updates. Set it to 0 to run *Gantry* once and then exit. When this is a non-zero value, after an updating, *Gantry* will sleep until the next scheduled update. The actual sleep time is this value minus time spent on updating services. |
 | TZ                    |      | Set timezone for time in logs. |
 
@@ -68,34 +68,34 @@ You can configure the most behaviors of *Gantry* via environment variables.
 | Environment Variable  | Default | Description |
 |-----------------------|---------|-------------|
 | GANTRY_SERVICES_EXCLUDED         | | A space separated list of services names that are excluded from updating. |
-| GANTRY_SERVICES_EXCLUDED_FILTERS | `label=gantry.services.excluded=true` | A space separated list of [filters](https://docs.docker.com/engine/reference/commandline/service_ls/#filter), e.g. `label=project=project-a`. Exclude services which match the given filters from updating. Note that multiple filters will be logical **ANDED**. The default value allows you to add label `gantry.services.excluded=true` to services to exclude them from updating. |
-| GANTRY_SERVICES_FILTERS          | | A space separated list of [filters](https://docs.docker.com/engine/reference/commandline/service_ls/#filter) that are accepted by `docker service ls --filter` to select services to update, e.g. `label=project=project-a`. Note that multiple filters will be logical **ANDED**. |
+| GANTRY_SERVICES_EXCLUDED_FILTERS | `label=gantry.services.excluded=true` | A space separated list of [filters](https://docs.docker.com/engine/reference/commandline/service_ls/#filter), e.g. `label=project=project-a`. Exclude services which match the given filters from updating. The default value allows you to add label `gantry.services.excluded=true` to services to exclude them from updating. Note that multiple filters will be logical **ANDED**. |
+| GANTRY_SERVICES_FILTERS          | | A space separated list of [filters](https://docs.docker.com/engine/reference/commandline/service_ls/#filter) that are accepted by `docker service ls --filter` to select services to update, e.g. `label=project=project-a`. Note that multiple filters will be logical **ANDED**. Also see [How to filters multiple services by name](docs/faq.md#how-to-filters-multiple-services-by-name). |
 | GANTRY_SERVICES_SELF             | | This is optional. When running as a docker service, *Gantry* will try to find the service name of itself automatically, and update itself firstly. The manifest inspection will be always performed on the *Gantry* service to avoid an infinity loop of updating itself. This can be used to ask *Gantry* to update another service firstly. |
 
 ### To check if new images are available
 
 | Environment Variable  | Default | Description |
 |-----------------------|---------|-------------|
-| GANTRY_MANIFEST_CMD         | buildx | Valid values are `buildx`, `manifest`, and `none`.<br>Set which command for manifest inspection. Also see FAQ section [when to set `GANTRY_MANIFEST_CMD`](docs/faq.md#when-to-set-gantry_manifest_cmd).<ul><li>[`docker buildx imagetools inspect`](https://docs.docker.com/engine/reference/commandline/buildx_imagetools_inspect/)</li><li>[`docker manifest inspect`](https://docs.docker.com/engine/reference/commandline/manifest_inspect/)</li></ul>Set to `none` to skip checking the manifest. As a result of skipping, `docker service update` always runs. In case you add `--force` to `GANTRY_UPDATE_OPTIONS`, you also want to disable the inspection. |
+| GANTRY_MANIFEST_CMD         | buildx | Valid values are `buildx`, `manifest`, and `none`.<br>Set which command for manifest inspection. Also see FAQ section [when to set `GANTRY_MANIFEST_CMD`](docs/faq.md#when-to-set-gantry_manifest_cmd).<ul><li>[`docker buildx imagetools inspect`](https://docs.docker.com/engine/reference/commandline/buildx_imagetools_inspect/)</li><li>[`docker manifest inspect`](https://docs.docker.com/engine/reference/commandline/manifest_inspect/)</li></ul>Set to `none` to skip checking the manifest. As a result of skipping, `docker service update` always runs. In case you add `--force` to `GANTRY_UPDATE_OPTIONS`, you also want to disable the inspection. You can apply a different value to a particular service via [labels](#labels). |
 | GANTRY_MANIFEST_NUM_WORKERS | 1      | The maximum number of `GANTRY_MANIFEST_CMD` that can run in parallel. |
-| GANTRY_MANIFEST_OPTIONS     |        | [Options](https://docs.docker.com/engine/reference/commandline/buildx_imagetools_inspect/#options) added to the `docker buildx imagetools inspect` or [options](https://docs.docker.com/engine/reference/commandline/manifest_inspect/#options) to `docker manifest inspect`, depending on `GANTRY_MANIFEST_CMD` value, for all services. Also see [Labels](#labels) about adding options to a particular service. |
+| GANTRY_MANIFEST_OPTIONS     |        | [Options](https://docs.docker.com/engine/reference/commandline/buildx_imagetools_inspect/#options) added to the `docker buildx imagetools inspect` or [options](https://docs.docker.com/engine/reference/commandline/manifest_inspect/#options) to `docker manifest inspect`, depending on `GANTRY_MANIFEST_CMD` value, for all services. You can apply a different value to a particular service via [labels](#labels). |
 
 ### To add options to services update
 
 | Environment Variable  | Default | Description |
 |-----------------------|---------|-------------|
-| GANTRY_ROLLBACK_ON_FAILURE    | true  | Set to `true` to enable rollback when updating fails. Set to `false` to disable the rollback. |
-| GANTRY_ROLLBACK_OPTIONS       |       | [Options](https://docs.docker.com/engine/reference/commandline/service_update/#options) added to the `docker service update --rollback` command for all services. Also see [Labels](#labels) about adding options to a particular service. |
-| GANTRY_UPDATE_JOBS            | false | Set to `true` to update replicated-job or global-job. Set to `false` to disable updating jobs. *Gantry* adds additional options to `docker service update` when there is [no running tasks](docs/faq.md#how-to-update-services-with-no-running-tasks). |
+| GANTRY_ROLLBACK_ON_FAILURE    | true  | Set to `true` to enable rollback when updating fails. Set to `false` to disable the rollback. You can apply a different value to a particular service via [labels](#labels). |
+| GANTRY_ROLLBACK_OPTIONS       |       | [Options](https://docs.docker.com/engine/reference/commandline/service_update/#options) added to the `docker service update --rollback` command for all services. You can apply a different value to a particular service via [labels](#labels). |
+| GANTRY_UPDATE_JOBS            | false | Set to `true` to update replicated-job or global-job. Set to `false` to disable updating jobs. *Gantry* adds additional options to `docker service update` when there is [no running tasks](docs/faq.md#how-to-update-services-with-no-running-tasks). You can apply a different value to a particular service via [labels](#labels). |
 | GANTRY_UPDATE_NUM_WORKERS     | 1     | The maximum number of updates that can run in parallel. |
-| GANTRY_UPDATE_OPTIONS         |       | [Options](https://docs.docker.com/engine/reference/commandline/service_update/#options) added to the `docker service update` command for all services. Also see [Labels](#labels) about adding options to a particular service. |
-| GANTRY_UPDATE_TIMEOUT_SECONDS | 300   | Error out if updating of a single service takes longer than the given time. |
+| GANTRY_UPDATE_OPTIONS         |       | [Options](https://docs.docker.com/engine/reference/commandline/service_update/#options) added to the `docker service update` command for all services. You can apply a different value to a particular service via [labels](#labels). |
+| GANTRY_UPDATE_TIMEOUT_SECONDS | 300   | Error out if updating of a single service takes longer than the given time. You can apply a different value to a particular service via [labels](#labels). |
 
 ### After updating
 
 | Environment Variable  | Default | Description |
 |-----------------------|---------|-------------|
-| GANTRY_CLEANUP_IMAGES           | true  | Set to `true` to clean up the updated images. Set to `false` to disable the cleanup. Before cleaning up, *Gantry* will try to remove any *exited* and *dead* containers that are using the images. |
+| GANTRY_CLEANUP_IMAGES           | true  | Set to `true` to clean up the updated images on all hosts. Set to `false` to disable the cleanup. Before cleaning up, *Gantry* will try to remove any *exited* and *dead* containers that are using the images. |
 | GANTRY_CLEANUP_IMAGES_OPTIONS   |       | [Options](https://docs.docker.com/engine/reference/commandline/service_create/#options) added to the `docker service create` command to create a global job for images removal. You can use this to add a label to the service or the containers. |
 | GANTRY_NOTIFICATION_APPRISE_URL |       | Enable notifications on service update with [apprise](https://github.com/caronc/apprise-api). This must point to the notification endpoint (e.g. `http://apprise:8000/notify`) |
 | GANTRY_NOTIFICATION_CONDITION   | all   | Valid values are `all` and `on-change`. Specifies the conditions under which notifications are sent. Set to `all` to send notifications every run. Set to `on-change` to send notifications only when there are updates or errors. |
@@ -130,8 +130,8 @@ You need to tell *Gantry* to use a named config rather than the default one when
 
 Labels can be added to services to modify the behavior of *Gantry* for particular services. When *Gantry* sees the following labels on a service, it will modify the Docker command line only for that service. The value on the label overrides the global environment variables.
 
-| Labels  | Description |
-|---------|-------------|
+| Label  | Description |
+|--------|-------------|
 | `gantry.auth.config=<config-name>`       | See [Authentication](#authentication). |
 | `gantry.services.excluded=true`          | Exclude the services from updating if you are using the default [`GANTRY_SERVICES_EXCLUDED_FILTERS`](#to-select-services). |
 | `gantry.manifest.cmd=<command>`          | Override [`GANTRY_MANIFEST_CMD`](#to-check-if-new-images-are-available) |

examples/README.md

@@ -2,9 +2,12 @@
 
 ## [cronjob](./cronjob)
 
-Run *gantry* at the specific time.
+Run *Gantry* at a specific time.
 
 ## [prune-and-watchtower](./prune-and-watchtower)
 
 Remove unused containers, networks and images. Update standalone docker containers. Update docker swarm services.
 
+## [webhook](./webhook)
+
+Launch *Gantry* via webhook.

examples/cronjob/README.md

@@ -1,4 +1,4 @@
 # cronjob
 
-Use [*swarm-cronjob*](https://github.com/crazy-max/swarm-cronjob) to launch *gantry* at a given time.
+Use [*swarm-cronjob*](https://github.com/crazy-max/swarm-cronjob) to launch [*Gantry*](https://github.com/shizunge/gantry) at a specific time.
 

examples/webhook/README.md

@@ -0,0 +1,31 @@
+# webhook
+
+This example describes how to launch [*Gantry*](https://github.com/shizunge/gantry) via [adnanh/webhook](https://github.com/adnanh/webhook).
+
+## Setup
+
+We leverage a dockerized webhook image [lwlook/webhook](https://hub.docker.com/r/lwlook/webhook) which is based on the offical Docker image. This allows us to launch the *Gantry* service with simple docker commands.
+
+[hooks.json](./hooks.json) defines the webhook's behavior. It parses incoming payloads and transforms them into environment variables like `GANTRY_SERVICES_EXCLUDED`, `GANTRY_SERVICES_EXCLUDED_FILTERS` and `GANTRY_SERVICES_FILTERS`. These variables are then used by [run_gantry.sh](./run_gantry.sh) to control *Gantry* behaviors. which means you can update different services by passing different payloads to the webhook. Refer to the [adnanh/webhook](https://github.com/adnanh/webhook) repository for more advanced webhook configurations, including securing the webhook with `trigger-rule`.
+
+[run_gantry.sh](./run_gantry.sh) is responsible for launching the *Gantry* service.
+
+## Test
+
+Use the following command to deploy the Docker Compose stack that includes the webhook service.
+
+```
+docker stack deploy --detach=true --prune --with-registry-auth --compose-file ./docker-compose.yml webhook
+```
+
+Use curl to send a POST request to the webhook endpoint. This request tells the *Gantry* to only update the service named "webhook_webhook".
+
+```
+curl -X POST localhost:9000/hooks/run-gantry -H "Content-Type: application/json"  -d '{"GANTRY_SERVICES_FILTERS":"name=webhook_webhook"}'
+```
+
+Check the webhook service logs to confirm if the webhook was triggered correctly.
+
+```
+docker service logs webhook_webhook
+```

examples/webhook/docker-compose.yml

@@ -0,0 +1,60 @@
+version: "3.8"
+
+services:
+  webhook:
+    image: lwlook/webhook:latest
+    command:
+      - -verbose
+      - -hooks=/hooks.json
+      - -hotreload
+    ports:
+      - "9000:9000"
+    configs:
+      - source: hooks_json
+        target: /hooks.json
+      - source: run_gantry
+        target: /run_gantry.sh
+        mode: 0550
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock
+    deploy:
+      placement:
+        constraints:
+          - node.role==manager
+
+  # Note: run_gantry.sh does not use this service by default.
+  # This service is left here to demonstrate a potential approach for reusing the same service
+  # by scaling its replicas instead of starting a new service each webhook request.
+  # See function resume_gantry in run_gantry.sh.
+  # Pros:
+  #   * This approach can work together with other launching methods like crazymax/swarm-cronjob.
+  # Cons:
+  #   * Concurrency Issues: Sending webhook requests too frequently can increase the chance of the
+  #     webhook failing to launch Gantry correctly for some requests due to the existing service
+  #     potentially handling a previous command.
+  gantry:
+    image: shizunge/gantry:latest
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock
+    environment:
+      - "GANTRY_NODE_NAME={{.Node.Hostname}}"
+      - "GANTRY_SLEEP_SECONDS=0"
+    deploy:
+      replicas: 0
+      placement:
+        constraints:
+          - node.role==manager
+      restart_policy:
+        condition: none
+      labels:
+        # The label can be used to find this service.
+        # This can be used together with resume_gantry function in the run_gantry.sh
+        - webhook.run-gantry=true
+
+configs:
+  hooks_json:
+    name: hooks_json
+    file: ./hooks.json
+  run_gantry:
+    name: run_gantry
+    file: ./run_gantry.sh

examples/webhook/hooks.json

@@ -0,0 +1,25 @@
+[
+  {
+    "id": "run-gantry",
+    "execute-command": "/run_gantry.sh",
+    "command-working-directory": "/",
+    "pass-environment-to-command":
+    [
+      {
+        "source": "payload",
+        "name": "GANTRY_SERVICES_EXCLUDED",
+        "envname": "GANTRY_SERVICES_EXCLUDED"
+      },
+      {
+        "source": "payload",
+        "name": "GANTRY_SERVICES_EXCLUDED_FILTERS",
+        "envname": "GANTRY_SERVICES_EXCLUDED_FILTERS"
+      },
+      {
+        "source": "payload",
+        "name": "GANTRY_SERVICES_FILTERS",
+        "envname": "GANTRY_SERVICES_FILTERS"
+      }
+    ]
+  }
+]