Deploys project on GKE 1.15.7

CHANGELOG.md

@@ -5,6 +5,13 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
+### Changed
+- Upgraded the Google Marketplace Tools container image used by the Deployer from 0.7.0 to 0.10.0. The newer version of these tools provide an adaptive Kubectl client version (tools read the Kubernetes server version, then select a matching kubectl binary).
+- Upgraded the Google Marketplace Tools submodule to 0.10.0.
+- Upgraded the version of Helm used by the deployer from 2.6.1 to 2.16.1 to eliminate this Helm bug: https://github.com/helm/helm/issues/2998
+- Deleted x-google-marketplace section for tester.image in schema.yaml to be consistent for Google Marketplace Tools v0.10.0
+- Added a build.sh flag (`-p` or `--persist`) to persist the application deployment after testing.
+- Fixed the deployment's `deploy-info` annotations to use keys that are valid JSON keys (i.e. with quotes).
 
 ## [1.3.4](https://github.com/cyberark/conjur-google-cloud-launcher/releases/tag/v1.3.4) - 2019-01-08
 ### Changed

DEVELOPMENT.md

@@ -19,13 +19,30 @@
 
 The project URL is: https://console.cloud.google.com/home/dashboard?organizationId=854380395992&project=conjur-cloud-launcher-onboard
 
+Before proceeding, make sure that:
+* You are authorized to access to the `conjur-cloud-launcher-onboard` GCP project.
+* Your current GCP project is set to `conjur-cloud-launcher-onboard`:
+```
+gcloud config set project conjur-cloud-launcher-onboard
+```
+* Your current GCP cluster is set to the desired cluster in the `conjur-cloud-launcher-onboard` project:
+```
+gcloud container cluster list
+gcloud container clusters get-credentials <CLUSTER NAME>
+```
+
 ## Cluster Setup
 
 0. Run the following command to create the Application CRD: `$ make crd/install`.
 
-1. Create the namespace from `setenv.sh`: `kubectl create ns "$(whoami)"`
+1. Create the namespace from `setenv.sh`, and set to that namespace:
+```
+   export NAMESPACE="$(whoami)"
+   kubectl create ns "$NAMESPACE"
+   kubectl config set-context --current --namespace="$NAMESPACE"
+```
 
-2. Run the following to create the app: `$ make app/install`.
+2. Run the following to create the app: `$ make app/install-test`.
 
 3. Run the following to watch the app: `$ make app/watch`.
 
@@ -48,6 +65,7 @@ This is done automatically by Jenkins in the build pipeline.
 
 ## Testing
 
-`./build.sh` will automatically test the application with the configured kubectl context.
-It will launch the application in a custom namespace, test it, and then delete the namespace.
-This step is also automatically done by the build pipeline.
+The `build.sh` script can be run a couple of ways:
+
+* `./build.sh` will automatically test the application with the configured kubectl context. It will launch the application in a custom namespace, test it, and then delete the namespace. This step is also automatically done by the build pipeline.
+* `./build.sh -p` (or `./build.sh --persist`) will automatically test the application with the configured kubectl context. It will launch the application in a custom namespace, test it, and leave the application running.

Jenkinsfile

@@ -8,6 +8,10 @@ pipeline {
     buildDiscarder(logRotator(numToKeepStr: '30'))
   }
 
+  triggers {
+    cron(getDailyCronString())
+  }
+
   environment {
     REGISTRY = 'gcr.io/conjur-cloud-launcher-onboard'
   }

Makefile

@@ -64,6 +64,8 @@ app/build:: .build/conjur/deployer \
                         .build/var/REGISTRY \
                         .build/var/TAG \
                         | .build/conjur
+	# Note: print_target displays a highlighted (in yellow) message
+	# indicating the target that is being built.
 	$(call print_target, $@)
 	docker build \
 	    --build-arg REGISTRY="$(REGISTRY)" \

README.md

@@ -89,7 +89,7 @@ export NAMESPACE=conjur
 Configure the container images:
 
 ```shell
-export TAG_VERSION=1.3.4
+export TAG_VERSION=1.4.0
 export IMAGE_CONJUR="gcr.io/cloud-marketplace/cyberark/conjur-open-source:$TAG_VERSION"
 export IMAGE_POSTGRES="gcr.io/cloud-marketplace/cyberark/conjur-open-source/postgres:$TAG_VERSION"
 export IMAGE_NGINX="gcr.io/cloud-marketplace/cyberark/conjur-open-source/nginx:$TAG_VERSION"
@@ -112,22 +112,21 @@ for i in "IMAGE_CONJUR" "IMAGE_POSTGRES" "IMAGE_NGINX"; do
 done
 ```
 
-The Conjur data key is generated by the
-deployer and does not need to be created
-beforehand.
-
 #### Create namespace in your Kubernetes cluster
 
 We recommend running Conjur in its own namespace.
 If you use a different namespace than the `default`, run the command below to create a new namespace:
 
 ```shell
 kubectl create namespace "$NAMESPACE"
+kubectl config set-context --current --namespace="$NAMESPACE"
 ```
 
-#### Install the application with Helm to your Kubernetes cluster
+#### Install the application with Helm (v2) to your Kubernetes cluster
+
+These instructions assume that your local `helm` client is version 2.
 
-This project uses the upstream [cyberark/conjur-oss Helm chart](https://github.com/cyberark/conjur-oss-helm-chart).
+This project uses the upstream [cyberark/conjur-oss Helm chart](https://github.com/cyberark/conjur-oss-helm-chart). (You do not need to clone or helm install this repo directly; this will be done indirectly via the helm install of conjur below.)
 
 Use `helm` to deploy the application to your Kubernetes cluster:
 
@@ -138,7 +137,8 @@ See [conjur-oss/values.yaml](https://github.com/cyberark/conjur-oss-helm-chart/b
 for all available upstream Helm chart parameters and their defaults.
 
 ```shell
-helm install ./conjur
+helm dependency update ./conjur
+helm install conjur --set conjur-oss.dataKey="$(docker run --rm cyberark/conjur data-key generate)" ./conjur
 ```
 
 #### View the app in the Google Cloud Console
@@ -264,11 +264,12 @@ Delete the application release using Helm:
 ```sh-session
 # Find the release
 $ helm list | grep conjur
-reeling-greyhound	1       	Fri Jul 20 16:36:03 2018	DEPLOYED	conjur-1.0.0
+
+conjur	conjur   	1       	2020-03-09 15:36:14.293351857 -0400 EDT	deployed	conjur-1.3.7
 
 # Delete the release
-$ helm delete reeling-greyhound
-release "reeling-greyhound" deleted
+$ helm delete conjur
+release "conjur" uninstalled
 ```
 
 ## License

app.Makefile

@@ -41,6 +41,9 @@ endef
 .PHONY: .build/app/dev
 .build/app/dev: .build/var/MARKETPLACE_TOOLS_TAG \
               | .build/app
+	# Note: print_target displays a highlighted (in yellow) message
+	# indicating the target that is being built.
+	$(call print_target, $@)
 	docker run \
 	    "gcr.io/cloud-marketplace-tools/k8s/dev:$(MARKETPLACE_TOOLS_TAG)" \
 	    cat /scripts/dev > "$@"

apptest/deployer/schema.yaml

@@ -2,8 +2,6 @@ properties:
   tester.image:
     type: string
     default: $REGISTRY/tester:$TAG
-    x-google-marketplace:
-      type: IMAGE
 
   conjur-oss.ssl.hostname:
     type: string

build.sh

@@ -9,12 +9,16 @@ make crd/install
 gcloud auth configure-docker
 
 chart_dir=""
+build_target="app/verify"
 
 while [ "$1" != "" ]; do
     case $1 in
         -c | --chart-dir )  shift
                             chart_dir="${1}"
                             ;;
+        # Use the -p | --persist flag to keep the application running
+        -p | --persist )    build_target="app/install-test"
+                            ;;
         * )                 >&2 echo "Unknown argument: ${1}"
                             exit 1
                             ;;
@@ -39,13 +43,13 @@ if [ "${chart_dir}" != "" ]; then
 fi
 
 echo "Getting the desired marketplace Docker image..."
-MARKETPLACE_TOOLS_TAG="0.7.0"
+MARKETPLACE_TOOLS_TAG="0.10.0"
 LOCAL_MARKETPLACE_TOOLS_TAG="local-$USER"
 docker pull "gcr.io/cloud-marketplace-tools/k8s/dev:$MARKETPLACE_TOOLS_TAG"
 docker tag "gcr.io/cloud-marketplace-tools/k8s/dev:$MARKETPLACE_TOOLS_TAG" \
            "gcr.io/cloud-marketplace-tools/k8s/dev:$LOCAL_MARKETPLACE_TOOLS_TAG"
 
 echo "Building/verifying app..."
-make -j4 -e app/verify
+make -j4 -e "$build_target"
 
 echo "Done!"

conjur/templates/application.yaml

@@ -7,7 +7,7 @@ metadata:
   annotations:
     kubernetes-engine.cloud.google.com/icon: >-
       data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAIMAAACDCAYAAACunahmAAAACXBIWXMAAAsSAAALEgHS3X78AAAON0lEQVR4Ae2dT2hbyR3HR04cObEbyY66Xm8Klqho6ckyOfTSruX2UOhBVqD0avnSSw+rgGgpgdhOSenBsAqU0p4sL+nu3iKll0KhkSgUylbd59JdaCsqOWSTzdpJJLvx39gu4/xmGE/m/ZH09Oc9/T4gsJ+e3pv33nd+85uZ32+e5/j4mCAIpa8LyoB0CSgGhINiQDgoBoSDYkA4KAaEg2JAOCgGhINiQDgoBoSDYkA4KAaE07NiePvm/fEuKEZX0ZOzlrF0MbG7t/erL9bXNwghSW1pJtsFxeo4PSWGWLoYJYSkCSETu3t75Iv1dfZVAUShdbaEnaUnxBBLF4Mgghm2TRIDYwVEUe1EOTuNq8UQSxf99OHCxyd+pyMGSo0QsqAtzaTbVtAuwbVioH4BfaiEEKWjaCAGxhohJKEtzeRbWc5uwnViiKWLEWgSpoz2syAGRgFEUbGznN2Ia8QATQIVwayV/esQA+M2NB+u9SdcMc4QSxdpc1CxKoQGeYeeI5LKJVp6MR3E0ZYhli7GwRrUPYDUgGUQWYVeh6v8CUeKAbqKGTO/wIgmxcDIgShc4U84SgzgFyyAyW4Km8RAoCtKrVPa6f6EY3yGWLqYBL+gaSHYDB2/mCeEaE73J7reMohDyHYe10bLIFOAXofj/ImutgzRn+c/PDo6um+3ECj9h9uk33No92EJ+DH3I6lcvBUHbyVdLYZqrfbmZ48f72xube3aedz+x8Xdi5+8t/e9oX/Xgv3Pd+w8NoxcTjtxJvRsF5TBkOPj4/PVWo1sbm3VApcu+Qa83oaPdWbzIfGWfl/zvNw9mafo9xx6J89/Rr7qffpkdXdsdOPlYDNFdfycRteLgXF0dOSjbfy5/v6NwKVLgbNnrRfds7dJBv5zb6Nvez0gT1hRLvbtjn77Qpk8fnnxyT923xzdPjpXb/FcMdvpGDEw9g8OAo8+/5wMDQ5u+H2+QF+ffkvnOdwj/Y8+qvU//ogKIGB27LGzm6OjQ1vbpb3AwSd7o6+JRoGr4iAcJwbG/168CLzY3t4d9vuPhwYHz8vfn13/54638icPOT608lA5feT4wte86yR07lnt77uX+x8dXLyg2G3NjRFSjhUDeeVPDDx7/pzUNjc3Lo2MBKg/ceIX/PcPG579LVNLYES/59D3zfMPyKZ34Mnfdr4yWjscIGyASVuaWWj1tXUCR4uBcXh4GKD+xOj+g0cjG399y0qTYBXqT3xnsET+tfflP366N/ojN09luys6eufpQasO/XXv+l/cHtOAeRMIB8WAcFAMCAfFgHBQDC0gksr5I6lcxGnl7lkx9PX1/QaGkW0FZivpiKTjZi1dMc5QJyeh77+7895JN3Hx5q0MRE81HEJHXonAUoh+N9NLYjgZQv7g/TunhpDnb1ynQSjRxZu3EvAw6xq+pk1CPSH63UwvNBN0CHnxg/fvBGUhiMzfuE4tBA20XbR64Egql2xDiH7bcLtlWAFrYGlqef7GdbrfAjQdpxJ1RSKpXBSis121xoNbxVAAETQ0tTx/4zqt7fHFm7dOxV9GUrmmQ/S7GVeJ4fCcjxzvD/74w5Xf/tqO44E/Ebm++MuflfdHpiEK2rW4Qgwej2f7S0NDBwNvfWt8u+/tn8TSxe17ySsZO479Kvz9Gz+t17F0Io4Xw4DXuzEyPCyGwdF2fBlS8pP3klcaairAL7A9RL+bcawYzpw5wwJa9GIXaLv+cSxdXAFRWHIiwS/QdR7djOO6lh6PZ3dkeHjn8thYwGKkNO32VSBTWxcYQqb7lHtRCMRpYqBBsJfHxgZUMY8mnKTAxdLFCmRunwLS4jS3O4hmOKKZEMLjmw1no/7E3Vi6eNL1fPDwod+OoWi30NVi8J47t+/z+YiBX9AoJ/5E+67EGXR1MzH6xhs/HPB6F2FI2SnQsl5zYgS1I9ZngMU5FuyeA3jw8KGdhyNOz6xy2mIdtvb9bRSDKzKrnLqMT0PTzTI2iMFVmVWOXeBLWP214e5gE2JwZWaV49eBVK0LbZUGxbACqfeuS6hx06KgdfsTdYrBscvzWMWNywUnoedh6k9YFEMN/AJbZkG7GdeFvd1LXklD+NptGw5HxziCvSAE0gOvGDCMTDKwDK5a7NMqvfLyEWXMokIMPfdaAZFeey3RKX9CEEPPvnBEpKcyqgR/Qsykug1+QU8LgfTq2+so3/3Fn7//9NmzT3vNLzCiZ8WAvA5mYSMcFAPCQTEgHBQDwkExIBwUA8JBMfQAdGkhWFTEEBSDyxHWksibCQLF4H4SsLoM/USNrtZyEk04GEpAzKEcSUSne9OlSjkP+2WEkPZptl06Fi3Uffb7UqUch+15g+ymAlzQQqlSPjWEHA6GFizGQk4L5RTLoHc+el2Gwa5Q8+TjhPSGuSGfU6+sa3CNGVUMRSSVOzVcrC3NeAzKkNOWZuLwzOg1ZM0Cd00tQzgY8oeDIRoCvqwTUkZjD++Hg6Ek/J8Ukl5euyB6PGF7Dfa3whSITAsHQ+1YY5Ge7y4IzQhV+Rt9peE4nHc5ksppVtp5yBqXH/IaKwPN4dCWZqJWAnSsNBNZQQS0tlwlhAwTQiYJIXNwYspJ4kipUq4KN2NccTOTQlxBUq7lAH298LT0YZlVPpiGVqH6nfjRy2uQ97sKx6LMh4OhoOpH8CBUgbhJKw/ShAmzigLnyEohfvQexRtJ5DFsJqBpYGb7dqlSFgunQS2lhYmK5pT+HQ6GcnCj6M3M0IcONZqZSNo86Km1qmhe8mBV3jFo+1S/M0WnKcuDyfaBuFUC1BOlDxYFtRIud00QqRyAkzQ4B9EJAI43msxjZhlYDS9IQuBQS6DTriYUzYXYPBiZUjNVtzx9DSyc7k2FWimm969Ku1jNqdBoZBVEV8WlvFKzoF453XCumSgtMweSWYW6A0LpzQTLcpceB/wOpuIE3Gw9/IrmJSKYZL3y+MEx1CuT5RsFVsjIN0lKDysNFos9oHHav68z2yrRRJbY7WYDd632JhoKAIHmogCiYkLImXnosL9er2K1VCnr1boJkx6CR7VRISC/FG6vusmiZatB2y0vEJpUOHcy6UgqV4Vzyia/YPJbxqq2NGPVEdelHeszsFVRfBaaBwa9CWIt9kOtozdrAnwQ1XFqRqbdACMBzclOLqz0IrbtWXDY6MDOmvDdFF1T2qQN10v6qaenNUHL1GrLwC4sLj0cy4DjqEFNT5s0D4y8qvZD9/VdWvtoM6LoiWilStlwYEUHVgODwoM8NX4iIT+k2Ugqp7dcQLKBrmYBorTrsci0O0qaEYSZA8luRALa0I5SqpTFoFVld68RqIBARBHBEYyqmkcY4KlnSYBZ6ILqcQ0+p05j8diy07rczHsuzMTAaqcPunavnYjW1nAwVG3HQBCMbrYMsFpxYTwjq6gEjbTNRpZBg8jsnLDt5NwWxirk3geBpqqhZ2HYTICJnxNGHz8GhzAP6o0IZjVrY21NKJw60aFc1THftNdiFOGrHB4XgWuOwjVOQCWglqOqGGSq6bxkJAj3jEEHodImA0HMt2L3cwJ6KLpCos0IWKq84Oz6QBDBegeeTEcgYWDoqqDAKRg4mhEKXjCbBKmTcaFHIfcsCk0M91qiVClrwjkmhN6E7Mdk2RiB9MkII7NEGITSBR6cfF2z4Kwa/U5TWCuflVlKGUu9CegK0j58XNGeZeHmGZGEHoGZQ8T206OiM3ydsejgatLf00bXHA6GJll5aHMx9IN0WupmGl13VLKUrJbKZeXHoIKKpHKT0j0Qa7eyvFR8dC7D5N6ZgnkTCAfjGRAOigHhoBgQDooB4aAYEA6KAeGgGBAOigHhdOx9E4rRTM0o6AWCUuPCKFsVRj+Vo5oQKVWVZjpV++SN5itgn4oYr6naJv2GDiEHxWl4tk2xuwZl6PhK9G0XA4ggowrvCgdDaxASlxe2Gb1r+t1wMLQCUdbyzZxnvzeIjGLBuUZD2fMwH5Ix2SbCAokXFNtU1GAyrKOr0re1mRBiIgmEvg+XKmUaihaC//3iZA0IIQ9CoA99ku4Pv5mEbbNC5LSKeaO4yA4gh+XPQRGMrqEttM0ywANZhtnPU7UATP0CC6kXfpaBWcM52SSzmUUIaV82me6lk07BbjDFOmH5FQi9Y69O6AjttAzMZMb1zKEoBPAR6DT5ikF+BZtiX4FQOLlNLkDQiM9CYGrHEATSUcvQFjGA+ZsyCEpRwWq5lfwDto/KMrD8jSkLqXIdQRBxRy1XuywD6zXUXTv1egtW9xFC2UgX+g9MCFkh3L5j9MQ4A1ijRfhXFdfYNmhYnviBN+wGDfJO20a7xMAu0jD0S4Ve0quIlX2ge1noAv9hUfiw0Li4kV/ULtoiBlD8KiTAWDXT7IHV4zOYPWQ7/Qcj66L7HRUl+5j4Om2nE72JjF5NFrdDjyMHvQTdmwXfzULanuGgDYiSHct0cQ+hPLL51hU2bJuQAmL1ysMCZ1U9obbTNjHAUPMKRD7TVH6emENvBNRUTcqNSMCNX6bbxRsGv0nDGMOq1doF5VC+pYYuLyCdhwlY7gExC5QVhQp/s++smn1eSSzu3zLaHhBrsuRODYajs8L+bEGKKWEfIgxnF6DNPdUtA+esoJduJ2SFL7LhailTnCXS8GWGpDLlDTKrVmFgrSr85mSJIhg9lctSgUpimtfRSto+N0FvPNT+OISTsxD6PEw8VaX96f9RyNgSJ7c0kzD9gkkoO5sj4U1AqVKOQO1mAsqrHDtYbiCqM2eiN1dilAaQhE+00ZxWO8BQ+SaR1nHQy+twBCgGhIPBLQgHxYBwUAwIB8WAcFAMCAfFgHBQDAgHxYBwUAwIB8WAcFAMCAfFgHBQDMgrCCH/B72jI0AbGfj3AAAAAElFTkSuQmCC
-    marketplace.cloud.google.com/deploy-info: '{partner_id: "cyberark", product_id: "conjur-open-source", partner_name: "CyberArk"}'
+    marketplace.cloud.google.com/deploy-info: '{"partner_id": "cyberark", "product_id": "conjur-open-source", "partner_name": "CyberArk"}'
   labels:
     app.kubernetes.io/name: "{{ .Release.Name }}"
 spec:

crd.Makefile

@@ -7,7 +7,9 @@ include common.Makefile
 # Installs the application CRD on the cluster.
 .PHONY: crd/install
 crd/install:
-	kubectl apply -f "https://raw.githubusercontent.com/GoogleCloudPlatform/marketplace-k8s-app-tools/master/crd/app-crd.yaml"
+	# Ignore errors on kubectl apply. `AlreadyExists` Errors can occur if
+	# another parallel test is doing a kubectl apply at the same time.
+	-kubectl apply -f "https://raw.githubusercontent.com/GoogleCloudPlatform/marketplace-k8s-app-tools/master/crd/app-crd.yaml"
 
 
 # Uninstalls the application CRD from the cluster.

deployer/Dockerfile

@@ -5,7 +5,7 @@ RUN echo "cachebuster-20190118" >/dev/null \
     && apt-get update \
     && apt-get install -y --no-install-recommends gettext curl
 
-RUN curl -L -o /tmp/helm.tgz https://storage.googleapis.com/kubernetes-helm/helm-v2.6.1-linux-amd64.tar.gz && \
+RUN curl -L -o /tmp/helm.tgz https://storage.googleapis.com/kubernetes-helm/helm-v2.16.1-linux-amd64.tar.gz && \
     cd /tmp && \
     tar xvzf helm.tgz && \
     cp linux-amd64/helm /usr/local/bin/helm && \
@@ -43,7 +43,7 @@ RUN cat /tmp/test/schema.yaml \
     && mv /tmp/test/schema.yaml.new /tmp/test/schema.yaml
 
 
-FROM gcr.io/cloud-marketplace-tools/k8s/deployer_helm:0.7.0
+FROM gcr.io/cloud-marketplace-tools/k8s/deployer_helm:0.10.0
 COPY --from=build /tmp/conjur.tar.gz /data/chart/
 COPY --from=build /tmp/test/conjur.tar.gz /data-test/chart/
 COPY --from=build /tmp/schema.yaml /data/