Skip to content
This repository has been archived by the owner on Jan 9, 2021. It is now read-only.

Deploys project on GKE 1.15.7 #27

Merged
merged 1 commit into from
Mar 11, 2020
Merged

Deploys project on GKE 1.15.7 #27

merged 1 commit into from
Mar 11, 2020

Conversation

diverdane
Copy link
Contributor

@diverdane diverdane commented Mar 9, 2020

This change fixes a problem whereby deployment of the marketplace
app would fail on GKE clusters with Kubernetes versions >= 1.15.

The main problem was that we were using the Google Marketplace Tools
Version 0.7.0, and this uses a version of kubectl (1.12) that does
not know how to authenticate with Kubernetes version 1.15 or newer.

The changes included are as follows:

  • Upgraded the Google Marketplace Tools container image used by the Deployer from 0.7.0 to 0.10.0. The newer version of these tools provide an adaptive Kubectl client version (tools read the Kubernetes server version, then select a matching kubectl binary).
  • Upgraded the Google Marketplace Tools submodule to 0.10.0.
  • Upgraded the version of Helm used by the deployer from 2.6.1 to 2.16.1 to eliminate this Helm bug: helm dependency update fails after helm upgrade helm/helm#2998
  • Deleted x-google-marketplace section for tester.image in schema.yaml to be consistent for Google Marketplace Tools v0.10.0
  • Added a build.sh flag (-p or --persist) to persist the application deployment after testing.
  • Fixed the deployment's deploy-info annotations to use keys that are valid JSON keys (i.e. with quotes).
  • Fixed up a bunch of inaccuracies in README.md and DEVELOPMENT.md.

Fixes Issue #25 "Project fails to deploy on GKE 1.15.7-gke.23"
Fixes Issue #26 "GCP marketplace integration has automated tests against the latest GKE versions"

@diverdane diverdane self-assigned this Mar 9, 2020
@diverdane diverdane requested a review from sgnn7 March 9, 2020 22:18
Copy link
Contributor

@sgnn7 sgnn7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@diverdane Just a few small comment but great investigative work on this. We may need to consider bumping the version right after this commit (or as part of it).

DEVELOPMENT.md Outdated Show resolved Hide resolved
DEVELOPMENT.md Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
app.Makefile Show resolved Hide resolved
build.sh Outdated Show resolved Hide resolved
@diverdane diverdane force-pushed the 25_deploy_k8s_1_15 branch 3 times, most recently from 56058b7 to b8be637 Compare March 10, 2020 18:26
@diverdane
Copy link
Contributor Author

@sgnn7 - I made the changes that you suggested.
I also added a parallel CI stage so that our CI tests our Marketplace deploy on:

  • A gke-1.12.10 cluster (Kubernetes 1.12.10)
  • A gke-1.15.9 cluster (Kubernetes 1.15.9)
    (I don't think I have access to the CI Conjur, so the secrets.yml section for gke-1.15.9 uses a string for the GKE cluster name, rather than a Conjur variable.)

@diverdane diverdane force-pushed the 25_deploy_k8s_1_15 branch 2 times, most recently from f40a84a to a096a75 Compare March 10, 2020 18:51
kubectl apply -f "https://raw.githubusercontent.com/GoogleCloudPlatform/marketplace-k8s-app-tools/master/crd/app-crd.yaml"
# Ignore errors on kubectl apply. `AlreadyExists` Errors can occur if
# another parallel test is doing a kubectl apply at the same time.
-kubectl apply -f "https://raw.githubusercontent.com/GoogleCloudPlatform/marketplace-k8s-app-tools/master/crd/app-crd.yaml"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this is a risky move but maybe it will be fine. I'll leave it as-is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I did not like adding this, but the first time I ran the 2 test cases in parallel, I saw an AlreadyExists error in one test. I didn't expect to see this sort of error when kubectl apply ... is being used, but there's apparently a race condition where the 2 instances of kubectl apply ... realize that a create is needed, and they both try the create at the same time. It's too bad there's no --ignore-already-created flag or similar to whatever the flag is for kubectl create ....

ci/secrets.yml Outdated
GCLOUD_ZONE: !var ci/google-conjur-cloud-launcher-onboard/gcloud-zone

gke-1-15-9:
GCLOUD_CLUSTER_NAME: onboarding-cluster-1-15-9
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending resolution of convo on slack

README.md Outdated
@@ -89,7 +89,7 @@ export NAMESPACE=conjur
Configure the container images:

```shell
export TAG_VERSION=1.3.4
export TAG_VERSION=1.3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think pinning to full semver version might be better than to the minor version since you won't be able to know what version of components you're pulling/writing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, there are no tags using the full semver version in our registry: https://console.cloud.google.com/gcr/images/cloud-marketplace/GLOBAL/cyberark/conjur-open-source?gcrImageListsize=30
We can change this when we make a new release.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I switched this to the full semver 1.4.0 in anticipation of us making a release 1.4.0 and then publishing it to Google Cloud Marketplace.

@diverdane diverdane force-pushed the 25_deploy_k8s_1_15 branch 2 times, most recently from 1004060 to 998e8a2 Compare March 10, 2020 22:17
This change fixes a problem whereby deployment of the marketplace
app would fail on GKE clusters with Kubernetes versions >= 1.15.

The main problem was that we were using the Google Marketplace Tools
Version 0.7.0, and this uses a version of kubectl (1.12) that does
not know how to authenticate with Kubernetes version 1.15 or newer.

The changes included are as follows:

* Upgraded the Google Marketplace Tools container image used by the Deployer from 0.7.0 to 0.10.0. The newer version of these tools provide an adaptive Kubectl client version (tools read the Kubernetes server version, then select a matching kubectl binary).
* Upgraded the Google Marketplace Tools submodule to 0.10.0.
* Upgraded the version of Helm used by the deployer from 2.6.1 to 2.16.1 to eliminate this Helm bug: helm/helm#2998
* Deleted x-google-marketplace section for tester.image in schema.yaml to be consistent for Google Marketplace Tools v0.10.0
* Added a build.sh flag (`-p` or `--persist`) to persist the application deployment after testing.
* Fixed the deployment's `deploy-info` annotations to use keys that are valid JSON keys (i.e. with quotes).

Fixes Issue #25 "Project fails to deploy on GKE 1.15.7-gke.23"
Copy link
Contributor

@sgnn7 sgnn7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@diverdane Great work on this! LGTM!

@sgnn7 sgnn7 merged commit 4198a69 into master Mar 11, 2020
@sgnn7 sgnn7 deleted the 25_deploy_k8s_1_15 branch March 11, 2020 15:47
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants