Attempt to use latest Kubernetes / etcd fails

[pete0emerson]

I tried to move to Kubernetes 1.0.6 and etcd 2.2.0, and terraform throws up at it. I'm just dipping my toes into CoreOS / Kubernetes / terraform, so I don't yet know what's going on, but thought I'd post here in case something pops out at you that's a simple fix to get on the latest versions. I'll keep digging and issue a pull request if I figure it out. I'm going to upgrade Kubernetes and etcd individually to narrow down the issue, once I've done that I'll add details here. Thanks for creating and sharing this repo!

Here's my changes to variables.tf:

bash-3.2$ git diff variables.tf
diff --git a/variables.tf b/variables.tf
index 93a5c33..a7d1ba4 100644
--- a/variables.tf
+++ b/variables.tf
@@ -35,10 +35,10 @@ variable "ami" {
 }

 variable "kubernetes_version" {
-    default = "0.17.1"
+    default = "1.0.6"
 }

 variable "etcd_version" {
     description = "version of etcd to use"
-    default = "2.0.11"
+    default = "2.2.0"
 }

Here's the end of the terraform run:

aws_instance.master (remote-exec): + for S in etcd flanneld kube-apiserver kube-scheduler kube-controller-manager
aws_instance.master (remote-exec): + start kube-apiserver
aws_instance.master (remote-exec): + systemctl daemon-reload
aws_instance.master (remote-exec): + systemctl enable kube-apiserver.service
aws_instance.master (remote-exec): Created symlink from /etc/systemd/system/multi-user.target.wants/kube-apiserver.service to /etc/systemd/system/kube-apiserver.service.
aws_instance.master (remote-exec): + systemctl start kube-apiserver.service
aws_instance.master (remote-exec): + for S in etcd flanneld kube-apiserver kube-scheduler kube-controller-manager
aws_instance.master (remote-exec): + start kube-scheduler
aws_instance.master (remote-exec): + systemctl daemon-reload
aws_instance.master (remote-exec): + systemctl enable kube-scheduler.service
aws_instance.master (remote-exec): Created symlink from /etc/systemd/system/multi-user.target.wants/kube-scheduler.service to /etc/systemd/system/kube-scheduler.service.
aws_instance.master (remote-exec): + systemctl start kube-scheduler.service
aws_instance.master (remote-exec): Job for kube-scheduler.service failed. See "systemctl status kube-scheduler.service" and "journalctl -xe" for details.
Error applying plan:

1 error(s) occurred:

* Script exited with non-zero exit status: 1

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.
make: *** [apply] Error 1

Per the terraform output, here's some more info:

core@ip-172-20-250-24 ~ $ systemctl status kube-scheduler.service`
● kube-scheduler.service - Kubernetes Scheduler
   Loaded: loaded (/etc/systemd/system/kube-scheduler.service; enabled; vendor preset: disabled)
   Active: activating (start-pre) since Wed 2015-10-14 17:57:11 ; 1min 14s ago
     Docs: https://github.com/GoogleCloudPlatform/kubernetes
  Control: 2686 (wupiao)
   Memory: 380.0K
   CGroup: /system.slice/kube-scheduler.service
           └─control
             ├─2686 /bin/bash /opt/bin/wupiao http://127.0.0.1:8080/api/v1beta3/nodes
             └─2854 sleep 1

Oct 14 17:58:16 ip-172-20-250-24.us-west-2.compute.internal wupiao[2686]: .
Oct 14 17:58:17 ip-172-20-250-24.us-west-2.compute.internal wupiao[2686]: .
Oct 14 17:58:18 ip-172-20-250-24.us-west-2.compute.internal wupiao[2686]: .
Oct 14 17:58:19 ip-172-20-250-24.us-west-2.compute.internal wupiao[2686]: .
Oct 14 17:58:20 ip-172-20-250-24.us-west-2.compute.internal wupiao[2686]: .
Oct 14 17:58:21 ip-172-20-250-24.us-west-2.compute.internal wupiao[2686]: .
Oct 14 17:58:22 ip-172-20-250-24.us-west-2.compute.internal wupiao[2686]: .
Oct 14 17:58:23 ip-172-20-250-24.us-west-2.compute.internal wupiao[2686]: .
Oct 14 17:58:24 ip-172-20-250-24.us-west-2.compute.internal wupiao[2686]: .
Oct 14 17:58:25 ip-172-20-250-24.us-west-2.compute.internal wupiao[2686]: .

core@ip-172-20-250-24 ~ $ journalctl -xe
Oct 14 17:59:05 ip-172-20-250-24.us-west-2.compute.internal kube-apiserver[1044]: I1014 17:59:05.825634    1044 handlers.go:13
Oct 14 17:59:06 ip-172-20-250-24.us-west-2.compute.internal wupiao[2886]: .
Oct 14 17:59:06 ip-172-20-250-24.us-west-2.compute.internal kube-apiserver[1044]: I1014 17:59:06.830323    1044 handlers.go:13
Oct 14 17:59:07 ip-172-20-250-24.us-west-2.compute.internal wupiao[2886]: .
Oct 14 17:59:07 ip-172-20-250-24.us-west-2.compute.internal kube-apiserver[1044]: I1014 17:59:07.835027    1044 handlers.go:13
Oct 14 17:59:08 ip-172-20-250-24.us-west-2.compute.internal wupiao[2886]: .
Oct 14 17:59:08 ip-172-20-250-24.us-west-2.compute.internal kube-apiserver[1044]: I1014 17:59:08.839659    1044 handlers.go:13
Oct 14 17:59:09 ip-172-20-250-24.us-west-2.compute.internal wupiao[2886]: .
Oct 14 17:59:09 ip-172-20-250-24.us-west-2.compute.internal kube-apiserver[1044]: I1014 17:59:09.844357    1044 handlers.go:13
Oct 14 17:59:10 ip-172-20-250-24.us-west-2.compute.internal wupiao[2886]: .
Oct 14 17:59:10 ip-172-20-250-24.us-west-2.compute.internal kube-apiserver[1044]: I1014 17:59:10.849213    1044 handlers.go:13
Oct 14 17:59:11 ip-172-20-250-24.us-west-2.compute.internal wupiao[2886]: .
Oct 14 17:59:11 ip-172-20-250-24.us-west-2.compute.internal kube-apiserver[1044]: I1014 17:59:11.854015    1044 handlers.go:13
Oct 14 17:59:12 ip-172-20-250-24.us-west-2.compute.internal wupiao[2886]: .
Oct 14 17:59:12 ip-172-20-250-24.us-west-2.compute.internal kube-apiserver[1044]: I1014 17:59:12.858681    1044 handlers.go:13
Oct 14 17:59:13 ip-172-20-250-24.us-west-2.compute.internal wupiao[2886]: .
Oct 14 17:59:13 ip-172-20-250-24.us-west-2.compute.internal kube-apiserver[1044]: I1014 17:59:13.863374    1044 handlers.go:13
Oct 14 17:59:14 ip-172-20-250-24.us-west-2.compute.internal wupiao[2886]: .
Oct 14 17:59:14 ip-172-20-250-24.us-west-2.compute.internal kube-apiserver[1044]: I1014 17:59:14.867960    1044 handlers.go:13
Oct 14 17:59:15 ip-172-20-250-24.us-west-2.compute.internal wupiao[2886]: .
Oct 14 17:59:15 ip-172-20-250-24.us-west-2.compute.internal kube-apiserver[1044]: I1014 17:59:15.872613    1044 handlers.go:13
Oct 14 17:59:16 ip-172-20-250-24.us-west-2.compute.internal wupiao[2886]: .
Oct 14 17:59:16 ip-172-20-250-24.us-west-2.compute.internal kube-apiserver[1044]: I1014 17:59:16.877244    1044 handlers.go:13
Oct 14 17:59:17 ip-172-20-250-24.us-west-2.compute.internal wupiao[2886]: .
Oct 14 17:59:17 ip-172-20-250-24.us-west-2.compute.internal kube-apiserver[1044]: I1014 17:59:17.881835    1044 handlers.go:13
Oct 14 17:59:18 ip-172-20-250-24.us-west-2.compute.internal wupiao[2886]: .
Oct 14 17:59:18 ip-172-20-250-24.us-west-2.compute.internal kube-apiserver[1044]: I1014 17:59:18.886516    1044 handlers.go:13```

[pete0emerson]

Not surprisingly, moving to the latest version of etcd (2.2.0 as of 10/14/2015) worked. I think I'll incrementally bump up the kubernetes version until I can break it.

[pete0emerson]

Kubernetes version 0.20.2 works, where "works" is defined as "terraform completed successfully".

[bakins]

I'll need to dig into the latest version of kubernetes. A lot has changed in last few months.