systemd units: dep cleanup

[tjkirch]

commit e4dd6170ac3d37ca8fd1bd21740111d5272bfb4e
Author: Tom Kirchner <[email protected]>
Date:   Mon Oct 21 11:43:01 2019 -0700

    Join systemd dependency lines
    
    This is consistent with other units, and arguably easier to find/read the
    dependency list if it's on a single line.
    
    Also add comments to clarify one of the dependencies.

commit b23d21a5cbf908d6c0ef110a10daca4ad69c5e6d
Author: Tom Kirchner <[email protected]>
Date:   Mon Oct 21 11:42:07 2019 -0700

    Remove network.target from apiserver.service dependencies
    
    Since we're now binding to a Unix-domain socket, we no longer need any network
    interfaces up to bind to them.

commit 62e2c91e31d863ddb96e88d94bc97574885692ac
Author: Tom Kirchner <[email protected]>
Date:   Mon Oct 21 11:40:04 2019 -0700

    Remove prepare-var-lib-thar.service
    
    systemd automatically creates directories that represent the mount points
    (Where=) of mount units.
    
    We don't change or remove the mkdir in prepare-local.service because /var is a
    bind mount and we want to be careful about that behavior.

commit 5ccacebeeaa307156e082eff88e2cbb371aaac17
Author: Tom Kirchner <[email protected]>
Date:   Mon Oct 21 11:39:30 2019 -0700

    Use BindsTo= for dependencies that *must* be up

---

Fixes #143

Testing done:

Still joins cluster, pods still run.

All systemd units happy, system is State: running.

Detailed status on services with bigger changes:

bash-5.0# systemctl status kubelet.service
● kubelet.service - Kubelet
   Loaded: loaded (/x86_64-thar-linux-gnu/sys-root/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2019-10-21 19:06:49 UTC; 3min 33s ago
     Docs: https://github.com/kubernetes/kubernetes
  Process: 3193 ExecStartPre=/sbin/iptables -P FORWARD ACCEPT (code=exited, status=0/SUCCESS)
  Process: 3335 ExecStartPre=/usr/bin/host-ctr -source ${POD_INFRA_CONTAINER_IMAGE} -pull-image-only -containerd-socket /run/containerd/containerd.sock -namespace k8s.io (code=exited, status=0/SUCCESS)
 Main PID: 3427 (kubelet)
    Tasks: 18
   Memory: 137.6M
      CPU: 5.031s
   CGroup: /system.slice/kubelet.service
           └─3427 /usr/bin/kubelet --cloud-provider aws --config /etc/kubernetes/kubelet/config --kubeconfig /etc/kubernetes/kubelet/kubeconfig --container-runtime=remote --container-runtime-endpoint=unix:
///run/containerd/containerd.sock --network-plugin cni --root-dir /var/lib/kubelet --cert-dir /var/lib/kubelet/pki --volume-plugin-dir /var/lib/kubelet/plugins/volume/exec --node-ip 192.168.116.152 --nod
e-labels  --register-with-taints  --pod-infra-container-image 602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pause-amd64:3.1

bash-5.0# systemctl status host-containers@control
● [email protected] - Host container: control
   Loaded: loaded (/x86_64-thar-linux-gnu/sys-root/usr/lib/systemd/system/[email protected]; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2019-10-21 19:06:48 UTC; 4min 42s ago
 Main PID: 3192 (host-ctr)
    Tasks: 20
   Memory: 25.3M
   CGroup: /system.slice/system-host\x2dcontainers.slice/[email protected]
           └─3192 /usr/bin/host-ctr -ctr-id=control -source=328549459982.dkr.ecr.us-west-2.amazonaws.com/thar-control:v0.1 -superpowered=false

/var and /var/lib/thar mounts still OK; from admin container:

/dev/xvdb1 on /.thar/rootfs/var type ext4 (rw,nosuid,nodev,noatime)
/dev/xvda10 on /.thar/rootfs/local/var/lib/thar type ext4 (rw,nosuid,nodev,noatime)
/dev/xvda10 on /.thar/rootfs/var/lib/thar type ext4 (rw,nosuid,nodev,noatime)

[ec2-user@ip-192-168-116-152 ~]$ ls /.thar/rootfs/var /.thar/rootfs/var/lib/thar
/.thar/rootfs/var:
cache  lib  log  run  spool  tmp

/.thar/rootfs/var/lib/thar:
datastore  lost+found  moondog.ran  updog

And now, if we stop containerd, kubelet will stop because of the BindsTo:

bash-5.0# systemctl stop containerd
bash-5.0# systemctl status kubelet 
● kubelet.service - Kubelet
   Loaded: loaded (/x86_64-thar-linux-gnu/sys-root/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Mon 2019-10-21 19:15:59 UTC; 1s ago
     Docs: https://github.com/kubernetes/kubernetes
  Process: 3193 ExecStartPre=/sbin/iptables -P FORWARD ACCEPT (code=exited, status=0/SUCCESS)
  Process: 3335 ExecStartPre=/usr/bin/host-ctr -source ${POD_INFRA_CONTAINER_IMAGE} -pull-image-only -containerd-socket /run/containerd/containerd.sock -namespace k8s.io (code=exited, status=0/SUCCESS)
  Process: 3427 ExecStart=/usr/bin/kubelet --cloud-provider aws --config /etc/kubernetes/kubelet/config --kubeconfig /etc/kubernetes/kubelet/kubeconfig --container-runtime=remote --container-runtime-endp
oint=unix:///run/containerd/containerd.sock --network-plugin cni --root-dir /var/lib/kubelet --cert-dir /var/lib/kubelet/pki --volume-plugin-dir /var/lib/kubelet/plugins/volume/exec --node-ip ${NODE_IP} 
--node-labels ${NODE_LABELS} --register-with-taints ${NODE_TAINTS} --pod-infra-container-image ${POD_INFRA_CONTAINER_IMAGE} (code=exited, status=0/SUCCESS)
 Main PID: 3427 (code=exited, status=0/SUCCESS)
      CPU: 11.273s

[bcressey]

Does the kubelet behavior fix #289 ?

[tjkirch]

Does the kubelet behavior fix #289 ?

Nope, still a big delay and log messages about containerd/"libcontai*".

Pushed a fix for the nit regarding a period, though.

[zmrow]

⚡️