Add k8s event to the resource owning the pod(s) being terminated.

[pims]

Hi @linki,

Thoughts on also adding the termination event to the top most owner of a pod in addition to the pod itself?

We're attempting to add visibility for application owners when their pods get terminated – and new pods are starting. The terminated pods kind of disappear from the current view of the deployment, which adds a bit of work to find if any pod has been terminated through ChaosKube.

I know that it's possible to look at events in that namespace to find out what happened, but I believe that adding an event to the deployment/parent of the pod would greatly help in surfacing the actions of ChaosKube.

If that sounds like a useful feature, I'll happily submit a PR.

[linki]

Hi @pims,

that sounds very useful to me.

What about duplicated events in this case? When I'm doing kubectl get events -w would the event show up twice (for the Pod and the Deployment)? Does it make sense to drop the event on the Pod? Do you know how other Kubernetes components deal with this?

[pims]

I need to do some more research on how other project implement this kind of functionality.
I’d like to avoid duplicate events if possible, but I didn't want to introduce breaking changes.

My guess is that most of the complexity comes from navigating the tree of owners.
I'll work on a PoC and we can continue the design discussion in that PR.

[linki]

Sounds good!

[pims]

After prototyping a solution to this, I am no longer convinced this is a good idea.
It drastically extends the permissions granted to Chaoskube and is somewhat convoluted code.

One alternative would be to add annotations on the pod itself that points to any resource that should be notified.

[linki]

Do you mean that in order to reach a Pod's Deployment you have to read the owning ReplicaSet to get its owner?

How about adding the event only on the direct owner for now?

[pims]

Do you mean that in order to reach a Pod's Deployment you have to read the owning ReplicaSet to get its owner?

Yes, which requires doing a Get on the parent and possibly on the parent's parent.

How about adding the event only on the direct owner for now?

That'd be easier, but in the case of a Deployment, I'm not sure how useful it is to add an event to the ReplicaSet controlling the current deployment.

What do you think?