CAKC017 error does not remove "client.pem" causing continual failures

[rysmith0315]

Summary

The sidecar container startup has been modified to restart the authenticator process instead of letting the container fail. This is being done to allow the Pod to remain available during brief interruptions of service to the follower. In this case, the authenticator timed out waiting for the cert to arrive and terminated. The cert was injected shortly after the process terminated.
Once the authenticator was restarted the file from the previous run was still in the container. The new instance of the authenticator process will no longer match the key used to generate the client.pem file and will continue to fail until the pod is bounced.

We know this is due to how we run the sidecar but its necessary for availability.

INFO: 2023/11/14 20:50:39.960247 authenticator.go:84: CAKC040 Authenticating as user 'host/conjur/authn-k8s///'
ERROR: 2023/11/14 20:50:40.191386 client.go:28: CAKC017 Failed to parse key-pair from pem. Reason: tls: private key does not match public key

Steps to Reproduce

1. Open a terminal session for the authenticator container 2. create a file /etc/conjur/ssl/client.pem with either a bad pem of just trash (doesn't matter) 3. execute "ps" to find the authenticator process id and kill it 4. execute the authenticator process and observe the CAKC017 error 5. delete /etc/conjur/ssl/client.pem and observe the authenticator recover and request a new cert

Expected Results

Delete /etc/conjur/ssl/client.pem whenever a CAKC017 error occurs

Actual Results

/etc/conjur/ssl/client.pem remains and continues to be read instead of requesting a new cert

Reproducible

• [X ] Always
• Sometimes
• Non-Reproducible

Version/Tag number

0.25.0

Environment setup

Running in Openshift with a custom startup script for the sidecar
" command:
- /bin/sh
- '-c'
- "#!/bin/sh\nJITTER=$(( $RANDOM % 10 * 10 ))\nDELAY=$(( $CONJUR_RESTART_DELAY + $JITTER ))\necho "start authenticator"\nuntil /usr/local/bin/authenticator; do\n\techo "authenticator crashed with exit code $? Respawning in $DELAY seconds" >&2\n\tfor i in seq $DELAY -10 1 ; do echo "$i" ; sleep 10 ; done\ndone"
"

Additional Information

[jodyhuntatx]

hey Ryan! Thanks for logging this. We've escalated this for evaluation. On the face of it, seems like a legit enhancement.