RFC: does a CAA-provisioned agent-config.toml undermine confidentiality?

[mkulke]

Currently we have agent-config.toml defined as an entry write_files directive. we do this to set image-registry-auth from a CAA daemonset. I assume this is a problem, the agent could be configured in certain ways to disable security features (e.g. policy=off, even if doesn't exist yet, it might be introduced in the future).

this also illustrates why cloud-init as part of the stack is problematic. it will provision files to any location without restrictions, including a malicious agent-config.toml.

There are several options to address this:

• provision agent-config.toml via initdata
  • upside: registry secrets will be measured
  • downside: leaking implementation details (peerpod kata-agent config options) to userland
• provision registry secrets via initdata
  • upside: registry secrets will be measured
  • downside: needs some extra glue to provide registry secrets to kata-agent
  • downside: registry secrets in cleartext in k8s annotation
• provision registry secrets directly via write_files
  • downside: needs some extra glue to provide registry secrets to kata-agent

I'm leaning towards the last option atm. opinions?