How to use edgex with spiffe service on multiple nodes?

[LavenderQAQ]

Hello, I'm from the openyurt project, I'm trying to deploy a secure version of edgex on k8s, but I'm having some problems.

Edgex's service by default requires a shared volume to distribute tokens (almost every service has /tmp/edgex/secrets mounted), and edgex-init is required to generate scripts when security versions are started. In the environment is not enabled PVC k8s unable to use a shared file system to complete the operation, but we found at spiffe.

Could you please tell me how I can test it using edgex-compose and whether it can get rid of the constraints of the shared file system?

[lenny-goodell]

Please see the Helm example here for reference on K8s deployment.
https://github.com/edgexfoundry/edgex-examples/tree/main/deployment/helm

[bnevis-i]

I also see a possible future where we use Kubernetes service account authentication to talk to a cluster-based Vault instance. In this scenario, we could have an init container that pre-seeds the token, and getting rid of the pre-shared secrets volume that way. There's also an issue filed for that

[bnevis-i]

@LavenderQAQ A simpler stopgap method to achieve what you want might be to write a file system watcher, that when an init script for Vault token is written to disk, slurps it up, sticks it in a configmap or secret via the kubernetes API, and then the remote target container consumes it via a secretRef or configmap Ref, where Kubernetes is doing the hard work of distribution.

[LavenderQAQ]

@bnevis-i Yeah, yeah, it looks like we need the init container for script distribution. This would be much easier if you had a cluster-level vault(for token distribution). For now, I'd like to use an init container for script distribution. Of course I need to discuss this with the openyurt community first, and if this goes well I would be willing to help edgex-helm repo.

[LavenderQAQ]

@bnevis-i We did consider writing a file system watcher before😆, but we feel this one really didn't look that beautiful. We're still trying not to do this as much as possible.

[LavenderQAQ]

@bnevis-i Personally, I currently feel that the init container distributing scripts and using spiffe to distribute tokens seems like a good solution. It seems a little more elegant🧐.