Look Ma, No secrets in my Kubernetes Cluster!

Kubernetes Secrets are major maintenance and security headache.

Traditionally, 3-4 approaches exists for secret mgmt.

1. Define secrets in Git as plain-text and deploy them. Restrict access to git repo.
2. Use git level encryption like git-crypt or Mozilla SOPS and encrypt the files which contain secrets
3. Create encrypted secret values using Sealed Secrets. Store them in git. Use sealed-secrets operator in cluster which reads encrypted CRD values and decrypt them and create k8s secrets for you.
4. Use External Secrets operator to manage secrets in cloud secret management tools

All 4 approaches have trade-offs like Access control issues, git history cleanup, no-encryption at runtime,

Secret-Store-CSI takes different approach. It allows you to make use of standard cloud based secret-mgmt solutions like AWS secrets manager or established players like vault to manage secrets and access to them. Secret-Store-CSI is a Container Storage Interface and so it directly mounts secrets as a storage volume in the POD! There is no actual Kubernetes secret resource created at all. It also syncs any change in secret in external secret manager in the pod.

In the demo -

1. Secret-store-CSI is deployed on a k8s cluster in AWS.
2. AWS kube2iam restricts access to aws secrets manager to only the operator pod.
3. Demo app uses secret value provided by the operator.
4. We change secret in secret mgr and we see app get reloaded (due to stakator-reloader) and uses new value

Speaker: Vijay Dharap, Tech Lead at Kubermatic