K08: Secrets Management Failures

📋 What Is It?

Secrets Management Failures occur when sensitive material - database passwords, API keys, TLS private keys, cloud credentials, tokens - is created, stored, distributed, or consumed in a cluster so that an unintended party can read it. The core misunderstanding: a Kubernetes Secret is base64-encoded, not encrypted, and by default it sits in etcd as recoverable plaintext unless encryption-at-rest is on. The cluster is a credential hub, so one leaked secret often unlocks systems far outside it - and exposure is permanent, so rotation, not deletion, is the remedy.

K08 OWASP Rank

base64 ≠ 🔒 Root Cause

Permanent Exposure

⚠️ Common Exploits

API Reads: a permissive token runs get/list secrets; base64 is no obstacle
Token Harvest: the automounted SA token in a popped pod is a free API credential
Env-Var Dump: secrets in env vars leak via /proc, logs, crash dumps
etcd / Snapshot Read: unencrypted datastore yields every Secret at once
Image Mining: keys baked into layers recovered offline, never expire
Git History: committed secrets live forever, scanned continuously
ConfigMaps & Logs: connection strings with embedded passwords in the clear

🔴 Attack Flow

1. Locate - where do secrets live? etcd, API, env vars, images, Git, CI
↓
2. Access - reach the weakest copy: a token, snapshot, public image, commit
↓
3. Extract - decode base64, dump env, pull layers, grep history
↓
4. Escalate - use creds against DBs, cloud APIs, registries
↓
5. IMPACT: Blast radius beyond the cluster - cloud-account breach!

❌ Vulnerable Code

# Bad: base64 is encoding, NOT encryption - trivially reversible
kind: Secret
data:
  password: U3VwZXJTZWNyZXQxMjM=   ← echo -n 'SuperSecret123' | base64
$ kubectl get secret db-creds -o jsonpath='{.data.password}' | base64 -d
SuperSecret123

# Bad: secret hardcoded as an env var - leaks via /proc, logs, crash dumps
env:
  - name: DB_PASSWORD
    value: "SuperSecret123"       ← plaintext in the pod spec

# Bad: a credential in a ConfigMap - no confidentiality semantics at all
kind: ConfigMap
data:
  DATABASE_URL: "postgres://app:SuperSecret123@db:5432/prod"

# Bad: key baked into an image layer forever (leaks in `docker history`)
ENV AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# Bad: broad RBAC - any bound identity reads every secret in the namespace
rules:
  - resources: ["secrets"]
    verbs: ["get", "list", "watch"]   ← plus token automounted everywhere
                

✅ Secure Code

# Good: etcd encryption-at-rest via an external KMS provider
kind: EncryptionConfiguration
resources:
  - resources: ["secrets"]
    providers:
      - kms: { apiVersion: v2, name: cluster-kms }   ← KEK held outside etcd
      - identity: {}                              ← read-only fallback, never first
# Then re-encrypt: kubectl get secrets -A -o json | kubectl replace -f -

# Good: deliver as a read-only file with tight mode, not an env var
volumes:
  - name: db
    secret:
      secretName: db-creds
      defaultMode: 0400          ← read-only, owner only

# Good: source from an external manager at runtime (Secrets Store CSI)
kind: SecretProviderClass
spec:
  provider: vault           ← no static K8s Secret to leak; supports rotation

# Good: least-privilege - one named secret, read-only, no list/watch
rules:
  - resources: ["secrets"]
    resourceNames: ["db-creds"]   ← not all secrets, just this one
    verbs: ["get"]

# Good: no free API token in the pod
automountServiceAccountToken: false

# Good: GitOps - commit ciphertext only (Sealed Secrets / SOPS)
$ kubeseal --format yaml < secret.yaml > sealed-secret.yaml
                

✓ Prevention Checklist

Enable etcd encryption-at-rest, ideally KMS-backed
Access-control and encrypt etcd snapshots/backups
Source secrets from an external manager (Vault, CSI, ESO)
Never commit plaintext secrets - use Sealed Secrets / SOPS
Keep secrets out of images, ConfigMaps, and Dockerfiles
Prefer read-only file mounts over environment variables
Scope RBAC - narrow get, avoid cluster-wide list secrets
Set automountServiceAccountToken: false by default
Rotate on a schedule and after any suspected exposure
Prefer short-lived, dynamically issued credentials
Scan images, repos, and CI for secrets (gitleaks, trufflehog)
Scrub secrets from application and build logs

🔍 Detection & Tools

Kubernetes Tools:

Trivy kubeaudit kubescape Falco Polaris

Secret Scanning & Managers:

gitleaks trufflehog Vault Sealed Secrets SOPS External Secrets

How to Test:

kubectl auth can-i list secrets --all-namespaces - should be no for most
Scan images: trivy image --scanners secret myapp:tag
Grep Git history: gitleaks detect --source .
Check for env-var secrets in pod specs and ConfigMaps
Confirm etcd encryption-at-rest is enabled (not identity only)

🌍 Real-World

Credentials Leaked Through Source Control: manifests/Helm values with plaintext creds committed to a repo. History is permanent; automated scanners crawl public forges continuously and abuse exposed cloud keys within minutes.
Secrets Recovered From Container Images: build-time secrets baked into layers, or a copied .env/.git directory - anyone who can pull the image extracts them offline.
Whole-Cluster Exposure via etcd: etcd without encryption-at-rest, snapshots stored without strong access control - a single backup yields every credential in the cluster.

Described as recurring classes of failure, not specific attributed breaches - no invented CVEs or statistics. The durable truth: secrets leak by default and the leaks are permanent.

📌 Quick Tips

DO NOT treat base64 as encryption
DO NOT commit secrets to Git or bake them into images
DO NOT put credentials in ConfigMaps or logs
DO enable etcd encryption-at-rest (KMS)
DO source secrets from an external manager at runtime
DO mount files over env vars and scope RBAC tightly
DO rotate on exposure - deletion never suffices

📜 Related Risks

Connected OWASP Kubernetes Top 10:

K03 Overly Permissive RBAC - broad reads on secrets
K06 Broken Authentication - protects the API/etcd doors
K02 Supply-Chain - secrets baked into images
K05 Inadequate Logging & Monitoring - detecting harvest

🔐 Secrets Management Failures