Vault seals itself on every restart. That’s by design — it protects the encryption keys from being read off disk. In production on AWS or GCP, you point Vault at KMS and it unseals automatically. On a self-hosted Kubernetes cluster, you have no KMS. Without a solution, every pod restart requires manual intervention.
The Problem
After any Vault pod restart — rolling update, node reboot, OOM kill — the pod comes up sealed. Sealed Vault means no secrets. No secrets means every workload that depends on Vault starts failing. On a self-managed cluster there’s no operator watching for this. The pod shows Running in Kubernetes but Vault itself is non-functional.
kubectl exec -n vault vault-0 -- vault status
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed true ← everything is broken until this is false
The Approach
Store the unseal key in a Kubernetes Secret. Mount it into the Vault pod as a volume. Use a postStart lifecycle hook to unseal automatically after the process starts.
This mirrors what cloud KMS does — Vault reads a key from an external source on startup and unseals itself. The difference is the key lives in etcd instead of KMS.
Implementation
1. Create the secret after vault operator init
kubectl create secret generic vault-unseal-keys \
--namespace vault \
--from-literal=key="<unseal-key-from-init>"
2. Mount it into the Vault pod via Helm values
server:
volumes:
- name: unseal-key
secret:
secretName: vault-unseal-keys
optional: true # first install: secret doesn't exist yet
volumeMounts:
- name: unseal-key
mountPath: /vault/unseal
readOnly: true
optional: true is important. On first install the secret doesn’t exist — Vault needs to start so you can run vault operator init to get the key. Without optional: true the pod won’t start because the secret is missing.
3. Add a postStart hook
server:
postStart:
- /bin/sh
- -c
- |
sleep 15
if [ -f /vault/unseal/key ] && \
vault status 2>/dev/null | grep -q "Sealed.*true"; then
vault operator unseal "$(cat /vault/unseal/key)"
fi
The sleep 15 gives Vault time to start its listener before the hook runs. The conditional checks that the key file exists and that Vault is actually sealed — so the hook is safe to run on already-unsealed pods.
The Bootstrap Sequence
First run:
- Helm installs Vault pod —
optional: truemeans no secret needed yet - Pod starts sealed (expected — not initialized)
vault operator init -key-shares=1 -key-threshold=1— generates unseal key- Create
vault-unseal-keysSecret with the key vault operator unseal— manually, once- Every subsequent restart: postStart hook handles it automatically
The Tradeoff
The unseal key is in etcd, protected by Kubernetes RBAC, but not in a HSM. Anyone with cluster admin access can read it. For a homelab or internal platform this is acceptable. For a multi-tenant cluster handling regulated data, use cloud KMS or Vault’s built-in Transit unseal (another Vault instance as the key source).
The key-shares=1 -key-threshold=1 setup (single key, single shard) is also a simplification. Production uses multiple key shares distributed across operators so no single person can unseal alone. For a single-operator cluster it adds no real security benefit.
Takeaway
postStart lifecycle hooks run after the container process starts but before Kubernetes marks the container ready. That window is exactly what you need for unseal — Vault is running, not yet receiving traffic, safe to unseal. The optional: true volume mount breaks the chicken-and-egg problem on first install. Together these make Vault self-healing across restarts without requiring a cloud key management service.