From 7cc9cde5b18bf520edad25dbbe6d3c1cedf5a8eb Mon Sep 17 00:00:00 2001 From: Viktor Barzin Date: Mon, 29 Jun 2026 15:32:22 +0000 Subject: [PATCH] external-secrets: enable ESO Vault token cache to cut sdc write churn Add --enable-vault-token-cache to the ESO controller (a graduated, non-experimental flag in chart 2.6.0). Until now ESO authenticated to Vault with login -> lookup-self -> revoke-self on *every* secret fetch. Across 92 ExternalSecrets refreshing every 15m that measured ~0.22 logins/s + ~0.22 revoke-self/s on the active Vault member, and each cycle is a token create+revoke (plus its lease) written to the Raft log on all three members. Those fsync-heavy writes land on the contended PVE RAID1 7200rpm HDD (sdc) -- one of the write sources behind the recurring control-plane flaps (code-oflt write-reduction). The eso kubernetes-auth role already issues a 240h periodic, unlimited- use token, so the churn was pure waste: ESO discarded a perfectly good token after a single use. With token caching ESO mints one token and reuses/renews it, collapsing logins from ~13/min to a handful per token lifetime. Verified live: vault cache initialized, 112/113 ExternalSecrets Ready (the one failure, instagram-poster, is pre-existing data drift unrelated to auth), logins dropped to ~0 after warm-up. Co-Authored-By: Claude Opus 4.8 --- stacks/external-secrets/main.tf | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/stacks/external-secrets/main.tf b/stacks/external-secrets/main.tf index 5356a437..11583fac 100644 --- a/stacks/external-secrets/main.tf +++ b/stacks/external-secrets/main.tf @@ -28,6 +28,14 @@ resource "helm_release" "external_secrets" { values = [yamlencode({ installCRDs = true + # --enable-vault-token-cache (graduated in chart 2.6.0): ESO reuses ONE Vault token + # (the `eso` k8s-auth role mints a 240h periodic token) instead of login+lookup+revoke + # on every secret fetch. Cuts the dominant Vault Raft write-churn on the contended PVE + # sdc HDD: baseline was ~0.22 login/s + ~0.22 revoke-self/s across 92 ExternalSecrets@15m, + # each cycle a token create+revoke (+lease) written ×3 members. (code-oflt write-reduction) + extraArgs = { + "enable-vault-token-cache" = "" + } })] }