viktor/infra
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

matrix: auto-reload Synapse on DB credential rotation (Reloader)

Browse source 
Synapse injects the Postgres password into homeserver.yaml only at
startup (inject-db-password initContainer). matrix-db-creds is rotated
by Vault via ESO (15m refresh), so each rotation left the running pod
with a stale password and Synapse DB auth failed silently until a
manual rollout restart. Found today via Loki: ~12.9k/hr 'password
authentication failed for user matrix' lines; secret password verified
working against the DB while the 10-day-old pod held the pre-rotation
value.

Add the explicit secret.reloader.stakater.com/reload annotation so
Reloader rolls the deployment whenever the secret changes (explicit
form, not auto/search, because the secret is referenced only in an
initContainer env var). Live pod already restarted to restore service;
this prevents recurrence on the next rotation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Viktor Barzin 2026-06-05 21:48:48 +00:00
parent e7ece3eaf9
commit d661d074ef
1 changed files with 12 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

12
stacks/matrix/main.tf
View file

@ -98,6 +98,18 @@ resource "kubernetes_deployment" "matrix" {
app = "matrix"
tier = local.tiers.aux
}
annotations = {
# Synapse reads the DB password ONLY at startup: the inject-db-password
# initContainer seds matrix-db-creds into homeserver.yaml. That secret is
# rotated by Vault via the ESO above (15m refresh), so without an
# auto-reload the running pod keeps a stale password and Synapse's DB
# auth fails on every rotation until a manual `rollout restart` (observed
# 2026-06-05). Reloader watches the named secret and rolls the deployment
# when it changes. Explicit form (not auto/search) because the secret is
# referenced only in an initContainer env var, not a mount/envFrom, so
# Reloader's reference auto-discovery is unreliable here.
"secret.reloader.stakater.com/reload" = "matrix-db-creds"
}
}
spec {
replicas = 1