infra/docs/architecture
Viktor Barzin 51313ee088 kured: fix sentinel-gate OOM — 256Mi limit + self-restart leak guard
The k8s-master gate pod OOM-killed child kubectls 149x/7d (accelerating:
0/day → 15 → 134) while master sat in pending-reboot. Root cause: only the
pending-reboot node's gate pod runs the kubectl-heavy hot path each cycle,
and the immortal bash loop slowly leaks (kubectl forks + Check-4 process
substitution) past the 64Mi cgroup limit. PID 1 bash survives each kill, so
the pod never restarts — just silent oom_events.

Fix: raise limit 64Mi→256Mi (headroom for ~30-50Mi kubectl forks) + add a
MAX_ITER=72 self-exit (~6h) so kubelet restarts the pod fresh and the leak
can never accumulate, regardless of how long a node stays pending-reboot.

Docs: post-mortem + automated-upgrades.md gate note.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-31 14:49:04 +00:00
..
agent-task-tracking.md Add agent task tracking documentation 2026-04-15 17:11:26 +00:00
authentication.md docs/auth: sync to current auth enum (required/app/public/none) 2026-05-11 19:28:42 +00:00
automated-upgrades.md kured: fix sentinel-gate OOM — 256Mi limit + self-restart leak guard 2026-05-31 14:49:04 +00:00
backup-dr.md backup-dr docs: refresh diagrams for daily/immich-only architecture 2026-05-26 20:00:31 +00:00
chrome-service.md chrome-service: open NP for Traefik → noVNC sidecar (port 6080) 2026-05-07 18:40:11 +00:00
ci-cd.md [forgejo] Phases 3+4+5: cutover, decommission, docs sweep 2026-05-07 18:30:02 +00:00
compute.md immich: GPU-accelerate video transcoding (NVENC + NVDEC) 2026-05-29 18:05:34 +00:00
databases.md redis: revert 3-node Sentinel HA to single standalone instance [ci skip] 2026-05-30 17:49:43 +00:00
dns.md phpipam-pfsense-import: every 5min → hourly 2026-04-26 22:48:43 +00:00
homepage.md add homepage auto-discovery documentation [ci skip] 2026-03-25 13:06:43 +02:00
incident-response.md [claude-agent-service] Migrate all pipelines from DevVM SSH to K8s HTTP 2026-04-18 10:12:02 +00:00
llama-cpp.md infra/llama-cpp: benchmark report + -fa flag fix 2026-05-10 15:03:16 +00:00
mailserver.md monitoring: bring EmailRoundtripStale threshold docs in sync with for:20m 2026-04-21 22:39:46 +00:00
monitoring.md security(wave1): W1.1 audit-log shipping LIVE + W1.5 trusted-registries Enforce LIVE 2026-05-19 06:37:54 +00:00
multi-tenancy.md add architecture documentation for all infrastructure subsystems [ci skip] 2026-03-24 00:55:25 +02:00
networking.md traefik: uncap download duration (writeTimeout 60s->0), upload window 3600s [ci skip] 2026-05-30 17:46:59 +00:00
overview.md gpu: schedule off NFD label, not k8s-node1 hostname 2026-04-22 13:43:07 +00:00
secrets.md docs: comprehensive audit and update of all architecture docs and runbooks [ci skip] 2026-04-06 13:21:05 +03:00
security.md security(wave1): W1.7 analysis snapshot — observation data → allowlist plan 2026-05-22 15:22:25 +00:00
storage.md storage docs: document the per-VM SCSI-LUN cap (proxmox-csi) 2026-05-26 02:56:27 +00:00
vpn.md docs: Technitium DNS IP — 10.0.20.101 → 10.0.20.201 2026-05-23 08:53:52 +00:00
wave1-egress-observation-2026-05-22.md security(wave1): W1.7 analysis snapshot — observation data → allowlist plan 2026-05-22 15:22:25 +00:00