Vaultwarden + 18 pods got stuck for 7h on 2026-05-26 when k8s-node4 went
down: surviving workloads piled onto node1 and hit the
csi.proxmox.sinextra.dev/max-volume-attachments=28 cap. The Proxmox VM also
had 5 stale scsi entries (PVCs long-migrated to other nodes but never
removed from VM config), which bypassed the K8s scheduler safety until the
plugin returned 'no free lun found' at attach time.
Three new alerts on the kube_volumeattachment_info count per node:
- warning at 24/28 (>= 85%), 10m
- critical at 27/28 (1 slot left), 3m
- critical at 28/28 (cap reached), 1m
Also whitelisted kube_volumeattachment_info — the metric was being dropped
by the disk-write-reduction filter (id=559) and the alert queries returned
zero series until it's kept.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
aded77d5ab