infra/state/stacks/monitoring
Viktor Barzin 375a3d91d5 [monitoring] Exclude websocket protocol from HighServiceLatency alert
Traefik records websocket connection lifetimes (minutes to hours) as
"request duration." When websockets close, the full lifetime pollutes
the average latency metric — Authentik showed 6.7s avg (201s websocket
avg) vs 0.065s actual HTTP avg. This caused ~90 false alerts/day across
12 services (Authentik, Vaultwarden, Terminal, HA, etc.).

Changes:
- Add protocol!="websocket" filter to HighServiceLatency alert expr
- Raise minimum traffic threshold from 0.01 to 0.05 rps to filter
  statistical noise from services with <3 req/min
- Remove .githooks/pre-commit file-size hook (blocked state commits)

Validated against 7-day historical data: 637 breaches → ~2 with both
filters applied (99.7% reduction).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 21:51:19 +00:00
..
terraform.tfstate.enc [monitoring] Exclude websocket protocol from HighServiceLatency alert 2026-04-15 21:51:19 +00:00