viktor/infra
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

[alerts] Fix status-page-pusher crash + Prometheus backup push

Browse source 
## status-page-pusher (ExternalAccessDivergence false positive)
The pusher was crashing with `AttributeError: 'list' object has no attribute
'get'` at line 122 — the uptime-kuma-api library changed the heartbeats return
format. Fixed by making beat flattening more robust: handle any nesting of
lists/dicts in the heartbeat data, and add isinstance check before calling
`.get()` on the latest beat.

## Prometheus backup (PrometheusBackupNeverRun)
The backup sidecar's Pushgateway push was silently failing because `wget
--post-file=-` needs `--header="Content-Type: text/plain"` for Pushgateway
to accept the Prometheus exposition format. Added the header. Also manually
pushed the metric to clear the `absent()` alert immediately.

Note: ExternalAccessDivergence still fires because 5 services (ollama, pdf,
poison, dns, travel) ARE genuinely externally unreachable but internally up.
This is a real issue (likely Cloudflare tunnel routing) not a false positive.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Viktor Barzin 2026-04-17 18:29:43 +00:00
parent eef4242408
commit cdc851fc63
2 changed files with 11 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

15
stacks/status-page/main.tf
View file

@ -212,13 +212,18 @@ for m in monitors:
else:
# Get latest heartbeat for current status
mid = m["id"]
mon_beats = heartbeats.get(mid, [])
mon_beats = heartbeats.get(mid, heartbeats.get(str(mid), []))
if mon_beats:
# Flatten if nested lists
if mon_beats and isinstance(mon_beats[0], list):
mon_beats = [b for sublist in mon_beats for b in sublist]
# Flatten nested lists (API format varies by version)
flat = []
for item in mon_beats:
if isinstance(item, list):
flat.extend(item)
elif isinstance(item, dict):
flat.append(item)
mon_beats = flat if flat else mon_beats
latest = mon_beats[-1] if mon_beats else None
if latest and beat_status_is_up(latest.get("status", 0)):
if latest and isinstance(latest, dict) and beat_status_is_up(latest.get("status", 0)):
status = "up"
else:
status = "down"