infra/stacks/rybbit
Viktor Barzin 5b49634fe0
All checks were successful
ci/woodpecker/push/default Pipeline was successful
rybbit/crowdsec-cf-sync: stop Cloudflare Lists-API retry-storm (429 self-DoS)
The edge-ban sync was failing every 2 min on Cloudflare HTTP 429
(rate-limited) and never recovering, leaving the crowdsec_ban list empty.

Root cause: backoff_limit=2 made k8s re-run a failing pod up to 3x within
seconds, so each */2 cycle fired a burst of POSTs into Cloudflare's
per-60s Lists-API write limit. That kept the throttle perpetually tripped
(it stopped clearing even after minutes of quiet) — a self-inflicted DoS.

Two changes make the sync gentle and self-healing:
- backoff_limit 2 -> 0: one attempt per */2 cycle (the schedule IS the
  retry cadence), no rapid-fire burst.
- lapi_kv_sync.py: treat a CF 429 as a soft-skip (exit 0, retry next
  cycle) like the existing LAPI fail-safe, instead of fail-loud + k8s
  retry. Any other CF error still fails loud.

Found during a cluster health check (AIOStreams CSI + pfSense SSH issues
handled separately).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 15:23:42 +00:00
..
worker fix: restore tree dropped by 6d224861; land stem95su gdrive-sync (10m) [ci skip] 2026-06-09 08:45:33 +00:00
.terraform.lock.hcl eso: complete migration — chart 2.6.0, all CRs on v1, 1.35 gate cleared 2026-06-23 09:55:51 +00:00
crowdsec_edge.tf rybbit/crowdsec-cf-sync: stop Cloudflare Lists-API retry-storm (429 self-DoS) 2026-06-27 15:23:42 +00:00
lapi_kv_sync.py rybbit/crowdsec-cf-sync: stop Cloudflare Lists-API retry-storm (429 self-DoS) 2026-06-27 15:23:42 +00:00
main.tf ESO: add force_conflicts to all ExternalSecret manifests (fleet sweep) 2026-06-25 21:28:11 +00:00
providers.tf fix: restore tree dropped by 6d224861; land stem95su gdrive-sync (10m) [ci skip] 2026-06-09 08:45:33 +00:00
secrets fix: restore tree dropped by 6d224861; land stem95su gdrive-sync (10m) [ci skip] 2026-06-09 08:45:33 +00:00
terragrunt.hcl fix: restore tree dropped by 6d224861; land stem95su gdrive-sync (10m) [ci skip] 2026-06-09 08:45:33 +00:00