viktor/infra
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
infra/stacks/changedetection
History
Viktor Barzin ce79bd5c04 Add node hang instrumentation and scale down chromium services 
- Add journald collection to Alloy (loki.source.journal) for kernel OOM,
  panic, hung task, and soft lockup detection — ships system logs off-node
  so they survive hard resets
- Add 5 Loki alerting rules (KernelOOMKiller, KernelPanic, KernelHungTask,
  KernelSoftLockup, ContainerdDown) evaluating against node-journal logs
- Fix Loki ruler config: correct rules mount path (/var/loki/rules/fake),
  add alertmanager_url and enable_api
- Add Prometheus alerts: NodeMemoryPressureTrending (>85%), NodeExporterDown,
  NodeHighIOWait (>30%)
- Add caretta tolerations for control-plane and GPU nodes
- Scale down chromium-based services to 0 for cluster stability:
  f1-stream, flaresolverr, changedetection, resume/printer
2026-03-13 22:20:28 +00:00
..
.terraform.lock.hcl [ci skip] Move Terraform modules into stack directories 2026-02-22 14:38:14 +00:00
backend.tf [ci skip] Move Terraform modules into stack directories 2026-02-22 14:38:14 +00:00
main.tf Add node hang instrumentation and scale down chromium services 2026-03-13 22:20:28 +00:00
providers.tf [ci skip] phase 5+6: update CI pipelines for SOPS, add sensitive=true to secret vars 2026-03-07 14:30:36 +00:00
secrets [ci skip] Move Terraform modules into stack directories 2026-02-22 14:38:14 +00:00
terragrunt.hcl [ci skip] Phase 3: Create 66 service stacks and migrate state 2026-02-22 13:56:34 +00:00
tiers.tf [ci skip] add auto-generated tiers.tf, planning docs, and helm chart cache 2026-03-06 23:55:57 +00:00