viktor/infra
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
infra/scripts
History
Viktor Barzin 6950b8f197 cluster-health #43: tighten PVE thermal threshold to 65 C 
Per Viktor: healthy baseline range is 55-65 C; anything above 65 C is a
signal a VM/workload is using too much CPU and warrants investigation.

Previous thresholds were calibrated to the hardware's TjMax (75/83 C) —
that was too lax, since cluster-load-driven elevation arrives a long
time before throttling. The 65 C cutoff matches the live Prometheus
baseline (Apr 20-May 8 2026: peak 61-69 C, avg 51-55 C) and the
session-observed correlation: above 65 C means the cluster is doing
sustained work that should be looked at, even if hardware is still
nowhere near its limit.

Updated:
  PASS  < 65 C   (within 55-65 baseline)
  WARN  65-82 C  (elevated; check top kvm processes for the culprit)
  FAIL  >= 83 C  (at/above TjMax — throttling imminent)

Verified live: 67 C now WARN (was PASS under the 75 C threshold).
2026-05-22 14:17:01 +00:00
..
server_safe_poweroff move helper scripts in scripts dir [ci skip] 2025-10-11 17:14:59 +00:00
check-ingress-auth-comments.py infra/scripts/tg: enforce ingress_factory auth-comment convention 2026-05-22 14:16:44 +00:00
cluster_healthcheck.sh cluster-health #43: tighten PVE thermal threshold to 65 C 2026-05-22 14:17:01 +00:00
cluster_manager.py chore: add untracked stacks, scripts, and agent configs 2026-04-15 09:33:06 +00:00
daily-backup.service backup: fix daily-backup silent failures, postiz pg_dump CronJob, doc reconcile 2026-05-10 11:12:39 +00:00
daily-backup.sh scripts: timeout rsync + sqlite calls in daily-backup 2026-05-22 14:16:42 +00:00
daily-backup.timer rename weekly-backup → daily-backup across scripts, timers, services, and docs [ci skip] 2026-04-13 18:37:04 +00:00
extend_vm_storage.sh [ci skip] expand k8s worker nodes to 256G, update inventory and extend script 2026-02-28 16:00:16 +00:00
forgejo-migrate-orphan-images.sh [forgejo] Migration script: exclude empty repos, all-images full mode 2026-05-07 23:29:34 +00:00
frigate-bulk-classify.js [ci skip] sync tfstate and add frigate helper scripts 2026-02-12 23:11:23 +00:00
frigate-inspect.mjs [ci skip] sync tfstate and add frigate helper scripts 2026-02-12 23:11:23 +00:00
gen_service_stacks.py cleanup: remove calibre and audiobookshelf stacks after ebooks migration [ci skip] 2026-03-25 23:56:07 +02:00
graceful-db-maintenance.sh add pod dependency management via Kyverno init container injection 2026-03-15 19:17:57 +00:00
image_pull.sh chore: add untracked stacks, scripts, and agent configs 2026-04-15 09:33:06 +00:00
image_pull_remote.sh chore: add untracked stacks, scripts, and agent configs 2026-04-15 09:33:06 +00:00
kill_ns.sh move helper scripts in scripts dir [ci skip] 2025-10-11 17:14:59 +00:00
lvm-pvc-snapshot.sh [backup] Fix lvm-pvc-snapshot Pushgateway push (stdout pollution in cmd_prune_count) 2026-04-25 14:30:58 +00:00
lvm-pvc-snapshot.timer add 3-2-1 backup pipeline: weekly PVC file copy, NFS mirror, pfsense, offsite sync 2026-04-06 14:53:28 +03:00
migrate-state-to-pg [infra] Migrate Terraform state from local SOPS to PostgreSQL backend 2026-04-16 19:33:12 +00:00
migrate_service_state.sh cleanup: remove calibre and audiobookshelf stacks after ebooks migration [ci skip] 2026-03-25 23:56:07 +02:00
nfs-change-tracker.service consolidate offsite backup: inotify change tracking, deduplicate Synology paths [ci skip] 2026-04-13 18:06:20 +00:00
node_registry_manager.sh some nits on the registry manager script - note it is still not working correctly [ci skip] 2025-10-17 19:23:43 +00:00
offsite-sync-backup.service rename weekly-backup → daily-backup across scripts, timers, services, and docs [ci skip] 2026-04-13 18:37:04 +00:00
offsite-sync-backup.sh rename weekly-backup → daily-backup across scripts, timers, services, and docs [ci skip] 2026-04-13 18:37:04 +00:00
offsite-sync-backup.timer switch backup + offsite sync from weekly to daily — RPO 7d → 1d [ci skip] 2026-04-13 18:24:38 +00:00
parse-postmortem-todos.sh fix: use sh instead of bash in pipeline (Alpine compat) 2026-04-14 17:29:14 +00:00
pfsense-haproxy-bootstrap.php mailserver: split healthcheck path off PROXY-aware listeners + book-search uses ClusterIP 2026-05-05 19:45:33 +00:00
pfsense-nat-mailserver-haproxy-flip.php [mailserver] Phase 4+5 — pfSense HAProxy cutover for all 4 mail ports [ci skip] 2026-04-19 12:24:50 +00:00
pfsense-nat-mailserver-haproxy-unflip.php [mailserver] Phase 4+5 — pfSense HAProxy cutover for all 4 mail ports [ci skip] 2026-04-19 12:24:50 +00:00
postmortem-pipeline.sh [claude-agent-service] Migrate all pipelines from DevVM SSH to K8s HTTP 2026-04-18 10:12:02 +00:00
pve-nfs-exports fix(post-mortem): add /etc/exports to git, NFS health check in daily-backup, document CSI requirements [PM-2026-04-14] 2026-04-14 18:08:24 +00:00
renew_worker_certs.sh move helper scripts in scripts dir [ci skip] 2025-10-11 17:14:59 +00:00
setup-containerd-pullthrough.sh chore: add untracked stacks, scripts, and agent configs 2026-04-15 09:33:06 +00:00
setup-forgejo-containerd-mirror.sh [forgejo] Phase 0 of registry consolidation: prepare Forgejo OCI registry 2026-05-07 23:29:33 +00:00
setup-task-pipeline.sh [ci skip] add Forgejo task pipeline for OpenClaw AI agent 2026-03-07 21:11:07 +00:00
setup_containerd_mirrors.sh add upstream fallback to containerd registry mirrors 2026-04-02 11:05:30 +03:00
state-sync [infra] Migrate Terraform state from local SOPS to PostgreSQL backend 2026-04-16 19:33:12 +00:00
stop_storage_services.sh cleanup: remove calibre and audiobookshelf stacks after ebooks migration [ci skip] 2026-03-25 23:56:07 +02:00
task-processor.sh [ci skip] add Forgejo task pipeline for OpenClaw AI agent 2026-03-07 21:11:07 +00:00
tg infra/scripts/tg: enforce ingress_factory auth-comment convention 2026-05-22 14:16:44 +00:00
update-istio-injection.sh move helper scripts in scripts dir [ci skip] 2025-10-11 17:14:59 +00:00
update_k8s.sh k8s-version-upgrade: retry kubeadm apply on static-pod-hash timeout 2026-05-22 14:17:00 +00:00
update_node.sh k8s-version-upgrade: automated kubeadm/kubelet/kubectl upgrade pipeline 2026-05-22 14:16:42 +00:00
upgrade_state.sh upgrade-state: filter transient registry digest-check errors 2026-05-22 14:17:00 +00:00
vault-kubeconfig remove SOPS pipeline, deploy ESO + Vault DB/K8s engines 2026-03-15 16:37:38 +00:00
woodpecker-register-forgejo-repo.sh [woodpecker] Programmatic Forgejo repo registration 2026-05-10 11:12:36 +00:00