viktor/infra
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

monitoring(alloy): drop goflow2 + vpa logs from Loki to cut sdc write wear

Browse source 
goflow2 emits ~8 GB/day of per-flow NetFlow JSON to stdout (~64% of all cluster
log volume) but only its Prometheus aggregate metrics are used; vpa is ~1.3
GB/day of Goldilocks/VPA recommender chatter. Both are low-value and were
landing in Loki (PVC on the contended sdc HDD). Drop them at the Alloy relabel.
Reversible (remove the drop rule). Loki ingestion drops ~73%.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Viktor Barzin 2026-06-05 17:44:47 +00:00
parent dbe115910f
commit 5b5b855528
1 changed files with 13 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

13
stacks/monitoring/modules/monitoring/alloy.yaml
View file

@ -37,6 +37,19 @@ alloy:
discovery.relabel "pod_logs" {
targets = discovery.kubernetes.pod.targets
// Drop high-volume, low-value producers from Loki to cut sdc write wear
// (the log PVC is on the contended sdc HDD). goflow2 emits one JSON line
// per NetFlow record to stdout (~8 GB/day, ~64% of all cluster logs) but
// we only use its Prometheus aggregate metrics, not the per-flow logs;
// vpa = Goldilocks/VPA recommender chatter (~1.3 GB/day). Both reversible
// — remove this rule to ship them again. (Added 2026-06-05.)
rule {
source_labels = ["__meta_kubernetes_namespace", "__meta_kubernetes_pod_name"]
separator = "/"
regex = "monitoring/goflow2-.*|vpa/.*"
action = "drop"
}
// Label creation - "namespace" field from "__meta_kubernetes_namespace"
rule {
source_labels = ["__meta_kubernetes_namespace"]