apiserver: enable audit logging (low-write Metadata) + ship to Loki
Some checks failed
ci/woodpecker/push/default Pipeline failed
ci/woodpecker/push/build-cli Pipeline was successful

Resource changes/deletions are now attributable (the novelapp deletion this week
was untraceable because apiserver audit was off). Low-write policy: drops
reads/noise, Metadata level on mutations, omitStages RequestReceived. Wired into
the kube-apiserver static-pod manifest + kubeadm-config (v1beta4
extraArgs/extraVolumes -> survives kubeadm upgrade) on k8s-master; Alloy tails
/var/log/kubernetes/audit/audit.log -> Loki {job=kubernetes-audit}.

Root cause that had silently blocked this AND OIDC for weeks: a stray
kube-apiserver.yaml.bak inside /etc/kubernetes/manifests/ was a duplicate
static-pod manifest kubelet ran instead of the real one, dropping every flag
added to the real manifest. Removed it. Runbook added.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Viktor Barzin 2026-06-06 16:50:34 +00:00
parent 3696ff5922
commit 551412488b
3 changed files with 137 additions and 3 deletions

View file

@ -222,11 +222,14 @@ alloy:
forward_to = [loki.write.default.receiver]
}
// Kubernetes audit log collection from /var/log/kubernetes/audit.log
// Requires alloy.mounts.varlog=true to mount /var/log from the host
// Kubernetes audit log collection from /var/log/kubernetes/audit/audit.log
// (kube-apiserver --audit-log-path on k8s-master; rotated siblings stay in
// the audit/ subdir). Requires alloy.mounts.varlog=true to mount /var/log
// from the host. Enabled 2026-06-06 once apiserver audit actually started
// writing — see infra/scripts/k8s-apiserver-audit-policy.yaml.
local.file_match "audit_logs" {
path_targets = [{
__path__ = "/var/log/kubernetes/audit.log",
__path__ = "/var/log/kubernetes/audit/audit.log",
job = "kubernetes-audit",
node = env("HOSTNAME"),
}]