viktor/infra
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

kyverno: exclude postiz namespace from Keel auto-update injection

Browse source 
Postiz was generating hourly Slack spam and a wedged rollout, both
Keel-driven:
- Bundled redis StatefulSets run docker.io/bitnamilegacy/redis; Keel
  tried 7.4.0->7.4.1/7.4.2 every poll but require-trusted-registries
  denies bitnamilegacy/* (only bitnami/* allowlisted) -> endless
  deny/retry/Slack-ping loop.
- Keel bumped postiz-app v2.21.7->v2.21.8 on 2026-05-26; the surge pod
  couldn't schedule under the 3Gi tier-4-aux quota, wedging the rollout
  for 3 days.

postiz Terraform state is heavily drifted (~2/30 resources tracked), so
per-workload opt-out can't be applied from the postiz stack. Durable
guard is here (clean kyverno state). Operational steps applied live via
kubectl (postiz stack can't apply): removed keel.sh/enrolled=true from
the namespace, set keel.sh/policy=never (annotation+label) on all 4
workloads, rolled postiz back to the running v2.21.7. Keel restarted
(scale 0->1) to drop postiz-app from its in-memory tracker; confirmed it
no longer tracks postiz.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Viktor Barzin 2026-05-29 19:15:21 +00:00
parent ae72ad51bb
commit 0f26bf030b
1 changed files with 19 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

19
stacks/kyverno/modules/kyverno/keel-annotations.tf
View file

@ -110,6 +110,24 @@ resource "kubectl_manifest" "policy_inject_keel_annotations" {
# cnpg-system + dbaas (state-coupled), nvidia (pinned to
# 570.195.03 until NVIDIA ships ubuntu26.04 images per
# code-8vr0), kube-system (k8s built-ins).
#
# 2026-05-29: ADDED postiz. Two Keel failure modes, both
# unfixable while postiz stays enrolled:
# 1. Bundled redis StatefulSets run docker.io/bitnamilegacy/
# redis (the Broadcom archive repo). Keel hourly resolves
# newer patch tags (7.4.0→7.4.1/7.4.2) and tries to roll,
# but require-trusted-registries (security-policies.tf)
# denies bitnamilegacy/* (only bitnami/* is allowlisted).
# Endless deny→retry→Slack-ping loop.
# 2. Keel bumped postiz-app v2.21.7→v2.21.8 (2026-05-26); the
# surge pod can't schedule under the 3Gi tier-4-aux quota,
# wedging the rollout for 3 days (rolled back to v2.21.7).
# postiz Terraform state is heavily drifted (~2/30 resources
# tracked — memory id=2798/2840), so per-workload opt-out can't
# be applied from the postiz stack. Namespace exclude here
# (clean kyverno state) is the reliable guard. Workloads also
# carry keel.sh/policy=never (annotation+label) set via kubectl
# since the postiz stack can't apply.
namespaces = [
"keel",
"calico-system",
@ -118,6 +136,7 @@ resource "kubectl_manifest" "policy_inject_keel_annotations" {
"nvidia",
"kube-system",
"tigera-operator",
"postiz",
]
}
},