viktor/infra
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

t3-afk: fix crashloop — exclude from Keel at the deployment level
All checks were successful
ci/woodpecker/push/default Pipeline was successful
Details

Browse source 
Keel "patch"-downgraded the image docker.io/library/node:24 -> library/node:24.0.2,
which is below t3@0.0.27's required node >=24.10, so `t3 serve` exited silently and
the pod crash-looped (~160 restarts / 13h).

Root cause: keel.sh/policy=never was on the POD-TEMPLATE labels, but Keel reads the
policy at the DEPLOYMENT level. The cluster's Kyverno inject-keel-annotations is
opt-out, so it stamped policy=patch and Keel acted on it.

Fix: set keel.sh/policy=never as a deployment-level annotation; ignore_changes the
Kyverno-injected keel.sh/pollSchedule + keel.sh/trigger annotations; the image stays
TF-owned (apply reverted Keel's downgrade). Pod now 1/1, t3 serve 200.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Viktor Barzin 2026-06-15 10:32:38 +00:00
parent 214638216b
commit 82a0c5aedf
1 changed files with 19 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

25
stacks/t3-afk/main.tf
View file

@ -131,6 +131,16 @@ resource "kubernetes_deployment" "t3_afk" {
name = "t3-afk"
namespace = kubernetes_namespace.t3_afk.metadata[0].name
labels = local.labels
# keel.sh/policy=never must be a DEPLOYMENT-level annotation — that's where
# Keel reads it. (A pod-template label is ignored by Keel, which is why the
# earlier attempt failed.) The cluster's Kyverno inject-keel-annotations
# policy is opt-OUT: it stamps policy=patch on any workload that doesn't
# carry its own keel.sh/policy — and Keel then "patch"-downgraded
# node:24 -> node:24.0.2 (below t3@0.0.27's required node >=24.10), which
# crash-looped `t3 serve`. ADR 0003 (Keel-excluded).
annotations = {
"keel.sh/policy" = "never"
}
}
spec {
@ -146,11 +156,7 @@ resource "kubernetes_deployment" "t3_afk" {
template {
metadata {
labels = merge(local.labels, {
# Belt-and-braces: this namespace isn't Keel-enrolled, but pin the
# churny pre-1.0 T3 explicitly out of any auto-upgrade. ADR 0003.
"keel.sh/policy" = "never"
})
labels = local.labels
}
spec {
@ -312,7 +318,14 @@ resource "kubernetes_deployment" "t3_afk" {
}
lifecycle {
ignore_changes = [spec[0].template[0].spec[0].dns_config] # KYVERNO_LIFECYCLE_V1
ignore_changes = [
spec[0].template[0].spec[0].dns_config, # KYVERNO_LIFECYCLE_V1
# Kyverno's inject-keel-annotations stamps pollSchedule/trigger alongside
# the policy; we own keel.sh/policy=never above, but ignore these two so
# they don't perpetually drift the plan.
metadata[0].annotations["keel.sh/pollSchedule"],
metadata[0].annotations["keel.sh/trigger"],
]
}
}