forgejo: pin to v11.0.14 + disable Keel (image-rewrite incident 2026-05-24)
Some checks failed
ci/woodpecker/push/build-cli Pipeline failed
ci/woodpecker/push/default Pipeline was successful

On 2026-05-24T15:35:37Z Keel's force-policy rewrote the image tag from
`11.0.14 → 1.18` (codeberg.org/forgejo/forgejo). v1.18 is a Gitea-era
Forgejo (Forgejo forked from Gitea at 1.18 and used pre-Forgejo
versioning early on); the DB had already been migrated to schema 305
by 11.0.14, and 1.18 only knows up to migration 231 → pod refused to
start ("Your database (migration version: 305) is for a newer Gitea,
you can not use the newer database for this old Gitea release (231)").
Exact replay of the 2026-05-16 force-policy tag-rewriting bug
(memory id=1933).

Changes:
- Pin image to explicit `:11.0.14` (latest 11.x, published 2026-05-12)
- Add `keel.sh/policy: "never"` deploy annotation — overrides the
  Kyverno-stamped `force` policy via the chart's `+()` anchor semantics
  (memory id=1972). Keel will no longer touch this workload.
- Drop KEEL_IGNORE_IMAGE from `lifecycle.ignore_changes` (TF owns the
  image now). Restore it if you flip Keel back to `force`.
- Add the KEEL_LIFECYCLE_V1 trio (`kubernetes.io/change-cause`,
  `deployment.kubernetes.io/revision`, `keel.sh/update-time` on the
  pod template) so future TF applies don't fight K8s rollout metadata.

Verified: new pod on v11.0.14 came up Running 1/1.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Viktor Barzin 2026-05-24 22:06:59 +00:00
parent 5a0e4b3dac
commit 5cdac421c2

View file

@ -61,6 +61,12 @@ resource "kubernetes_deployment" "forgejo" {
app = "forgejo"
tier = local.tiers.edge
}
annotations = {
# Keel disabled here its `force` policy rewrote the image tag
# from 11.0.14 1.18 on 2026-05-24 (same bug as memory id=1933).
# TF owns the tag now; bump it manually here when upgrading.
"keel.sh/policy" = "never"
}
}
spec {
replicas = 1
@ -89,7 +95,14 @@ resource "kubernetes_deployment" "forgejo" {
}
container {
name = "forgejo"
image = "codeberg.org/forgejo/forgejo:11"
# Pinned to 11.0.14 (latest 11.x as of 2026-05-12) was on
# floating `:11`. On 2026-05-24T15:35:37Z Keel force-policy
# rewrote the tag from `11.0.14 1.18` (Gitea-era Forgejo
# v1.18), exact replay of the 2026-05-16 force-policy
# tag-rewriting incident (memory id=1933). The pod crashlooped
# because the DB had already been migrated to schema 305 by
# 11.0.14 and v1.18 only knows up to migration 231.
image = "codeberg.org/forgejo/forgejo:11.0.14"
env {
name = "USER_UID"
value = 1000
@ -182,10 +195,16 @@ resource "kubernetes_deployment" "forgejo" {
lifecycle {
ignore_changes = [
spec[0].template[0].spec[0].dns_config, # KYVERNO_LIFECYCLE_V1
spec[0].template[0].spec[0].container[0].image, # KEEL_IGNORE_IMAGE Keel manages tag updates
metadata[0].annotations["keel.sh/policy"],
# KEEL_IGNORE_IMAGE removed 2026-05-24 Keel is disabled for this
# workload now (keel.sh/policy=never annotation above), so TF owns
# the image tag. Restore this ignore_changes line if you flip
# keel.sh/policy back to `force` later.
metadata[0].annotations["keel.sh/match-tag"],
metadata[0].annotations["keel.sh/trigger"],
metadata[0].annotations["keel.sh/pollSchedule"], # KYVERNO_LIFECYCLE_V2
metadata[0].annotations["kubernetes.io/change-cause"],
metadata[0].annotations["deployment.kubernetes.io/revision"],
spec[0].template[0].metadata[0].annotations["keel.sh/update-time"],
]
}
}