k8s-version-upgrade: compat gate must not false-block patch upgrades

The compat gate compared every addon's matrix ceiling against the target
k8s minor unconditionally. That is correct for a minor JUMP, but it also
blocked patch upgrades within the minor the cluster is ALREADY running:
ESO v0.12's matrix ceiling is 1.31, the cluster runs 1.34.9, so a target of
1.34.10 (a patch) was refused with "external-secrets supports k8s <= 1.31;
target 1.34 exceeds it" — even though the running cluster is itself proof ESO
0.12 works on 1.34. That silently defeats autonomous patching (it would have
bitten the moment a 1.34.10 was published).

Fix: a target at or below the running minor crosses into no new k8s minor, so
every installed addon is already empirically proven on it — check_addons now
returns no reasons when target_minor <= running_minor. Added running_minor()
(oldest kubelet across nodes, mirroring the detector; RUNNING_K8S env override
for tests) and pass it in. Minor jumps are unchanged: 1.34->1.35 still blocks
on ESO 0.12 + kyverno 1.16. removed-API + containerd checks are naturally
inert for patches (no API removal / containerd floor inside a minor) and keep
running as defence. Added test_compat_gate.py (8 cases) covering both paths.

Verified end-to-end against live Prometheus: target 1.34.10 -> EXIT 0 (safe),
target 1.35.6 -> EXIT 2 (blocked on ESO+kyverno).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

stacks/k8s-version-upgrade/scripts/compat-gate.py

@@ -46,7 +46,31 @@ def kget(args):
         return ""
 
 
-def check_addons(matrix, tgt):
+def running_minor():
+    """Oldest kubelet minor across all nodes, as a (major, minor) tuple.
+
+    Mirrors the detector's "oldest kubelet" choice so a partially-upgraded
+    cluster is judged by its lowest node, not its newest. RUNNING_K8S overrides
+    for local testing. None if undeterminable (treated as a minor jump → the
+    addon checks run in full, fail-safe)."""
+    env = os.environ.get("RUNNING_K8S")
+    if env:
+        return minor(env)
+    out = kget(["get", "nodes", "-o",
+                "jsonpath={range .items[*]}{.status.nodeInfo.kubeletVersion}{\"\\n\"}{end}"])
+    minors = [minor(line) for line in out.splitlines() if minor(line)]
+    return min(minors) if minors else None
+
+
+def check_addons(matrix, tgt, running):
+    # A target at or below the RUNNING minor (a patch, or a same/lower minor)
+    # crosses into no new k8s minor, so every installed addon is already
+    # empirically proven on it — addon ceilings only constrain a true minor jump.
+    # Without this guard an addon whose matrix ceiling sits below the running
+    # minor (e.g. ESO 0.12 → 1.31 on a cluster already running 1.34) would
+    # false-block legitimate patch upgrades, defeating autonomous patching.
+    if running and tgt <= running:
+        return []
     reasons = []
     for a in matrix.get("addons", []):
         img = kget(["-n", a["namespace"], "get", a["kind"], a["resource"],
@@ -127,7 +151,8 @@ def main():
         print(f"could not parse compat matrix JSON: {e}")
         sys.exit(3)
 
-    reasons = (check_addons(matrix, tgt)
+    running = running_minor()
+    reasons = (check_addons(matrix, tgt, running)
                + check_removed_apis(tgt)
                + check_containerd(matrix, tgt))
     if reasons: