The monitoring stack apply was create-failing on every push with `configmaps
"alert-digest-script" already exists` + `secrets "alert-digest" already exists`
(modules/monitoring/alert_digest.tf) — both resources exist in-cluster but fell
out of Terraform state, so apply tried to CREATE them and errored. Pre-existing
(failed on pipelines 203 AND 204, NOT caused by the t3 alert-rules change). Add
import {} blocks (TF 1.5+ adoption per AGENTS.md) so apply imports + reconciles
instead of failing. Idempotent once imported; safe to remove after a green apply.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Before auto-tracking t3 nightly builds (Viktor's call, risk accepted), stand up
the detection that was missing on 2026-06-09 — when an auto-pulled nightly broke
pairing for ALL users and nothing alerted. Viktor's explicit requirement: make
sure session auth keeps working and revert if the pairing fallback/failure rate
climbs. This is phase 0 (detection) of that work.
- t3-dispatch: exchangeCredential now reports WHICH pairing endpoint answered,
and autoPair logs every outcome (paired user=.. endpoint=.. fallback=..) — so
the real-user browser-session->bootstrap fallback rate is observable. A
non-zero rate flags that a build moved the pairing API (the 2026-06-09 class).
- Loki ruler alerts (devvm journal -> Alertmanager -> Slack): T3PairingBroken
(real users failing to pair), T3PairFallbackHigh (build moved the pairing API),
T3AutoUpdateRolledBack / RollbackFailed / Frozen (enforcer outcomes). Closes
the post-mortem's open "nothing monitors end-to-end pairing" detection gap.
The existing t3-probe only checks GET /api/auth/session==200, which stays 200
even when pairing is dead, so it never caught the outage class.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two small doc additions that also re-include these stacks in Woodpecker's
changed-stack detection. The earlier 2-commit push left chrome-service out of the
HEAD~1..HEAD diff so its ignore_changes fix never applied; the monitoring apply was
separately blocked by a stuck prometheus pending-upgrade (now cleared).
- chrome-service: note the live pod's container order had drifted from this file's
order, so a TF apply reorders them (containers[0] differs live-vs-TF until the
apply lands) -- documents the confusion this caused during diagnosis.
- mam-farming: cross-ref the grabber script that emits mam_grabber_last_run_timestamp.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
MAMFarmingStuck fired whenever the freeleech grabber added 0 torrents in 4h, but
grabbing 0 is normal: the grabber searches a random catalogue offset each run and
legitimately finds nothing when freeleech is dry (account ratio was a healthy
37.5; the alert even misreported it as "0.00" because $value was the grabbed
count, not the ratio). The alert's real intent was to catch the grabber not
running at all (CronJob Forbid-blocked / wedged), but increase(grabbed[4h])==0
cannot distinguish "didn't run" from "ran, nothing to grab" since Pushgateway
serves the last pushed value forever.
The grabber now heartbeats mam_grabber_last_run_timestamp on every completed run
(main success, ratio/mouse skip, and qBittorrent-unreachable paths). The alert
fires only when that heartbeat is >4h stale — the true stuck condition. Cookie
expiry and qBittorrent-down keep their own dedicated alerts.
Surfaced by /cluster-health as a false-firing alert.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The chrome-service container (container[0]) runs the pinned Microsoft Playwright
image, which ships chromium under /ms-playwright. Its image was still listed in
the deployment's lifecycle ignore_changes — a leftover KEEL_IGNORE from before
ADR-0002 #29 moved the novnc container to TF management. With that field ignored,
a stray clobber of container[0] to ghcr chrome-service-novnc:latest (which has no
chromium there) stuck permanently: the container crash-looped ~12h on "chromium
binary not found under /ms-playwright" (273 restarts) and TF could not revert it.
Remove container[0].image from ignore_changes so Terraform pins it to local.image
and re-asserts it on every apply. Both containers are TF-managed now (novnc since
ADR-0002 #29); Keel is inert (policy=never), so nothing should fight TF here.
Surfaced by /cluster-health. Live state was already restored transiently via
kubectl set image; this commit makes the fix durable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CI pipeline 198 failed: the pinned goauthentik/authentik provider has no data "authentik_application" source, so terraform failed the whole authentik plan and applied NOTHING (state unchanged). Replace the data-source lookups with the live pbm_uuid (Vault app) and group_uuid (Allow Login Users) as literals; authentik_policy_binding is supported (used in guest.tf).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Audit found the Vault Authentik application had no authorization binding, so any authenticated identity (incl. a future self-enrolled TripIt External user) could complete Vault OIDC login and get a built-in default-policy token. Bind it to 'Allow Login Users' — existing homelab users inherit that group via its children (verified User.all_groups() includes the parent), parentless TripIt External users are excluded. Closes the only OIDC app the forward-auth fence does not cover.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Viktor wants people outside the homelab to self-register to TripIt with email + a passkey (no password), kept separate from the rest of the homelab. Adds the empty, parentless 'TripIt External' Authentik group and a first-position branch in the catch-all policy that admits those users to tripit.viktorbarzin.me only and denies every other forward-auth host. Inert on apply (group empty => matches no existing user => no lockout). An adversarial review found the fence is forward-auth-only, so the runbook records the OIDC-app containment audit (every sensitive app already requires a trusted group External users won't hold), the Vault->Allow Login Users binding that closes the one open OIDC app, the SMTP prerequisite for email verification, and the before/after access-matrix verification. Flows/SMTP/Vault binding are UI steps per the runbook; the push that applies the catch-all edit must be human-watched (CI auto-applies the authentik stack).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of "the agent never commits": the issue-implementer CLAUDE.md was
subPath-mounted at /home/node/.claude/CLAUDE.md, which made /home/node/.claude
root-owned. The agent (uid 1000) then couldn't create its Bash session-env
there, so EVERY Bash/git call failed (Write/Edit worked, so it silently edited
but never committed). Found by reading the agent transcripts from
state.sqlite -> projection_thread_messages.
Fix: don't mount anything into ~/.claude (it's not honored by T3's SDK anyway).
Behaviour is injected via the dispatch message preamble by the control plane;
files/issue-implementer-CLAUDE.md kept as the canonical source text.
Verified post-fix: a preamble-dispatched task edited README and COMMITTED
(073ab28) unattended.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The bare `t3 serve` behind Authentik showed the manual /pair#token screen, which
didn't connect. Mirror the devvm t3-dispatch: a small stdlib-Node sidecar fronts
t3 serve, and on a cookieless (already Authentik-gated) document load it mints a
pairing credential (`t3 auth pairing create`) and exchanges it at
/api/auth/browser-session for the t3_session cookie, then 302s back. Everything
else — including WebSocket upgrades for the live cockpit — reverse-proxies to
:3773. The Service now targets the sidecar (:8080).
Verified: cookieless GET -> 302 + Set-Cookie t3_session; cookied GET -> 200 SPA.
Matches the t3.viktorbarzin.me experience (Authentik login -> straight into the
cockpit).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Keel "patch"-downgraded the image docker.io/library/node:24 -> library/node:24.0.2,
which is below t3@0.0.27's required node >=24.10, so `t3 serve` exited silently and
the pod crash-looped (~160 restarts / 13h).
Root cause: keel.sh/policy=never was on the POD-TEMPLATE labels, but Keel reads the
policy at the DEPLOYMENT level. The cluster's Kyverno inject-keel-annotations is
opt-out, so it stamped policy=patch and Keel acted on it.
Fix: set keel.sh/policy=never as a deployment-level annotation; ignore_changes the
Kyverno-injected keel.sh/pollSchedule + keel.sh/trigger annotations; the image stays
TF-owned (apply reverted Keel's downgrade). Pod now 1/1, t3 serve 200.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The docker.io fix created the deployment, but wait_for_rollout (default true)
then hung on the OOMing pod and the apply failed — leaving the deployment in
the cluster but NOT in terraform state, so every later apply hit
'deployments.apps "anisette" already exists'. Deleted that orphan and set
wait_for_rollout=false (mirrors tts/llama-cpp slow-start services); readiness
probe still gates Service traffic.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice #2 of claude-agent-service PRD #1 (AFK implementation pipeline). Dedicated
in-cluster T3 Code instance the control plane dispatches issues into; runs the
issue-implementer agent in a git worktree with a live cockpit. Applied + live
2026-06-14 (9 resources).
Pilot-fast: stock docker.io/library/node:24 + install pinned t3@0.0.27 + Claude
CLI at startup onto an SSD-NFS PVC. Authentik-gated ingress. issue-implementer
behaviour ships as a user-level ~/.claude/CLAUDE.md (T3 hardcodes the system
prompt; settingSources loads it) and forbids plan-mode/clarifying-questions so
unattended threads don't stall. Keel-excluded (ADR 0003). wait_for_rollout=false
(slow first start). Image fully-qualified for the Kyverno trusted-registries
allowlist; container mem limit 4Gi (tier-aux LimitRange cap).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The pod CrashLooped with OOMKilled (exit 137): anisette downloads and
initializes Apple's CoreADI provisioning library on startup, spiking past the
128Mi limit before it can bind :6969 (empty logs, liveness 'connection
refused'). Bump request 256Mi / limit 512Mi; steady state is much lower.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First apply was denied at admission — a bare dadoum/anisette-v3-server@sha256
ref isn't in the trusted-registries allowlist (only enumerated DockerHub
user-repo prefixes are). docker.io/* IS allowlisted, so use the explicit
registry prefix; still pulls via the 10.0.20.10 pull-through cache.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Deploy a small stateless anisette-data server so the TripIt iOS Shell can be
sideloaded with SideStore using a free Apple ID, without brokering the
Apple-ID auth dance through a public third-party anisette server (which would
see every login). SideStore points at a stable internal endpoint we control.
- Image: Dadoum/anisette-v3-server, the de-facto standard anisette-v3 server
for SideStore/AltStore. Upstream ships only a mutable :latest (no GitHub
releases / semver / sha tags), so pinned by manifest digest instead of a tag
per the "never :latest" rule. Pulled from DockerHub via the registry-VM
pull-through cache like echo/cyberchef. Diun watches :latest (notify-only) so
a new upstream build prompts a digest re-pin.
- Stateless: emptyDir backs the provisioning-library cache dir (regenerable
download; upstream issue #23 means it doesn't preserve client auth across
restarts anyway) — no PVC, no Vault secret.
- Internal-only endpoint http://anisette.viktorbarzin.lan (auth=none,
allow_local_access_only, ssl_redirect off) — SideStore is a native client
that can't do the Authentik cookie dance, same reasoning as android-emulator's
adb. The .lan CNAME is auto-created by technitium-ingress-dns-sync; never
publicly exposed.
Mirrors the echo/networking-toolbox/android-emulator stack pattern. Service
catalog updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The commit that enabled real city cover photos (a69847a0,
CITY_IMAGE_PROVIDER=wikipedia, #47) was committed to master but its CI run
skipped the tripit stack apply (changed-stack diff race — same class as the
prior "re-apply after pipeline race" fixes). The env never landed in-cluster,
so the provider stayed on its fake 1x1-PNG default and every trip/stay cover
rendered blank/placeholder in prod. This comment touch forces CI to re-apply
the tripit stack; terraform then reconciles the drift (desired HCL already
has the env) so the deployment picks up CITY_IMAGE_PROVIDER=wikipedia.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
My previous commit referenced the new limiter as `health-rate-limit@kubernetescrd`,
omitting the namespace prefix. Traefik CRD middleware refs are
`<namespace>-<name>@kubernetescrd`, and the Middleware lives in the `traefik` ns,
so the router couldn't resolve it — Traefik failed the whole
health.viktorbarzin.me router and returned 404 on every path (the app + pod were
healthy throughout; verified via port-forward).
Correct it to `traefik-health-rate-limit@kubernetescrd`, matching the working
traefik-tripit-rate-limit / traefik-actualbudget-rate-limit references.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Viktor hit 429s browsing the redesigned health app. The default shared limiter
is 10 req/s / burst 50, but each page load is the shell (JS chunks + two
self-hosted Geist woff2) plus a 5-8 call API burst, so fast tab-to-tab
navigation from one client IP overruns burst 50 — Traefik 429s the tail and the
affected cards/pages render empty.
Give health its own limiter (average 100, burst 1000) and skip the default,
exactly as tripit/immich/actualbudget/ha-sofia already do for the same
parallel-burst pattern. Attached via the ingress_factory escape hatch
(skip_default_rate_limit + extra_middlewares).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Documents the 2026-06-13 right-sizing review: Kuma is already lean (~1 check/s, 227 monitors mostly at 300s, 77MB on shared MySQL, 30d retention); the 'scraping too much' concern traced to a fixed socket.io login-timeout incident, not load. Records the deliberate decisions (keep per-service [External] monitors over canaries; keep datastore on shared mysql.dbaas) with rejected alternatives + rationale, plus the known internal-sync no-prune gap (stale Goldilocks monitor cleaned up by hand).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add health-test.viktorbarzin.lan (auth=none, allow_local_access_only,
anti-AI off) pointing at the same health deployment, plus a
DEV_AUTH_EMAIL=vbarzin@gmail.com env on the container. Lets automated
E2E / Playwright / manual screenshots reach the live app without the
Authentik SSO redirect, for testing — while the public
health.viktorbarzin.me ingress stays auth=required (forward-auth fails
closed, so the public path always carries the real X-authentik-email
header and never hits the DEV_AUTH_EMAIL fallback). LAN-only, no public
exposure. Decision recorded in health repo ADR-0008.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The internal + external monitor-sync CronJobs intermittently failed with socketio.exceptions.TimeoutError on api.login(), firing JobFailed -> Slack noise (and leaving monitor sync stale). Kuma 2.3.2 itself is healthy (1/1, 30m CPU); its single Node event loop just briefly stalls under ~300 monitors so the socket.io login handshake occasionally exceeds the client timeout. Wrap connect+login in a 5-attempt / 15s-backoff retry (disconnecting the half-open client between tries) so a transient stall no longer fails the whole job. Applied to both sync scripts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Yesterday's Forgejo 3Gi->4Gi OOM fix pushed its tier-3-edge namespace quota (requests.memory=4Gi) to 100%, firing KubeQuotaAlmostFull + the healthcheck resourcequota check. Forgejo is the git + OCI-registry backbone and legitimately needs ~4Gi, so the edge tier's 4Gi ceiling is too tight. Opt the namespace out of the auto tier quota (resource-governance/custom-quota=true) and define a forgejo-specific ResourceQuota at requests.memory=8Gi, so the 4Gi pod sits at ~50% with headroom. Same opt-out pattern dbaas uses. Re-tiering was rejected: tier 1-cluster is also 4Gi, and 0-core (8Gi) would over-classify Forgejo's priority/eviction.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
k8s-portal was the last in-cluster image build; it now builds on GHA and
pushes ghcr.io/viktorbarzin/k8s-portal:latest, which is PRIVATE (infra repo
default). To pull it: add k8s-portal to the sync-ghcr-credentials Kyverno
allowlist (clones the ghcr-credentials Secret into the namespace) and
reference that secret via imagePullSecrets on the deployment — same wiring
as tripit/recruiter-responder. Completes the no-local-builds migration so
nothing builds container images on the cluster anymore (ADR-0002).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Recovered the real manifest + resolved lockfile (lockfileVersion 3, 71 pkgs)
from the running pod. A parent .gitignore force-ignored package.json, so the
git source tree was incomplete and the image only ever built manually. Now
reproducible on GHA (ADR-0002 no-local-builds).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
package-lock.json was never committed to either lineage — npm ci needs it,
so the build only ever worked from a manual devvm build with a local lock.
npm install resolves from package.json, unblocking the GHA build (ADR-0002).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The last in-cluster image build. GHA build-k8s-portal.yml builds
ghcr.io/viktorbarzin/k8s-portal:latest+sha (path-filtered on the Dockerfile
dir); Keel (force/poll/match-tag) rolls the deployment. Stack image repointed
to ghcr (ignore_changed); .woodpecker/k8s-portal.yml deleted.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Inline note on why the four backup CronJobs moved 10s->600s (bda1bdcb): a 10s deadline silently dropped the 2026-06-13 midnight full-backup run, firing PostgreSQLBackupStale. bda1bdcb rode in the same push as a forgejo change that failed CI on a namespace-quota error, so that pipeline failed before the dbaas apply took effect (live deadline was still 10s). This dbaas-only commit re-triggers the dbaas apply at a clean master so the 600s deadline actually goes live.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 3Gi->6Gi bump in ff3cc44a was rejected by the forgejo namespace tier-quota (requests.memory capped at 4Gi). With Guaranteed QoS the 6Gi request exceeded quota; FailedCreate left forgejo with 0 pods for ~6 min (git remote + OCI registry outage) until I patched the live Deployment back to a schedulable 4Gi. 4Gi is the most the quota allows and is still a headroom bump over the OOM-prone 3Gi. To go higher the tier-quota must be raised in the same change. This reconciles TF to the live 4Gi so the pending/next apply is a no-op rather than reverting to the quota-busting 6Gi.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Forgejo OOMKilled twice on 2026-06-13 at the 3Gi cap (exit 137), briefly taking the git remote and OCI registry down and spiking ingress TTFB to 4.7s and the 4xx rate to 51%. Steady-state is ~2.2Gi but it spiked into the cap (true demand above 3.2Gi). The 2026-06-09 bump to 3Gi was sized for tripit buildkit registry pushes, but that driver is gone now that the Forgejo registry was frozen and emptied today (ADR-0002, images on ghcr), so the spike is git ops / the integrity-probe catalog walk / a possible leak. 6Gi gives headroom on the critical git backbone while we watch whether working-set keeps climbing (which would indicate a leak).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The daily full PostgreSQL backup silently skipped its 2026-06-13 00:00 run, leaving the last full dump 37h old and firing the critical PostgreSQLBackupStale alert. Root cause: startingDeadlineSeconds was 10s on all four dbaas backup CronJobs, so when the CronJob controller was more than 10s late to the midnight tick (many IO-heavy backups all fire at 00:00, the known etcd-starvation window) the run was dropped entirely instead of starting late. 600s lets a brief controller lag still launch the job. Applied to all four (mysql + pg, full + per-db) since they share the footgun and the midnight contention.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ADR-0002 is fully landed (issues #11-#32 closed): every owned image now
builds on GitHub Actions and pushes to ghcr.io/viktorbarzin/<name>, with
Woodpecker reduced to deploy-only. The Forgejo container registry is frozen
and emptied; there are no in-cluster image builds or CI test runs anywhere.
The docs still described the old hybrid topology (DockerHub builds,
Woodpecker-native owned-app builds, the per-pattern migration lists, the
tripit-only pilot framing), which would mislead future sessions and
incident response.
This brings the docs to the completed reality (closes#33):
- docs/architecture/ci-cd.md: full rewrite as the canonical CI/CD reference —
the fleet GHA->ghcr->Woodpecker-deploy pattern, public/private ghcr package
split, infra-owned image workflows (incl. infra-ci on ghcr), the frozen
Forgejo registry, what Woodpecker still runs, and the #31 decommissions.
- .claude/CLAUDE.md: rewrite the "CI/CD Architecture" section to the
fleet-wide final state; FIX the stale claim that claude-memory-mcp builds
to DockerHub (it is GHA->ghcr); note owned images now live on ghcr and the
Forgejo registry is frozen/break-glass near the image-registry bullet.
- .claude/reference/service-catalog.md: f1-stream is GHA->ghcr + Woodpecker
deploy-only (was "Woodpecker-native build->deploy").
- stacks/{tuya-bridge,android-emulator}/variables.tf + stacks/terminal/main.tf:
cosmetic description/comment updates (forgejo -> ghcr; terminal-lobby has no
CI pipeline). Description/comment text only — no stack logic changed.
Historical records (docs/post-mortems/*, docs/plans/*) and ADR-0002 itself
are left untouched as point-in-time records.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
openclaw's install-nextcloud-todos-plugin init still pulled forgejo
nextcloud-todos (would ImagePullBackOff on restart once the forgejo
registry is wiped) -> ghcr:latest. f1-stream stack base (KEEL_IGNORE'd,
live already ghcr via set-image) repointed for fresh-create correctness.
Clears the last LIVE forgejo viktor/* refs before the registry reclaim.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
novnc's image was ignore_changed (KEEL_IGNORE) but nothing manages its
tag (keel.sh/policy=never), so the earlier forgejo->ghcr repoint never
took. Removing container[1].image from ignore_changes lets terragrunt
own novnc=ghcr:latest and roll it. container[0]/[2] (pinned playwright)
stay ignored.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both now built by GHA → public ghcr. Repoint stack image bases
forgejo→ghcr:latest (terragrunt-managed, imagePullPolicy Always picks up
rebuilds). android var default api36-v8 -> latest.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Flips the planning workspace's Stay cover photos from the fake provider to live Wikipedia lead-image fetches (downloaded into STORAGE_DIR, served by the backend, editable per Stay). Part of the new-trip flow feature: every picked destination city gets a banner-ready cover. HOLD-ORDER: pushed only after the tripit image containing CityImageMode.wikipedia rolled out.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Service was already scaled 0/0 and unused (Viktor: 'not used anymore').
Live resources destroyed via scripts/tg destroy (10 resources: deployment,
namespace, service, anubis-travel + PDB/cm/svc/secret, ingress, TLS).
Removing the stack dir; old Woodpecker build (repo 5) deactivated
separately. The harmless legacy 'travel' CNAME->apex in config.tfvars is
left (now 404s; removing it would trigger a full-platform apply).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Formalizing x402-gateway CI (was a manual no-CI image). The deployment
lives in the traefik module; its image was NOT in ignore_changes, so a
set-image deploy would be reverted on the next traefik apply — added it
(KEEL_IGNORE_IMAGE). Base repointed to ghcr:latest; the GHA deploy
set-images the :sha8. Public ghcr package = no pull secret. Inert on the
live pod (image now ignored); rolling cutover keeps forwardAuth up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GHA now builds+pushes ghcr.io/viktorbarzin/claude-memory-mcp (public).
Image is KEEL_IGNORE_IMAGE (set-image managed), so this apply is inert
on the live pod; the stale :17 default literal is corrected to :latest.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GHA now builds+pushes ghcr.io/viktorbarzin/claude-agent-service (public
package, anonymous pulls). Repointed: claude-agent-service (deployment +
git-init/seed-beads-agent inits), claude-breakglass, ci-pipeline-health,
beads-server CronJobs, k8s-version-upgrade (tag var 2fd7670d -> latest —
the Forgejo registry lost that sha; node caches were the only thing
keeping those CronJobs alive). publish-gate: vendor-contact emails
(licensing@/legal@/security@/sales@) ruled license-boilerplate, not PII.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
broker-sync is a CronJob-only consumer (no deployment): new --no-deploy
mode skips Woodpecker registration and renders build.yml without the
deploy job — :latest+Always CronJobs pick up builds on the next run.
wealthfolio stack: ghcr-credentials pull secret + image base repoint.
The wealthfolio-sync image regains a reproducible rebuild path.
Closes: code-62tm
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Second half of the recruiter-responder off-infra migration: the first GHA
build has published ghcr.io/viktorbarzin/recruiter-responder:{1d99a8d5,latest},
so the openclaw plugin-install init container can now follow the ghcr
:latest. The forgejo-side build pipeline was removed by the onboarding
commit, so the old forgejo :latest tag is frozen and would silently serve
stale plugin code. Deferred from the first commit on purpose - flipping it
before the package existed would have wedged the openclaw rollout on
ImagePullBackOff.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Migrating recruiter-responder off in-cluster Woodpecker builds: GHA will
build and push ghcr.io/viktorbarzin/recruiter-responder (PRIVATE package).
This commit lands the pull-side prerequisites BEFORE the first off-infra
build fires:
- stacks/recruiter-responder: image base forgejo -> ghcr (inert on the live
Deployment - both containers are ignore_changes'd; the Woodpecker deploy
moves the tag) + ghcr-credentials imagePullSecrets on the Deployment
(covers the recruiter-responder container AND the alembic-migrate init
container, which share the image).
- stacks/openclaw: ghcr-credentials imagePullSecrets on the openclaw
Deployment - its install-recruiter-plugin init container consumes the
:latest tag of this image. The image ref itself flips to ghcr in a
follow-up once the first GHA build has created the package (flipping now
would ImagePullBackOff on a not-yet-existing package and wedge the apply).
- stacks/kyverno: allowlist openclaw in sync-ghcr-credentials so the pull
secret is cloned into that namespace too.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment-only touch so the changed-stack detection applies
stacks/fire-planner from the current master tree. Pipeline 150 (commit
f18dfa4c — the ghcr image base + ghcr-credentials migration for issue
#26) was auto-killed when the concurrent nextcloud-todos push superseded
it, and pipeline 151 diffed from f18dfa4c onward so the fire-planner
stack changes were never applied (cronjobs still point at the forgejo
image, pod specs lack ghcr-credentials).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The nextcloud-todos build moved off-infra: GHA builds on the public
GitHub mirror and pushes ghcr.io/viktorbarzin/nextcloud-todos (public
package, anonymous pulls); Woodpecker repo 207 is deploy-only. First
ghcr image (:19c22d8c) is already built, deployed and rolled out, so
this repoint lands after the image exists. Both deployment image refs
(main + alembic-migrate init) are ignore_changes'd — no live churn,
the base matters only on resource (re)create. Old image was pulled
from a Forgejo registry package that no longer exists (pods survived
on node image cache only).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Migrating fire-planner off in-cluster Woodpecker builds to GitHub
Actions -> ghcr.io (ADR-0002, issue #26). The image base moves
forgejo.viktorbarzin.me/viktor/fire-planner ->
ghcr.io/viktorbarzin/fire-planner (a PRIVATE ghcr package), so the
deployment, all three cronjobs (recompute, col-refresh,
examples-weekly) and the examples bulk job gain the ghcr-credentials
imagePullSecret (the kyverno sync-ghcr-credentials allowlist already
covers the fire-planner namespace). registry-credentials stays
alongside so the currently-running sha-pinned forgejo image can still
be pulled until the first ghcr deploy lands; the cronjob images are TF
literals and flip to ghcr :latest on this apply.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment-only touch of both stacks so the changed-stack detection applies
them from the current master tree. Two pipelines went wrong in sequence
during the parallel ADR-0002 wave-2 migrations (issues #23/#24):
- pipeline 146 (instagram-poster stack prep, commit 29c69250) was
auto-killed when the concurrent payslip-ingest push superseded it, so
its apply never ran;
- restarting it as pipeline 148 inherited CI_PREV_COMMIT_SHA = the NEW
branch head (6928ce0b) with the OLD checkout (29c69250) — a reverse
diff that re-applied stacks/payslip-ingest from the pre-migration
tree, stripping the ghcr image base + ghcr-credentials pull secrets
that pipeline 147 had just applied (2 resources reverted).
This commit restores the committed payslip-ingest config exactly as
issue #24 landed it and finally applies the instagram-poster ghcr prep
from issue #23. Lesson encoded in the comments: do not restart killed
infra pipelines after master has moved — re-trigger with a touch commit
instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Prep for moving payslip-ingest's image build off-infra to GitHub Actions ->
ghcr.io (ADR-0002 wave 2, issue #24). One stack commit before onboarding:
- image base repointed forgejo.viktorbarzin.me/viktor/payslip-ingest ->
ghcr.io/viktorbarzin/payslip-ingest (private ghcr package)
- ghcr-credentials imagePullSecrets added on the Deployment AND the
actualbudget-payroll-sync CronJob pod specs (namespace is already in the
kyverno sync-ghcr-credentials allowlist; secret verified present)
- the CronJob's SHA pin is retired: terragrunt image_tag 4f70681d -> latest
plus explicit imagePullPolicy Always on the cron container, per the fleet
convention for owned-app CronJobs — one less set-image target, and the
cron can never go back to pulling the dead Forgejo tag
The Deployment keeps KEEL_IGNORE_IMAGE; its concrete :sha8 tag is set by
the Woodpecker deploy pipeline after each GHA build.
Closes: nothing yet — the repo-side onboarding (offinfra-onboard) follows.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
