5d22b449f9
Stage 1 of moving private images off the registry:2 container at registry.viktorbarzin.me:5050 (which has hit distribution#3324 corruption 3x in 3 weeks) onto Forgejo's built-in OCI registry. No cutover risk — pods still pull from the existing registry until Phase 3. What changes: * Forgejo deployment: memory 384Mi→1Gi, PVC 5Gi→15Gi (cap 50Gi). Explicit FORGEJO__packages__ENABLED + CHUNKED_UPLOAD_PATH (defensive, v11 default-on). * ingress_factory: max_body_size variable was declared but never wired in after the nginx→Traefik migration. Now creates a per-ingress Buffering middleware when set; default null = no limit (preserves existing behavior). Forgejo ingress sets max_body_size=5g to allow multi-GB layer pushes. * Cluster-wide registry-credentials Secret: 4th auths entry for forgejo.viktorbarzin.me, populated from Vault secret/viktor/ forgejo_pull_token (cluster-puller PAT, read:package). Existing Kyverno ClusterPolicy syncs cluster-wide — no policy edits. * Containerd hosts.toml redirect: forgejo.viktorbarzin.me → in-cluster Traefik LB 10.0.20.200 (avoids hairpin NAT for in-cluster pulls). Cloud-init for new VMs + scripts/setup-forgejo-containerd-mirror.sh for existing nodes. * Forgejo retention CronJob (0 4 * * *): keeps newest 10 versions per package + always :latest. First 7 days dry-run (DRY_RUN=true); flip the local in cleanup.tf after log review. * Forgejo integrity probe CronJob (*/15): same algorithm as the existing registry-integrity-probe. Existing Prometheus alerts (RegistryManifestIntegrityFailure et al) made instance-aware so they cover both registries during the bake. * Docs: design+plan in docs/plans/, setup runbook in docs/runbooks/. Operational note — the apply order is non-trivial because the new Vault keys (forgejo_pull_token, forgejo_cleanup_token, secret/ci/global/forgejo_*) must exist BEFORE terragrunt apply in the kyverno + monitoring + forgejo stacks. The setup runbook documents the bootstrap sequence. Phase 1 (per-project dual-push pipelines) follows in subsequent commits. Bake clock starts when the last project goes dual-push. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
109 lines
3 KiB
Bash
109 lines
3 KiB
Bash
#!/bin/sh
|
|
# Forgejo container-package retention.
|
|
#
|
|
# For each container package owned by ${FORGEJO_OWNER}, keep newest
|
|
# ${KEEP_LAST_N} versions + always keep tag "latest". Deletes the rest via
|
|
# DELETE /api/v1/packages/{owner}/container/{name}/{version}.
|
|
#
|
|
# DRY_RUN=true logs what would be deleted but issues no DELETE calls.
|
|
#
|
|
# Required env:
|
|
# FORGEJO_HOST e.g. http://forgejo.forgejo.svc.cluster.local
|
|
# FORGEJO_OWNER e.g. viktor
|
|
# FORGEJO_USER PAT owner (write:package scope)
|
|
# FORGEJO_TOKEN PAT
|
|
# KEEP_LAST_N integer (default 10)
|
|
# DRY_RUN true|false (default true)
|
|
|
|
set -eu
|
|
|
|
apk add --no-cache curl jq >/dev/null
|
|
|
|
OWNER="${FORGEJO_OWNER}"
|
|
KEEP="${KEEP_LAST_N:-10}"
|
|
DRY="${DRY_RUN:-true}"
|
|
BASE="${FORGEJO_HOST%/}/api/v1"
|
|
|
|
AUTH_HEADER="Authorization: token $FORGEJO_TOKEN"
|
|
|
|
echo "Forgejo cleanup: owner=$OWNER keep_last=$KEEP dry_run=$DRY"
|
|
echo "API base: $BASE"
|
|
|
|
# Page through ALL container packages.
|
|
TMPDIR=$(mktemp -d)
|
|
trap 'rm -rf "$TMPDIR"' EXIT
|
|
ALL="$TMPDIR/all.json"
|
|
echo "[]" > "$ALL"
|
|
|
|
PAGE=1
|
|
while :; do
|
|
RESP=$(curl -sf -H "$AUTH_HEADER" \
|
|
"$BASE/packages/$OWNER?type=container&limit=50&page=$PAGE")
|
|
COUNT=$(echo "$RESP" | jq 'length')
|
|
if [ "$COUNT" = "0" ]; then break; fi
|
|
jq -s '.[0] + .[1]' "$ALL" <(echo "$RESP") > "$TMPDIR/merged.json"
|
|
mv "$TMPDIR/merged.json" "$ALL"
|
|
PAGE=$((PAGE + 1))
|
|
# Safety: never run away.
|
|
if [ "$PAGE" -gt 100 ]; then break; fi
|
|
done
|
|
|
|
TOTAL=$(jq 'length' "$ALL")
|
|
echo "Found $TOTAL package version(s)."
|
|
|
|
if [ "$TOTAL" = "0" ]; then
|
|
echo "Nothing to do."
|
|
exit 0
|
|
fi
|
|
|
|
# Group by name and process each group.
|
|
NAMES=$(jq -r '.[].name' "$ALL" | sort -u)
|
|
|
|
DEL=0
|
|
KEPT=0
|
|
|
|
for NAME in $NAMES; do
|
|
# All versions of this name, sorted by created_at descending.
|
|
jq --arg n "$NAME" '
|
|
[.[] | select(.name == $n)]
|
|
| sort_by(.created_at) | reverse
|
|
' "$ALL" > "$TMPDIR/$NAME.json"
|
|
|
|
N_VERSIONS=$(jq 'length' "$TMPDIR/$NAME.json")
|
|
echo "[$NAME] $N_VERSIONS version(s)"
|
|
|
|
# Build the keep set: top $KEEP + anything tagged 'latest'.
|
|
jq -r --argjson keep "$KEEP" '
|
|
[.[0:$keep][].version] + [.[] | select(.version == "latest") | .version]
|
|
| unique
|
|
| .[]
|
|
' "$TMPDIR/$NAME.json" > "$TMPDIR/$NAME.keep"
|
|
|
|
# Build the delete set.
|
|
jq -r '.[].version' "$TMPDIR/$NAME.json" \
|
|
| grep -vxFf "$TMPDIR/$NAME.keep" > "$TMPDIR/$NAME.delete" || true
|
|
|
|
D_COUNT=$(wc -l < "$TMPDIR/$NAME.delete" | tr -d ' ')
|
|
K_COUNT=$(wc -l < "$TMPDIR/$NAME.keep" | tr -d ' ')
|
|
echo " keep=$K_COUNT delete=$D_COUNT"
|
|
KEPT=$((KEPT + K_COUNT))
|
|
|
|
while IFS= read -r VER; do
|
|
[ -z "$VER" ] && continue
|
|
URL="$BASE/packages/$OWNER/container/$NAME/$VER"
|
|
if [ "$DRY" = "true" ]; then
|
|
echo " DRY_RUN would DELETE $URL"
|
|
else
|
|
HTTP=$(curl -s -o /dev/null -w '%{http_code}' \
|
|
-X DELETE -H "$AUTH_HEADER" "$URL" || echo "000")
|
|
if [ "$HTTP" = "204" ] || [ "$HTTP" = "200" ]; then
|
|
echo " deleted $NAME:$VER"
|
|
else
|
|
echo " FAIL $NAME:$VER HTTP $HTTP"
|
|
fi
|
|
fi
|
|
DEL=$((DEL + 1))
|
|
done < "$TMPDIR/$NAME.delete"
|
|
done
|
|
|
|
echo "Summary: kept=$KEPT to_delete=$DEL dry_run=$DRY"
|