CI: migrate Docker build/push from Woodpecker to GitHub Actions

Was: Woodpecker built+pushed to DockerHub, then `kubectl set image` patched
the four Deployments to a pinned numeric tag. With Deployments pinned to
:51 (immutable tag), Keel polled forever and never saw a digest bump — and
no DockerHub pull-secret meant Keel hit 401 on the private repo at every
poll. The 4-Deployment setup also had a latent ImagePullBackOff risk: if a
node was replaced, fresh pulls would fail.

Now: GHA builds+pushes (.github/workflows/build-{api,frontend}.yml) on push
to master. Cluster Deployments reference :latest with an imagePullSecret
sourced from Vault via ESO (codified in infra/stacks/real-estate-crawler/
main.tf, separate commit). Keel polls :latest, sees the new digest after
each GHA build, and rolls all four Deployments.

- .github/workflows/build-api.yml: pytest (unit + integration/regression/
  e2e/test_listing_geojson) + buildx push viktorbarzin/realestatecrawler
  to {<8-char-sha>, latest}.
- .github/workflows/build-frontend.yml: vitest (all 4 ex-shards in one
  run) + Vite build with VITE_MAPBOX_TOKEN from GHA secret + buildx push
  viktorbarzin/immoweb to {<8-char-sha>, latest}.
- .woodpecker/{api,frontend}.yml renamed to
  .woodpecker/build-fallback-{api,frontend}.yml with `event: deployment`
  so they no longer fire on push — kept as manual-only fallback if GHA
  is down (CLAUDE.md convention from the 10 already-migrated projects).
- .claude/CLAUDE.md: Git Workflow section updated to reflect GHA as
  primary + the dockerhub-pull-secret wiring.

GHA repo secrets DOCKERHUB_TOKEN and MAPBOX_TOKEN populated from Vault
fields viktor.dockerhub_registry_password and ci/global.wrongmove-mapbox-token
respectively (DOCKERHUB_USERNAME=viktorbarzin was already set).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Viktor Barzin 2026-05-18 19:11:31 +00:00
parent 01a940b9b6
commit c2acbf5d2e
5 changed files with 176 additions and 6 deletions

View file

@ -81,8 +81,10 @@ See `.env.sample` for the full list. Key ones:
## Git Workflow
- CI: Woodpecker CI (`.woodpecker/`) builds Docker images on push to `master`, deploys to K8s.
- Linting: GitHub Actions runs Ruff on PR diffs.
- **CI: GitHub Actions** (`.github/workflows/build-api.yml`, `build-frontend.yml`) builds + pushes Docker images to DockerHub on `master` push. **Keel** in the cluster watches `:latest` on `viktorbarzin/realestatecrawler` and `viktorbarzin/immoweb` and rolls the four `realestate-crawler-*` Deployments on digest change. No Woodpecker deploy POST — Keel is the rollout mechanism.
- Pull-secret on the namespace: `dockerhub-pull-secret`, synced from Vault `secret/viktor.dockerhub_registry_password` via ExternalSecret (codified in `infra/stacks/real-estate-crawler/main.tf`). Required because the DockerHub repos are private.
- Fallback: `.woodpecker/build-fallback-{api,frontend}.yml` (event: `deployment`, manual-only) preserves the in-cluster build path if GHA is down.
- Linting: GitHub Actions runs Ruff on PR diffs (`.github/workflows/ruff.yml`).
- Keep commits focused — one logical change per commit.
- Group related files (e.g., code + its tests) in the same commit.

83
.github/workflows/build-api.yml vendored Normal file
View file

@ -0,0 +1,83 @@
name: Build and Push API
# Migrated from .woodpecker/api.yml on 2026-05-18. GHA builds + pushes to
# DockerHub; Keel polls and rolls the cluster Deployments. No POST to
# Woodpecker — Keel is the deploy mechanism now.
on:
push:
branches: [master]
paths-ignore:
- 'frontend/**'
- '**.md'
- '.woodpecker/**'
concurrency:
group: build-api-${{ github.ref }}
cancel-in-progress: true
env:
IMAGE: viktorbarzin/realestatecrawler
jobs:
test-and-build:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.13'
cache: 'pip'
- name: System deps
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends libglib2.0-0
- name: Python deps (mirror .woodpecker/api.yml install list)
run: |
python -m pip install --upgrade pip
pip install --quiet \
pytest pytest-asyncio pytest-cov httpx fakeredis aioresponses \
fastapi uvicorn sqlmodel sqlalchemy alembic pyjwt cryptography \
celery redis click aiohttp aiohttp-socks pillow numpy pytesseract \
opentelemetry-api opentelemetry-sdk opentelemetry-exporter-prometheus \
opentelemetry-instrumentation-fastapi opentelemetry-instrumentation-sqlalchemy \
python-dotenv webauthn apprise tenacity prometheus-client \
email-validator opencv-python-headless tqdm pandas cachetools watchdog
- name: Unit tests
run: pytest tests/unit/ -v --tb=short
- name: Integration / regression / e2e tests
run: pytest tests/integration/ tests/regression/ tests/e2e/ tests/test_listing_geojson.py -v --tb=short
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
username: viktorbarzin
password: ${{ secrets.DOCKERHUB_TOKEN }}
# github.sha is the commit hash (40-char hex, controlled by git, not
# external input) — using it via env to satisfy the workflow-injection
# lint hook and keep things uniform.
- id: meta
env:
GIT_SHA: ${{ github.sha }}
run: echo "sha=$(printf '%s' "$GIT_SHA" | cut -c1-8)" >> "$GITHUB_OUTPUT"
# Pushes both :<8-char-sha> (traceability) and :latest (Keel watches this).
# Deployments reference :latest; Keel detects the digest bump and rolls.
- uses: docker/build-push-action@v6
with:
context: .
file: Dockerfile
target: production
platforms: linux/amd64
push: true
tags: |
${{ env.IMAGE }}:${{ steps.meta.outputs.sha }}
${{ env.IMAGE }}:latest
cache-from: type=gha,scope=api
cache-to: type=gha,scope=api,mode=max

77
.github/workflows/build-frontend.yml vendored Normal file
View file

@ -0,0 +1,77 @@
name: Build and Push Frontend
# Migrated from .woodpecker/frontend.yml on 2026-05-18. GHA builds + pushes
# to DockerHub; Keel polls :latest and rolls the realestate-crawler-ui
# Deployment. The .env.production file carries VITE_MAPBOX_TOKEN at build
# time (Vite reads it from the build context) — same shape as the
# Woodpecker pipeline, just sourced from a GHA secret instead.
on:
push:
branches: [master]
paths:
- 'frontend/**'
- '.github/workflows/build-frontend.yml'
concurrency:
group: build-frontend-${{ github.ref }}
cancel-in-progress: true
env:
IMAGE: viktorbarzin/immoweb
jobs:
test-and-build:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '24'
cache: 'npm'
cache-dependency-path: frontend/package-lock.json
- name: Install deps
working-directory: frontend
run: npm ci
# Vitest runs all 4 Woodpecker shards in one job — single GHA runner
# is plenty fast and avoids matrix-cost x4.
- name: Vitest
working-directory: frontend
run: npx vitest run --reporter=verbose
# Writes .env.production for Vite's build-time read. Secret is piped
# via env (not `${{ }}` directly in run:) so shell metacharacters in
# the value, if any, can't break out.
- name: Write .env.production for Vite build
env:
MAPBOX_TOKEN: ${{ secrets.MAPBOX_TOKEN }}
run: printf 'VITE_MAPBOX_TOKEN=%s\n' "$MAPBOX_TOKEN" > frontend/.env.production
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
username: viktorbarzin
password: ${{ secrets.DOCKERHUB_TOKEN }}
- id: meta
env:
GIT_SHA: ${{ github.sha }}
run: echo "sha=$(printf '%s' "$GIT_SHA" | cut -c1-8)" >> "$GITHUB_OUTPUT"
- uses: docker/build-push-action@v6
with:
context: frontend
file: frontend/Dockerfile
target: production
platforms: linux/amd64
push: true
tags: |
${{ env.IMAGE }}:${{ steps.meta.outputs.sha }}
${{ env.IMAGE }}:latest
cache-from: type=gha,scope=frontend
cache-to: type=gha,scope=frontend,mode=max

View file

@ -1,6 +1,10 @@
# Fallback pipeline — only fires on manual `deployment` event after GHA
# (.github/workflows/build-api.yml) became the primary builder on
# 2026-05-18. Kept around in case GHA is down and a build needs to ship
# from inside the cluster. `event: deployment` means it never auto-fires
# on push.
when:
- event: push
branch: master
- event: deployment
clone:
git:

View file

@ -1,6 +1,10 @@
# Fallback pipeline — only fires on manual `deployment` event after GHA
# (.github/workflows/build-frontend.yml) became the primary builder on
# 2026-05-18. Kept around in case GHA is down and a build needs to ship
# from inside the cluster. `event: deployment` means it never auto-fires
# on push.
when:
- event: push
branch: master
- event: deployment
clone:
git: