CI: migrate Docker build/push from Woodpecker to GitHub Actions
Was: Woodpecker built+pushed to DockerHub, then `kubectl set image` patched
the four Deployments to a pinned numeric tag. With Deployments pinned to
:51 (immutable tag), Keel polled forever and never saw a digest bump — and
no DockerHub pull-secret meant Keel hit 401 on the private repo at every
poll. The 4-Deployment setup also had a latent ImagePullBackOff risk: if a
node was replaced, fresh pulls would fail.
Now: GHA builds+pushes (.github/workflows/build-{api,frontend}.yml) on push
to master. Cluster Deployments reference :latest with an imagePullSecret
sourced from Vault via ESO (codified in infra/stacks/real-estate-crawler/
main.tf, separate commit). Keel polls :latest, sees the new digest after
each GHA build, and rolls all four Deployments.
- .github/workflows/build-api.yml: pytest (unit + integration/regression/
e2e/test_listing_geojson) + buildx push viktorbarzin/realestatecrawler
to {<8-char-sha>, latest}.
- .github/workflows/build-frontend.yml: vitest (all 4 ex-shards in one
run) + Vite build with VITE_MAPBOX_TOKEN from GHA secret + buildx push
viktorbarzin/immoweb to {<8-char-sha>, latest}.
- .woodpecker/{api,frontend}.yml renamed to
.woodpecker/build-fallback-{api,frontend}.yml with `event: deployment`
so they no longer fire on push — kept as manual-only fallback if GHA
is down (CLAUDE.md convention from the 10 already-migrated projects).
- .claude/CLAUDE.md: Git Workflow section updated to reflect GHA as
primary + the dockerhub-pull-secret wiring.
GHA repo secrets DOCKERHUB_TOKEN and MAPBOX_TOKEN populated from Vault
fields viktor.dockerhub_registry_password and ci/global.wrongmove-mapbox-token
respectively (DOCKERHUB_USERNAME=viktorbarzin was already set).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
parent
01a940b9b6
commit
c2acbf5d2e
5 changed files with 176 additions and 6 deletions
|
|
@ -81,8 +81,10 @@ See `.env.sample` for the full list. Key ones:
|
|||
|
||||
## Git Workflow
|
||||
|
||||
- CI: Woodpecker CI (`.woodpecker/`) builds Docker images on push to `master`, deploys to K8s.
|
||||
- Linting: GitHub Actions runs Ruff on PR diffs.
|
||||
- **CI: GitHub Actions** (`.github/workflows/build-api.yml`, `build-frontend.yml`) builds + pushes Docker images to DockerHub on `master` push. **Keel** in the cluster watches `:latest` on `viktorbarzin/realestatecrawler` and `viktorbarzin/immoweb` and rolls the four `realestate-crawler-*` Deployments on digest change. No Woodpecker deploy POST — Keel is the rollout mechanism.
|
||||
- Pull-secret on the namespace: `dockerhub-pull-secret`, synced from Vault `secret/viktor.dockerhub_registry_password` via ExternalSecret (codified in `infra/stacks/real-estate-crawler/main.tf`). Required because the DockerHub repos are private.
|
||||
- Fallback: `.woodpecker/build-fallback-{api,frontend}.yml` (event: `deployment`, manual-only) preserves the in-cluster build path if GHA is down.
|
||||
- Linting: GitHub Actions runs Ruff on PR diffs (`.github/workflows/ruff.yml`).
|
||||
- Keep commits focused — one logical change per commit.
|
||||
- Group related files (e.g., code + its tests) in the same commit.
|
||||
|
||||
|
|
|
|||
83
.github/workflows/build-api.yml
vendored
Normal file
83
.github/workflows/build-api.yml
vendored
Normal file
|
|
@ -0,0 +1,83 @@
|
|||
name: Build and Push API
|
||||
|
||||
# Migrated from .woodpecker/api.yml on 2026-05-18. GHA builds + pushes to
|
||||
# DockerHub; Keel polls and rolls the cluster Deployments. No POST to
|
||||
# Woodpecker — Keel is the deploy mechanism now.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [master]
|
||||
paths-ignore:
|
||||
- 'frontend/**'
|
||||
- '**.md'
|
||||
- '.woodpecker/**'
|
||||
|
||||
concurrency:
|
||||
group: build-api-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
IMAGE: viktorbarzin/realestatecrawler
|
||||
|
||||
jobs:
|
||||
test-and-build:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.13'
|
||||
cache: 'pip'
|
||||
|
||||
- name: System deps
|
||||
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends libglib2.0-0
|
||||
|
||||
- name: Python deps (mirror .woodpecker/api.yml install list)
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install --quiet \
|
||||
pytest pytest-asyncio pytest-cov httpx fakeredis aioresponses \
|
||||
fastapi uvicorn sqlmodel sqlalchemy alembic pyjwt cryptography \
|
||||
celery redis click aiohttp aiohttp-socks pillow numpy pytesseract \
|
||||
opentelemetry-api opentelemetry-sdk opentelemetry-exporter-prometheus \
|
||||
opentelemetry-instrumentation-fastapi opentelemetry-instrumentation-sqlalchemy \
|
||||
python-dotenv webauthn apprise tenacity prometheus-client \
|
||||
email-validator opencv-python-headless tqdm pandas cachetools watchdog
|
||||
|
||||
- name: Unit tests
|
||||
run: pytest tests/unit/ -v --tb=short
|
||||
|
||||
- name: Integration / regression / e2e tests
|
||||
run: pytest tests/integration/ tests/regression/ tests/e2e/ tests/test_listing_geojson.py -v --tb=short
|
||||
|
||||
- uses: docker/setup-buildx-action@v3
|
||||
|
||||
- uses: docker/login-action@v3
|
||||
with:
|
||||
username: viktorbarzin
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
# github.sha is the commit hash (40-char hex, controlled by git, not
|
||||
# external input) — using it via env to satisfy the workflow-injection
|
||||
# lint hook and keep things uniform.
|
||||
- id: meta
|
||||
env:
|
||||
GIT_SHA: ${{ github.sha }}
|
||||
run: echo "sha=$(printf '%s' "$GIT_SHA" | cut -c1-8)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Pushes both :<8-char-sha> (traceability) and :latest (Keel watches this).
|
||||
# Deployments reference :latest; Keel detects the digest bump and rolls.
|
||||
- uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
file: Dockerfile
|
||||
target: production
|
||||
platforms: linux/amd64
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.IMAGE }}:${{ steps.meta.outputs.sha }}
|
||||
${{ env.IMAGE }}:latest
|
||||
cache-from: type=gha,scope=api
|
||||
cache-to: type=gha,scope=api,mode=max
|
||||
77
.github/workflows/build-frontend.yml
vendored
Normal file
77
.github/workflows/build-frontend.yml
vendored
Normal file
|
|
@ -0,0 +1,77 @@
|
|||
name: Build and Push Frontend
|
||||
|
||||
# Migrated from .woodpecker/frontend.yml on 2026-05-18. GHA builds + pushes
|
||||
# to DockerHub; Keel polls :latest and rolls the realestate-crawler-ui
|
||||
# Deployment. The .env.production file carries VITE_MAPBOX_TOKEN at build
|
||||
# time (Vite reads it from the build context) — same shape as the
|
||||
# Woodpecker pipeline, just sourced from a GHA secret instead.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [master]
|
||||
paths:
|
||||
- 'frontend/**'
|
||||
- '.github/workflows/build-frontend.yml'
|
||||
|
||||
concurrency:
|
||||
group: build-frontend-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
IMAGE: viktorbarzin/immoweb
|
||||
|
||||
jobs:
|
||||
test-and-build:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '24'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install deps
|
||||
working-directory: frontend
|
||||
run: npm ci
|
||||
|
||||
# Vitest runs all 4 Woodpecker shards in one job — single GHA runner
|
||||
# is plenty fast and avoids matrix-cost x4.
|
||||
- name: Vitest
|
||||
working-directory: frontend
|
||||
run: npx vitest run --reporter=verbose
|
||||
|
||||
# Writes .env.production for Vite's build-time read. Secret is piped
|
||||
# via env (not `${{ }}` directly in run:) so shell metacharacters in
|
||||
# the value, if any, can't break out.
|
||||
- name: Write .env.production for Vite build
|
||||
env:
|
||||
MAPBOX_TOKEN: ${{ secrets.MAPBOX_TOKEN }}
|
||||
run: printf 'VITE_MAPBOX_TOKEN=%s\n' "$MAPBOX_TOKEN" > frontend/.env.production
|
||||
|
||||
- uses: docker/setup-buildx-action@v3
|
||||
|
||||
- uses: docker/login-action@v3
|
||||
with:
|
||||
username: viktorbarzin
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- id: meta
|
||||
env:
|
||||
GIT_SHA: ${{ github.sha }}
|
||||
run: echo "sha=$(printf '%s' "$GIT_SHA" | cut -c1-8)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: frontend
|
||||
file: frontend/Dockerfile
|
||||
target: production
|
||||
platforms: linux/amd64
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.IMAGE }}:${{ steps.meta.outputs.sha }}
|
||||
${{ env.IMAGE }}:latest
|
||||
cache-from: type=gha,scope=frontend
|
||||
cache-to: type=gha,scope=frontend,mode=max
|
||||
|
|
@ -1,6 +1,10 @@
|
|||
# Fallback pipeline — only fires on manual `deployment` event after GHA
|
||||
# (.github/workflows/build-api.yml) became the primary builder on
|
||||
# 2026-05-18. Kept around in case GHA is down and a build needs to ship
|
||||
# from inside the cluster. `event: deployment` means it never auto-fires
|
||||
# on push.
|
||||
when:
|
||||
- event: push
|
||||
branch: master
|
||||
- event: deployment
|
||||
|
||||
clone:
|
||||
git:
|
||||
|
|
@ -1,6 +1,10 @@
|
|||
# Fallback pipeline — only fires on manual `deployment` event after GHA
|
||||
# (.github/workflows/build-frontend.yml) became the primary builder on
|
||||
# 2026-05-18. Kept around in case GHA is down and a build needs to ship
|
||||
# from inside the cluster. `event: deployment` means it never auto-fires
|
||||
# on push.
|
||||
when:
|
||||
- event: push
|
||||
branch: master
|
||||
- event: deployment
|
||||
|
||||
clone:
|
||||
git:
|
||||
Loading…
Add table
Add a link
Reference in a new issue