Migrate CI from Drone to Woodpecker

Replace .drone.yml with .woodpecker/ pipeline configs (frontend.yml, api.yml).
Convert Drone env vars to Woodpecker equivalents (CI_PIPELINE_NUMBER, CI_COMMIT_SHA),
use woodpeckerci/plugin-git for clone with retry, woodpeckerci/plugin-slack for
notifications, and plugins/docker for image builds. Update all docs and skills.
This commit is contained in:
Viktor Barzin 2026-02-23 22:00:09 +00:00
parent 2357722e80
commit c7c3331d30
No known key found for this signature in database
GPG key ID: 0EB088298288D958
9 changed files with 190 additions and 448 deletions

View file

@ -33,7 +33,7 @@ See `.claude/skills/` for detailed skills on dev environment, building, and depl
- **Backend:** Python 3.13, FastAPI, SQLModel (SQLAlchemy 2), Celery + Redis, pytesseract/OpenCV
- **Frontend:** React 19, TypeScript, Vite, Tailwind CSS, Radix UI, Mapbox GL
- **Database:** MySQL 9 (prod) / SQLite (local dev), Alembic migrations
- **Infrastructure:** Docker Compose (dev), Kubernetes (prod), Drone CI
- **Infrastructure:** Docker Compose (dev), Kubernetes (prod), Woodpecker CI
## Code Conventions
@ -81,7 +81,7 @@ See `.env.sample` for the full list. Key ones:
## Git Workflow
- CI: Drone CI builds Docker images on push to `master`, deploys to K8s.
- CI: Woodpecker CI (`.woodpecker/`) builds Docker images on push to `master`, deploys to K8s.
- Linting: GitHub Actions runs Ruff on PR diffs.
- Keep commits focused — one logical change per commit.
- Group related files (e.g., code + its tests) in the same commit.

View file

@ -99,7 +99,7 @@ docker push viktorbarzin/immoweb:latest
## CI/CD Note
Drone CI automatically builds and pushes images on push to `master` (see `.drone.yml`).
Woodpecker CI automatically builds and pushes images on push to `master` (see `.woodpecker/`).
The manual process above is for when you need to build/push outside of CI, such as:
- Hotfix deployments
- Testing image builds locally before pushing

View file

@ -151,7 +151,7 @@ kubectl port-forward deployment/realestate-crawler-api 5001:5001 -n realestate-c
## Notes
- Drone CI handles automated deployments on push to `master` (see `.drone.yml`)
- Woodpecker CI handles automated deployments on push to `master` (see `.woodpecker/`)
- Use manual deployment for hotfixes, testing, or deploying from non-master branches
- The K8s cluster is at `10.0.20.100:6443` (context: `kubernetes-admin@kubernetes`)
- If pods aren't picking up new `:latest` images, check the `kubernetes-latest-tag-image-pull` skill

View file

@ -1,397 +0,0 @@
kind: pipeline
type: kubernetes
name: frontend
clone:
disable: true
concurrency:
limit: 1
timeout: 20
trigger:
branch:
- master
event:
- push
steps:
- name: clone
image: alpine/git
commands:
- |
for i in 1 2 3 4 5; do
git clone --depth=50 "$DRONE_REMOTE_URL" . && exit 0
echo "Clone attempt $i failed, retrying in 5s..."
sleep 5
done
echo "Clone failed after 5 attempts"
exit 1
- git checkout "$DRONE_COMMIT"
- name: install-frontend-deps
image: node:24-alpine
depends_on:
- clone
environment:
NODE_OPTIONS: "--max-old-space-size=1024"
commands:
- cd frontend && npm ci
- name: test-shard-1
image: node:24-alpine
depends_on:
- install-frontend-deps
environment:
NODE_OPTIONS: "--max-old-space-size=1024"
commands:
- cd frontend && npx vitest run --reporter=verbose --shard=1/4
- name: test-shard-2
image: node:24-alpine
depends_on:
- install-frontend-deps
environment:
NODE_OPTIONS: "--max-old-space-size=1024"
commands:
- cd frontend && npx vitest run --reporter=verbose --shard=2/4
- name: test-shard-3
image: node:24-alpine
depends_on:
- install-frontend-deps
environment:
NODE_OPTIONS: "--max-old-space-size=1024"
commands:
- cd frontend && npx vitest run --reporter=verbose --shard=3/4
- name: test-shard-4
image: node:24-alpine
depends_on:
- install-frontend-deps
environment:
NODE_OPTIONS: "--max-old-space-size=1024"
commands:
- cd frontend && npx vitest run --reporter=verbose --shard=4/4
- name: build-frontend-image
image: plugins/kaniko
depends_on:
- clone
resources:
limits:
memory: 2048MiB
settings:
username: viktorbarzin
password:
from_secret: dockerhub-token
repo: viktorbarzin/immoweb
dockerfile: frontend/Dockerfile
context: frontend
target: production
enable_cache: true
cache_repo: viktorbarzin/immoweb-cache
tags:
- "build-${DRONE_BUILD_NUMBER}"
- name: publish-frontend-image
image: alpine
depends_on:
- test-shard-1
- test-shard-2
- test-shard-3
- test-shard-4
- build-frontend-image
environment:
DOCKERHUB_TOKEN:
from_secret: dockerhub-token
commands:
- apk add --no-cache skopeo
- 'skopeo copy --src-creds "viktorbarzin:$DOCKERHUB_TOKEN" --dest-creds "viktorbarzin:$DOCKERHUB_TOKEN" "docker://docker.io/viktorbarzin/immoweb:build-${DRONE_BUILD_NUMBER}" "docker://docker.io/viktorbarzin/immoweb:${DRONE_BUILD_NUMBER}"'
- 'skopeo copy --src-creds "viktorbarzin:$DOCKERHUB_TOKEN" --dest-creds "viktorbarzin:$DOCKERHUB_TOKEN" "docker://docker.io/viktorbarzin/immoweb:build-${DRONE_BUILD_NUMBER}" "docker://docker.io/viktorbarzin/immoweb:latest"'
- name: Update deployment
image: alpine
depends_on:
- publish-frontend-image
commands:
- apk add --no-cache curl jq
- |
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
IMAGE="viktorbarzin/immoweb:${DRONE_BUILD_NUMBER}"
RESTART_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)
API="https://kubernetes:6443/apis/apps/v1/namespaces/realestate-crawler/deployments"
DEPLOY="realestate-crawler-ui"
CONTAINER=$(curl -sfk "$API/$DEPLOY" \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/json" | jq -r '.spec.template.spec.containers[0].name')
echo "Patching $DEPLOY (container=$CONTAINER) to image $IMAGE with restartedAt=$RESTART_AT..."
curl -sf -X PATCH "$API/$DEPLOY" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/strategic-merge-patch+json" \
-k -d "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"kubectl.kubernetes.io/restartedAt\":\"$RESTART_AT\"}},\"spec\":{\"containers\":[{\"name\":\"$CONTAINER\",\"image\":\"$IMAGE\"}]}}}}" \
| jq '{name: .metadata.name, generation: .metadata.generation, image: .spec.template.spec.containers[0].image}'
- name: verify-deploy
image: alpine
depends_on:
- Update deployment
commands:
- apk add --no-cache curl jq
- |
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
EXPECTED_IMAGE="viktorbarzin/immoweb:${DRONE_BUILD_NUMBER}"
PODS_API="https://kubernetes:6443/api/v1/namespaces/realestate-crawler/pods?labelSelector=app%3Drealestate-crawler-ui"
for i in $(seq 1 60); do
RAW=$(curl -sfk "$PODS_API" \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/json")
# Debug: show all pod images and status on first attempt
if [ "$i" -eq 1 ]; then
echo "DEBUG: All pods for realestate-crawler-ui:"
echo "$RAW" | jq -r '[.items[] | {name: .metadata.name, image: .spec.containers[0].image, ready: (.status.containerStatuses[]? | .ready), phase: .status.phase}] | .[] | " \(.name) image=\(.image) ready=\(.ready) phase=\(.phase)"' 2>/dev/null || echo " (no pods found)"
fi
RESULT=$(echo "$RAW" | \
jq --arg img "$EXPECTED_IMAGE" '[.items[] | select(
(.status.containerStatuses[]? | .ready == true) and
(.spec.containers[]? | .image | endswith($img))
) | {name: .metadata.name, image: .spec.containers[0].image, started: .status.startTime}]')
COUNT=$(echo "$RESULT" | jq 'length')
echo "Attempt $i/60: $COUNT pod(s) ready with image matching $EXPECTED_IMAGE"
if [ "$COUNT" -gt 0 ]; then
echo "$RESULT" | jq -r '.[] | " \(.name) image=\(.image) started=\(.started)"'
echo "New pod is live!"
exit 0
fi
sleep 5
done
echo "ERROR: No new ready pod with image $EXPECTED_IMAGE appeared within 5 minutes"
exit 1
---
kind: pipeline
type: kubernetes
name: api
clone:
disable: true
concurrency:
limit: 1
timeout: 20
trigger:
branch:
- master
event:
- push
steps:
- name: clone
image: alpine/git
commands:
- |
for i in 1 2 3 4 5; do
git clone --depth=50 "$DRONE_REMOTE_URL" . && exit 0
echo "Clone attempt $i failed, retrying in 5s..."
sleep 5
done
echo "Clone failed after 5 attempts"
exit 1
- git checkout "$DRONE_COMMIT"
- name: install-api-deps
image: python:3.13-slim
depends_on:
- clone
commands:
- apt-get update && apt-get install -y --no-install-recommends libglib2.0-0
- python -m venv .venv
- .venv/bin/pip install --quiet --upgrade pip
- >-
.venv/bin/pip install --quiet
pytest pytest-asyncio pytest-cov httpx fakeredis aioresponses
fastapi uvicorn sqlmodel sqlalchemy alembic pyjwt cryptography
celery redis click aiohttp aiohttp-socks pillow numpy pytesseract
opentelemetry-api opentelemetry-sdk opentelemetry-exporter-prometheus
opentelemetry-instrumentation-fastapi opentelemetry-instrumentation-sqlalchemy
python-dotenv webauthn apprise tenacity prometheus-client
email-validator opencv-python-headless tqdm pandas cachetools watchdog
- name: test-unit
image: python:3.13-slim
depends_on:
- install-api-deps
commands:
- apt-get update && apt-get install -y --no-install-recommends libglib2.0-0
- .venv/bin/pytest tests/unit/ -v --tb=short
- name: test-integration
image: python:3.13-slim
depends_on:
- install-api-deps
commands:
- apt-get update && apt-get install -y --no-install-recommends libglib2.0-0
- .venv/bin/pytest tests/integration/ tests/regression/ tests/e2e/ tests/test_listing_geojson.py -v --tb=short
- name: build-api-image
image: plugins/docker
depends_on:
- clone
environment:
DOCKER_BUILDKIT: 1
settings:
username: viktorbarzin
password:
from_secret: dockerhub-token
repo: viktorbarzin/realestatecrawler
dockerfile: Dockerfile
context: .
target: production
cache_from:
- viktorbarzin/realestatecrawler:latest
- viktorbarzin/realestatecrawler:builder
tags:
- "build-${DRONE_BUILD_NUMBER}"
- name: publish-api-image
image: alpine
depends_on:
- test-unit
- test-integration
- build-api-image
environment:
DOCKERHUB_TOKEN:
from_secret: dockerhub-token
commands:
- apk add --no-cache skopeo
- 'skopeo copy --src-creds "viktorbarzin:$DOCKERHUB_TOKEN" --dest-creds "viktorbarzin:$DOCKERHUB_TOKEN" "docker://docker.io/viktorbarzin/realestatecrawler:build-${DRONE_BUILD_NUMBER}" "docker://docker.io/viktorbarzin/realestatecrawler:${DRONE_BUILD_NUMBER}"'
- 'skopeo copy --src-creds "viktorbarzin:$DOCKERHUB_TOKEN" --dest-creds "viktorbarzin:$DOCKERHUB_TOKEN" "docker://docker.io/viktorbarzin/realestatecrawler:build-${DRONE_BUILD_NUMBER}" "docker://docker.io/viktorbarzin/realestatecrawler:latest"'
- 'skopeo copy --src-creds "viktorbarzin:$DOCKERHUB_TOKEN" --dest-creds "viktorbarzin:$DOCKERHUB_TOKEN" "docker://docker.io/viktorbarzin/realestatecrawler:build-${DRONE_BUILD_NUMBER}" "docker://docker.io/viktorbarzin/realestatecrawler:builder"'
- name: Update deployment
image: alpine
depends_on:
- publish-api-image
commands:
- apk add --no-cache curl jq
- |
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
IMAGE="viktorbarzin/realestatecrawler:${DRONE_BUILD_NUMBER}"
RESTART_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)
API="https://kubernetes:6443/apis/apps/v1/namespaces/realestate-crawler/deployments"
for DEPLOY in realestate-crawler-api realestate-crawler-celery realestate-crawler-celery-beat; do
# Check if deployment is paused and get container name
STATUS=$(curl -sfk "$API/$DEPLOY" \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/json")
CONTAINER=$(echo "$STATUS" | jq -r '.spec.template.spec.containers[0].name')
PAUSED=$(echo "$STATUS" | jq -r '.spec.paused // false')
echo "Patching $DEPLOY (container=$CONTAINER, paused=$PAUSED) to image $IMAGE..."
# Strategic merge: update image, set restartedAt, ensure not paused
curl -sf -X PATCH "$API/$DEPLOY" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/strategic-merge-patch+json" \
-k -d "{\"spec\":{\"paused\":null,\"template\":{\"metadata\":{\"annotations\":{\"kubectl.kubernetes.io/restartedAt\":\"$RESTART_AT\"}},\"spec\":{\"containers\":[{\"name\":\"$CONTAINER\",\"image\":\"$IMAGE\"}]}}}}" \
| jq '{name: .metadata.name, generation: .metadata.generation, image: .spec.template.spec.containers[0].image, paused: .spec.paused, restartedAt: .spec.template.metadata.annotations["kubectl.kubernetes.io/restartedAt"]}'
done
- name: verify-deploy
image: alpine
depends_on:
- Update deployment
commands:
- apk add --no-cache curl jq
- |
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
EXPECTED_IMAGE="viktorbarzin/realestatecrawler:${DRONE_BUILD_NUMBER}"
BASE_API="https://kubernetes:6443/api/v1/namespaces/realestate-crawler/pods"
DEPLOY_API="https://kubernetes:6443/apis/apps/v1/namespaces/realestate-crawler/deployments"
for DEPLOY in realestate-crawler-api realestate-crawler-celery realestate-crawler-celery-beat; do
echo "Verifying $DEPLOY..."
PODS_API="$BASE_API?labelSelector=app%3D$DEPLOY"
RS_API="https://kubernetes:6443/apis/apps/v1/namespaces/realestate-crawler/replicasets?labelSelector=app%3D$DEPLOY"
# Check deployment status (spec, conditions, paused, replicas)
DEPLOY_STATUS=$(curl -sfk "$DEPLOY_API/$DEPLOY" \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/json")
echo " Deployment spec image: $(echo "$DEPLOY_STATUS" | jq -r '.spec.template.spec.containers[0].image')"
echo " Deployment paused: $(echo "$DEPLOY_STATUS" | jq -r '.spec.paused // false')"
echo " Deployment status: replicas=$(echo "$DEPLOY_STATUS" | jq -r '.status.replicas // 0') updated=$(echo "$DEPLOY_STATUS" | jq -r '.status.updatedReplicas // 0') ready=$(echo "$DEPLOY_STATUS" | jq -r '.status.readyReplicas // 0') unavailable=$(echo "$DEPLOY_STATUS" | jq -r '.status.unavailableReplicas // 0')"
echo " Conditions:"
echo "$DEPLOY_STATUS" | jq -r '.status.conditions[]? | " \(.type): \(.status) (\(.reason // "unknown"))"' 2>/dev/null || echo " (none)"
# Check ReplicaSets
echo " ReplicaSets:"
curl -sfk "$RS_API" \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/json" | \
jq -r '.items[] | " \(.metadata.name) desired=\(.spec.replicas) ready=\(.status.readyReplicas // 0) image=\(.spec.template.spec.containers[0].image)"' 2>/dev/null || echo " (none)"
FOUND=0
for i in $(seq 1 60); do
RAW=$(curl -sfk "$PODS_API" \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/json")
# Debug: show all pod images and status periodically
if [ "$i" -eq 1 ] || [ "$i" -eq 10 ] || [ "$i" -eq 30 ]; then
echo " DEBUG (attempt $i): All pods for $DEPLOY:"
echo "$RAW" | jq -r '[.items[] | {
name: .metadata.name,
image: .spec.containers[0].image,
ready: ([.status.containerStatuses[]? | .ready] | first // "unknown"),
phase: .status.phase,
restarts: ([.status.containerStatuses[]? | .restartCount] | first // 0),
reason: ([.status.containerStatuses[]? | .state | to_entries[] | .value.reason // empty] | first // "running")
}] | .[] | " \(.name) image=\(.image) ready=\(.ready) phase=\(.phase) restarts=\(.restarts) reason=\(.reason)"' 2>/dev/null || echo " (no pods or parse error)"
fi
RESULT=$(echo "$RAW" | \
jq --arg img "$EXPECTED_IMAGE" '[.items[] | select(
(.status.containerStatuses[]? | .ready == true) and
(.spec.containers[]? | .image | endswith($img))
) | {name: .metadata.name, image: .spec.containers[0].image, started: .status.startTime}]')
COUNT=$(echo "$RESULT" | jq 'length' 2>/dev/null || echo 0)
echo " Attempt $i/60: $COUNT pod(s) ready with image matching $EXPECTED_IMAGE"
if [ "$COUNT" -gt 0 ] 2>/dev/null; then
echo "$RESULT" | jq -r '.[] | " \(.name) image=\(.image) started=\(.started)"'
echo "$DEPLOY is live!"
FOUND=1
break
fi
sleep 5
done
if [ "$FOUND" -ne 1 ]; then
echo " FINAL DEBUG: All pods for $DEPLOY:"
echo "$RAW" | jq -r '[.items[] | {
name: .metadata.name,
image: .spec.containers[0].image,
ready: ([.status.containerStatuses[]? | .ready] | first // "unknown"),
phase: .status.phase,
restarts: ([.status.containerStatuses[]? | .restartCount] | first // 0),
reason: ([.status.containerStatuses[]? | .state | to_entries[] | .value.reason // empty] | first // "running")
}] | .[] | " \(.name) image=\(.image) ready=\(.ready) phase=\(.phase) restarts=\(.restarts) reason=\(.reason)"' 2>/dev/null || echo " (no pods or parse error)"
echo "ERROR: No new ready pod for $DEPLOY with image $EXPECTED_IMAGE appeared within 5 minutes"
exit 1
fi
done

View file

@ -28,20 +28,24 @@ steps:
- name: test-unit
image: python:3.13-slim
depends_on: [install-api-deps]
depends_on:
- install-api-deps
commands:
- apt-get update && apt-get install -y --no-install-recommends libglib2.0-0
- .venv/bin/pytest tests/unit/ -v --tb=short
- name: test-integration
image: python:3.13-slim
depends_on: [install-api-deps]
depends_on:
- install-api-deps
commands:
- apt-get update && apt-get install -y --no-install-recommends libglib2.0-0
- .venv/bin/pytest tests/integration/ tests/regression/ tests/e2e/ tests/test_listing_geojson.py -v --tb=short
- name: build-api-image
image: woodpeckerci/plugin-docker-buildx
image: plugins/docker
environment:
DOCKER_BUILDKIT: 1
settings:
username: viktorbarzin
password:
@ -50,12 +54,18 @@ steps:
dockerfile: Dockerfile
context: .
target: production
cache_from:
- viktorbarzin/realestatecrawler:latest
- viktorbarzin/realestatecrawler:builder
tags:
- "build-${CI_PIPELINE_NUMBER}"
- name: publish-api-image
image: alpine
depends_on: [test-unit, test-integration, build-api-image]
depends_on:
- test-unit
- test-integration
- build-api-image
environment:
DOCKERHUB_TOKEN:
from_secret: dockerhub-token
@ -67,41 +77,120 @@ steps:
- name: update-deployment
image: alpine
depends_on: [publish-api-image]
depends_on:
- publish-api-image
commands:
- apk add --no-cache curl jq
- |
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
IMAGE="viktorbarzin/realestatecrawler:${CI_PIPELINE_NUMBER}"
RESTART_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)
API="https://10.0.20.100:6443/apis/apps/v1/namespaces/realestate-crawler/deployments"
API="https://kubernetes:6443/apis/apps/v1/namespaces/realestate-crawler/deployments"
for DEPLOY in realestate-crawler-api realestate-crawler-celery realestate-crawler-celery-beat; do
CONTAINER=$(curl -sfk "$API/$DEPLOY" -H "Authorization: Bearer $TOKEN" -H "Accept: application/json" | jq -r '.spec.template.spec.containers[0].name')
echo "Patching $DEPLOY (container=$CONTAINER) to image $IMAGE..."
curl -sf -X PATCH "$API/$DEPLOY" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/strategic-merge-patch+json" -k \
-d "{\"spec\":{\"paused\":null,\"template\":{\"metadata\":{\"annotations\":{\"kubectl.kubernetes.io/restartedAt\":\"$RESTART_AT\"}},\"spec\":{\"containers\":[{\"name\":\"$CONTAINER\",\"image\":\"$IMAGE\"}]}}}}" \
| jq '{name: .metadata.name, generation: .metadata.generation, image: .spec.template.spec.containers[0].image}'
STATUS=$(curl -sfk "$API/$DEPLOY" \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/json")
CONTAINER=$(echo "$STATUS" | jq -r '.spec.template.spec.containers[0].name')
PAUSED=$(echo "$STATUS" | jq -r '.spec.paused // false')
echo "Patching $DEPLOY (container=$CONTAINER, paused=$PAUSED) to image $IMAGE..."
curl -sf -X PATCH "$API/$DEPLOY" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/strategic-merge-patch+json" \
-k -d "{\"spec\":{\"paused\":null,\"template\":{\"metadata\":{\"annotations\":{\"kubectl.kubernetes.io/restartedAt\":\"$RESTART_AT\"}},\"spec\":{\"containers\":[{\"name\":\"$CONTAINER\",\"image\":\"$IMAGE\"}]}}}}" \
| jq '{name: .metadata.name, generation: .metadata.generation, image: .spec.template.spec.containers[0].image, paused: .spec.paused, restartedAt: .spec.template.metadata.annotations["kubectl.kubernetes.io/restartedAt"]}'
done
- name: verify-deploy
image: alpine
depends_on: [update-deployment]
depends_on:
- update-deployment
commands:
- apk add --no-cache curl jq
- |
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
EXPECTED_IMAGE="viktorbarzin/realestatecrawler:${CI_PIPELINE_NUMBER}"
BASE_API="https://10.0.20.100:6443/api/v1/namespaces/realestate-crawler/pods"
BASE_API="https://kubernetes:6443/api/v1/namespaces/realestate-crawler/pods"
DEPLOY_API="https://kubernetes:6443/apis/apps/v1/namespaces/realestate-crawler/deployments"
for DEPLOY in realestate-crawler-api realestate-crawler-celery realestate-crawler-celery-beat; do
echo "Verifying $DEPLOY..."
PODS_API="$BASE_API?labelSelector=app%3D$DEPLOY"
RS_API="https://kubernetes:6443/apis/apps/v1/namespaces/realestate-crawler/replicasets?labelSelector=app%3D$DEPLOY"
DEPLOY_STATUS=$(curl -sfk "$DEPLOY_API/$DEPLOY" \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/json")
echo " Deployment spec image: $(echo "$DEPLOY_STATUS" | jq -r '.spec.template.spec.containers[0].image')"
echo " Deployment paused: $(echo "$DEPLOY_STATUS" | jq -r '.spec.paused // false')"
echo " Deployment status: replicas=$(echo "$DEPLOY_STATUS" | jq -r '.status.replicas // 0') updated=$(echo "$DEPLOY_STATUS" | jq -r '.status.updatedReplicas // 0') ready=$(echo "$DEPLOY_STATUS" | jq -r '.status.readyReplicas // 0') unavailable=$(echo "$DEPLOY_STATUS" | jq -r '.status.unavailableReplicas // 0')"
echo " Conditions:"
echo "$DEPLOY_STATUS" | jq -r '.status.conditions[]? | " \(.type): \(.status) (\(.reason // "unknown"))"' 2>/dev/null || echo " (none)"
echo " ReplicaSets:"
curl -sfk "$RS_API" \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/json" | \
jq -r '.items[] | " \(.metadata.name) desired=\(.spec.replicas) ready=\(.status.readyReplicas // 0) image=\(.spec.template.spec.containers[0].image)"' 2>/dev/null || echo " (none)"
FOUND=0
for i in $(seq 1 60); do
COUNT=$(curl -sfk "$PODS_API" -H "Authorization: Bearer $TOKEN" -H "Accept: application/json" | \
jq --arg img "$EXPECTED_IMAGE" '[.items[] | select((.status.containerStatuses[]? | .ready == true) and (.spec.containers[]? | .image | endswith($img)))] | length')
echo " Attempt $i/60: $COUNT pod(s) ready"
if [ "$COUNT" -gt 0 ]; then echo "$DEPLOY is live!"; FOUND=1; break; fi
RAW=$(curl -sfk "$PODS_API" \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/json")
if [ "$i" -eq 1 ] || [ "$i" -eq 10 ] || [ "$i" -eq 30 ]; then
echo " DEBUG (attempt $i): All pods for $DEPLOY:"
echo "$RAW" | jq -r '[.items[] | {
name: .metadata.name,
image: .spec.containers[0].image,
ready: ([.status.containerStatuses[]? | .ready] | first // "unknown"),
phase: .status.phase,
restarts: ([.status.containerStatuses[]? | .restartCount] | first // 0),
reason: ([.status.containerStatuses[]? | .state | to_entries[] | .value.reason // empty] | first // "running")
}] | .[] | " \(.name) image=\(.image) ready=\(.ready) phase=\(.phase) restarts=\(.restarts) reason=\(.reason)"' 2>/dev/null || echo " (no pods or parse error)"
fi
RESULT=$(echo "$RAW" | \
jq --arg img "$EXPECTED_IMAGE" '[.items[] | select(
(.status.containerStatuses[]? | .ready == true) and
(.spec.containers[]? | .image | endswith($img))
) | {name: .metadata.name, image: .spec.containers[0].image, started: .status.startTime}]')
COUNT=$(echo "$RESULT" | jq 'length' 2>/dev/null || echo 0)
echo " Attempt $i/60: $COUNT pod(s) ready with image matching $EXPECTED_IMAGE"
if [ "$COUNT" -gt 0 ] 2>/dev/null; then
echo "$RESULT" | jq -r '.[] | " \(.name) image=\(.image) started=\(.started)"'
echo "$DEPLOY is live!"
FOUND=1
break
fi
sleep 5
done
if [ "$FOUND" -ne 1 ]; then echo "ERROR: $DEPLOY not ready within 5 minutes"; exit 1; fi
if [ "$FOUND" -ne 1 ]; then
echo " FINAL DEBUG: All pods for $DEPLOY:"
echo "$RAW" | jq -r '[.items[] | {
name: .metadata.name,
image: .spec.containers[0].image,
ready: ([.status.containerStatuses[]? | .ready] | first // "unknown"),
phase: .status.phase,
restarts: ([.status.containerStatuses[]? | .restartCount] | first // 0),
reason: ([.status.containerStatuses[]? | .state | to_entries[] | .value.reason // empty] | first // "running")
}] | .[] | " \(.name) image=\(.image) ready=\(.ready) phase=\(.phase) restarts=\(.restarts) reason=\(.reason)"' 2>/dev/null || echo " (no pods or parse error)"
echo "ERROR: No new ready pod for $DEPLOY with image $EXPECTED_IMAGE appeared within 5 minutes"
exit 1
fi
done
- name: slack
image: woodpeckerci/plugin-slack
settings:
webhook:
from_secret: slack-webhook-url
channel: general
when:
- status: [success, failure]

View file

@ -19,7 +19,8 @@ steps:
- name: test-shard-1
image: node:24-alpine
depends_on: [install-frontend-deps]
depends_on:
- install-frontend-deps
environment:
NODE_OPTIONS: "--max-old-space-size=1024"
commands:
@ -27,7 +28,8 @@ steps:
- name: test-shard-2
image: node:24-alpine
depends_on: [install-frontend-deps]
depends_on:
- install-frontend-deps
environment:
NODE_OPTIONS: "--max-old-space-size=1024"
commands:
@ -35,7 +37,8 @@ steps:
- name: test-shard-3
image: node:24-alpine
depends_on: [install-frontend-deps]
depends_on:
- install-frontend-deps
environment:
NODE_OPTIONS: "--max-old-space-size=1024"
commands:
@ -43,14 +46,15 @@ steps:
- name: test-shard-4
image: node:24-alpine
depends_on: [install-frontend-deps]
depends_on:
- install-frontend-deps
environment:
NODE_OPTIONS: "--max-old-space-size=1024"
commands:
- cd frontend && npx vitest run --reporter=verbose --shard=4/4
- name: build-frontend-image
image: woodpeckerci/plugin-docker-buildx
image: plugins/docker
settings:
username: viktorbarzin
password:
@ -59,12 +63,19 @@ steps:
dockerfile: frontend/Dockerfile
context: frontend
target: production
cache_from:
- viktorbarzin/immoweb:latest
tags:
- "build-${CI_PIPELINE_NUMBER}"
- name: publish-frontend-image
image: alpine
depends_on: [test-shard-1, test-shard-2, test-shard-3, test-shard-4, build-frontend-image]
depends_on:
- test-shard-1
- test-shard-2
- test-shard-3
- test-shard-4
- build-frontend-image
environment:
DOCKERHUB_TOKEN:
from_secret: dockerhub-token
@ -75,36 +86,75 @@ steps:
- name: update-deployment
image: alpine
depends_on: [publish-frontend-image]
depends_on:
- publish-frontend-image
commands:
- apk add --no-cache curl jq
- |
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
IMAGE="viktorbarzin/immoweb:${CI_PIPELINE_NUMBER}"
RESTART_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)
API="https://10.0.20.100:6443/apis/apps/v1/namespaces/realestate-crawler/deployments"
API="https://kubernetes:6443/apis/apps/v1/namespaces/realestate-crawler/deployments"
DEPLOY="realestate-crawler-ui"
CONTAINER=$(curl -sfk "$API/$DEPLOY" -H "Authorization: Bearer $TOKEN" -H "Accept: application/json" | jq -r '.spec.template.spec.containers[0].name')
echo "Patching $DEPLOY (container=$CONTAINER) to image $IMAGE..."
curl -sf -X PATCH "$API/$DEPLOY" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/strategic-merge-patch+json" -k \
-d "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"kubectl.kubernetes.io/restartedAt\":\"$RESTART_AT\"}},\"spec\":{\"containers\":[{\"name\":\"$CONTAINER\",\"image\":\"$IMAGE\"}]}}}}" \
CONTAINER=$(curl -sfk "$API/$DEPLOY" \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/json" | jq -r '.spec.template.spec.containers[0].name')
echo "Patching $DEPLOY (container=$CONTAINER) to image $IMAGE with restartedAt=$RESTART_AT..."
curl -sf -X PATCH "$API/$DEPLOY" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/strategic-merge-patch+json" \
-k -d "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"kubectl.kubernetes.io/restartedAt\":\"$RESTART_AT\"}},\"spec\":{\"containers\":[{\"name\":\"$CONTAINER\",\"image\":\"$IMAGE\"}]}}}}" \
| jq '{name: .metadata.name, generation: .metadata.generation, image: .spec.template.spec.containers[0].image}'
- name: verify-deploy
image: alpine
depends_on: [update-deployment]
depends_on:
- update-deployment
commands:
- apk add --no-cache curl jq
- |
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
EXPECTED_IMAGE="viktorbarzin/immoweb:${CI_PIPELINE_NUMBER}"
PODS_API="https://10.0.20.100:6443/api/v1/namespaces/realestate-crawler/pods?labelSelector=app%3Drealestate-crawler-ui"
PODS_API="https://kubernetes:6443/api/v1/namespaces/realestate-crawler/pods?labelSelector=app%3Drealestate-crawler-ui"
for i in $(seq 1 60); do
RESULT=$(curl -sfk "$PODS_API" -H "Authorization: Bearer $TOKEN" -H "Accept: application/json" | \
jq --arg img "$EXPECTED_IMAGE" '[.items[] | select((.status.containerStatuses[]? | .ready == true) and (.spec.containers[]? | .image | endswith($img))) | {name: .metadata.name, image: .spec.containers[0].image}]')
RAW=$(curl -sfk "$PODS_API" \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/json")
if [ "$i" -eq 1 ]; then
echo "DEBUG: All pods for realestate-crawler-ui:"
echo "$RAW" | jq -r '[.items[] | {name: .metadata.name, image: .spec.containers[0].image, ready: (.status.containerStatuses[]? | .ready), phase: .status.phase}] | .[] | " \(.name) image=\(.image) ready=\(.ready) phase=\(.phase)"' 2>/dev/null || echo " (no pods found)"
fi
RESULT=$(echo "$RAW" | \
jq --arg img "$EXPECTED_IMAGE" '[.items[] | select(
(.status.containerStatuses[]? | .ready == true) and
(.spec.containers[]? | .image | endswith($img))
) | {name: .metadata.name, image: .spec.containers[0].image, started: .status.startTime}]')
COUNT=$(echo "$RESULT" | jq 'length')
echo "Attempt $i/60: $COUNT pod(s) ready with image $EXPECTED_IMAGE"
if [ "$COUNT" -gt 0 ]; then echo "New pod is live!"; exit 0; fi
echo "Attempt $i/60: $COUNT pod(s) ready with image matching $EXPECTED_IMAGE"
if [ "$COUNT" -gt 0 ]; then
echo "$RESULT" | jq -r '.[] | " \(.name) image=\(.image) started=\(.started)"'
echo "New pod is live!"
exit 0
fi
sleep 5
done
echo "ERROR: No pod with image $EXPECTED_IMAGE appeared within 5 minutes"; exit 1
echo "ERROR: No new ready pod with image $EXPECTED_IMAGE appeared within 5 minutes"
exit 1
- name: slack
image: woodpeckerci/plugin-slack
settings:
webhook:
from_secret: slack-webhook-url
channel: general
when:
- status: [success, failure]

View file

@ -51,7 +51,7 @@ Minimize Docker build times and production image sizes for both the backend (Pyt
- Add non-root nginx user.
- Add `HEALTHCHECK`.
### 4. CI Pipeline (Drone)
### 4. CI Pipeline (Woodpecker)
- Enable BuildKit in the API pipeline (`DOCKER_BUILDKIT=1` in `plugins/docker` settings).
- Frontend pipeline (Kaniko) already has good caching; no changes needed.

View file

@ -767,17 +767,17 @@ git commit -m "Expand frontend .dockerignore to exclude build artifacts"
---
### Task 14: Enable BuildKit in Drone CI API pipeline
### Task 14: Enable BuildKit in Woodpecker CI API pipeline
**Files:**
- Modify: `.drone.yml`
- Modify: `.woodpecker/api.yml`
**Step 1: Add DOCKER_BUILDKIT environment to API build step**
In the `.drone.yml` file, under the `api` pipeline's "Build and test API image" step, add the `build_args` setting:
In the `.woodpecker/api.yml` file, under the "build-api-image" step, add the `build_args` setting:
```yaml
- name: Build and test API image
- name: build-api-image
image: plugins/docker
environment:
DOCKER_BUILDKIT: 1
@ -794,14 +794,14 @@ In the `.drone.yml` file, under the `api` pipeline's "Build and test API image"
tags:
- latest
- builder
- "${DRONE_BUILD_NUMBER}"
- "${CI_PIPELINE_NUMBER}"
```
**Step 2: Commit**
```bash
git add .drone.yml
git commit -m "Enable BuildKit in Drone API pipeline"
git add .woodpecker/api.yml
git commit -m "Enable BuildKit in Woodpecker API pipeline"
```
---

View file

@ -4,7 +4,7 @@ Date: 2026-02-21
## Problem
The Drone CI pipeline is slow. Both the frontend and backend pipelines install dependencies multiple times per build and pull Docker cache layers from Docker Hub instead of the local registry at `10.0.20.10:5000`.
The Woodpecker CI pipeline is slow. Both the frontend and backend pipelines install dependencies multiple times per build and pull Docker cache layers from Docker Hub instead of the local registry at `10.0.20.10:5000`.
## Root Causes
@ -23,7 +23,7 @@ Switch all `cache_from` and `cache_repo` references from Docker Hub to `10.0.20.
### 2. Merge Tests into Dockerfile (Frontend)
Replace the separate "Run frontend tests" Drone step + Kaniko build with a single multi-stage Dockerfile build:
Replace the separate "Run frontend tests" Woodpecker step + Docker build with a single multi-stage Dockerfile build:
- `deps` stage: `npm ci` (cached if `package-lock.json` unchanged)
- `test` stage: runs `vitest run` (fails the build if tests fail)