Replace .drone.yml with .woodpecker/ pipeline configs (frontend.yml, api.yml).
Convert Drone env vars to Woodpecker equivalents (CI_PIPELINE_NUMBER, CI_COMMIT_SHA),
use woodpeckerci/plugin-git for clone with retry, woodpeckerci/plugin-slack for
notifications, and plugins/docker for image builds. Update all docs and skills.

2026-02-23 22:00:09 +00:00

3 KiB

Raw Blame History

CI Pipeline Optimization — Design Document

Date: 2026-02-21

Problem

The Woodpecker CI pipeline is slow. Both the frontend and backend pipelines install dependencies multiple times per build and pull Docker cache layers from Docker Hub instead of the local registry at 10.0.20.10:5000.

Root Causes

Frontend npm ci runs twice: once in the test step (bare node image), once inside the Kaniko Dockerfile build. 560MB of node_modules downloaded each time.
Backend builds 2 Docker images: one for the :test target, one for the final image. Both execute the builder stage which runs pip install -r requirements.txt (1874 lines, hash-pinned).
Docker cache pulled from Docker Hub: cache_from and cache_repo point to Docker Hub. Local registry at 10.0.20.10:5000 already exists and has all images but is not used for caching.

Solution

1. Local Registry for Docker Layer Caching

Switch all cache_from and cache_repo references from Docker Hub to 10.0.20.10:5000. The local registry already has viktorbarzin/realestatecrawler and viktorbarzin/immoweb with builder, test, and latest tags. Local network pulls are near-instant vs Docker Hub.

Docker Hub remains the "production" registry (k8s pulls from there)
Local registry becomes the "CI cache" registry

2. Merge Tests into Dockerfile (Frontend)

Replace the separate "Run frontend tests" Woodpecker step + Docker build with a single multi-stage Dockerfile build:

deps stage: npm ci (cached if package-lock.json unchanged)
test stage: runs vitest run (fails the build if tests fail)
builder stage: runs vite build
Final stage: nginx serving built assets

This eliminates the double npm ci.

3. Merge Tests into Dockerfile (Backend)

Replace the separate "Cache test image" + "Run backend tests" + "Build API image" steps with a single Docker build:

builder stage: pip install -r requirements.txt (cached if requirements unchanged)
runtime-base stage: system dependencies
test stage: runs pytest tests/ -x -q (fails the build if tests fail)
Final stage: production image with app code

This eliminates the double pip install and double image build.

4. Simplified Pipeline Steps

Frontend: 5 steps to 4

Before: clone, npm ci + vitest, Kaniko build (npm ci again), deploy, verify After: clone, Kaniko build (npm ci + vitest + vite build), deploy, verify

Backend: 5 steps to 4

Before: clone, build :test image (pip install), pytest, build final image (pip install again), deploy + verify After: clone, build image (pip install + pytest + final), deploy, verify

Expected Impact

npm ci runs once instead of twice per frontend build
pip install runs once instead of twice per backend build
Docker layer cache from local network instead of Docker Hub
On cache hit (deps unchanged): dependency install steps are completely skipped (cached layers)

Constraints

Tests must always run (gate deployment)
Final images must still be pushed to Docker Hub for k8s to pull
Local registry is HTTP-only at 10.0.20.10:5000

3 KiB Raw Blame History