From 68859ae57773866e8b048f781d707feeaa051c16 Mon Sep 17 00:00:00 2001
From: Viktor Barzin <viktorbarzin@meta.com>
Date: Sat, 21 Feb 2026 15:07:50 +0000
Subject: [PATCH] docs: add CI pipeline optimization design document

Merge tests into Dockerfiles to eliminate double dependency installs,
switch Docker layer caching to local registry at 10.0.20.10:5000.
---
 ...6-02-21-ci-pipeline-optimization-design.md | 69 +++++++++++++++++++
 1 file changed, 69 insertions(+)
 create mode 100644 docs/plans/2026-02-21-ci-pipeline-optimization-design.md

diff --git a/docs/plans/2026-02-21-ci-pipeline-optimization-design.md b/docs/plans/2026-02-21-ci-pipeline-optimization-design.md
new file mode 100644
index 0000000..5e22c77
--- /dev/null
+++ b/docs/plans/2026-02-21-ci-pipeline-optimization-design.md
@@ -0,0 +1,69 @@
+# CI Pipeline Optimization — Design Document
+
+Date: 2026-02-21
+
+## Problem
+
+The Drone CI pipeline is slow. Both the frontend and backend pipelines install dependencies multiple times per build and pull Docker cache layers from Docker Hub instead of the local registry at `10.0.20.10:5000`.
+
+## Root Causes
+
+1. **Frontend `npm ci` runs twice**: once in the test step (bare node image), once inside the Kaniko Dockerfile build. 560MB of node_modules downloaded each time.
+2. **Backend builds 2 Docker images**: one for the `:test` target, one for the final image. Both execute the `builder` stage which runs `pip install -r requirements.txt` (1874 lines, hash-pinned).
+3. **Docker cache pulled from Docker Hub**: `cache_from` and `cache_repo` point to Docker Hub. Local registry at `10.0.20.10:5000` already exists and has all images but is not used for caching.
+
+## Solution
+
+### 1. Local Registry for Docker Layer Caching
+
+Switch all `cache_from` and `cache_repo` references from Docker Hub to `10.0.20.10:5000`. The local registry already has `viktorbarzin/realestatecrawler` and `viktorbarzin/immoweb` with `builder`, `test`, and `latest` tags. Local network pulls are near-instant vs Docker Hub.
+
+- Docker Hub remains the "production" registry (k8s pulls from there)
+- Local registry becomes the "CI cache" registry
+
+### 2. Merge Tests into Dockerfile (Frontend)
+
+Replace the separate "Run frontend tests" Drone step + Kaniko build with a single multi-stage Dockerfile build:
+
+- `deps` stage: `npm ci` (cached if `package-lock.json` unchanged)
+- `test` stage: runs `vitest run` (fails the build if tests fail)
+- `builder` stage: runs `vite build`
+- Final stage: nginx serving built assets
+
+This eliminates the double `npm ci`.
+
+### 3. Merge Tests into Dockerfile (Backend)
+
+Replace the separate "Cache test image" + "Run backend tests" + "Build API image" steps with a single Docker build:
+
+- `builder` stage: `pip install -r requirements.txt` (cached if requirements unchanged)
+- `runtime-base` stage: system dependencies
+- `test` stage: runs `pytest tests/ -x -q` (fails the build if tests fail)
+- Final stage: production image with app code
+
+This eliminates the double pip install and double image build.
+
+### 4. Simplified Pipeline Steps
+
+**Frontend: 5 steps to 4**
+
+Before: clone, npm ci + vitest, Kaniko build (npm ci again), deploy, verify
+After: clone, Kaniko build (npm ci + vitest + vite build), deploy, verify
+
+**Backend: 5 steps to 4**
+
+Before: clone, build :test image (pip install), pytest, build final image (pip install again), deploy + verify
+After: clone, build image (pip install + pytest + final), deploy, verify
+
+## Expected Impact
+
+- `npm ci` runs once instead of twice per frontend build
+- `pip install` runs once instead of twice per backend build
+- Docker layer cache from local network instead of Docker Hub
+- On cache hit (deps unchanged): dependency install steps are completely skipped (cached layers)
+
+## Constraints
+
+- Tests must always run (gate deployment)
+- Final images must still be pushed to Docker Hub for k8s to pull
+- Local registry is HTTP-only at `10.0.20.10:5000`