diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index af780d1b..54b51441 100755 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -104,15 +104,13 @@ have `ignore_changes` on `…container[0].image` (KEEL_IGNORE_IMAGE) so CI `:latest` + `imagePullPolicy: Always` (fresh pod each run) instead of a deploy step. **Never** `set image`/`rollout restart` operator-managed StatefulSets (memory id=740). Reference impls: `tuya_bridge/.woodpecker.yml`, -`job-hunter`, `f1-stream` (viktor/f1-stream, extracted from this monorepo -2026-06-04). This reverses decision #12 of +`job-hunter`. This reverses decision #12 of `docs/plans/2026-05-16-auto-upgrade-apps-design.md` for owned (not upstream) images. **Flow (GHA-migrated apps)**: `git push → GHA build+push DockerHub (8-char SHA) → POST Woodpecker API → kubectl set image` -**Migrated to GHA** (9): Website, k8s-portal, claude-memory-mcp, apple-health-data, audiblez-web, plotting-book, insta2spotify, audiobook-search, council-complaints -**Woodpecker-native owned-app build** (Forgejo registry, build->deploy in one `.woodpecker.yml`): tuya_bridge, job-hunter, f1-stream (extracted to viktor/f1-stream 2026-06-04; Woodpecker repo id 166) +**Migrated to GHA** (10): Website, k8s-portal, f1-stream, claude-memory-mcp, apple-health-data, audiblez-web, plotting-book, insta2spotify, audiobook-search, council-complaints **Woodpecker-only**: travel_blog (1.4GB content too large for GHA), infra pipelines (terragrunt apply, certbot, build-cli — need cluster access) **Per-project files**: @@ -121,7 +119,7 @@ images. - `.woodpecker/build-fallback.yml` — Old full build pipeline preserved (event: `deployment` — never auto-fires) **Woodpecker API**: Uses **numeric repo IDs** (`/api/repos/2/pipelines`), NOT owner/name paths (those return HTML). -Repo IDs: infra=1, Website=2, finance=3, health=4, travel_blog=5, webhook-handler=6, audiblez-web=9, plotting-book=43, claude-memory-mcp=78, infra-onboarding=79, council-complaints=TBD (f1-stream's old GHA-era id 10 is defunct; it's now a Woodpecker-native build at repo id 166) +Repo IDs: infra=1, Website=2, finance=3, health=4, travel_blog=5, webhook-handler=6, audiblez-web=9, f1-stream=10, plotting-book=43, claude-memory-mcp=78, infra-onboarding=79, council-complaints=TBD **Woodpecker YAML gotchas**: - Commands with `${VAR}:${VAR}` must be **quoted** — unquoted `:` triggers YAML map parsing when vars are empty diff --git a/.claude/reference/service-catalog.md b/.claude/reference/service-catalog.md index 5741b227..86016d62 100644 --- a/.claude/reference/service-catalog.md +++ b/.claude/reference/service-catalog.md @@ -46,7 +46,7 @@ | nextcloud | File sync/share | nextcloud | | calibre | E-book management (may be merged into ebooks stack) | calibre | | onlyoffice | Document editing | onlyoffice | -| f1-stream | F1 streaming (uses chrome-service for hmembeds verifier); source in own repo `viktor/f1-stream` (extracted 2026-06-04), Woodpecker-native build->deploy | f1-stream | +| f1-stream | F1 streaming (uses chrome-service for hmembeds verifier) | f1-stream | | chrome-service | Headed Chromium WebSocket pool (`ws://chrome-service.chrome-service.svc:3000/`) for sibling services driving anti-bot embeds | chrome-service | | rybbit | Analytics | rybbit | | isponsorblocktv | SponsorBlock for TV | isponsorblocktv | diff --git a/docs/architecture/ci-cd.md b/docs/architecture/ci-cd.md index 75699744..4c0c020b 100644 --- a/docs/architecture/ci-cd.md +++ b/docs/architecture/ci-cd.md @@ -58,9 +58,10 @@ graph LR ### Project Migration Status -**Migrated to GHA (8 projects)**: +**Migrated to GHA (9 projects)**: - Website - k8s-portal +- f1-stream - claude-memory-mcp - apple-health-data - audiblez-web @@ -68,14 +69,6 @@ graph LR - insta2spotify - book-search (audiobook-search) -**Woodpecker-native owned-app builds** (build + push to the Forgejo private -registry + `kubectl set image` rollout, all in one `.woodpecker.yml`; Keel -stays enrolled as a redundant net): -- `tuya_bridge`, `job-hunter`, `f1-stream` -- `f1-stream` was extracted from this monorepo into its own repo - (`viktor/f1-stream`) on 2026-06-04; its Woodpecker repo id is 166 (the old - GHA-era id 10 is defunct). - **Woodpecker-only (infra + large apps)**: - `travel_blog`: 5.7GB content directory exceeds GHA limits - Infra pipelines: require cluster access (terragrunt apply, certbot, build-cli) @@ -99,6 +92,7 @@ Woodpecker API uses numeric IDs (not owner/name): | travel_blog | 5 | | webhook-handler | 6 | | audiblez-web | 9 | +| f1-stream | 10 | | plotting-book | 43 | | claude-memory-mcp | 78 | | infra-onboarding | 79 | diff --git a/docs/plans/2026-06-04-f1-stream-extraction-design.md b/docs/plans/2026-06-04-f1-stream-extraction-design.md deleted file mode 100644 index 1ccc7909..00000000 --- a/docs/plans/2026-06-04-f1-stream-extraction-design.md +++ /dev/null @@ -1,78 +0,0 @@ -# f1-stream extraction + productionization — design (2026-06-04) - -## Problem - -`f1-stream` (FastAPI backend serving a SvelteKit SPA; ~15 pluggable stream -extractors + a Playwright/chrome-service playback verifier) lived **inside** -the infra monorepo at `infra/stacks/f1-stream/files/`. It had: - -- no standalone repo — source coupled to the Terraform stack; -- **no real CI** — only a manual `redeploy.sh` doing a local `docker buildx` - push to DockerHub (`viktorbarzin/f1-stream`) + `kubectl rollout restart`; -- no README, no tests, a loose unpinned `requirements.txt`, no semver tags; -- a stale CI claim in docs ("migrated to GHA, Woodpecker repo id 10") that did - not match reality (no GHA workflow ever existed for it). - -## Goal - -Extract the app into its own Forgejo repo `viktor/f1-stream` and productionize -it, mirroring the established owned-app pattern (`tuya_bridge`, `job-hunter`, -`tripit`, `travel-agent`). - -## Decisions (with rationale) - -- **Registry → Forgejo private** (`forgejo.viktorbarzin.me/viktor/f1-stream`), - matching the fleet standard. Needs the `registry-credentials` pull secret - (Kyverno-synced to every namespace) on the deployment. -- **Packaging → Poetry + ruff + mypy** (replaces the loose pip - `requirements.txt`). Python **package stays `backend`** — imports are - `from backend.x` and the entrypoint is `uvicorn backend.main:app`; renaming - would churn every module + the Dockerfile + the staticfiles path. Python - **3.13 kept** (the live image already runs it; tripit's 3.12 pin is for - zxing-cpp/pymupdf, which f1-stream lacks). -- **Tests → pragmatic pure-logic only**. The extractors + verifier are - network/browser-bound; full coverage is brittle. Unit-test the deterministic - core: `m3u8_rewriter` (incl. the EXT-X tag rewriters), the `proxy` HLS - parsers, `schedule` parsing/status, the extractor `registry`. 63 tests. -- **CI → single `.woodpecker.yml`**: `lint-and-test` (ruff + mypy + pytest on - `python:3.13-slim`) → `build-and-push` (buildx → Forgejo, tags `latest` + - `${CI_COMMIT_SHA:0:8}`) → `deploy` (`kubectl set image` + `rollout status`). - **Keel stays enrolled** as a redundant net. This is the `tuya_bridge` - "build drives the rollout" model + a `travel-agent`-style test gate. - - A Slack-notify step was prototyped but **dropped**: the - `environment: { from_secret }` form is rejected by this Woodpecker - version's pipeline-struct decoder (`yaml: did not find expected key`), and - the canonical owned-app refs (`tuya_bridge`, `job-hunter`) have no Slack - step. Deploy success is confirmed by `rollout status`. -- **Versioning → first git tag `v2.0.1`** (continuity with the existing image - lineage; a fresh `v0.1.0` on a production 2.x app would mislead - monitoring/homepage). Deviates deliberately from the `v0.1.0` precedent of - tripit/travel-agent. -- **Runtime stays root** (matching the prior working image) to avoid a - non-root regression on the `/data` NFS write path and the Playwright browser - cache. Non-root is a possible future hardening. - -## Terraform delta (the only infra change) - -`infra/stacks/f1-stream/main.tf`: - -- image `viktorbarzin/f1-stream:latest` (DockerHub) → - `forgejo.viktorbarzin.me/viktor/f1-stream:${var.image_tag}` (new - `var.image_tag`, default `latest`); -- add `image_pull_secrets { name = "registry-credentials" }` to the pod spec; -- delete `files/` (source now lives in the standalone repo) and `redeploy.sh`. - -The image field is in the deployment's `ignore_changes` (KEEL_IGNORE_IMAGE), so -the live tag is managed by CI/Keel, not Terraform. Everything else — namespace, -ExternalSecrets (`f1-stream-secrets`, `chrome-service-client-secrets`), NFS data -volume, Anubis PoW policy, `ingress_factory`, homepage + x402 annotations, -Discord + chrome-service env — is unchanged. - -## Blast radius - -- The `f1-stream` K8s service is the only consumer; no other stack references - `viktorbarzin/f1-stream` or the `files/` dir (verified: no `path.module` / - `archive_file` / `null_resource` references the dir). -- Adding `imagePullSecrets` triggers one Recreate rollout that pulls the - *current* (still-DockerHub, public) image — safe; CI then switches it to the - Forgejo image. diff --git a/docs/plans/2026-06-04-f1-stream-extraction-plan.md b/docs/plans/2026-06-04-f1-stream-extraction-plan.md deleted file mode 100644 index ba773802..00000000 --- a/docs/plans/2026-06-04-f1-stream-extraction-plan.md +++ /dev/null @@ -1,54 +0,0 @@ -# f1-stream extraction + productionization — plan (2026-06-04) - -Companion to `2026-06-04-f1-stream-extraction-design.md`. - -## Steps - -1. **Scaffold** `/home/wizard/code/f1-stream/` — copy `backend/`, `frontend/`, - `Dockerfile`, `.dockerignore` from `infra/stacks/f1-stream/files/` by name - (exclude the `.claude/` marker + `redeploy.sh`); add `README.md`, - `.gitignore`. ✅ -2. **Poetry conversion** — `pyproject.toml` (dist `f1-stream` v2.0.1, - `packages=[{include="backend"}]`, pinned deps), `poetry.lock`, ruff/mypy/ - pytest config (E501 per-file-ignored on the embedded-JS/scraper modules). - Rewrite the Dockerfile to a Poetry multi-stage build (Poetry 2.1.3 to match - the lock; python:3.13; keep Chromium libs + `playwright install chromium`; - keep `backend/` + `frontend/build/` siblings under `/app`). ✅ -3. **Tests** — 63 pytest unit tests over the pure-logic core. ✅ -4. **CI** — single `.woodpecker.yml` (lint+test → buildx push to Forgejo → - `kubectl set image` + rollout). ✅ -5. **Create + push** — Forgejo repo `viktor/f1-stream` (private), commit, push - `master`, tag `v2.0.1`. ✅ -6. **Enable in Woodpecker** — activate via - `scripts/woodpecker-register-forgejo-repo.sh` (Woodpecker repo id 166); - org-level `forgejo_user`/`forgejo_push_token` secrets apply. ✅ -7. **Repoint Terraform** — `main.tf` image → Forgejo + `var.image_tag` + - `image_pull_secrets`; `tg apply`. ✅ -8. **Untrack from infra** — `git rm -r stacks/f1-stream/files`; add - `/f1-stream/` to the monorepo root `.gitignore`. ✅ -9. **Docs** — fix the stale "GHA / repo id 10" claim in `.claude/CLAUDE.md` + - `docs/architecture/ci-cd.md`; update `service-catalog.md`; this design/plan - pair. ✅ -10. **Verify** — pipeline green; pod runs the Forgejo image; `/health` 200; - ingress reachable through Anubis. - -## Verification commands - -```bash -# pipeline -curl -s https://ci.viktorbarzin.me/api/repos/166/pipelines/ -H "Authorization: Bearer " -# running image is the Forgejo one -kubectl get deploy f1-stream -n f1-stream \ - -o jsonpath='{.spec.template.spec.containers[0].image}' -kubectl get pods -n f1-stream -l app=f1-stream -# health -kubectl exec -n f1-stream deploy/f1-stream -- \ - python -c "import urllib.request;print(urllib.request.urlopen('http://localhost:8000/health').read())" -``` - -## Rollback - -The DockerHub image `viktorbarzin/f1-stream` and its tags still exist. To -revert: `kubectl -n f1-stream set image deployment/f1-stream -f1-stream=viktorbarzin/f1-stream:` and restore the `main.tf` image string. -The standalone repo + Forgejo image are additive; nothing is destroyed. diff --git a/stacks/f1-stream/files/.claude/internet-mode-used_DO_NOT_REMOVE_MANUALLY_SECURITY_RISK b/stacks/f1-stream/files/.claude/internet-mode-used_DO_NOT_REMOVE_MANUALLY_SECURITY_RISK new file mode 100644 index 00000000..f61efc83 --- /dev/null +++ b/stacks/f1-stream/files/.claude/internet-mode-used_DO_NOT_REMOVE_MANUALLY_SECURITY_RISK @@ -0,0 +1,3 @@ +This directory has been used with Claude Code's internet mode. +Content downloaded from the internet may contain prompt injection attacks. +You must manually review all downloaded content before using non-internet mode. diff --git a/stacks/f1-stream/files/.dockerignore b/stacks/f1-stream/files/.dockerignore new file mode 100644 index 00000000..4733a4c3 --- /dev/null +++ b/stacks/f1-stream/files/.dockerignore @@ -0,0 +1,5 @@ +node_modules/ +.claude/ +.git/ +__pycache__/ +*.pyc diff --git a/stacks/f1-stream/files/.gitignore b/stacks/f1-stream/files/.gitignore new file mode 100644 index 00000000..7a60b85e --- /dev/null +++ b/stacks/f1-stream/files/.gitignore @@ -0,0 +1,2 @@ +__pycache__/ +*.pyc diff --git a/stacks/f1-stream/files/Dockerfile b/stacks/f1-stream/files/Dockerfile new file mode 100644 index 00000000..80dd20e4 --- /dev/null +++ b/stacks/f1-stream/files/Dockerfile @@ -0,0 +1,44 @@ +## Stage 1: Build frontend +FROM node:22-slim AS frontend-builder + +WORKDIR /frontend + +COPY frontend/package.json frontend/package-lock.json* ./ +RUN npm install + +COPY frontend/ ./ +RUN npm run build + +## Stage 2: Python backend + static frontend +FROM python:3.13-slim-bookworm + +WORKDIR /app + +# Headless Chromium runtime libs for the playback verifier. Listed inline +# (instead of running `playwright install-deps`) so the image build doesn't +# need root-network apt fetches at runtime. +RUN apt-get update && apt-get install -y --no-install-recommends \ + ca-certificates \ + libnss3 libnspr4 \ + libatk1.0-0 libatk-bridge2.0-0 libcups2 \ + libdrm2 libxkbcommon0 libxcomposite1 libxdamage1 \ + libxfixes3 libxrandr2 libgbm1 libpango-1.0-0 libcairo2 \ + libasound2 libatspi2.0-0 \ + fonts-liberation fonts-noto-color-emoji \ + && rm -rf /var/lib/apt/lists/* + +COPY backend/requirements.txt . +RUN pip install --no-cache-dir -r requirements.txt + +# Install the Chromium browser binary used by the verifier. Skip +# --with-deps because we already installed the system libs above. +RUN playwright install chromium + +COPY backend/ ./backend/ + +# Copy built frontend into the image +COPY --from=frontend-builder /frontend/build ./frontend/build + +EXPOSE 8000 + +CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8000"] diff --git a/stacks/f1-stream/files/backend/__init__.py b/stacks/f1-stream/files/backend/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/stacks/f1-stream/files/backend/embed_proxy.py b/stacks/f1-stream/files/backend/embed_proxy.py new file mode 100644 index 00000000..34ccb28c --- /dev/null +++ b/stacks/f1-stream/files/backend/embed_proxy.py @@ -0,0 +1,359 @@ +"""Embed iframe-stripping reverse proxy. + +Serves third-party embed pages (e.g. https://hmembeds.one/embed/{hash}, +https://pooembed.eu/embed/{slug}) through our origin so we can: + +1. Strip X-Frame-Options and Content-Security-Policy: frame-ancestors headers, + so the embed loads in our + {:else} + + {/if} + + +
+
+ + + + setVolume(i, e)} + class="w-16 h-1 accent-f1-red" + aria-label="Volume" + /> + +
+ + +
+
+ + + {#if player.error} +
+ {player.error} +
+ {/if} + + {/each} + + {/if} + + + {#if loading} +
+
+ Loading streams... +
+ {:else if errorMsg} +
+

Failed to load streams: {errorMsg}

+ +
+ {:else if streamsData} +
+

+ Available Streams + ({streamsData.count}) +

+
+ {#if players.length > 0} + {players.length}/{MAX_PLAYERS} streams active + {/if} + +
+
+ + {#if streamsData.streams.length === 0} +
+

No streams available right now.

+

Streams appear when a session is live. Check the schedule for upcoming sessions.

+ + View Schedule + +
+ {:else} +
+ {#each streamsData.streams as stream, i} + {@const active = isStreamActive(stream.stream_type === 'embed' ? stream.embed_url : stream.url)} +
+
+
+ {stream.site_name || stream.site_key || 'Unknown'} + {#if stream.is_live} + Live + {/if} + {#if stream.stream_type === 'embed'} + Embed + {/if} + {#if active} + Playing + {/if} +
+
+ {#if stream.title} + {stream.title} + {/if} + {#if stream.quality} + {stream.quality} + {/if} + {#if stream.response_time_ms != null} + + {stream.response_time_ms}ms + + {/if} +
+
+ +
+ {#if !active} + + {:else} + Active + {/if} +
+
+ {/each} +
+ {/if} + {/if} + diff --git a/stacks/f1-stream/files/frontend/svelte.config.js b/stacks/f1-stream/files/frontend/svelte.config.js new file mode 100644 index 00000000..9088ef3b --- /dev/null +++ b/stacks/f1-stream/files/frontend/svelte.config.js @@ -0,0 +1,19 @@ +import adapter from '@sveltejs/adapter-static'; + +/** @type {import('@sveltejs/kit').Config} */ +const config = { + kit: { + adapter: adapter({ + pages: 'build', + assets: 'build', + fallback: 'index.html', + precompress: false, + strict: true + }), + paths: { + base: '' + } + } +}; + +export default config; diff --git a/stacks/f1-stream/files/frontend/vite.config.js b/stacks/f1-stream/files/frontend/vite.config.js new file mode 100644 index 00000000..a39ec5c1 --- /dev/null +++ b/stacks/f1-stream/files/frontend/vite.config.js @@ -0,0 +1,10 @@ +import { sveltekit } from '@sveltejs/kit/vite'; +import tailwindcss from '@tailwindcss/vite'; +import { defineConfig } from 'vite'; + +export default defineConfig({ + plugins: [ + tailwindcss(), + sveltekit() + ] +}); diff --git a/stacks/f1-stream/files/redeploy.sh b/stacks/f1-stream/files/redeploy.sh new file mode 100755 index 00000000..e436a6ce --- /dev/null +++ b/stacks/f1-stream/files/redeploy.sh @@ -0,0 +1,7 @@ +#!/usr/bin/env bash +set -e + +docker buildx build --platform linux/amd64 --provenance=false \ + -t viktorbarzin/f1-stream:v2.0.1 -t viktorbarzin/f1-stream:latest \ + --push . +kubectl -n f1-stream rollout restart deployment f1-stream diff --git a/stacks/f1-stream/main.tf b/stacks/f1-stream/main.tf index 9cbe6ce5..ff64af71 100644 --- a/stacks/f1-stream/main.tf +++ b/stacks/f1-stream/main.tf @@ -6,15 +6,6 @@ variable "nfs_server" { type = string } variable "discord_f1_guild_id" { type = string } variable "discord_f1_channel_ids" { type = string } -# Image tag for the Forgejo-registry image. CI (.woodpecker.yml in -# viktor/f1-stream) builds + pushes `latest` and ``, then drives the -# rollout via `kubectl set image`. Keel stays enrolled as a redundant net, so -# the running tag is managed outside Terraform (see KEEL_IGNORE_IMAGE below). -variable "image_tag" { - type = string - default = "latest" -} - resource "kubernetes_namespace" "f1-stream" { metadata { name = "f1-stream" @@ -22,7 +13,7 @@ resource "kubernetes_namespace" "f1-stream" { "istio-injection" : "disabled" tier = local.tiers.aux "chrome-service.viktorbarzin.me/client" = "true" - "keel.sh/enrolled" = "true" + "keel.sh/enrolled" = "true" } } lifecycle { @@ -127,7 +118,7 @@ resource "kubernetes_deployment" "f1-stream" { } spec { container { - image = "forgejo.viktorbarzin.me/viktor/f1-stream:${var.image_tag}" + image = "viktorbarzin/f1-stream:latest" image_pull_policy = "Always" name = "f1-stream" resources { @@ -185,11 +176,6 @@ resource "kubernetes_deployment" "f1-stream" { claim_name = module.nfs_data_host.claim_name } } - # Pull the (private) Forgejo-registry image. Kyverno syncs - # registry-credentials into every namespace. - image_pull_secrets { - name = "registry-credentials" - } } } }