trading/docs/plans/2026-05-21-meet-kevin-revival-plan.md
Viktor Barzin 8f616e6487 add Meet Kevin revival implementation plan
22-task plan across 6 phases (data layer, watcher service stages,
API gateway, dashboard, container/deps, K8s revival). Each task
includes TDD checkboxes, file paths, acceptance criteria, and
commit commands. References the design doc (ab382af) for schemas,
prompts, and the Terraform diff.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 19:01:24 +00:00

40 KiB
Raw Blame History

Meet Kevin Revival Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Ship the v1 Meet Kevin pipeline (RSS poll → caption extraction → Claude Sonnet 4.6 structured analysis → Postgres → ticker-centric UI) inside the existing viktor/trading repo, and revive the K8s stack with the 3 expensive services disabled and the new watcher container added.

Architecture: Single long-running async service (services/meet_kevin_watcher/) walks each video through 3 sequential stages and writes directly to Postgres. No Redis streams in v1. The existing api_gateway gains 11 new routes; the dashboard gains 5 new pages under /meet-kevin/*. The K8s revival is a single infra/stacks/trading-bot/main.tf edit (uncomment block + remove 3 disabled containers + add 1 new container + extend ExternalSecret + bump common env).

Tech Stack: Python 3.12, FastAPI, SQLAlchemy 2.0 (async), Pydantic v2, Alembic, feedparser, yt-dlp>=2026.04 (PyPI not apt), anthropic>=0.40 SDK with tool-use forcing for structured output + prompt caching, React 19 + TanStack Query + recharts + Tailwind 4, Terraform/Terragrunt, Vault KV + ExternalSecret.

Reference: spec at 2026-05-21-meet-kevin-revival-design.md (commit ab382af). The spec is the source of truth for schemas, prompts, endpoint shapes, and the Terraform diff — this plan sequences the implementation and pins down acceptance criteria per step.

Workspace: all backend + dashboard work happens inside /home/wizard/code/trading-bot/ (the cloned trading repo). The K8s revival in Phase 6 happens inside /home/wizard/code/infra/. Commits use me@viktorbarzin.me as the author email (personal-repo convention).

Pattern crib-sheet (read once before starting):

  • Service main loop pattern: services/news_fetcher/main.pyasyncio.Event shutdown, signal handlers, setup_telemetry, graceful close
  • Config pattern: services/news_fetcher/config.py — extends shared.config.BaseConfig, TRADING_ env prefix
  • Model pattern: shared/models/news.py — SQLAlchemy 2.0 Mapped[]/mapped_column() style, TimestampMixin from base.py
  • Schema pattern: shared/schemas/news.py — Pydantic v2 BaseModel, Field(ge=…, le=…)
  • Migration pattern: alembic/versions/b2c3d4e5f6a7_add_fundamentals_table.pyop.create_table with explicit columns, no autogen
  • API route pattern: services/api_gateway/routes/news.pyAPIRouter(prefix="/api/..."), _user=Depends(get_current_user), request.app.state.db_session_factory
  • Test pattern: tests/services/test_news_fetcher.py — pytest-asyncio auto mode, MagicMock/AsyncMock, committed fixture files
  • Dashboard page pattern: dashboard/src/pages/NewsFeed.tsx — TanStack Query, axios via api/client.ts, Tailwind on bg-slate-800 cards
  • Sidebar nav pattern: dashboard/src/components/Layout.tsx's navItems array

Phase 1 — Data layer

Task 1: Pydantic schemas for the Meet Kevin pipeline

Files:

  • Create: shared/schemas/meet_kevin.py
  • Modify: tests/test_schemas.py (append cases under a # --- Meet Kevin schemas --- section)

Required exports (match these names exactly — they're consumed by later tasks):

  • Enums: TickerAction (buy/sell/hold/watch/avoid), TimeHorizon (intraday/days/weeks/months/long_term/unspecified), MarketOutlook (bullish/neutral/bearish/mixed), VideoStatus (discovered/captioned/analyzed/failed/skipped), TranscriptSource (captions_manual/captions_auto/none)
  • LLM tool-input: MeetKevinTickerMention (symbol uppercased via @field_validator, conviction Field(ge=0.0, le=1.0)), MeetKevinAnalysis (market_outlook_direction, market_outlook_reasoning, macro_themes:list[str], key_risks:list[str], summary, tickers:list[MeetKevinTickerMention])
  • API response shapes: TranscriptSegment, VideoSummary, VideoDetail, StockSummary, StockMention, StockTimeline, TimelineBucket, PipelineHealth

Full field-level schemas in the spec under "Data model" and "LLM output schema".

  • Step 1: Write failing tests in tests/test_schemas.py covering: ticker mention validates, symbol auto-uppercases, conviction out of [0,1] raises pydantic.ValidationError, full MeetKevinAnalysis validates with one ticker entry.

  • Step 2: Run tests, verify they fail with ModuleNotFoundError:

python -m pytest tests/test_schemas.py -v -k "meet_kevin"
  • Step 3: Implement shared/schemas/meet_kevin.py per the spec. Use Pydantic v2 (field_validator, model_config).

  • Step 4: Re-run tests, verify pass.

  • Step 5: Commit

git add shared/schemas/meet_kevin.py tests/test_schemas.py
git -c user.email=me@viktorbarzin.me commit -m "feat: add Meet Kevin pydantic schemas (analysis + API shapes)"

Task 2: SQLAlchemy models

Files:

  • Create: shared/models/meet_kevin.py
  • Modify: shared/models/__init__.py (add 5 imports + __all__ entries)
  • Modify: tests/test_models.py (append)

Tables (all use BigInteger PKs, TimestampMixin, JSONB for json columns, SAEnum with named types per the spec):

  • KevinChannel (table kevin_channels) — youtube_channel_id unique, poll_enabled bool, poll_interval_seconds int default 10800, daily_cost_cap_usd Numeric(8,2) default 5.00, last_polled_at
  • KevinVideo (table kevin_videos) — channel_id FK, youtube_video_id unique+indexed, title, description, published_at indexed, duration_seconds, thumbnail_url, status enum + index, failure_reason, retry_count default 0, processed_at
  • KevinTranscript (table kevin_transcripts) — video_id unique FK, source enum, language, raw_text, segments_json JSONB, word_count
  • KevinAnalysis (table kevin_analyses) — video_id FK indexed, model, prompt_version, market_outlook_direction enum, market_outlook_reasoning, macro_themes_json + key_risks_json JSONB, summary, raw_response_json JSONB, prompt_tokens/completion_tokens int, cost_usd Numeric(10,4)
  • KevinStockMention (table kevin_stock_mentions) — video_id + analysis_id FKs, symbol indexed, action enum, conviction Numeric(4,3), time_horizon enum, rationale_quote, video_timestamp_seconds nullable

Each table establishes Mapped[]/mapped_column() relationships so the ORM nav matches the spec (videos → transcript, analyses, mentions).

  • Step 1: Add test_meet_kevin_models_importable to tests/test_models.py asserting KevinChannel.__tablename__ == "kevin_channels" (etc.) and that KevinVideo.__table__.c.status is an Enum containing "discovered".

  • Step 2: Verify failure.

  • Step 3: Implement shared/models/meet_kevin.py.

  • Step 4: Update shared/models/__init__.py — import the 5 classes and add them to __all__.

  • Step 5: Re-run, verify pass.

  • Step 6: Commit

git add shared/models/meet_kevin.py shared/models/__init__.py tests/test_models.py
git -c user.email=me@viktorbarzin.me commit -m "feat: add Meet Kevin SQLAlchemy models (5 tables)"

Task 3: Alembic migration

Files:

  • Create: alembic/versions/c3d4e5f6a7b8_add_meet_kevin_tables.py

Requirements:

  • revision = "c3d4e5f6a7b8", down_revision = "b2c3d4e5f6a7" (current head from python -m alembic heads)
  • Create 5 named ENUM types: kevin_video_status, kevin_transcript_source, kevin_market_outlook, kevin_ticker_action, kevin_time_horizon
  • op.create_table for all 5 tables matching the SQLAlchemy models exactly (column types, defaults, indexes)
  • Composite index ix_kevin_stock_mentions_symbol_created_at on (symbol, created_at DESC) for ticker-timeline queries
  • Partial index ix_kevin_videos_status_pending on status WHERE status IN ('discovered', 'captioned') for the pipeline scan
  • Seed the one channel: INSERT INTO kevin_channels (youtube_channel_id, title) VALUES ('UCUvvj5lwue7PspotMDjk5UA', 'Meet Kevin')
  • downgrade() reverses everything including the 5 enum types

Pattern model: alembic/versions/b2c3d4e5f6a7_add_fundamentals_table.py.

  • Step 1: Confirm current head
python -m alembic heads
# expected: b2c3d4e5f6a7 (head)
  • Step 2: Write the migration file.

  • Step 3: Apply against the docker-compose Postgres

docker compose up -d postgres
TRADING_DATABASE_URL=postgresql+asyncpg://trading:trading@localhost:5432/trading \
python -m alembic upgrade head
# expected: "Running upgrade b2c3d4e5f6a7 -> c3d4e5f6a7b8"
  • Step 4: Verify 5 tables exist:
docker compose exec -T postgres psql -U trading -d trading -c "\dt kevin_*"
# expected: 5 rows
  • Step 5: Round-trip check (downgrade -1 shows 0 rows; upgrade head restores).

  • Step 6: Commit

git add alembic/versions/c3d4e5f6a7b8_add_meet_kevin_tables.py
git -c user.email=me@viktorbarzin.me commit -m "feat: add Alembic migration for Meet Kevin tables"

Phase 2 — Watcher service stages

Task 4: Service scaffold + config

Files:

  • Create: services/meet_kevin_watcher/__init__.py (empty)
  • Create: services/meet_kevin_watcher/config.py

MeetKevinWatcherConfig extends BaseConfig, exposing (defaults shown):

  • meet_kevin_channel_id: str = "UCUvvj5lwue7PspotMDjk5UA"

  • meet_kevin_poll_interval_seconds: int = 10800

  • meet_kevin_max_caption_retries: int = 5

  • meet_kevin_max_llm_retries: int = 3

  • meet_kevin_llm_model: str = "claude-sonnet-4-6"

  • meet_kevin_prompt_version: str = "v1"

  • meet_kevin_daily_cost_cap_usd: float = 5.0

  • anthropic_api_key: str = ""

  • meet_kevin_workdir: str = "/tmp/meet_kevin_captions"

  • otel_service_name: str = "meet-kevin-watcher"

  • otel_metrics_port: int = 9097 (next free port after market_data's 9096)

  • Step 1: Create the dir, empty __init__.py, and the config file.

  • Step 2: Verify importable: python -c "from services.meet_kevin_watcher.config import MeetKevinWatcherConfig; print(MeetKevinWatcherConfig().otel_metrics_port)"9097.

  • Step 3: Commit

git add services/meet_kevin_watcher/__init__.py services/meet_kevin_watcher/config.py
git -c user.email=me@viktorbarzin.me commit -m "feat: scaffold meet_kevin_watcher service + config"

Task 5: RSS poller

Files:

  • Create: services/meet_kevin_watcher/rss_poller.py
  • Create: tests/services/meet_kevin_watcher/__init__.py (empty)
  • Create: tests/services/meet_kevin_watcher/test_rss_poller.py
  • Create: tests/fixtures/meet_kevin_rss.xml (real fixture)

Exports:

  • @dataclass(frozen=True) DiscoveredVideo(youtube_video_id, title, description, published_at, thumbnail_url)

  • parse_feed(xml_bytes: bytes) -> list[DiscoveredVideo] — uses stdlib xml.etree.ElementTree, returns [] on parse error/empty

  • async fetch_feed(channel_id: str, client: httpx.AsyncClient) -> bytes — GET with 15s timeout, returns b"" on HTTP error

  • Atom namespaces: {"a": "http://www.w3.org/2005/Atom", "y": "http://www.youtube.com/xml/schemas/2015", "m": "http://search.yahoo.com/mrss/"}

  • Feed URL template: https://www.youtube.com/feeds/videos.xml?channel_id={channel_id}

  • Step 1: Capture real fixture

mkdir -p tests/fixtures
curl -sL "https://www.youtube.com/feeds/videos.xml?channel_id=UCUvvj5lwue7PspotMDjk5UA" \
  -o tests/fixtures/meet_kevin_rss.xml
test -s tests/fixtures/meet_kevin_rss.xml || { echo "FAIL: empty fixture"; exit 1; }
  • Step 2: Write failing tests asserting:

    • parse_feed(fixture.read_bytes()) returns ≥1 DiscoveredVideo with 11-char video_id, title, published_at, thumbnail starting with https
    • parse_feed(b"") and parse_feed(b"<?xml version='1.0'?><not-a-feed/>") both return []
  • Step 3: Verify failure (ModuleNotFoundError).

  • Step 4: Implement rss_poller.py.

  • Step 5: Verify pass.

  • Step 6: Commit

git add services/meet_kevin_watcher/rss_poller.py \
        tests/services/meet_kevin_watcher/__init__.py \
        tests/services/meet_kevin_watcher/test_rss_poller.py \
        tests/fixtures/meet_kevin_rss.xml
git -c user.email=me@viktorbarzin.me commit -m "feat(meet-kevin): RSS poller for YouTube uploads"

Task 6: Caption extractor

Files:

  • Create: services/meet_kevin_watcher/caption_extractor.py
  • Create: tests/services/meet_kevin_watcher/test_caption_extractor.py
  • Create: tests/fixtures/sample.srt (3-segment SRT mentioning NVDA)

Exports:

  • @dataclass(frozen=True) CaptionResult(source, language, raw_text, segments, word_count)
  • parse_srt(text: str) -> list[dict] — segments with {start: float, end: float, text: str}. Handles SRT and yt-dlp's webvtt-converted variants; [] on empty/malformed.
  • async extract_captions(video_id: str, workdir: str, sub_lang_glob: str = "en.*") -> CaptionResult | None — calls yt-dlp subprocess via the internal _run_yt_dlp helper, then parses any <video_id>*.srt left in workdir. Prefers manual subs (no .auto. in filename) over auto-generated. Cleans up SRT files after parsing. Returns None if no SRT was produced.
  • Internal async _run_yt_dlp(cmd: list[str], cwd: str) -> int — wraps asyncio.create_subprocess_exec for testability (mocked in tests).

yt-dlp invocation:

yt-dlp --write-auto-sub --write-sub --sub-lang en.* --skip-download --convert-subs srt \
       -o "<video_id>.%(ext)s" "https://www.youtube.com/watch?v=<video_id>"
  • Step 1: Create tests/fixtures/sample.srt with 3 segments (one mentioning NVDA).

  • Step 2: Write tests:

    • parse_srt extracts 3 segments with correct start/end/text
    • parse_srt("") returns []
    • extract_captions returns a CaptionResult when yt-dlp writes an SRT (mock _run_yt_dlp to return 0, pre-place SRT in tmp_path)
    • extract_captions returns None when no SRT lands
  • Step 3: Verify failure.

  • Step 4: Implement the module.

  • Step 5: Verify pass.

  • Step 6: Commit

git add services/meet_kevin_watcher/caption_extractor.py \
        tests/services/meet_kevin_watcher/test_caption_extractor.py \
        tests/fixtures/sample.srt
git -c user.email=me@viktorbarzin.me commit -m "feat(meet-kevin): caption extractor via yt-dlp"

Task 7: LLM analyzer (Claude Sonnet 4.6 with tool-use forcing)

Files:

  • Create: services/meet_kevin_watcher/llm_analyzer.py
  • Create: tests/services/meet_kevin_watcher/test_llm_analyzer.py

Exports:

  • @dataclass(frozen=True) LlmCallResult(analysis: MeetKevinAnalysis, raw_response: dict, prompt_tokens, completion_tokens, cost_usd: Decimal)
  • SYSTEM_PROMPT (module constant) — analyst-style system instructions per the spec's "LLM output schema" section; MUST mention submit_analysis (the tool name) and the structured contract
  • compute_cost_usd(model: str, input_tokens: int, output_tokens: int) -> Decimal — uses _PRICING dict keyed by model; pinned prices in code: "claude-sonnet-4-6": (Decimal("3"), Decimal("15")) ($/M input, $/M output); unknown model → Decimal("0") + warning log
  • class LlmAnalyzer(client: AsyncAnthropic, model: str, prompt_version: str) — single method async analyze(title, description, published_at, transcript_text, transcript_segments) -> LlmCallResult

Anthropic call shape:

  • client.messages.create(model=..., max_tokens=4096, system=[{type: "text", text: SYSTEM_PROMPT, cache_control: {type: "ephemeral"}}], tools=[ANALYSIS_TOOL], tool_choice={type: "tool", name: "submit_analysis"}, messages=[{role: "user", content: <metadata + transcript>}])

  • ANALYSIS_TOOL defines a JSON schema mirroring MeetKevinAnalysis (see spec)

  • Response: pick the first tool_use content block, validate block.input via MeetKevinAnalysis.model_validate(...). Raise ValueError if no tool_use block (caller marks failed).

  • Step 1: Write tests using MagicMock/AsyncMock for anthropic.AsyncAnthropic:

    • Happy path: tool_use response with valid input → returns LlmCallResult with parsed MeetKevinAnalysis, correct token counts, cost_usd > 0
    • No tool_use block in response → ValueError("tool_use")
    • compute_cost_usd("claude-sonnet-4-6", 1_000_000, 1_000_000) == Decimal("18.0000")
    • compute_cost_usd("unknown-model", 1000, 1000) == Decimal("0")
    • SYSTEM_PROMPT contains "submit_analysis" and "ticker"
  • Step 2: Verify failure.

  • Step 3: Implement the module.

  • Step 4: Verify pass.

  • Step 5: Commit

git add services/meet_kevin_watcher/llm_analyzer.py \
        tests/services/meet_kevin_watcher/test_llm_analyzer.py
git -c user.email=me@viktorbarzin.me commit -m "feat(meet-kevin): Claude Sonnet 4.6 LLM analyzer (tool-use forcing + prompt cache)"

Task 8: Pipeline orchestrator + service main loop

Files:

  • Create: services/meet_kevin_watcher/pipeline.py
  • Create: services/meet_kevin_watcher/main.py
  • Create: tests/services/meet_kevin_watcher/test_pipeline.py

Exports from pipeline.py:

  • @dataclass(frozen=True) PipelineDeps(extract_captions, analyze, daily_cost_used, model, prompt_version, daily_cost_cap_usd: Decimal, workdir) — dependency injection container so the pipeline is unit-testable
  • async process_one_video(video: KevinVideo, session: AsyncSession, deps: PipelineDeps) -> str — runs the next pipeline stage and returns the new status. Status transitions:
    • discoveredextract_captions(). None → status=failed, failure_reason="no_captions". Otherwise insert KevinTranscript, advance to captioned.
    • captioned → check daily_cost_used() < daily_cost_cap_usd. If over cap, leave as captioned (skip). Otherwise call analyze(), on success insert KevinAnalysis + N KevinStockMention rows, advance to analyzed and set processed_at. On exception, increment retry_count; mark failed after 3 retries.
  • async daily_cost_used(session) -> DecimalSUM(cost_usd) from KevinAnalysis where created_at >= date_trunc('day', now())

main.py orchestrator (model: services/news_fetcher/main.py):

  • async run() entry point: load config, set up logging + OTEL + counters (meet_kevin.videos_discovered, meet_kevin.captions_extracted, meet_kevin.llm_calls, meet_kevin.llm_cost_usd), create DB engine + session factory, instantiate AsyncAnthropic + LlmAnalyzer, build PipelineDeps, install SIGTERM/SIGINT signal handlers on shutdown_event.

  • Main loop: poll RSS for each poll_enabled channel (use INSERT … ON CONFLICT (youtube_video_id) DO NOTHING RETURNING id to dedupe), then walk all videos with status in ("discovered", "captioned") ordered by published_at ASC, call process_one_video for each, commit per-video.

  • After each iteration, await asyncio.wait_for(shutdown_event.wait(), timeout=poll_interval_seconds) so shutdown is responsive.

  • On exit: await anthropic.close(), await engine.dispose().

  • if __name__ == "__main__": asyncio.run(run())

  • Step 1: Write 3 pipeline tests using MagicMock for KevinVideo + AsyncMock for session:

    • Happy path: discoveredcaptionedanalyzed, both deps awaited once
    • No captions: returns "failed", sets failure_reason="no_captions", analyze NOT awaited
    • Cost cap exceeded: status stays "captioned", analyze NOT awaited
  • Step 2: Verify failure.

  • Step 3: Implement pipeline.py then main.py.

  • Step 4: Verify all watcher tests pass

python -m pytest tests/services/meet_kevin_watcher/ -v
# expected: ~14 passing (2 RSS + 4 caption + 5 LLM + 3 pipeline)
  • Step 5: Commit
git add services/meet_kevin_watcher/pipeline.py \
        services/meet_kevin_watcher/main.py \
        tests/services/meet_kevin_watcher/test_pipeline.py
git -c user.email=me@viktorbarzin.me commit -m "feat(meet-kevin): pipeline orchestrator + service main loop"

Phase 3 — API gateway

Task 9: /api/meet-kevin/* routes

Files:

  • Create: services/api_gateway/routes/meet_kevin.py
  • Modify: services/api_gateway/main.py — import + app.include_router(meet_kevin_router) next to the other route registrations
  • Create: tests/api_gateway/routes/__init__.py (empty, if missing)
  • Create: tests/api_gateway/routes/test_meet_kevin.py

Router pattern (model: services/api_gateway/routes/news.py):

  • router = APIRouter(prefix="/api/meet-kevin", tags=["meet-kevin"])
  • All endpoints behind _user=Depends(get_current_user)
  • DB access via request.app.state.db_session_factory

Endpoints to implement (full request/response shapes in the spec's "API surface" section):

  1. GET /health — counts by status from kevin_videos, last poll timestamp from kevin_channels, daily cost from SUM(kevin_analyses.cost_usd) WHERE created_at >= date_trunc('day', now()), cap from channel row
  2. GET /channels — list all KevinChannel rows
  3. PATCH /channels/{id} — accept body fields poll_enabled, poll_interval_seconds, daily_cost_cap_usd
  4. GET /videos?status=&q=&page=&per_page= — paginated, ordered by published_at DESC, with top_tickers populated by joining latest kevin_stock_mentions per video (ordered by conviction DESC), and outlook/one_line_summary from the latest kevin_analyses row per video
  5. GET /videos/{id} — single video + analysis (latest by created_at) + first 5 ticker mentions ordered by conviction DESC + transcript_available: bool
  6. GET /videos/{id}/transcript — return segments_json + source + language; 404 if missing
  7. POST /videos/{id}/reprocess?stage=captions|analysis|auto — reset status: failed→discovered for captions/auto; captioned|failed|analyzed→captioned for analysis; reset retry_count and failure_reason
  8. GET /stocks — distinct symbols with mention_count, last_seen_at, latest_action (most recent mention), latest_conviction, avg_conviction
  9. GET /stocks/{symbol} — all mentions for the uppercased symbol joined with their videos, ordered by published_at DESC; 404 if none
  10. GET /stocks/{symbol}/timeline?bucket=day|weekdate_trunc(bucket, created_at) aggregation: avg_conviction, mention_count, net_action_score (sum of conviction for buy minus sum of conviction for sell, others = 0)
  11. GET /dashboard — latest analyzed video + its analysis + top 5 mentions, plus top conviction last 7 days (SELECT symbol, max(conviction), count() ... GROUP BY symbol LIMIT 10), plus 14-day outlook trend (date_trunc('day', kevin_analyses.created_at) × market_outlook_direction grouping)
  • Step 1: Write 3 API tests using FastAPI TestClient:

    • GET /api/meet-kevin/health → 200 with counts_by_status, daily_cost_usd, daily_cost_cap_usd keys
    • GET /api/meet-kevin/videos (empty DB) → {"videos": [], "total": 0, "page": 1, "per_page": 20}
    • GET /api/meet-kevin/stocks (empty DB) → {"stocks": []}

    Use monkeypatch.setenv("TRADING_DEV_MODE_AUTH_BYPASS", "true") if the repo has that flag, otherwise stub get_current_user via dependency override.

  • Step 2: Verify failure (404 — router not registered yet).

  • Step 3: Implement the route module.

  • Step 4: Register the router in main.py.

  • Step 5: Verify pass.

  • Step 6: Commit

git add services/api_gateway/routes/meet_kevin.py \
        services/api_gateway/main.py \
        tests/api_gateway/routes/test_meet_kevin.py \
        tests/api_gateway/routes/__init__.py 2>/dev/null
git -c user.email=me@viktorbarzin.me commit -m "feat(api): /api/meet-kevin/* routes (11 endpoints)"

Phase 4 — Dashboard

The repo has no FE unit test suite. For each FE task, verification is npm run build (zero TS/ESLint errors) followed by manual browser checks once the backend is up (Task 17).

Task 10: TypeScript types + API client

Files:

  • Create: dashboard/src/types/meetKevin.ts
  • Create: dashboard/src/api/meetKevin.ts

Types (one-to-one with the spec's API response shapes): TickerAction, TimeHorizon, MarketOutlook, VideoStatus, VideoSummary, TickerMention, VideoAnalysis, VideoDetail, TranscriptSegment, Transcript, StockSummary, StockMention, PipelineHealth, DashboardData.

API client (meetKevinApi) — thin wrappers on the shared axios client from dashboard/src/api/client.ts. Methods: health, dashboard, listVideos(params), getVideo(id), getTranscript(id), reprocess(id, stage), listStocks, getStock(symbol), getStockTimeline(symbol, bucket). Each method calls client.get/post with the path under /meet-kevin/... (the shared client already has /api as its base).

  • Step 1: Create the two files.

  • Step 2: Verify TS compiles

cd dashboard && npm run build && cd ..
  • Step 3: Commit
git add dashboard/src/types/meetKevin.ts dashboard/src/api/meetKevin.ts
git -c user.email=me@viktorbarzin.me commit -m "feat(dashboard): Meet Kevin TypeScript types + API client"

Task 11: Reusable Meet Kevin components

Files:

  • Create: dashboard/src/components/meetKevin/ActionChip.tsx
  • Create: dashboard/src/components/meetKevin/ConvictionBar.tsx
  • Create: dashboard/src/components/meetKevin/YouTubeEmbed.tsx
  • Create: dashboard/src/components/meetKevin/index.ts (barrel export)

ActionChip — accepts action: TickerAction prop. Color-coded chip per action: buy=green, sell=red, hold=slate, watch=yellow, avoid=rose. Tailwind classes only, no new CSS.

ConvictionBar — accepts value: number (0-1). Renders a clipped div with a blue fill bar (bg-blue-400 over bg-slate-700), width = pct%.

YouTubeEmbedforwardRef<YouTubeEmbedHandle, {videoId, startSeconds?}>. Renders an <iframe> pointed at https://www.youtube-nocookie.com/embed/<id>?start=<n>&enablejsapi=1. useImperativeHandle exposes seekTo(seconds) which postMessages {event: "command", func: "seekTo", args: [seconds, true]} to the iframe's contentWindow. The component uses the standard 16:9 aspect ratio via pb-[56.25%].

  • Step 1: Implement all 4 files.

  • Step 2: Verify build

cd dashboard && npm run build && cd ..
  • Step 3: Commit
git add dashboard/src/components/meetKevin/
git -c user.email=me@viktorbarzin.me commit -m "feat(dashboard): reusable Meet Kevin components (ActionChip, ConvictionBar, YouTubeEmbed)"

Task 12: Home page (/meet-kevin)

File: Create dashboard/src/pages/meetKevin/Home.tsx

Behavior:

  • Uses useQuery keyed ['meet-kevin', 'dashboard'] calling meetKevinApi.dashboard, refetch every 60s.

  • Uses useQuery keyed ['meet-kevin', 'health'] calling meetKevinApi.health, refetch every 60s.

  • Empty state (no latest_video): "Waiting for first poll" card with last_poll_at formatted.

  • Otherwise: latest video hero card (thumbnail + outlook badge + title + ticker chips + one-line summary, linking to /meet-kevin/videos/<id>), top conviction list (last 7 days, with ConvictionBar per row, each row linking to /meet-kevin/stocks/<symbol>), and an outlook trendline recharts.LineChart over the 14-day data (map outlook bullish/neutral/mixed/bearish to +1/0/0/-1 for the numeric Y-axis).

  • Tailwind aesthetic: bg-slate-800 border border-slate-700 rounded-xl p-5 cards (matches existing dashboard).

  • Step 1: Implement the page.

  • Step 2: Build verification.

  • Step 3: Commit

git add dashboard/src/pages/meetKevin/Home.tsx
git -c user.email=me@viktorbarzin.me commit -m "feat(dashboard): Meet Kevin home page"

Task 13: Videos feed page (/meet-kevin/videos)

File: Create dashboard/src/pages/meetKevin/Videos.tsx

Behavior:

  • Filters: <select> for status, <input> for title search; both bump page to 1 on change.

  • useQuery calls meetKevinApi.listVideos({ status, q, page, per_page: 20 }).

  • Renders 2-column grid (md:grid-cols-2) of video cards: thumbnail (left), title + date + status badge + failure_reason + ticker chips.

  • Each card links to /meet-kevin/videos/<id>.

  • Pagination Prev/Next buttons (model: NewsFeed.tsx).

  • Step 1: Implement.

  • Step 2: Build verification.

  • Step 3: Commit

git add dashboard/src/pages/meetKevin/Videos.tsx
git -c user.email=me@viktorbarzin.me commit -m "feat(dashboard): Meet Kevin videos feed page"

Task 14: Video detail page (/meet-kevin/videos/:id)

File: Create dashboard/src/pages/meetKevin/VideoDetail.tsx

Behavior:

  • useParams<{id: string}>()videoId: number.

  • useQuery for getVideo(videoId); second useQuery for getTranscript(videoId) enabled only when detail.transcript_available.

  • useMutation for reprocess(videoId, stage) — visible button when status is failed or discovered; invalidates the video query on success.

  • Layout: title + published date, embedded <YouTubeEmbed ref={playerRef} videoId={v.youtube_video_id} />, tab strip (Analysis / Transcript / Raw JSON).

    • Analysis tab: outlook card (banner + reasoning), macro themes chips, key risks bulleted list, summary card, per-ticker grid — each ticker row has <ActionChip>, <ConvictionBar>, time horizon, italicized rationale_quote, and a "▶ MM:SS" button calling playerRef.current?.seekTo(video_timestamp_seconds).
    • Transcript tab: scrollable list of segments; clicking a segment seeks the iframe to Math.floor(start).
    • Raw JSON tab: <pre>{JSON.stringify(detail.analysis, null, 2)}</pre>.
  • Disable Transcript tab if transcript_available is false.

  • Step 1: Implement.

  • Step 2: Build verification.

  • Step 3: Commit

git add dashboard/src/pages/meetKevin/VideoDetail.tsx
git -c user.email=me@viktorbarzin.me commit -m "feat(dashboard): Meet Kevin video detail page (tabs + iframe + deep-links)"

Task 15: Stocks list + drill-down

Files:

  • Create: dashboard/src/pages/meetKevin/Stocks.tsx
  • Create: dashboard/src/pages/meetKevin/StockDetail.tsx

Stocks.tsx:

  • useQuery calls meetKevinApi.listStocks with 60s refetch.
  • Renders a table: Symbol (link to /meet-kevin/stocks/<symbol>), Mentions, Last seen, Latest (ActionChip + percent), Avg conviction (ConvictionBar).
  • Empty state: "No analyses yet" card.

StockDetail.tsx:

  • useParams<{symbol: string}>() uppercased.

  • Two useQuerys: getStock(sym) and getStockTimeline(sym, "day").

  • Layout: large $SYMBOL header + back link, line chart of net_action_score over time (recharts), then chronological mention list. Each mention card links to its source /meet-kevin/videos/<id>, shows date, ActionChip, time horizon, ConvictionBar, percent, video title, and italicized rationale_quote.

  • Step 1: Implement both pages.

  • Step 2: Build verification.

  • Step 3: Commit

git add dashboard/src/pages/meetKevin/Stocks.tsx dashboard/src/pages/meetKevin/StockDetail.tsx
git -c user.email=me@viktorbarzin.me commit -m "feat(dashboard): Meet Kevin stocks list + per-ticker drill-down"

Task 16: Wire routes + sidebar

Files:

  • Modify: dashboard/src/App.tsx — import the 5 page components and add 5 routes inside the protected <Route path="/" element={...}> block, BEFORE the catch-all redirect:
<Route path="meet-kevin" element={<MeetKevinHome />} />
<Route path="meet-kevin/videos" element={<MeetKevinVideos />} />
<Route path="meet-kevin/videos/:id" element={<MeetKevinVideoDetail />} />
<Route path="meet-kevin/stocks" element={<MeetKevinStocks />} />
<Route path="meet-kevin/stocks/:symbol" element={<MeetKevinStockDetail />} />
  • Modify: dashboard/src/components/Layout.tsx — append to navItems:
{ to: '/meet-kevin', label: 'Meet Kevin', icon: 'M15 10l4.553-2.276A1 1 0 0121 8.618v6.764a1 1 0 01-1.447.894L15 14M5 18h8a2 2 0 002-2V8a2 2 0 00-2-2H5a2 2 0 00-2 2v8a2 2 0 002 2z' },

(Heroicon outline play-circle path — matches the video theme.)

  • Step 1: Apply both edits.

  • Step 2: Build verification + verify nav link renders.

  • Step 3: Commit

git add dashboard/src/App.tsx dashboard/src/components/Layout.tsx
git -c user.email=me@viktorbarzin.me commit -m "feat(dashboard): wire Meet Kevin routes + sidebar entry"

Task 17: Manual QA against docker-compose

Verification only — no commit (unless smoke notes are added).

  • Step 1: Boot infra: docker compose up -d postgres redis, then python -m alembic upgrade head.

  • Step 2: Start api-gateway with TRADING_DEV_MODE_AUTH_BYPASS=true python -m services.api_gateway.main &.

  • Step 3: Insert one fake row set via psql so the UI has data to render (one kevin_videos analyzed, one kevin_analyses, one kevin_stock_mentions for NVDA with conviction 0.82). Use the INSERT … RETURNING id \\gset pattern with psql.

  • Step 4: Start the dashboard: cd dashboard && VITE_DEV_MODE=true VITE_API_BASE_URL=http://localhost:8000/api npm run dev.

  • Step 5: Walkthrough at http://localhost:5173/meet-kevin:

    • Home renders latest video card + NVDA in top conviction list
    • /meet-kevin/videos shows the fake row
    • /meet-kevin/videos/<id> opens the iframe; the NVDA ticker card's "▶" deep-link calls seekTo on the player
    • /meet-kevin/stocks shows NVDA
    • /meet-kevin/stocks/NVDA shows the timeline + mention card
  • Step 6: Cleanup: docker compose down; kill the api-gateway process.


Phase 5 — Container & dependencies

Task 18: Add meet_kevin extras + Dockerfile

Files:

  • Modify: pyproject.toml — under [project.optional-dependencies] add:
meet_kevin = ["yt-dlp>=2026.04", "feedparser>=6.0", "anthropic>=0.40", "httpx>=0.27"]
  • Modify: docker/Dockerfile.service — update the uv pip install --system --no-cache-dir ".[api,news,sentiment,trading,backtester]" line to also include meet_kevin:
uv pip install --system --no-cache-dir ".[api,news,sentiment,trading,backtester,meet_kevin]" && \\
  • Step 1: Apply both edits.

  • Step 2: Sanity-check locally: pip install -e ".[meet_kevin]" (should resolve cleanly).

  • Step 3: Commit

git add pyproject.toml docker/Dockerfile.service
git -c user.email=me@viktorbarzin.me commit -m "feat: add meet_kevin extras (yt-dlp, feedparser, anthropic)"

Task 19: Push → CI build

  • Step 1: git push origin master

  • Step 2: Verify Woodpecker picked up the latest pipeline (response of GET /api/repos/viktor/trading/pipelines shows running or pending matching the head commit).

  • Step 3: Wait for pipeline transition to terminal state via an until loop polling Woodpecker. Do not pre-sleep — let the condition drive cadence (~30s checks). Expected: success.

  • Step 4: Confirm DockerHub got a fresh tag matching the head SHA short-form (hub.docker.com/v2/repositories/viktorbarzin/trading-bot-service/tags).


Phase 6 — K8s revival

Task 20: Vault secrets

  • Step 1: vault login -method=oidc

  • Step 2: Patch secret/trading-bot:

read -rs ANTHROPIC_KEY; echo
vault kv patch secret/trading-bot \
  anthropic_api_key="$ANTHROPIC_KEY" \
  meet_kevin_channel_id="UCUvvj5lwue7PspotMDjk5UA"
unset ANTHROPIC_KEY
  • Step 3: Verify with vault kv get -field=meet_kevin_channel_id secret/trading-bot.

Task 21: Uncomment + edit infra/stacks/trading-bot/main.tf

Workspace: this happens in /home/wizard/code/infra/, NOT the trading repo. Required action: claim presence for the infra stack before applying (per /home/wizard/code/CLAUDE.md "Agent Presence" rule):

~/code/scripts/presence claim stack:trading-bot --purpose "Phase 6 revival — uncomment stack + add meet-kevin-watcher container"
  • Step 1: Remove the outer /* … */ block-comment (lines ~2 and ~631).

  • Step 2: In the external_secret resource's template.data and data = […], add 2 keys: TRADING_ANTHROPIC_API_KEYanthropic_api_key, and TRADING_MEET_KEVIN_CHANNEL_IDmeet_kevin_channel_id. Both reference key = "trading-bot" (the existing Vault path).

  • Step 3: In locals.common_env, add:

TRADING_MEET_KEVIN_POLL_INTERVAL_SECONDS = "10800"
TRADING_MEET_KEVIN_DAILY_COST_CAP_USD    = "5"
TRADING_MEET_KEVIN_LLM_MODEL             = "claude-sonnet-4-6"
TRADING_MEET_KEVIN_PROMPT_VERSION        = "v1"
  • Step 4: Delete the three worker containers from the trading-bot-workers Pod spec: news-fetcher, sentiment-analyzer, trade-executor.

  • Step 5: Add the new meet-kevin-watcher container to the same Pod spec (after market-data):

container {
  name              = "meet-kevin-watcher"
  image             = "viktorbarzin/trading-bot-service:latest"
  image_pull_policy = "Always"
  command           = ["python", "-m", "services.meet_kevin_watcher.main"]
  dynamic "env" {
    for_each = local.common_env
    content {
      name  = env.key
      value = env.value
    }
  }
  env { name = "TRADING_OTEL_METRICS_PORT"; value = "9097" }
  env_from { secret_ref { name = "trading-bot-secrets" } }
  env_from { secret_ref { name = "trading-bot-db-creds" } }
  resources {
    requests = { cpu = "10m",  memory = "128Mi" }
    limits   = {                memory = "256Mi" }
  }
}
  • Step 6: Final container order is signal-generator [0], learning-engine [1], market-data [2], meet-kevin-watcher [3]. Update lifecycle.ignore_changes to only ignore image on container indices 03 plus dns_config (drop the index-4 and index-5 lines that were there for the 6-container layout).

  • Step 7: scripts/tg plan — verify the plan creates the namespace + secrets + Deployments + Service + Ingress, with the new container in the workers pod.

  • Step 8: scripts/tg apply --non-interactive (expect Apply complete!).

  • Step 9: Verify K8s:

kubectl get ns trading-bot
kubectl -n trading-bot get pods,deploy,svc,ingress
kubectl -n trading-bot get pod -l app=trading-bot-workers -o jsonpath='{.items[0].spec.containers[*].name}'
# expected: signal-generator learning-engine market-data meet-kevin-watcher
  • Step 10: Commit + push the infra change
git -C /home/wizard/code/infra add stacks/trading-bot/main.tf
git -C /home/wizard/code/infra -c user.email=me@viktorbarzin.me \
  commit -m "trading-bot: revive K8s stack + add meet-kevin-watcher; remove 3 disabled workers"
git -C /home/wizard/code/infra push
  • Step 11: ~/code/scripts/presence release stack:trading-bot.

Task 22: Production smoke test

  • Step 1: Tail watcher logs
kubectl -n trading-bot logs -l app=trading-bot-workers -c meet-kevin-watcher --tail=200 -f

Expected within ~60s: Starting meet-kevin-watcher, then RSS poll: N new videos (likely up to 15 from first-run backfill).

  • Step 2: Poll until at least one video reaches status='analyzed' (use an until [ $(SELECT count(*)) -ge 1 ] loop with sleep 30s — no pre-sleep).

  • Step 3: Hit the API: curl -sk https://trading.viktorbarzin.me/api/meet-kevin/health (with appropriate auth cookie) — expect counts_by_status.analyzed >= 1, daily_cost_usd > 0.

  • Step 4: Open https://trading.viktorbarzin.me/meet-kevin in a browser and verify the home + videos + stocks pages render with real Meet Kevin data.


Self-review (run inline before publishing)

  • Spec coverage: each design-doc section maps to one or more tasks:

    • Channel + seed → Task 3
    • Data model (5 tables) → Tasks 2, 3
    • Pipeline 3 stages → Tasks 5, 6, 7, 8
    • LLM output schema → Tasks 1, 7
    • Daily cost cap → Task 8 (logic) + Task 9 (/health surface)
    • 11 API endpoints → Task 9
    • 5 dashboard pages → Tasks 12, 13, 14, 15
    • Container changes → Task 18
    • Vault secrets → Task 20
    • K8s revival (Terraform edit) → Task 21
    • Observability metrics → Task 8 main loop (setup_telemetry + counters)
    • Failure modes → Tasks 5-8 (per-stage retry / status logic) + Task 9 (reprocess endpoint)
    • First-run backfill (~$1.50 burst) → Tasks 8 + 22 (covered by the daily cost cap)
  • Placeholder scan: no TBD / "implement later" / "similar to Task N". Where a section references "see spec", the spec section is named explicitly and the spec is committed at ab382af (immutable for this plan).

  • Type consistency: schema/class/enum/table names match across tasks (MeetKevinAnalysis, KevinVideo, kevin_stock_mentions, TickerAction.SELL, env prefix TRADING_MEET_KEVIN_*, metrics port 9097).


Execution

Two options:

  1. Subagent-Driven (recommended) — fresh subagent per task with two-stage review between tasks. Uses superpowers:subagent-driven-development.
  2. Inline execution — work through tasks in this session with periodic checkpoints. Uses superpowers:executing-plans.