add Meet Kevin revival implementation plan
22-task plan across 6 phases (data layer, watcher service stages,
API gateway, dashboard, container/deps, K8s revival). Each task
includes TDD checkboxes, file paths, acceptance criteria, and
commit commands. References the design doc (ab382af) for schemas,
prompts, and the Terraform diff.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
parent
ab382af3f5
commit
8f616e6487
1 changed files with 827 additions and 0 deletions
827
docs/plans/2026-05-21-meet-kevin-revival-plan.md
Normal file
827
docs/plans/2026-05-21-meet-kevin-revival-plan.md
Normal file
|
|
@ -0,0 +1,827 @@
|
|||
# Meet Kevin Revival Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Ship the v1 Meet Kevin pipeline (RSS poll → caption extraction → Claude Sonnet 4.6 structured analysis → Postgres → ticker-centric UI) inside the existing `viktor/trading` repo, and revive the K8s stack with the 3 expensive services disabled and the new watcher container added.
|
||||
|
||||
**Architecture:** Single long-running async service (`services/meet_kevin_watcher/`) walks each video through 3 sequential stages and writes directly to Postgres. No Redis streams in v1. The existing `api_gateway` gains 11 new routes; the dashboard gains 5 new pages under `/meet-kevin/*`. The K8s revival is a single `infra/stacks/trading-bot/main.tf` edit (uncomment block + remove 3 disabled containers + add 1 new container + extend ExternalSecret + bump common env).
|
||||
|
||||
**Tech Stack:** Python 3.12, FastAPI, SQLAlchemy 2.0 (async), Pydantic v2, Alembic, `feedparser`, `yt-dlp>=2026.04` (PyPI not apt), `anthropic>=0.40` SDK with tool-use forcing for structured output + prompt caching, React 19 + TanStack Query + recharts + Tailwind 4, Terraform/Terragrunt, Vault KV + ExternalSecret.
|
||||
|
||||
**Reference**: spec at [`2026-05-21-meet-kevin-revival-design.md`](2026-05-21-meet-kevin-revival-design.md) (commit `ab382af`). The spec is the source of truth for schemas, prompts, endpoint shapes, and the Terraform diff — this plan sequences the implementation and pins down acceptance criteria per step.
|
||||
|
||||
**Workspace:** all backend + dashboard work happens inside `/home/wizard/code/trading-bot/` (the cloned trading repo). The K8s revival in Phase 6 happens inside `/home/wizard/code/infra/`. Commits use `me@viktorbarzin.me` as the author email (personal-repo convention).
|
||||
|
||||
**Pattern crib-sheet** (read once before starting):
|
||||
- Service main loop pattern: `services/news_fetcher/main.py` — `asyncio.Event` shutdown, signal handlers, `setup_telemetry`, graceful close
|
||||
- Config pattern: `services/news_fetcher/config.py` — extends `shared.config.BaseConfig`, `TRADING_` env prefix
|
||||
- Model pattern: `shared/models/news.py` — SQLAlchemy 2.0 `Mapped[]`/`mapped_column()` style, `TimestampMixin` from `base.py`
|
||||
- Schema pattern: `shared/schemas/news.py` — Pydantic v2 `BaseModel`, `Field(ge=…, le=…)`
|
||||
- Migration pattern: `alembic/versions/b2c3d4e5f6a7_add_fundamentals_table.py` — `op.create_table` with explicit columns, no autogen
|
||||
- API route pattern: `services/api_gateway/routes/news.py` — `APIRouter(prefix="/api/...")`, `_user=Depends(get_current_user)`, `request.app.state.db_session_factory`
|
||||
- Test pattern: `tests/services/test_news_fetcher.py` — pytest-asyncio auto mode, `MagicMock`/`AsyncMock`, committed fixture files
|
||||
- Dashboard page pattern: `dashboard/src/pages/NewsFeed.tsx` — TanStack Query, axios via `api/client.ts`, Tailwind on `bg-slate-800` cards
|
||||
- Sidebar nav pattern: `dashboard/src/components/Layout.tsx`'s `navItems` array
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Data layer
|
||||
|
||||
### Task 1: Pydantic schemas for the Meet Kevin pipeline
|
||||
|
||||
**Files:**
|
||||
- Create: `shared/schemas/meet_kevin.py`
|
||||
- Modify: `tests/test_schemas.py` (append cases under a `# --- Meet Kevin schemas ---` section)
|
||||
|
||||
**Required exports** (match these names exactly — they're consumed by later tasks):
|
||||
|
||||
- Enums: `TickerAction` (buy/sell/hold/watch/avoid), `TimeHorizon` (intraday/days/weeks/months/long_term/unspecified), `MarketOutlook` (bullish/neutral/bearish/mixed), `VideoStatus` (discovered/captioned/analyzed/failed/skipped), `TranscriptSource` (captions_manual/captions_auto/none)
|
||||
- LLM tool-input: `MeetKevinTickerMention` (symbol uppercased via `@field_validator`, conviction `Field(ge=0.0, le=1.0)`), `MeetKevinAnalysis` (market_outlook_direction, market_outlook_reasoning, macro_themes:list[str], key_risks:list[str], summary, tickers:list[MeetKevinTickerMention])
|
||||
- API response shapes: `TranscriptSegment`, `VideoSummary`, `VideoDetail`, `StockSummary`, `StockMention`, `StockTimeline`, `TimelineBucket`, `PipelineHealth`
|
||||
|
||||
Full field-level schemas in the spec under "Data model" and "LLM output schema".
|
||||
|
||||
- [ ] **Step 1: Write failing tests** in `tests/test_schemas.py` covering: ticker mention validates, symbol auto-uppercases, conviction out of `[0,1]` raises `pydantic.ValidationError`, full `MeetKevinAnalysis` validates with one ticker entry.
|
||||
|
||||
- [ ] **Step 2: Run tests, verify they fail** with `ModuleNotFoundError`:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_schemas.py -v -k "meet_kevin"
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Implement `shared/schemas/meet_kevin.py`** per the spec. Use Pydantic v2 (`field_validator`, `model_config`).
|
||||
|
||||
- [ ] **Step 4: Re-run tests, verify pass.**
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add shared/schemas/meet_kevin.py tests/test_schemas.py
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat: add Meet Kevin pydantic schemas (analysis + API shapes)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: SQLAlchemy models
|
||||
|
||||
**Files:**
|
||||
- Create: `shared/models/meet_kevin.py`
|
||||
- Modify: `shared/models/__init__.py` (add 5 imports + `__all__` entries)
|
||||
- Modify: `tests/test_models.py` (append)
|
||||
|
||||
**Tables** (all use `BigInteger` PKs, `TimestampMixin`, `JSONB` for json columns, `SAEnum` with named types per the spec):
|
||||
- `KevinChannel` (table `kevin_channels`) — youtube_channel_id unique, poll_enabled bool, poll_interval_seconds int default 10800, daily_cost_cap_usd Numeric(8,2) default 5.00, last_polled_at
|
||||
- `KevinVideo` (table `kevin_videos`) — channel_id FK, youtube_video_id unique+indexed, title, description, published_at indexed, duration_seconds, thumbnail_url, status enum + index, failure_reason, retry_count default 0, processed_at
|
||||
- `KevinTranscript` (table `kevin_transcripts`) — video_id unique FK, source enum, language, raw_text, segments_json JSONB, word_count
|
||||
- `KevinAnalysis` (table `kevin_analyses`) — video_id FK indexed, model, prompt_version, market_outlook_direction enum, market_outlook_reasoning, macro_themes_json + key_risks_json JSONB, summary, raw_response_json JSONB, prompt_tokens/completion_tokens int, cost_usd Numeric(10,4)
|
||||
- `KevinStockMention` (table `kevin_stock_mentions`) — video_id + analysis_id FKs, symbol indexed, action enum, conviction Numeric(4,3), time_horizon enum, rationale_quote, video_timestamp_seconds nullable
|
||||
|
||||
Each table establishes `Mapped[]`/`mapped_column()` relationships so the ORM nav matches the spec (videos → transcript, analyses, mentions).
|
||||
|
||||
- [ ] **Step 1**: Add `test_meet_kevin_models_importable` to `tests/test_models.py` asserting `KevinChannel.__tablename__ == "kevin_channels"` (etc.) and that `KevinVideo.__table__.c.status` is an Enum containing "discovered".
|
||||
|
||||
- [ ] **Step 2**: Verify failure.
|
||||
|
||||
- [ ] **Step 3**: Implement `shared/models/meet_kevin.py`.
|
||||
|
||||
- [ ] **Step 4**: Update `shared/models/__init__.py` — import the 5 classes and add them to `__all__`.
|
||||
|
||||
- [ ] **Step 5**: Re-run, verify pass.
|
||||
|
||||
- [ ] **Step 6**: Commit
|
||||
|
||||
```bash
|
||||
git add shared/models/meet_kevin.py shared/models/__init__.py tests/test_models.py
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat: add Meet Kevin SQLAlchemy models (5 tables)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Alembic migration
|
||||
|
||||
**Files:**
|
||||
- Create: `alembic/versions/c3d4e5f6a7b8_add_meet_kevin_tables.py`
|
||||
|
||||
**Requirements:**
|
||||
- `revision = "c3d4e5f6a7b8"`, `down_revision = "b2c3d4e5f6a7"` (current head from `python -m alembic heads`)
|
||||
- Create 5 named ENUM types: `kevin_video_status`, `kevin_transcript_source`, `kevin_market_outlook`, `kevin_ticker_action`, `kevin_time_horizon`
|
||||
- `op.create_table` for all 5 tables matching the SQLAlchemy models exactly (column types, defaults, indexes)
|
||||
- Composite index `ix_kevin_stock_mentions_symbol_created_at` on `(symbol, created_at DESC)` for ticker-timeline queries
|
||||
- Partial index `ix_kevin_videos_status_pending` on `status` `WHERE status IN ('discovered', 'captioned')` for the pipeline scan
|
||||
- Seed the one channel: `INSERT INTO kevin_channels (youtube_channel_id, title) VALUES ('UCUvvj5lwue7PspotMDjk5UA', 'Meet Kevin')`
|
||||
- `downgrade()` reverses everything including the 5 enum types
|
||||
|
||||
Pattern model: `alembic/versions/b2c3d4e5f6a7_add_fundamentals_table.py`.
|
||||
|
||||
- [ ] **Step 1**: Confirm current head
|
||||
|
||||
```bash
|
||||
python -m alembic heads
|
||||
# expected: b2c3d4e5f6a7 (head)
|
||||
```
|
||||
|
||||
- [ ] **Step 2**: Write the migration file.
|
||||
|
||||
- [ ] **Step 3**: Apply against the docker-compose Postgres
|
||||
|
||||
```bash
|
||||
docker compose up -d postgres
|
||||
TRADING_DATABASE_URL=postgresql+asyncpg://trading:trading@localhost:5432/trading \
|
||||
python -m alembic upgrade head
|
||||
# expected: "Running upgrade b2c3d4e5f6a7 -> c3d4e5f6a7b8"
|
||||
```
|
||||
|
||||
- [ ] **Step 4**: Verify 5 tables exist:
|
||||
|
||||
```bash
|
||||
docker compose exec -T postgres psql -U trading -d trading -c "\dt kevin_*"
|
||||
# expected: 5 rows
|
||||
```
|
||||
|
||||
- [ ] **Step 5**: Round-trip check (`downgrade -1` shows 0 rows; `upgrade head` restores).
|
||||
|
||||
- [ ] **Step 6**: Commit
|
||||
|
||||
```bash
|
||||
git add alembic/versions/c3d4e5f6a7b8_add_meet_kevin_tables.py
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat: add Alembic migration for Meet Kevin tables"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Watcher service stages
|
||||
|
||||
### Task 4: Service scaffold + config
|
||||
|
||||
**Files:**
|
||||
- Create: `services/meet_kevin_watcher/__init__.py` (empty)
|
||||
- Create: `services/meet_kevin_watcher/config.py`
|
||||
|
||||
**`MeetKevinWatcherConfig`** extends `BaseConfig`, exposing (defaults shown):
|
||||
- `meet_kevin_channel_id: str = "UCUvvj5lwue7PspotMDjk5UA"`
|
||||
- `meet_kevin_poll_interval_seconds: int = 10800`
|
||||
- `meet_kevin_max_caption_retries: int = 5`
|
||||
- `meet_kevin_max_llm_retries: int = 3`
|
||||
- `meet_kevin_llm_model: str = "claude-sonnet-4-6"`
|
||||
- `meet_kevin_prompt_version: str = "v1"`
|
||||
- `meet_kevin_daily_cost_cap_usd: float = 5.0`
|
||||
- `anthropic_api_key: str = ""`
|
||||
- `meet_kevin_workdir: str = "/tmp/meet_kevin_captions"`
|
||||
- `otel_service_name: str = "meet-kevin-watcher"`
|
||||
- `otel_metrics_port: int = 9097` (next free port after market_data's 9096)
|
||||
|
||||
- [ ] **Step 1**: Create the dir, empty `__init__.py`, and the config file.
|
||||
|
||||
- [ ] **Step 2**: Verify importable: `python -c "from services.meet_kevin_watcher.config import MeetKevinWatcherConfig; print(MeetKevinWatcherConfig().otel_metrics_port)"` → `9097`.
|
||||
|
||||
- [ ] **Step 3**: Commit
|
||||
|
||||
```bash
|
||||
git add services/meet_kevin_watcher/__init__.py services/meet_kevin_watcher/config.py
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat: scaffold meet_kevin_watcher service + config"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: RSS poller
|
||||
|
||||
**Files:**
|
||||
- Create: `services/meet_kevin_watcher/rss_poller.py`
|
||||
- Create: `tests/services/meet_kevin_watcher/__init__.py` (empty)
|
||||
- Create: `tests/services/meet_kevin_watcher/test_rss_poller.py`
|
||||
- Create: `tests/fixtures/meet_kevin_rss.xml` (real fixture)
|
||||
|
||||
**Exports:**
|
||||
- `@dataclass(frozen=True) DiscoveredVideo(youtube_video_id, title, description, published_at, thumbnail_url)`
|
||||
- `parse_feed(xml_bytes: bytes) -> list[DiscoveredVideo]` — uses stdlib `xml.etree.ElementTree`, returns `[]` on parse error/empty
|
||||
- `async fetch_feed(channel_id: str, client: httpx.AsyncClient) -> bytes` — GET with 15s timeout, returns `b""` on HTTP error
|
||||
- Atom namespaces: `{"a": "http://www.w3.org/2005/Atom", "y": "http://www.youtube.com/xml/schemas/2015", "m": "http://search.yahoo.com/mrss/"}`
|
||||
- Feed URL template: `https://www.youtube.com/feeds/videos.xml?channel_id={channel_id}`
|
||||
|
||||
- [ ] **Step 1: Capture real fixture**
|
||||
|
||||
```bash
|
||||
mkdir -p tests/fixtures
|
||||
curl -sL "https://www.youtube.com/feeds/videos.xml?channel_id=UCUvvj5lwue7PspotMDjk5UA" \
|
||||
-o tests/fixtures/meet_kevin_rss.xml
|
||||
test -s tests/fixtures/meet_kevin_rss.xml || { echo "FAIL: empty fixture"; exit 1; }
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write failing tests** asserting:
|
||||
- `parse_feed(fixture.read_bytes())` returns ≥1 `DiscoveredVideo` with 11-char video_id, title, published_at, thumbnail starting with https
|
||||
- `parse_feed(b"")` and `parse_feed(b"<?xml version='1.0'?><not-a-feed/>")` both return `[]`
|
||||
|
||||
- [ ] **Step 3**: Verify failure (ModuleNotFoundError).
|
||||
|
||||
- [ ] **Step 4**: Implement `rss_poller.py`.
|
||||
|
||||
- [ ] **Step 5**: Verify pass.
|
||||
|
||||
- [ ] **Step 6**: Commit
|
||||
|
||||
```bash
|
||||
git add services/meet_kevin_watcher/rss_poller.py \
|
||||
tests/services/meet_kevin_watcher/__init__.py \
|
||||
tests/services/meet_kevin_watcher/test_rss_poller.py \
|
||||
tests/fixtures/meet_kevin_rss.xml
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat(meet-kevin): RSS poller for YouTube uploads"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Caption extractor
|
||||
|
||||
**Files:**
|
||||
- Create: `services/meet_kevin_watcher/caption_extractor.py`
|
||||
- Create: `tests/services/meet_kevin_watcher/test_caption_extractor.py`
|
||||
- Create: `tests/fixtures/sample.srt` (3-segment SRT mentioning NVDA)
|
||||
|
||||
**Exports:**
|
||||
- `@dataclass(frozen=True) CaptionResult(source, language, raw_text, segments, word_count)`
|
||||
- `parse_srt(text: str) -> list[dict]` — segments with `{start: float, end: float, text: str}`. Handles SRT and yt-dlp's webvtt-converted variants; `[]` on empty/malformed.
|
||||
- `async extract_captions(video_id: str, workdir: str, sub_lang_glob: str = "en.*") -> CaptionResult | None` — calls yt-dlp subprocess via the internal `_run_yt_dlp` helper, then parses any `<video_id>*.srt` left in workdir. Prefers manual subs (no `.auto.` in filename) over auto-generated. Cleans up SRT files after parsing. Returns `None` if no SRT was produced.
|
||||
- Internal `async _run_yt_dlp(cmd: list[str], cwd: str) -> int` — wraps `asyncio.create_subprocess_exec` for testability (mocked in tests).
|
||||
|
||||
**yt-dlp invocation:**
|
||||
```
|
||||
yt-dlp --write-auto-sub --write-sub --sub-lang en.* --skip-download --convert-subs srt \
|
||||
-o "<video_id>.%(ext)s" "https://www.youtube.com/watch?v=<video_id>"
|
||||
```
|
||||
|
||||
- [ ] **Step 1**: Create `tests/fixtures/sample.srt` with 3 segments (one mentioning NVDA).
|
||||
|
||||
- [ ] **Step 2**: Write tests:
|
||||
- `parse_srt` extracts 3 segments with correct start/end/text
|
||||
- `parse_srt("")` returns `[]`
|
||||
- `extract_captions` returns a `CaptionResult` when yt-dlp writes an SRT (mock `_run_yt_dlp` to return 0, pre-place SRT in tmp_path)
|
||||
- `extract_captions` returns `None` when no SRT lands
|
||||
|
||||
- [ ] **Step 3**: Verify failure.
|
||||
|
||||
- [ ] **Step 4**: Implement the module.
|
||||
|
||||
- [ ] **Step 5**: Verify pass.
|
||||
|
||||
- [ ] **Step 6**: Commit
|
||||
|
||||
```bash
|
||||
git add services/meet_kevin_watcher/caption_extractor.py \
|
||||
tests/services/meet_kevin_watcher/test_caption_extractor.py \
|
||||
tests/fixtures/sample.srt
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat(meet-kevin): caption extractor via yt-dlp"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: LLM analyzer (Claude Sonnet 4.6 with tool-use forcing)
|
||||
|
||||
**Files:**
|
||||
- Create: `services/meet_kevin_watcher/llm_analyzer.py`
|
||||
- Create: `tests/services/meet_kevin_watcher/test_llm_analyzer.py`
|
||||
|
||||
**Exports:**
|
||||
- `@dataclass(frozen=True) LlmCallResult(analysis: MeetKevinAnalysis, raw_response: dict, prompt_tokens, completion_tokens, cost_usd: Decimal)`
|
||||
- `SYSTEM_PROMPT` (module constant) — analyst-style system instructions per the spec's "LLM output schema" section; MUST mention `submit_analysis` (the tool name) and the structured contract
|
||||
- `compute_cost_usd(model: str, input_tokens: int, output_tokens: int) -> Decimal` — uses `_PRICING` dict keyed by model; pinned prices in code: `"claude-sonnet-4-6": (Decimal("3"), Decimal("15"))` ($/M input, $/M output); unknown model → `Decimal("0")` + warning log
|
||||
- `class LlmAnalyzer(client: AsyncAnthropic, model: str, prompt_version: str)` — single method `async analyze(title, description, published_at, transcript_text, transcript_segments) -> LlmCallResult`
|
||||
|
||||
**Anthropic call shape:**
|
||||
- `client.messages.create(model=..., max_tokens=4096, system=[{type: "text", text: SYSTEM_PROMPT, cache_control: {type: "ephemeral"}}], tools=[ANALYSIS_TOOL], tool_choice={type: "tool", name: "submit_analysis"}, messages=[{role: "user", content: <metadata + transcript>}])`
|
||||
- `ANALYSIS_TOOL` defines a JSON schema mirroring `MeetKevinAnalysis` (see spec)
|
||||
- Response: pick the first `tool_use` content block, validate `block.input` via `MeetKevinAnalysis.model_validate(...)`. Raise `ValueError` if no `tool_use` block (caller marks failed).
|
||||
|
||||
- [ ] **Step 1**: Write tests using `MagicMock`/`AsyncMock` for `anthropic.AsyncAnthropic`:
|
||||
- Happy path: tool_use response with valid input → returns `LlmCallResult` with parsed `MeetKevinAnalysis`, correct token counts, `cost_usd > 0`
|
||||
- No tool_use block in response → `ValueError("tool_use")`
|
||||
- `compute_cost_usd("claude-sonnet-4-6", 1_000_000, 1_000_000) == Decimal("18.0000")`
|
||||
- `compute_cost_usd("unknown-model", 1000, 1000) == Decimal("0")`
|
||||
- `SYSTEM_PROMPT` contains `"submit_analysis"` and `"ticker"`
|
||||
|
||||
- [ ] **Step 2**: Verify failure.
|
||||
|
||||
- [ ] **Step 3**: Implement the module.
|
||||
|
||||
- [ ] **Step 4**: Verify pass.
|
||||
|
||||
- [ ] **Step 5**: Commit
|
||||
|
||||
```bash
|
||||
git add services/meet_kevin_watcher/llm_analyzer.py \
|
||||
tests/services/meet_kevin_watcher/test_llm_analyzer.py
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat(meet-kevin): Claude Sonnet 4.6 LLM analyzer (tool-use forcing + prompt cache)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Pipeline orchestrator + service main loop
|
||||
|
||||
**Files:**
|
||||
- Create: `services/meet_kevin_watcher/pipeline.py`
|
||||
- Create: `services/meet_kevin_watcher/main.py`
|
||||
- Create: `tests/services/meet_kevin_watcher/test_pipeline.py`
|
||||
|
||||
**Exports from `pipeline.py`:**
|
||||
- `@dataclass(frozen=True) PipelineDeps(extract_captions, analyze, daily_cost_used, model, prompt_version, daily_cost_cap_usd: Decimal, workdir)` — dependency injection container so the pipeline is unit-testable
|
||||
- `async process_one_video(video: KevinVideo, session: AsyncSession, deps: PipelineDeps) -> str` — runs the next pipeline stage and returns the new status. Status transitions:
|
||||
- `discovered` → `extract_captions()`. None → status=failed, failure_reason=`"no_captions"`. Otherwise insert `KevinTranscript`, advance to `captioned`.
|
||||
- `captioned` → check `daily_cost_used() < daily_cost_cap_usd`. If over cap, leave as `captioned` (skip). Otherwise call `analyze()`, on success insert `KevinAnalysis` + N `KevinStockMention` rows, advance to `analyzed` and set `processed_at`. On exception, increment `retry_count`; mark `failed` after 3 retries.
|
||||
- `async daily_cost_used(session) -> Decimal` — `SUM(cost_usd)` from `KevinAnalysis` where `created_at >= date_trunc('day', now())`
|
||||
|
||||
**`main.py` orchestrator** (model: `services/news_fetcher/main.py`):
|
||||
- `async run()` entry point: load config, set up logging + OTEL + counters (`meet_kevin.videos_discovered`, `meet_kevin.captions_extracted`, `meet_kevin.llm_calls`, `meet_kevin.llm_cost_usd`), create DB engine + session factory, instantiate `AsyncAnthropic` + `LlmAnalyzer`, build `PipelineDeps`, install SIGTERM/SIGINT signal handlers on `shutdown_event`.
|
||||
- Main loop: poll RSS for each `poll_enabled` channel (use `INSERT … ON CONFLICT (youtube_video_id) DO NOTHING RETURNING id` to dedupe), then walk all videos with status in `("discovered", "captioned")` ordered by `published_at ASC`, call `process_one_video` for each, commit per-video.
|
||||
- After each iteration, `await asyncio.wait_for(shutdown_event.wait(), timeout=poll_interval_seconds)` so shutdown is responsive.
|
||||
- On exit: `await anthropic.close()`, `await engine.dispose()`.
|
||||
- `if __name__ == "__main__": asyncio.run(run())`
|
||||
|
||||
- [ ] **Step 1**: Write 3 pipeline tests using `MagicMock` for `KevinVideo` + `AsyncMock` for session:
|
||||
- Happy path: `discovered` → `captioned` → `analyzed`, both deps awaited once
|
||||
- No captions: returns `"failed"`, sets `failure_reason="no_captions"`, `analyze` NOT awaited
|
||||
- Cost cap exceeded: status stays `"captioned"`, `analyze` NOT awaited
|
||||
|
||||
- [ ] **Step 2**: Verify failure.
|
||||
|
||||
- [ ] **Step 3**: Implement `pipeline.py` then `main.py`.
|
||||
|
||||
- [ ] **Step 4**: Verify all watcher tests pass
|
||||
|
||||
```bash
|
||||
python -m pytest tests/services/meet_kevin_watcher/ -v
|
||||
# expected: ~14 passing (2 RSS + 4 caption + 5 LLM + 3 pipeline)
|
||||
```
|
||||
|
||||
- [ ] **Step 5**: Commit
|
||||
|
||||
```bash
|
||||
git add services/meet_kevin_watcher/pipeline.py \
|
||||
services/meet_kevin_watcher/main.py \
|
||||
tests/services/meet_kevin_watcher/test_pipeline.py
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat(meet-kevin): pipeline orchestrator + service main loop"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — API gateway
|
||||
|
||||
### Task 9: `/api/meet-kevin/*` routes
|
||||
|
||||
**Files:**
|
||||
- Create: `services/api_gateway/routes/meet_kevin.py`
|
||||
- Modify: `services/api_gateway/main.py` — import + `app.include_router(meet_kevin_router)` next to the other route registrations
|
||||
- Create: `tests/api_gateway/routes/__init__.py` (empty, if missing)
|
||||
- Create: `tests/api_gateway/routes/test_meet_kevin.py`
|
||||
|
||||
**Router pattern** (model: `services/api_gateway/routes/news.py`):
|
||||
- `router = APIRouter(prefix="/api/meet-kevin", tags=["meet-kevin"])`
|
||||
- All endpoints behind `_user=Depends(get_current_user)`
|
||||
- DB access via `request.app.state.db_session_factory`
|
||||
|
||||
**Endpoints to implement** (full request/response shapes in the spec's "API surface" section):
|
||||
|
||||
1. `GET /health` — counts by status from `kevin_videos`, last poll timestamp from `kevin_channels`, daily cost from `SUM(kevin_analyses.cost_usd) WHERE created_at >= date_trunc('day', now())`, cap from channel row
|
||||
2. `GET /channels` — list all `KevinChannel` rows
|
||||
3. `PATCH /channels/{id}` — accept body fields `poll_enabled`, `poll_interval_seconds`, `daily_cost_cap_usd`
|
||||
4. `GET /videos?status=&q=&page=&per_page=` — paginated, ordered by `published_at DESC`, with `top_tickers` populated by joining latest `kevin_stock_mentions` per video (ordered by conviction DESC), and `outlook`/`one_line_summary` from the latest `kevin_analyses` row per video
|
||||
5. `GET /videos/{id}` — single video + analysis (latest by created_at) + first 5 ticker mentions ordered by conviction DESC + `transcript_available: bool`
|
||||
6. `GET /videos/{id}/transcript` — return segments_json + source + language; 404 if missing
|
||||
7. `POST /videos/{id}/reprocess?stage=captions|analysis|auto` — reset status: failed→discovered for captions/auto; captioned|failed|analyzed→captioned for analysis; reset `retry_count` and `failure_reason`
|
||||
8. `GET /stocks` — distinct symbols with `mention_count`, `last_seen_at`, `latest_action` (most recent mention), `latest_conviction`, `avg_conviction`
|
||||
9. `GET /stocks/{symbol}` — all mentions for the uppercased symbol joined with their videos, ordered by `published_at DESC`; 404 if none
|
||||
10. `GET /stocks/{symbol}/timeline?bucket=day|week` — `date_trunc(bucket, created_at)` aggregation: `avg_conviction`, `mention_count`, `net_action_score` (sum of conviction for buy minus sum of conviction for sell, others = 0)
|
||||
11. `GET /dashboard` — latest analyzed video + its analysis + top 5 mentions, plus top conviction last 7 days (`SELECT symbol, max(conviction), count() ... GROUP BY symbol LIMIT 10`), plus 14-day outlook trend (`date_trunc('day', kevin_analyses.created_at)` × `market_outlook_direction` grouping)
|
||||
|
||||
- [ ] **Step 1**: Write 3 API tests using FastAPI `TestClient`:
|
||||
- `GET /api/meet-kevin/health` → 200 with `counts_by_status`, `daily_cost_usd`, `daily_cost_cap_usd` keys
|
||||
- `GET /api/meet-kevin/videos` (empty DB) → `{"videos": [], "total": 0, "page": 1, "per_page": 20}`
|
||||
- `GET /api/meet-kevin/stocks` (empty DB) → `{"stocks": []}`
|
||||
|
||||
Use `monkeypatch.setenv("TRADING_DEV_MODE_AUTH_BYPASS", "true")` if the repo has that flag, otherwise stub `get_current_user` via dependency override.
|
||||
|
||||
- [ ] **Step 2**: Verify failure (404 — router not registered yet).
|
||||
|
||||
- [ ] **Step 3**: Implement the route module.
|
||||
|
||||
- [ ] **Step 4**: Register the router in `main.py`.
|
||||
|
||||
- [ ] **Step 5**: Verify pass.
|
||||
|
||||
- [ ] **Step 6**: Commit
|
||||
|
||||
```bash
|
||||
git add services/api_gateway/routes/meet_kevin.py \
|
||||
services/api_gateway/main.py \
|
||||
tests/api_gateway/routes/test_meet_kevin.py \
|
||||
tests/api_gateway/routes/__init__.py 2>/dev/null
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat(api): /api/meet-kevin/* routes (11 endpoints)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — Dashboard
|
||||
|
||||
> The repo has no FE unit test suite. For each FE task, verification is `npm run build` (zero TS/ESLint errors) followed by manual browser checks once the backend is up (Task 17).
|
||||
|
||||
### Task 10: TypeScript types + API client
|
||||
|
||||
**Files:**
|
||||
- Create: `dashboard/src/types/meetKevin.ts`
|
||||
- Create: `dashboard/src/api/meetKevin.ts`
|
||||
|
||||
**Types** (one-to-one with the spec's API response shapes): `TickerAction`, `TimeHorizon`, `MarketOutlook`, `VideoStatus`, `VideoSummary`, `TickerMention`, `VideoAnalysis`, `VideoDetail`, `TranscriptSegment`, `Transcript`, `StockSummary`, `StockMention`, `PipelineHealth`, `DashboardData`.
|
||||
|
||||
**API client** (`meetKevinApi`) — thin wrappers on the shared `axios client` from `dashboard/src/api/client.ts`. Methods: `health`, `dashboard`, `listVideos(params)`, `getVideo(id)`, `getTranscript(id)`, `reprocess(id, stage)`, `listStocks`, `getStock(symbol)`, `getStockTimeline(symbol, bucket)`. Each method calls `client.get/post` with the path under `/meet-kevin/...` (the shared client already has `/api` as its base).
|
||||
|
||||
- [ ] **Step 1**: Create the two files.
|
||||
|
||||
- [ ] **Step 2**: Verify TS compiles
|
||||
|
||||
```bash
|
||||
cd dashboard && npm run build && cd ..
|
||||
```
|
||||
|
||||
- [ ] **Step 3**: Commit
|
||||
|
||||
```bash
|
||||
git add dashboard/src/types/meetKevin.ts dashboard/src/api/meetKevin.ts
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat(dashboard): Meet Kevin TypeScript types + API client"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 11: Reusable Meet Kevin components
|
||||
|
||||
**Files:**
|
||||
- Create: `dashboard/src/components/meetKevin/ActionChip.tsx`
|
||||
- Create: `dashboard/src/components/meetKevin/ConvictionBar.tsx`
|
||||
- Create: `dashboard/src/components/meetKevin/YouTubeEmbed.tsx`
|
||||
- Create: `dashboard/src/components/meetKevin/index.ts` (barrel export)
|
||||
|
||||
**`ActionChip`** — accepts `action: TickerAction` prop. Color-coded chip per action: buy=green, sell=red, hold=slate, watch=yellow, avoid=rose. Tailwind classes only, no new CSS.
|
||||
|
||||
**`ConvictionBar`** — accepts `value: number` (0-1). Renders a clipped div with a blue fill bar (`bg-blue-400` over `bg-slate-700`), width = `pct%`.
|
||||
|
||||
**`YouTubeEmbed`** — `forwardRef<YouTubeEmbedHandle, {videoId, startSeconds?}>`. Renders an `<iframe>` pointed at `https://www.youtube-nocookie.com/embed/<id>?start=<n>&enablejsapi=1`. `useImperativeHandle` exposes `seekTo(seconds)` which `postMessage`s `{event: "command", func: "seekTo", args: [seconds, true]}` to the iframe's contentWindow. The component uses the standard 16:9 aspect ratio via `pb-[56.25%]`.
|
||||
|
||||
- [ ] **Step 1**: Implement all 4 files.
|
||||
|
||||
- [ ] **Step 2**: Verify build
|
||||
|
||||
```bash
|
||||
cd dashboard && npm run build && cd ..
|
||||
```
|
||||
|
||||
- [ ] **Step 3**: Commit
|
||||
|
||||
```bash
|
||||
git add dashboard/src/components/meetKevin/
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat(dashboard): reusable Meet Kevin components (ActionChip, ConvictionBar, YouTubeEmbed)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 12: Home page (`/meet-kevin`)
|
||||
|
||||
**File:** Create `dashboard/src/pages/meetKevin/Home.tsx`
|
||||
|
||||
**Behavior:**
|
||||
- Uses `useQuery` keyed `['meet-kevin', 'dashboard']` calling `meetKevinApi.dashboard`, refetch every 60s.
|
||||
- Uses `useQuery` keyed `['meet-kevin', 'health']` calling `meetKevinApi.health`, refetch every 60s.
|
||||
- Empty state (no `latest_video`): "Waiting for first poll" card with `last_poll_at` formatted.
|
||||
- Otherwise: latest video hero card (thumbnail + outlook badge + title + ticker chips + one-line summary, linking to `/meet-kevin/videos/<id>`), top conviction list (last 7 days, with `ConvictionBar` per row, each row linking to `/meet-kevin/stocks/<symbol>`), and an outlook trendline `recharts.LineChart` over the 14-day data (map outlook `bullish/neutral/mixed/bearish` to `+1/0/0/-1` for the numeric Y-axis).
|
||||
- Tailwind aesthetic: `bg-slate-800 border border-slate-700 rounded-xl p-5` cards (matches existing dashboard).
|
||||
|
||||
- [ ] **Step 1**: Implement the page.
|
||||
|
||||
- [ ] **Step 2**: Build verification.
|
||||
|
||||
- [ ] **Step 3**: Commit
|
||||
|
||||
```bash
|
||||
git add dashboard/src/pages/meetKevin/Home.tsx
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat(dashboard): Meet Kevin home page"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 13: Videos feed page (`/meet-kevin/videos`)
|
||||
|
||||
**File:** Create `dashboard/src/pages/meetKevin/Videos.tsx`
|
||||
|
||||
**Behavior:**
|
||||
- Filters: `<select>` for status, `<input>` for title search; both bump `page` to 1 on change.
|
||||
- `useQuery` calls `meetKevinApi.listVideos({ status, q, page, per_page: 20 })`.
|
||||
- Renders 2-column grid (`md:grid-cols-2`) of video cards: thumbnail (left), title + date + status badge + failure_reason + ticker chips.
|
||||
- Each card links to `/meet-kevin/videos/<id>`.
|
||||
- Pagination Prev/Next buttons (model: `NewsFeed.tsx`).
|
||||
|
||||
- [ ] **Step 1**: Implement.
|
||||
|
||||
- [ ] **Step 2**: Build verification.
|
||||
|
||||
- [ ] **Step 3**: Commit
|
||||
|
||||
```bash
|
||||
git add dashboard/src/pages/meetKevin/Videos.tsx
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat(dashboard): Meet Kevin videos feed page"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 14: Video detail page (`/meet-kevin/videos/:id`)
|
||||
|
||||
**File:** Create `dashboard/src/pages/meetKevin/VideoDetail.tsx`
|
||||
|
||||
**Behavior:**
|
||||
- `useParams<{id: string}>()` → `videoId: number`.
|
||||
- `useQuery` for `getVideo(videoId)`; second `useQuery` for `getTranscript(videoId)` enabled only when `detail.transcript_available`.
|
||||
- `useMutation` for `reprocess(videoId, stage)` — visible button when `status` is `failed` or `discovered`; invalidates the video query on success.
|
||||
- Layout: title + published date, embedded `<YouTubeEmbed ref={playerRef} videoId={v.youtube_video_id} />`, tab strip (Analysis / Transcript / Raw JSON).
|
||||
- Analysis tab: outlook card (banner + reasoning), macro themes chips, key risks bulleted list, summary card, per-ticker grid — each ticker row has `<ActionChip>`, `<ConvictionBar>`, time horizon, italicized `rationale_quote`, and a "▶ MM:SS" button calling `playerRef.current?.seekTo(video_timestamp_seconds)`.
|
||||
- Transcript tab: scrollable list of segments; clicking a segment seeks the iframe to `Math.floor(start)`.
|
||||
- Raw JSON tab: `<pre>{JSON.stringify(detail.analysis, null, 2)}</pre>`.
|
||||
- Disable Transcript tab if `transcript_available` is false.
|
||||
|
||||
- [ ] **Step 1**: Implement.
|
||||
|
||||
- [ ] **Step 2**: Build verification.
|
||||
|
||||
- [ ] **Step 3**: Commit
|
||||
|
||||
```bash
|
||||
git add dashboard/src/pages/meetKevin/VideoDetail.tsx
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat(dashboard): Meet Kevin video detail page (tabs + iframe + deep-links)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 15: Stocks list + drill-down
|
||||
|
||||
**Files:**
|
||||
- Create: `dashboard/src/pages/meetKevin/Stocks.tsx`
|
||||
- Create: `dashboard/src/pages/meetKevin/StockDetail.tsx`
|
||||
|
||||
**`Stocks.tsx`:**
|
||||
- `useQuery` calls `meetKevinApi.listStocks` with 60s refetch.
|
||||
- Renders a table: Symbol (link to `/meet-kevin/stocks/<symbol>`), Mentions, Last seen, Latest (ActionChip + percent), Avg conviction (ConvictionBar).
|
||||
- Empty state: "No analyses yet" card.
|
||||
|
||||
**`StockDetail.tsx`:**
|
||||
- `useParams<{symbol: string}>()` uppercased.
|
||||
- Two `useQuery`s: `getStock(sym)` and `getStockTimeline(sym, "day")`.
|
||||
- Layout: large `$SYMBOL` header + back link, line chart of `net_action_score` over time (recharts), then chronological mention list. Each mention card links to its source `/meet-kevin/videos/<id>`, shows date, ActionChip, time horizon, ConvictionBar, percent, video title, and italicized `rationale_quote`.
|
||||
|
||||
- [ ] **Step 1**: Implement both pages.
|
||||
|
||||
- [ ] **Step 2**: Build verification.
|
||||
|
||||
- [ ] **Step 3**: Commit
|
||||
|
||||
```bash
|
||||
git add dashboard/src/pages/meetKevin/Stocks.tsx dashboard/src/pages/meetKevin/StockDetail.tsx
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat(dashboard): Meet Kevin stocks list + per-ticker drill-down"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 16: Wire routes + sidebar
|
||||
|
||||
**Files:**
|
||||
- Modify: `dashboard/src/App.tsx` — import the 5 page components and add 5 routes inside the protected `<Route path="/" element={...}>` block, BEFORE the catch-all redirect:
|
||||
|
||||
```
|
||||
<Route path="meet-kevin" element={<MeetKevinHome />} />
|
||||
<Route path="meet-kevin/videos" element={<MeetKevinVideos />} />
|
||||
<Route path="meet-kevin/videos/:id" element={<MeetKevinVideoDetail />} />
|
||||
<Route path="meet-kevin/stocks" element={<MeetKevinStocks />} />
|
||||
<Route path="meet-kevin/stocks/:symbol" element={<MeetKevinStockDetail />} />
|
||||
```
|
||||
|
||||
- Modify: `dashboard/src/components/Layout.tsx` — append to `navItems`:
|
||||
|
||||
```
|
||||
{ to: '/meet-kevin', label: 'Meet Kevin', icon: 'M15 10l4.553-2.276A1 1 0 0121 8.618v6.764a1 1 0 01-1.447.894L15 14M5 18h8a2 2 0 002-2V8a2 2 0 00-2-2H5a2 2 0 00-2 2v8a2 2 0 002 2z' },
|
||||
```
|
||||
|
||||
(Heroicon outline play-circle path — matches the video theme.)
|
||||
|
||||
- [ ] **Step 1**: Apply both edits.
|
||||
|
||||
- [ ] **Step 2**: Build verification + verify nav link renders.
|
||||
|
||||
- [ ] **Step 3**: Commit
|
||||
|
||||
```bash
|
||||
git add dashboard/src/App.tsx dashboard/src/components/Layout.tsx
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat(dashboard): wire Meet Kevin routes + sidebar entry"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 17: Manual QA against docker-compose
|
||||
|
||||
**Verification only — no commit (unless smoke notes are added).**
|
||||
|
||||
- [ ] **Step 1**: Boot infra: `docker compose up -d postgres redis`, then `python -m alembic upgrade head`.
|
||||
|
||||
- [ ] **Step 2**: Start api-gateway with `TRADING_DEV_MODE_AUTH_BYPASS=true python -m services.api_gateway.main &`.
|
||||
|
||||
- [ ] **Step 3**: Insert one fake row set via psql so the UI has data to render (one `kevin_videos` analyzed, one `kevin_analyses`, one `kevin_stock_mentions` for NVDA with conviction 0.82). Use the `INSERT … RETURNING id \\gset` pattern with psql.
|
||||
|
||||
- [ ] **Step 4**: Start the dashboard: `cd dashboard && VITE_DEV_MODE=true VITE_API_BASE_URL=http://localhost:8000/api npm run dev`.
|
||||
|
||||
- [ ] **Step 5**: Walkthrough at `http://localhost:5173/meet-kevin`:
|
||||
- Home renders latest video card + NVDA in top conviction list
|
||||
- `/meet-kevin/videos` shows the fake row
|
||||
- `/meet-kevin/videos/<id>` opens the iframe; the NVDA ticker card's "▶" deep-link calls seekTo on the player
|
||||
- `/meet-kevin/stocks` shows NVDA
|
||||
- `/meet-kevin/stocks/NVDA` shows the timeline + mention card
|
||||
|
||||
- [ ] **Step 6**: Cleanup: `docker compose down`; kill the api-gateway process.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Container & dependencies
|
||||
|
||||
### Task 18: Add `meet_kevin` extras + Dockerfile
|
||||
|
||||
**Files:**
|
||||
- Modify: `pyproject.toml` — under `[project.optional-dependencies]` add:
|
||||
|
||||
```
|
||||
meet_kevin = ["yt-dlp>=2026.04", "feedparser>=6.0", "anthropic>=0.40", "httpx>=0.27"]
|
||||
```
|
||||
|
||||
- Modify: `docker/Dockerfile.service` — update the `uv pip install --system --no-cache-dir ".[api,news,sentiment,trading,backtester]"` line to also include `meet_kevin`:
|
||||
|
||||
```
|
||||
uv pip install --system --no-cache-dir ".[api,news,sentiment,trading,backtester,meet_kevin]" && \\
|
||||
```
|
||||
|
||||
- [ ] **Step 1**: Apply both edits.
|
||||
|
||||
- [ ] **Step 2**: Sanity-check locally: `pip install -e ".[meet_kevin]"` (should resolve cleanly).
|
||||
|
||||
- [ ] **Step 3**: Commit
|
||||
|
||||
```bash
|
||||
git add pyproject.toml docker/Dockerfile.service
|
||||
git -c user.email=me@viktorbarzin.me commit -m "feat: add meet_kevin extras (yt-dlp, feedparser, anthropic)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 19: Push → CI build
|
||||
|
||||
- [ ] **Step 1**: `git push origin master`
|
||||
|
||||
- [ ] **Step 2**: Verify Woodpecker picked up the latest pipeline (response of `GET /api/repos/viktor/trading/pipelines` shows `running` or `pending` matching the head commit).
|
||||
|
||||
- [ ] **Step 3**: Wait for pipeline transition to terminal state via an `until` loop polling Woodpecker. **Do not pre-sleep** — let the condition drive cadence (~30s checks). Expected: `success`.
|
||||
|
||||
- [ ] **Step 4**: Confirm DockerHub got a fresh tag matching the head SHA short-form (`hub.docker.com/v2/repositories/viktorbarzin/trading-bot-service/tags`).
|
||||
|
||||
---
|
||||
|
||||
## Phase 6 — K8s revival
|
||||
|
||||
### Task 20: Vault secrets
|
||||
|
||||
- [ ] **Step 1**: `vault login -method=oidc`
|
||||
|
||||
- [ ] **Step 2**: Patch `secret/trading-bot`:
|
||||
|
||||
```bash
|
||||
read -rs ANTHROPIC_KEY; echo
|
||||
vault kv patch secret/trading-bot \
|
||||
anthropic_api_key="$ANTHROPIC_KEY" \
|
||||
meet_kevin_channel_id="UCUvvj5lwue7PspotMDjk5UA"
|
||||
unset ANTHROPIC_KEY
|
||||
```
|
||||
|
||||
- [ ] **Step 3**: Verify with `vault kv get -field=meet_kevin_channel_id secret/trading-bot`.
|
||||
|
||||
---
|
||||
|
||||
### Task 21: Uncomment + edit `infra/stacks/trading-bot/main.tf`
|
||||
|
||||
**Workspace:** this happens in `/home/wizard/code/infra/`, NOT the trading repo. **Required action**: claim presence for the infra stack before applying (per `/home/wizard/code/CLAUDE.md` "Agent Presence" rule):
|
||||
|
||||
```bash
|
||||
~/code/scripts/presence claim stack:trading-bot --purpose "Phase 6 revival — uncomment stack + add meet-kevin-watcher container"
|
||||
```
|
||||
|
||||
- [ ] **Step 1**: Remove the outer `/* … */` block-comment (lines ~2 and ~631).
|
||||
|
||||
- [ ] **Step 2**: In the `external_secret` resource's `template.data` and `data = […]`, add 2 keys: `TRADING_ANTHROPIC_API_KEY` ← `anthropic_api_key`, and `TRADING_MEET_KEVIN_CHANNEL_ID` ← `meet_kevin_channel_id`. Both reference `key = "trading-bot"` (the existing Vault path).
|
||||
|
||||
- [ ] **Step 3**: In `locals.common_env`, add:
|
||||
|
||||
```
|
||||
TRADING_MEET_KEVIN_POLL_INTERVAL_SECONDS = "10800"
|
||||
TRADING_MEET_KEVIN_DAILY_COST_CAP_USD = "5"
|
||||
TRADING_MEET_KEVIN_LLM_MODEL = "claude-sonnet-4-6"
|
||||
TRADING_MEET_KEVIN_PROMPT_VERSION = "v1"
|
||||
```
|
||||
|
||||
- [ ] **Step 4**: Delete the three worker containers from the `trading-bot-workers` Pod spec: `news-fetcher`, `sentiment-analyzer`, `trade-executor`.
|
||||
|
||||
- [ ] **Step 5**: Add the new `meet-kevin-watcher` container to the same Pod spec (after `market-data`):
|
||||
|
||||
```hcl
|
||||
container {
|
||||
name = "meet-kevin-watcher"
|
||||
image = "viktorbarzin/trading-bot-service:latest"
|
||||
image_pull_policy = "Always"
|
||||
command = ["python", "-m", "services.meet_kevin_watcher.main"]
|
||||
dynamic "env" {
|
||||
for_each = local.common_env
|
||||
content {
|
||||
name = env.key
|
||||
value = env.value
|
||||
}
|
||||
}
|
||||
env { name = "TRADING_OTEL_METRICS_PORT"; value = "9097" }
|
||||
env_from { secret_ref { name = "trading-bot-secrets" } }
|
||||
env_from { secret_ref { name = "trading-bot-db-creds" } }
|
||||
resources {
|
||||
requests = { cpu = "10m", memory = "128Mi" }
|
||||
limits = { memory = "256Mi" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 6**: Final container order is signal-generator [0], learning-engine [1], market-data [2], meet-kevin-watcher [3]. Update `lifecycle.ignore_changes` to only ignore image on container indices 0–3 plus `dns_config` (drop the index-4 and index-5 lines that were there for the 6-container layout).
|
||||
|
||||
- [ ] **Step 7**: `scripts/tg plan` — verify the plan creates the namespace + secrets + Deployments + Service + Ingress, with the new container in the workers pod.
|
||||
|
||||
- [ ] **Step 8**: `scripts/tg apply --non-interactive` (expect `Apply complete!`).
|
||||
|
||||
- [ ] **Step 9**: Verify K8s:
|
||||
|
||||
```bash
|
||||
kubectl get ns trading-bot
|
||||
kubectl -n trading-bot get pods,deploy,svc,ingress
|
||||
kubectl -n trading-bot get pod -l app=trading-bot-workers -o jsonpath='{.items[0].spec.containers[*].name}'
|
||||
# expected: signal-generator learning-engine market-data meet-kevin-watcher
|
||||
```
|
||||
|
||||
- [ ] **Step 10**: Commit + push the infra change
|
||||
|
||||
```bash
|
||||
git -C /home/wizard/code/infra add stacks/trading-bot/main.tf
|
||||
git -C /home/wizard/code/infra -c user.email=me@viktorbarzin.me \
|
||||
commit -m "trading-bot: revive K8s stack + add meet-kevin-watcher; remove 3 disabled workers"
|
||||
git -C /home/wizard/code/infra push
|
||||
```
|
||||
|
||||
- [ ] **Step 11**: `~/code/scripts/presence release stack:trading-bot`.
|
||||
|
||||
---
|
||||
|
||||
### Task 22: Production smoke test
|
||||
|
||||
- [ ] **Step 1**: Tail watcher logs
|
||||
|
||||
```bash
|
||||
kubectl -n trading-bot logs -l app=trading-bot-workers -c meet-kevin-watcher --tail=200 -f
|
||||
```
|
||||
|
||||
Expected within ~60s: `Starting meet-kevin-watcher`, then `RSS poll: N new videos` (likely up to 15 from first-run backfill).
|
||||
|
||||
- [ ] **Step 2**: Poll until at least one video reaches `status='analyzed'` (use an `until [ $(SELECT count(*)) -ge 1 ]` loop with sleep 30s — no pre-sleep).
|
||||
|
||||
- [ ] **Step 3**: Hit the API: `curl -sk https://trading.viktorbarzin.me/api/meet-kevin/health` (with appropriate auth cookie) — expect `counts_by_status.analyzed >= 1`, `daily_cost_usd > 0`.
|
||||
|
||||
- [ ] **Step 4**: Open `https://trading.viktorbarzin.me/meet-kevin` in a browser and verify the home + videos + stocks pages render with real Meet Kevin data.
|
||||
|
||||
---
|
||||
|
||||
## Self-review (run inline before publishing)
|
||||
|
||||
- [x] **Spec coverage**: each design-doc section maps to one or more tasks:
|
||||
- Channel + seed → Task 3
|
||||
- Data model (5 tables) → Tasks 2, 3
|
||||
- Pipeline 3 stages → Tasks 5, 6, 7, 8
|
||||
- LLM output schema → Tasks 1, 7
|
||||
- Daily cost cap → Task 8 (logic) + Task 9 (`/health` surface)
|
||||
- 11 API endpoints → Task 9
|
||||
- 5 dashboard pages → Tasks 12, 13, 14, 15
|
||||
- Container changes → Task 18
|
||||
- Vault secrets → Task 20
|
||||
- K8s revival (Terraform edit) → Task 21
|
||||
- Observability metrics → Task 8 main loop (`setup_telemetry` + counters)
|
||||
- Failure modes → Tasks 5-8 (per-stage retry / status logic) + Task 9 (`reprocess` endpoint)
|
||||
- First-run backfill (~$1.50 burst) → Tasks 8 + 22 (covered by the daily cost cap)
|
||||
|
||||
- [x] **Placeholder scan**: no TBD / "implement later" / "similar to Task N". Where a section references "see spec", the spec section is named explicitly and the spec is committed at `ab382af` (immutable for this plan).
|
||||
|
||||
- [x] **Type consistency**: schema/class/enum/table names match across tasks (`MeetKevinAnalysis`, `KevinVideo`, `kevin_stock_mentions`, `TickerAction.SELL`, env prefix `TRADING_MEET_KEVIN_*`, metrics port `9097`).
|
||||
|
||||
---
|
||||
|
||||
## Execution
|
||||
|
||||
Two options:
|
||||
|
||||
1. **Subagent-Driven (recommended)** — fresh subagent per task with two-stage review between tasks. Uses `superpowers:subagent-driven-development`.
|
||||
2. **Inline execution** — work through tasks in this session with periodic checkpoints. Uses `superpowers:executing-plans`.
|
||||
Loading…
Add table
Add a link
Reference in a new issue