# Trading Bot — Project Knowledge

## Project Overview
Automated stock trading bot combining news sentiment analysis with technical strategies. Event-driven Python microservices communicating via Redis Streams, with a React/TypeScript dashboard.

## Architecture
- **7 microservices**: news-fetcher, sentiment-analyzer, signal-generator, trade-executor, learning-engine, api-gateway, dashboard
- **Shared libraries** in `shared/`: config, redis_streams, telemetry, db, models, schemas, broker abstraction, 9 strategies, fundamentals providers
- **Infra**: PostgreSQL+TimescaleDB, Redis Streams, Ollama (local LLM), Docker Compose
- **CI/CD**: Woodpecker pipeline → Docker build → Kubernetes deploy
- **Brokerage**: Alpaca (paper trading) via abstraction layer in `shared/broker/`

## Key Design Decisions
- Services use underscores in Python package names (e.g., `services/news_fetcher/`) but hyphens in docker-compose service names (e.g., `news-fetcher`)
- Redis Streams for inter-service messaging: `news:raw`, `news:scored`, `signals:generated`, `trades:executed`
- Sentiment analysis is tiered: FinBERT first, Ollama fallback when confidence < threshold
- Learning uses multi-armed bandit style weight adjustment with guardrails (min trades, max shift, weight floor)
- Auth is passkey/WebAuthn (py-webauthn on backend, @simplewebauthn/browser on frontend) with JWT sessions
- OpenTelemetry + Prometheus `/metrics` endpoints (no Prometheus/Grafana deployed — uses existing infra)

## Tech Stack
- **Backend**: Python 3.12, FastAPI, SQLAlchemy 2.0 (async, mapped_column style), Pydantic v2, alpaca-py
- **Frontend**: React 18, TypeScript, Vite, Tailwind CSS, TanStack Query, TradingView lightweight-charts, Recharts
- **ML**: transformers (FinBERT), ollama-python
- **Testing**: pytest, pytest-asyncio (asyncio_mode="auto"), React Testing Library
- **Database**: PostgreSQL 16 + TimescaleDB extension, Alembic for migrations

## Project Structure
```
trading-bot/
├── shared/              # Shared Python libraries
│   ├── config.py        # Pydantic BaseSettings (env_prefix="TRADING_")
│   ├── redis_streams.py # StreamPublisher + StreamConsumer
│   ├── telemetry.py     # OpenTelemetry setup
│   ├── db.py            # Async engine + sessionmaker
│   ├── models/          # SQLAlchemy models (16 tables)
│   ├── schemas/         # Pydantic v2 schemas (message types)
│   ├── broker/          # BaseBroker ABC + AlpacaBroker
│   ├── fundamentals/    # Alpha Vantage, FMP, Yahoo providers + cache
│   └── strategies/      # BaseStrategy + 9 implementations
├── services/
│   ├── news_fetcher/    # RSS + Reddit → news:raw
│   ├── sentiment_analyzer/ # FinBERT + Ollama → news:scored
│   ├── signal_generator/   # Weighted ensemble → signals:generated
│   ├── trade_executor/     # Risk mgmt + Alpaca → trades:executed
│   ├── learning_engine/    # Multi-armed bandit weight adjustment
│   └── api_gateway/        # FastAPI + WebAuthn + WebSocket
├── backtester/          # Historical replay engine
├── dashboard/           # React/TypeScript (Vite)
├── docker/              # Dockerfile.service, Dockerfile.dashboard, nginx.conf
├── scripts/             # seed_strategies.py, smoke_test.sh
├── tests/               # Unit + integration tests
├── alembic/             # Database migrations
├── docker-compose.yml
└── pyproject.toml       # Monorepo with optional dep groups
```

## Development Workflow
- Use `.venv`: `python3 -m venv .venv && source .venv/bin/activate`
- Install all deps: `pip install -e ".[api,news,sentiment,trading,backtester,dev]"`
- Run unit tests: `python -m pytest tests/ -v -m "not integration"`
- Run integration tests (requires docker): `python -m pytest tests/ -v -m integration`
- Dashboard dev: `cd dashboard && npm install && npm run dev`
- Dashboard build: `cd dashboard && npm run build`
- Full stack: `docker compose up -d`

## Test Coverage (246 unit tests)
- `tests/test_redis_streams.py` — 5 tests
- `tests/test_models.py` — 21 tests
- `tests/test_schemas.py` — 49 tests
- `tests/test_broker.py` — 18 tests
- `tests/test_strategies.py` — 24 tests
- `tests/test_backtester.py` — 13 tests
- `tests/services/test_news_fetcher.py` — 10 tests
- `tests/services/test_sentiment_analyzer.py` — 19 tests
- `tests/services/test_signal_generator.py` — 17 tests
- `tests/services/test_trade_executor.py` — 16 tests
- `tests/services/test_learning_engine.py` — 28 tests
- `tests/services/test_api_auth.py` — 13 tests
- `tests/services/test_api_routes.py` — 13 tests
- `tests/integration/` — 9 integration tests (news pipeline + trading flow)

## Important Patterns
- Each service extends `shared.config.BaseConfig` for its own settings
- Services use `StreamConsumer`/`StreamPublisher` for Redis Streams messaging
- Broker abstraction: all trading goes through `BaseBroker` interface (swap Alpaca for another broker without changing services)
- Strategy abstraction: all strategies implement `BaseStrategy.evaluate()` returning `TradeSignal | None`
- The pyproject.toml uses optional dependency groups per service: `[api]`, `[news]`, `[sentiment]`, `[trading]`, `[backtester]`
- The webauthn package on PyPI is called `webauthn` (not `py-webauthn`)

## Known Issues / TODO
- Code review was started but not completed — should be done before production use
- JWT test warnings about short HMAC keys (test-only, production keys should be 32+ bytes)
- Integration tests mock FinBERT and Alpaca — need real integration tests with live services
- Twitter/X source not implemented in news fetcher (only RSS + Reddit)
- Dashboard has no unit tests (only build verification)

## How This Was Built
- Built in 6 sprints using parallel subagent worktrees
- Each sprint had 2-3 agents working independently on non-overlapping files
- Worktree branches merged into master after each sprint
- Tests verified after every merge (246 tests, 0 failures)
- Agent teams didn't work well (idle notification issues) — switched to direct subagent Task calls with worktree isolation