# Codebase Structure

**Analysis Date:** 2025-02-23

## Directory Layout

```
trading-bot/
├── alembic/              # Database migration versions and configuration
│   ├── env.py            # Alembic runtime environment
│   ├── script.py.mako    # Migration template
│   └── versions/         # Individual migration files
├── backtester/           # Historical replay engine
│   ├── __init__.py
│   ├── config.py         # Backtester configuration
│   ├── data_loader.py    # Load bars from database
│   ├── engine.py         # Main replay loop with SimulatedBroker
│   ├── metrics.py        # Equity curve, Sharpe, drawdown calculations
│   └── simulated_broker.py  # Paper trading for backtests
├── dashboard/            # React/TypeScript frontend
│   ├── public/           # Static assets (favicon, manifest)
│   ├── src/
│   │   ├── api/          # API client (auth.ts, client.ts)
│   │   ├── assets/       # Images, fonts
│   │   ├── components/   # Reusable UI (PositionsTable, EquityCurve, etc.)
│   │   ├── hooks/        # Custom React hooks (useAuth, useWebSocket, usePortfolio)
│   │   ├── pages/        # Page components (Login, Portfolio, TradeLog, etc.)
│   │   ├── App.tsx       # Router and layout
│   │   ├── main.tsx      # Entry point
│   │   └── index.css     # Tailwind styles
│   ├── index.html        # HTML template
│   ├── package.json      # npm dependencies (Vite, React, TypeScript)
│   └── vite.config.ts    # Vite build config
├── docker/               # Dockerfiles and compose configs
│   ├── Dockerfile.service # Python service base image
│   ├── Dockerfile.dashboard # Node.js build + Nginx serve
│   └── nginx.conf        # Nginx reverse proxy config
├── docs/                 # Documentation and planning
│   └── plans/            # Implementation plans from GSD phases
├── scripts/              # Utility scripts
│   ├── seed_strategies.py  # Initialize default strategies in DB
│   ├── smoke_test.sh     # Health check / integration test script
│   └── __init__.py
├── services/             # Microservices (Python packages with underscores)
│   ├── api_gateway/      # FastAPI HTTP/WebSocket server
│   │   ├── main.py       # App factory, lifespan, router registration
│   │   ├── config.py     # ApiGatewayConfig (CORS, JWT secret, etc.)
│   │   ├── ws.py         # WebSocket endpoint
│   │   ├── auth/
│   │   │   ├── routes.py # Register/login/logout endpoints
│   │   │   ├── jwt.py    # JWT token encoding/decoding
│   │   │   └── middleware.py  # get_current_user dependency
│   │   ├── routes/       # REST endpoints organized by domain
│   │   │   ├── news.py   # GET /api/news
│   │   │   ├── signals.py  # GET /api/signals
│   │   │   ├── trades.py # GET /api/trades
│   │   │   ├── portfolio.py # GET /api/portfolio (live account)
│   │   │   ├── strategies.py # GET/PUT /api/strategies
│   │   │   ├── controls.py  # POST /api/controls/* (pause/resume/etc)
│   │   │   └── backtest.py  # POST /api/backtest
│   │   └── tasks/
│   │       └── portfolio_sync.py # Background task: sync account data
│   ├── news_fetcher/    # Polls RSS + Reddit
│   │   ├── main.py       # Entry point: fetch + deduplicate + publish
│   │   ├── config.py     # NewsFetcherConfig (feed URLs, poll intervals)
│   │   └── sources/
│   │       ├── rss.py    # RSSSource.fetch() via feedparser
│   │       └── reddit.py # RedditSource.fetch() via asyncpraw
│   ├── sentiment_analyzer/ # Score sentiment + extract tickers
│   │   ├── main.py       # Entry point: consume raw → score → publish
│   │   ├── config.py     # SentimentAnalyzerConfig (thresholds, model paths)
│   │   ├── ticker_extractor.py # Regex-based company name → ticker
│   │   └── analyzers/
│   │       ├── finbert.py # FinBERTAnalyzer (transformers model)
│   │       └── ollama_analyzer.py # OllamaAnalyzer (fallback LLM)
│   ├── signal_generator/  # Combine sentiment + market → signals
│   │   ├── main.py       # Entry point: consume scored articles + bars → ensemble
│   │   ├── config.py     # SignalGeneratorConfig (ensemble threshold)
│   │   ├── ensemble.py   # WeightedEnsemble orchestration
│   │   └── market_data.py # MarketDataManager: SMA/RSI calculation per ticker
│   ├── trade_executor/   # Risk checks + order submission
│   │   ├── main.py       # Entry point: consume signals → risk check → submit
│   │   ├── config.py     # TradeExecutorConfig (risk limits, position size formula)
│   │   └── risk_manager.py # RiskManager: position limits, market hours, cooldowns
│   ├── learning_engine/  # Evaluate trades + adjust weights
│   │   ├── main.py       # Entry point: consume executions → evaluate → adjust weights
│   │   ├── config.py     # LearningEngineConfig (adjustment parameters)
│   │   ├── evaluator.py  # TradeEvaluator: match opens to closes, calc P&L
│   │   └── weight_adjuster.py # WeightAdjuster: multi-armed bandit logic
│   ├── market_data/      # Fetch + stream OHLCV bars
│   │   ├── main.py       # Entry point: fetch from Alpaca → publish bars
│   │   ├── config.py     # MarketDataConfig (watchlist, timeframe, limits)
│   │   └── __main__.py   # Allows `python -m services.market_data`
│   └── __init__.py
├── shared/               # Shared Python libraries (imported by all services)
│   ├── __init__.py
│   ├── config.py         # BaseConfig: common settings (DB, Redis, logging)
│   ├── db.py             # create_db(): async engine + sessionmaker
│   ├── redis_streams.py  # StreamPublisher, StreamConsumer
│   ├── telemetry.py      # setup_telemetry(): OpenTelemetry + Prometheus
│   ├── broker/           # Brokerage abstraction layer
│   │   ├── base.py       # BaseBroker ABC (abstract interface)
│   │   └── alpaca_broker.py # AlpacaBroker: alpaca-py implementation
│   ├── models/           # SQLAlchemy ORM models
│   │   ├── __init__.py   # Imports all models so Alembic discovers them
│   │   ├── base.py       # Base (declarative), TimestampMixin
│   │   ├── auth.py       # User, UserCredential
│   │   ├── news.py       # Article, ArticleSentiment
│   │   ├── trading.py    # Signal, Trade, Position, Strategy, StrategyWeightHistory
│   │   ├── learning.py   # TradeOutcome, LearningAdjustment
│   │   └── timeseries.py # MarketData, PortfolioSnapshot, StrategyMetric
│   ├── schemas/          # Pydantic v2 message schemas
│   │   ├── __init__.py   # Imports all schemas
│   │   ├── base.py       # BaseSchema, TimestampSchema
│   │   ├── news.py       # RawArticle, ScoredArticle
│   │   ├── trading.py    # TradeSignal, TradeExecution, OrderRequest, etc.
│   │   └── learning.py   # TradeOutcomeSchema, WeightAdjustment
│   └── strategies/       # Trading strategy implementations
│       ├── __init__.py   # Exports BaseStrategy, concrete implementations
│       ├── base.py       # BaseStrategy ABC
│       ├── momentum.py    # MomentumStrategy
│       ├── mean_reversion.py # MeanReversionStrategy
│       └── news_driven.py # NewsDrivenStrategy
├── tests/                # Test suites
│   ├── __init__.py
│   ├── conftest.py       # Pytest fixtures (fake Redis, DB, configs)
│   ├── test_*.py         # Unit tests: models, schemas, broker, strategies, backtester
│   ├── services/         # Service-specific tests
│   │   ├── test_news_fetcher.py
│   │   ├── test_sentiment_analyzer.py
│   │   ├── test_signal_generator.py
│   │   ├── test_trade_executor.py
│   │   ├── test_learning_engine.py
│   │   ├── test_api_auth.py
│   │   ├── test_api_routes.py
│   │   ├── test_market_data.py
│   │   └── test_portfolio_sync.py
│   └── integration/      # Integration tests (require docker)
│       ├── test_news_pipeline.py
│       └── test_trading_flow.py
├── .claude/              # Project-specific Claude knowledge
│   └── CLAUDE.md         # Architecture notes, patterns, known issues
├── .env.example          # Example environment variables
├── alembic.ini           # Alembic configuration
├── docker-compose.yml    # Service orchestration (8 services + infra)
├── pyproject.toml        # Python package config, dependencies, test config
└── README.md             # Project overview
```

## Directory Purposes

**`alembic/`:**
- Purpose: Database schema migrations using Alembic (SQLAlchemy migration tool)
- Contains: Migration scripts (timestamped Python files in `versions/`)
- Workflow: `alembic revision -m "message"` creates migration, `alembic upgrade head` applies to DB

**`backtester/`:**
- Purpose: Historical replay engine for strategy testing
- Contains: Engine loop, simulated broker, metrics calculation
- Used by: API endpoint `/api/backtest` (triggered by dashboard)
- Workflow: Load historical bars from DB, replay trades with `SimulatedBroker`, output equity curve + metrics

**`dashboard/`:**
- Purpose: React TypeScript frontend for user interaction
- Contains: Pages (Login, Portfolio, TradeLog, News, etc.), components (charts, tables), API client, hooks
- Entry: `src/main.tsx` → `src/App.tsx`
- Build: `npm run build` outputs to `dist/`, served by Nginx in Docker

**`docker/`:**
- Purpose: Container images and deployment config
- Contains: `Dockerfile.service` (Python services base), `Dockerfile.dashboard` (Node build + Nginx)
- Used by: docker-compose.yml to build and run all services

**`scripts/`:**
- Purpose: Utility and setup scripts
- Key files:
  - `seed_strategies.py`: Inserts default strategies into DB (Momentum, MeanReversion, NewsDriven)
  - `smoke_test.sh`: Checks all service health endpoints

**`services/*/main.py`:**
- Purpose: Entry point for each microservice
- Pattern: Each service has own `main.py` that can be run as `python -m services.SERVICE_NAME`
- Lifecycle: Setup config → connect to Redis/DB → start polling/consuming → handle shutdown gracefully

**`services/*/config.py`:**
- Purpose: Service-specific configuration
- Pattern: Extends `BaseConfig`, adds service-specific settings
- Example: `NewsFetcherConfig` adds `rss_urls`, `reddit_subreddits`, `rss_poll_interval_seconds`

**`shared/models/`:**
- Purpose: SQLAlchemy ORM models (database schema)
- Key tables: `trades`, `signals`, `articles`, `article_sentiments`, `market_data`, `positions`, `users`, `strategies`, `strategy_weight_history`, `learning_adjustments`, `portfolio_snapshots`
- Pattern: Each model imports in `__init__.py` so Alembic can auto-discover for migrations

**`shared/schemas/`:**
- Purpose: Pydantic v2 message schemas for validation
- Used by: Services to validate messages on consume/publish via Redis Streams
- Naming: Schemas mirror the Redis Stream message types (RawArticle, ScoredArticle, TradeSignal, TradeExecution, etc.)

**`shared/strategies/`:**
- Purpose: Pluggable trading strategy implementations
- Pattern: All extend `BaseStrategy`, implement `async evaluate(ticker, market, sentiment) → TradeSignal | None`
- Used by: Signal generator via `WeightedEnsemble`

**`tests/`:**
- Purpose: Unit and integration tests
- Pattern: `test_*.py` files match modules in `shared/` and `services/`
- Marker: `@pytest.mark.integration` for tests requiring live services (redis, postgres)
- Run: `pytest tests/ -v -m "not integration"` for unit, or with docker running for integration

## Key File Locations

**Entry Points:**
- `services/news_fetcher/main.py`: Polls RSS/Reddit, publishes to `news:raw`
- `services/sentiment_analyzer/main.py`: Consumes `news:raw`, publishes to `news:scored`
- `services/signal_generator/main.py`: Consumes `news:scored` + `market:bars`, publishes to `signals:generated`
- `services/trade_executor/main.py`: Consumes `signals:generated`, submits orders, publishes to `trades:executed`
- `services/learning_engine/main.py`: Consumes `trades:executed`, adjusts strategy weights
- `services/market_data/main.py`: Fetches OHLCV bars, publishes to `market:bars`
- `services/api_gateway/main.py`: HTTP server listening on port 8000 (configurable)
- `dashboard/src/main.tsx`: React app entry, connects to API gateway

**Configuration:**
- `.env`: Runtime environment variables (database URL, Redis URL, API keys, etc.)
- `pyproject.toml`: Python project metadata, dependencies (split into optional groups per service)
- `docker-compose.yml`: Service orchestration, port mappings, environment variables
- `alembic.ini`: Database migration tool configuration

**Core Logic:**
- `services/signal_generator/ensemble.py`: Weighted ensemble combining strategies
- `services/trade_executor/risk_manager.py`: Risk checks before order submission
- `services/learning_engine/weight_adjuster.py`: Multi-armed bandit weight adjustment
- `backtester/engine.py`: Historical replay loop
- `shared/broker/alpaca_broker.py`: Alpaca order submission and account queries

**Testing:**
- `tests/conftest.py`: Pytest fixtures for mock Redis, DB, service configs
- `tests/test_models.py`: ORM model tests
- `tests/test_schemas.py`: Pydantic schema validation tests
- `tests/services/`: Service-specific unit tests
- `tests/integration/`: End-to-end pipeline tests (require docker)

## Naming Conventions

**Files:**
- Python service directories use underscores: `news_fetcher`, `api_gateway`, `signal_generator` (matches Docker Compose service names with hyphens replaced)
- Test files: `test_*.py` or `*_test.py` (pytest convention)
- Database migration files: `{timestamp}_{description}.py` (Alembic auto-generated)
- React components: PascalCase `.tsx` files (Login.tsx, Portfolio.tsx, PositionsTable.tsx)
- React hooks: `use*.ts` prefix (useAuth.ts, useWebSocket.ts, usePortfolio.ts)
- Config classes: `*Config` suffix extending `BaseConfig` (NewsFetcherConfig, ApiGatewayConfig)

**Directories:**
- Services under `services/`: lowercase with underscores (news_fetcher, api_gateway)
- Shared libraries under `shared/`: logical grouping (broker/, models/, schemas/, strategies/)
- Tests mirror source structure: `tests/services/test_*.py` for `services/*/` modules
- React pages: match route names (Portfolio.tsx for /portfolio, TradeLog.tsx for /trades)

## Where to Add New Code

**New Feature (End-to-End):**
- Service-specific code: `services/{SERVICE_NAME}/{module}.py`
- Shared models: `shared/models/{domain}.py` (if new entity type)
- Shared schemas: `shared/schemas/{domain}.py` (if new message type)
- Tests: `tests/services/test_{SERVICE_NAME}.py` and/or `tests/integration/`

**New Component/Module:**
- Service: Create `services/{service_name}/` directory with `__init__.py`, `main.py`, `config.py`
- Strategy: Extend `BaseStrategy` in `shared/strategies/{strategy_name}.py`
- Broker adapter: Extend `BaseBroker` in `shared/broker/{provider_name}_broker.py`
- API route: Add to `services/api_gateway/routes/{domain}.py`
- Dashboard page: Add `dashboard/src/pages/{PageName}.tsx` and route in `App.tsx`

**Utilities:**
- Shared helpers: `shared/{module}.py` (e.g., `shared/redis_streams.py`)
- Service-specific helpers: `services/{SERVICE_NAME}/{helper}.py`
- API utilities: `services/api_gateway/{utility}.py`

**Database:**
- New table: Add model in `shared/models/{domain}.py`
- Schema migration: Run `alembic revision -m "description"`, fill in migration file
- Persist data: Service calls `session.add(model_instance)` then `session.commit()`

**Tests:**
- Unit test: `tests/test_{module}.py` for shared, `tests/services/test_{service}.py` for services
- Integration test: `tests/integration/test_{flow}.py`
- Fixtures: Add to `tests/conftest.py` (database, Redis, mocked API responses)

## Special Directories

**`.planning/codebase/`:**
- Purpose: Generated codebase mapping documents (ARCHITECTURE.md, STRUCTURE.md, etc.)
- Generated by: `/gsd:map-codebase` command
- Committed: Yes, part of repo for CI/CD reference

**`.pytest_cache/`, `__pycache__/`:**
- Purpose: Pytest and Python runtime caches
- Generated: Yes (auto-created by pytest/Python)
- Committed: No (in .gitignore)

**`trading_bot.egg-info/`:**
- Purpose: Build metadata from `setuptools.build_meta`
- Generated: Yes (auto-created during install)
- Committed: No (in .gitignore)

**`dashboard/dist/`:**
- Purpose: Built React app (HTML, JS, CSS bundles)
- Generated: Yes (by `npm run build`)
- Committed: No (in .gitignore)