Context
-------
Phase 1 kickoff: Trading212 tags every ticker with `_EQ`, sometimes
preceded by a lowercase exchange letter ("l" = LSE) or `_US`. Raw
symbols like `VUAGl_EQ` are an implementation detail that would leak
into Wealthfolio and diverge from other providers (InvestEngine and
Schwab emit `VUAG` / `META`). The canonical form has to match across
providers so portfolio aggregation lines up.
Unlike the finance/ reference code, we do NOT restrict to a
SUPPORTED_TICKERS allowlist here — Wealthfolio is the source of truth,
everything gets imported, and the user decides what to track.
This change
-----------
- broker_sync/providers/trading212.py: pure `_normalise_ticker`
helper backed by a single regex that peels `(_US)?[a-z]?_EQ`. No
lookup tables — the rule covers all observed shapes.
- tests/providers/test_trading212_ticker.py: parametrised cases for
every mapping called out in the Phase 1 plan plus pass-through of
already-canonical symbols.
Test plan
---------
## Automated
- poetry run pytest -q → 41 passed in 0.46s
- poetry run mypy broker_sync tests → Success: no issues found in 22 source files
- poetry run ruff check . → All checks passed!
## Manual Verification
Not applicable — pure function, no external side effects.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Context
-------
Closes Phase 0 scaffolding. Image must build and run so infra can
schedule an initial no-op CronJob (the plan's Phase 0 exit criterion)
while Phase 0.5 / 0.75 / 1 land.
This change
-----------
- broker_sync/cli.py: typer app with two commands.
* `version` — prints __version__; used as the no-op CronJob
liveness check.
* `auth-spike` — Phase 0.5 end-to-end live probe: log in to
Wealthfolio, list accounts, exit 0 on success. Credentials read
from env (WF_BASE_URL/USERNAME/PASSWORD) so CronJob + ESO can
inject them without CLI flags.
- Dockerfile: multi-stage, Python 3.12-slim, non-root user 10001
with /data as the shared PVC mount. Poetry virtualenv baked into
/app/.venv, entrypoint is `broker-sync`, default command `version`.
- CLI test via typer.testing.CliRunner.
Test plan
---------
## Automated
- poetry run pytest -q → 32 passed
- poetry run mypy broker_sync tests → Success: no issues found in 19 source files
- poetry run ruff check . → All checks passed!
- poetry run broker-sync version → broker-sync 0.1.0
## Manual Verification
Docker build + run deferred — image will be built via GHA after the
repo is pushed to GitHub in a follow-up session; the pyproject install
has already been verified locally.
Context
-------
This is the Phase 0.5 deliverable — the hardest-to-validate unknown
in the plan. Wealthfolio auth is JWT HttpOnly cookie with a 5-req/min
login rate limit. CronJob pods are ephemeral, so we persist cookies
to disk between runs (shared PVC in production).
Plan stress-test also flagged: use the CSV import path, not per-row
JSON POST. Wealthfolio's UI uses /activities/import and its dedup
logic is battle-tested; CSVs double as audit artefacts we can replay.
This change
-----------
- WealthfolioSink (httpx async): login with username/password, persists
cookie dict to session_path on disk, attaches it as a Cookie header
on subsequent calls.
- 401 on a non-login endpoint triggers a single re-login + retry.
- ensure_account() is idempotent — GETs the account list first, only
POSTs /accounts if id is missing.
- import_activities() always runs /activities/import/check first; any
non-2xx there raises ImportValidationError and we never touch the
real import endpoint. Protects against half-written state when the
broker emits a symbol Wealthfolio doesn't know.
- httpx.MockTransport-based tests cover: login persistence, 401 on
login raises UnauthorizedError, session reuse from disk, 401 retry
path, ensure_account idempotency + creation, import dry-run-then-real
sequencing, halt on check failure.
Not yet covered (deferred):
- Multi-process file lock on session_path (single-process enough for
now; Phase 1 adds it when multiple CronJobs run concurrently).
- 429 jittered backoff (TBD when Wealthfolio actually rate-limits us).
Test plan
---------
## Automated
- poetry run pytest -q → 31 passed
- poetry run mypy broker_sync tests → Success: no issues found in 17 source files
- poetry run ruff check . → All checks passed!
## Manual Verification
Live auth spike against https://wealthfolio.viktorbarzin.me deferred
until the password is seeded into Vault at secret/broker-sync/wealthfolio
in a follow-up commit (needs Viktor's Vault session).
Context
-------
Every broker connector needs a uniform shape so the orchestrator can
fan out without knowing provider-specific details. Normalisation (GBP
conversion) lives outside providers on purpose — keeping providers
native-currency-emitters means we can re-normalise historical activity
when HMRC rates land without re-fetching from the broker.
This change
-----------
- providers/base.py: Provider Protocol with `accounts()` and async
`fetch(since, before)` iterator. No abstract base class — duck-typed
Protocol so each concrete provider stays independent.
- normaliser.py: takes a native Activity + FxCache, returns a copy
with amount_gbp/fx_rate_gbp/fx_rate_source filled in. Two modes:
qty*price for BUY/SELL, amount for DIVIDEND/DEPOSIT/etc.
- Namespace packages for providers/, providers/parsers/, sinks/ so
future modules slot in cleanly.
Test plan
---------
## Automated
- poetry run pytest -q → 23 passed
- poetry run mypy broker_sync tests → Success: no issues found in 14 source files
- poetry run ruff check . → All checks passed!
## Manual Verification
Not applicable at this layer.
Context
-------
FX for UK users has two lives: live ECB rates for portfolio display
(available same-day), and HMRC monthly/daily rates for CGT basis
(published after month-end). The plan keeps both in one cache table
with an upgradable `source` column, so a later reconciliation job can
replace ECB_LIVE values with HMRC_MONTHLY for the same date without
schema work.
This change
-----------
- FxCache: SQLite table (currency, on_date) -> (rate_gbp, source) with
ON CONFLICT UPDATE semantics so reconciliation is a single put().
- convert_to_gbp(): GBP short-circuits to identity; any other currency
must be in the cache (network fetch is the caller's responsibility,
separately implemented by the ECB and HMRC fetchers).
- Explicit LookupError on cache miss — deliberate, we do NOT want a
silent fallback that produces wrong cost-basis numbers.
Decisions deferred to later commits:
- Actual ECB daily reference-rate fetcher (eurofxref XML) — lands with
the Trading212 provider in Phase 1 when non-GBP trades first appear.
- HMRC monthly-rate fetcher + reconciliation CronJob — Phase 1 tail.
Test plan
---------
## Automated
- poetry run pytest tests/test_fx.py -v → 6 passed
- poetry run mypy broker_sync tests → Success: no issues found in 8 source files
- poetry run ruff check . → All checks passed!
## Manual Verification
Not applicable — no network yet.
Context
-------
Wealthfolio's activity `notes` field is user-editable via the UI, so
using it as the dedup key would let a single note-edit in Wealthfolio
cause the next sync to create a duplicate. Stress-testing the plan
flagged this as the top structural risk.
This change
-----------
- SQLite-backed store at `/data/broker_sync.db` in production; keyed on
(provider, account, external_id) so each provider's id space is
scoped to its own account.
- `INSERT OR IGNORE` makes record() idempotent — second call with the
same key is a no-op and preserves the original wealthfolio_activity_id
plus first_seen timestamp.
- `filter_new()` is the integration point: provider fetches activities,
hands them to the store, gets back only the unseen subset to submit
to the Wealthfolio sink.
- Wealthfolio activity id returned by the API is persisted alongside
each record so the HMRC FX reconciliation job can later PATCH the
original activity rather than creating a new one.
Test plan
---------
## Automated
- poetry run pytest tests/test_dedup.py -v → 6 passed
- poetry run mypy broker_sync tests → Success: no issues found in 6 source files
- poetry run ruff check . → All checks passed!
## Manual Verification
Not applicable for this layer — full end-to-end verification happens
once a provider + sink land (Phase 1 Trading212 and the auth spike).
Context
-------
New connector suite that syncs UK brokerage activity (Trading212,
InvestEngine, Schwab email-parsed, CSV drop-folder) into Wealthfolio.
Lives outside finance/ intentionally — finance/ is untouched per the
plan at ~/.claude/plans/let-s-work-on-linking-temporal-valiant.md.
This change
-----------
- Poetry project with httpx, typer, bs4, dev tools (pytest, mypy strict,
ruff, yapf).
- Canonical Activity + Account models with the 6 UK tax wrappers
(ISA/SIPP/GIA/LISA/JISA/WORKPLACE_PENSION) and the 12 Wealthfolio
activity types from docs/activities/activity-types.md on the upstream.
- Validation invariants: BUY/SELL need qty+price, DIVIDEND/DEPOSIT/etc
need amount — raises early so providers can't silently emit broken
rows.
- to_wealthfolio_csv_row() shape matches Wealthfolio's CSV import;
primary sink path per the plan.
Test plan
---------
## Automated
- poetry run pytest -q → 7 passed in 0.03s
- poetry run mypy broker_sync tests → Success: no issues found in 4 source files
- poetry run ruff check . → All checks passed!
- poetry run yapf --diff --recursive broker_sync tests → no diff
## Manual Verification
Not applicable — pure data model, no runtime behaviour.
Closes: code-thw.1
