832732a419
## Context
UK workplace pension at planviewer.fidelity.co.uk has no public API; the SPA
calls a private JSON backend at prd.wiciam.fidelity.co.uk/cvmfe/api/*. Viktor
confirmed in DevTools that an OPTIONS preflight lists auth headers
(ch, fid, rid, sid, tbid, theosreferer, ua). Full reverse-engineering of the
endpoint paths is pending Viktor's POST cURL paste for transactions +
holdings views.
Until those endpoints are captured, ship the scaffold: provider module, CLI
commands, tests, docs. This unblocks installing Playwright in the image and
lets Viktor run the one-off seed command on his laptop ahead of the data
integration.
## This change
- broker_sync/providers/fidelity_planviewer.py
- FidelityCreds namedtuple (storage_state_path, plan_id).
- FidelitySessionError (401 → re-seed), FidelityProviderConfigError.
- FidelityPlanViewerProvider: .accounts() returns a single
WORKPLACE_PENSION account, .fetch() raises until endpoints are wired.
- broker_sync/cli.py
- fidelity-seed: launches headed Chromium so Viktor can log in and tick
"Remember device", then dumps storage_state.json.
- fidelity-ingest: stub matching the invest-engine / trading212 CLI
shape; reads storage_state + plan_id, pipes through the shared pipeline.
- tests/providers/test_fidelity_planviewer.py
- Asserts the single-account shape + the loud-failure guard.
- docs/providers/fidelity-planviewer.md
- Architecture diagram, one-time seed procedure, backfill + monthly
commands, alert runbook.
- pyproject.toml
- playwright ^1.47 as a first-class dep (used only by fidelity-seed and
later by the session-refresh step in fidelity-ingest).
## What is NOT in this change
- Endpoint wiring in provider.fetch() — blocked on DevTools POST cURL.
- Infra CronJob + Vault secret + Prometheus alert — lands once the first
manual backfill succeeds and we know the Chromium image size is fine.
- Dockerfile Chromium install — same trigger.
## Verification
### Automated
$ poetry run pytest tests/providers/test_fidelity_planviewer.py -v
2 passed in 0.08s
$ poetry run pytest -q
122 passed, 1 skipped in 1.07s
$ poetry run mypy broker_sync/providers/fidelity_planviewer.py broker_sync/cli.py
Success: no issues found in 2 source files
$ poetry run ruff check broker_sync/providers/fidelity_planviewer.py broker_sync/cli.py tests/providers/test_fidelity_planviewer.py
All checks passed!
### Manual (Viktor, later)
1. poetry install && poetry run playwright install chromium
2. poetry run broker-sync fidelity-seed --out /tmp/state.json
3. Chromium opens → log in → tick "Remember device" → press Enter
4. vault kv patch secret/broker-sync fidelity_storage_state=@/tmp/state.json
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
111 lines
6 KiB
Markdown
111 lines
6 KiB
Markdown
# Fidelity UK PlanViewer provider
|
||
|
||
Viktor's UK workplace pension is hosted at `pv.planviewer.fidelity.co.uk`. There
|
||
is no public API for individual members — the provider reverse-engineers the
|
||
private JSON backend at `prd.wiciam.fidelity.co.uk/cvmfe/api/*` that the SPA
|
||
itself calls, and uses Playwright only to keep a long-lived login session
|
||
alive.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
┌─────────────┐ storage_state.json ┌──────────────────┐
|
||
│ Vault KV │◀─── (quarterly reseed) ───│ fidelity-seed │
|
||
│ broker-sync │ │ (headed browser) │
|
||
└──────┬──────┘ └──────────────────┘
|
||
│ ▲
|
||
│ loads on start │ Viktor runs once
|
||
▼ when session expires
|
||
┌────────────────────┐
|
||
│ Monthly CronJob │
|
||
│ broker-sync-fidelity│
|
||
└────────────┬────────┘
|
||
│ headless Chromium
|
||
▼
|
||
┌─────────────────────────────────┐ ┌────────────────────────────────┐
|
||
│ pv.planviewer.fidelity.co.uk │◀─────│ navigate dashboard → capture │
|
||
│ (SPA) │ │ fresh sid/fid/tbid/rid headers │
|
||
└─────────────────────────────────┘ └──────────────┬─────────────────┘
|
||
│
|
||
┌───────────▼─────────────┐
|
||
│ httpx JSON calls │
|
||
│ prd.wiciam.../cvmfe/api│
|
||
└───────────┬─────────────┘
|
||
│
|
||
┌────────────────────▼────────────────────┐
|
||
│ DEPOSIT × N (employee + employer) │
|
||
│ BUY × N (fund unit purchases, per date) │
|
||
└────────────────────┬────────────────────┘
|
||
│
|
||
┌────────────────▼────────────────┐
|
||
│ Wealthfolio account │
|
||
│ type = WORKPLACE_PENSION │
|
||
│ currency = GBP │
|
||
└──────────────────────────────────┘
|
||
```
|
||
|
||
## One-time seed (Viktor)
|
||
|
||
```bash
|
||
# on your laptop (macOS / Linux with a desktop):
|
||
cd broker-sync
|
||
poetry install
|
||
poetry run playwright install chromium
|
||
poetry run broker-sync fidelity-seed --out /tmp/fidelity_storage_state.json
|
||
# chromium opens — log in to PlanViewer, tick "Remember device", press Enter
|
||
|
||
# stage to Vault
|
||
vault kv patch secret/broker-sync \
|
||
fidelity_storage_state=@/tmp/fidelity_storage_state.json \
|
||
fidelity_plan_id=<your-plan-id>
|
||
|
||
rm /tmp/fidelity_storage_state.json # don't leave credentials lying around
|
||
```
|
||
|
||
Re-seed when the monthly CronJob fails with `FidelitySessionError` (expect
|
||
every 30-90 days, depending on how long Fidelity honours the remember-device
|
||
cookie).
|
||
|
||
## One-time backfill
|
||
|
||
```bash
|
||
kubectl -n broker-sync create job fidelity-backfill \
|
||
--from=cronjob/broker-sync-fidelity
|
||
kubectl -n broker-sync logs -f job/fidelity-backfill
|
||
# expect: fidelity-ingest: fetched=N new=N imported=N failed=0
|
||
```
|
||
|
||
## Monthly cron
|
||
|
||
- Schedule: `0 3 5 * *` (3am UTC on the 5th of each month — after mid-month payroll settles in Viktor's scheme)
|
||
- CronJob: `broker-sync-fidelity` in namespace `broker-sync`
|
||
- Resource: small, ≤512 MiB memory (Chromium for ~2 min, then idle)
|
||
- Alert: `BrokerSyncFidelityFailed` fires on 2 consecutive failures
|
||
|
||
## Runbook — `BrokerSyncFidelityFailed`
|
||
|
||
1. Check pod logs: `kubectl -n broker-sync logs job/broker-sync-fidelity-<timestamp>`.
|
||
2. If the error is `FidelitySessionError`: session expired, re-run the seed on
|
||
Viktor's laptop (see above).
|
||
3. If the error is a 404 / 5xx from `prd.wiciam.fidelity.co.uk`: likely an API
|
||
path change. Check DevTools for the new endpoint, update the provider, ship
|
||
a new image.
|
||
4. If Playwright can't launch Chromium: check that the image still has Chromium
|
||
installed (`playwright install chromium` at build time).
|
||
|
||
## Data model notes
|
||
|
||
- **Salary sacrifice scheme**: all employee + employer contributions are
|
||
pre-tax from gross salary. No HMRC basic-rate relief line.
|
||
- Emits two `DEPOSIT` per month (employee, employer) with `comment` carrying
|
||
the source tag `fidelity:<doc-id>:<source>` for audit.
|
||
- Emits one `BUY` per fund unit purchase, `symbol` = Fidelity fund code / ISIN.
|
||
Units × unit price should reconcile to the cash deposited ±pennies.
|
||
|
||
## Not yet implemented
|
||
|
||
- Endpoint paths: waiting on Viktor's DevTools POST cURL for transactions +
|
||
holdings views. Until pasted, `fidelity-ingest` raises
|
||
`FidelityProviderConfigError` to fail loudly.
|
||
- Infra: CronJob + Vault secret wiring + Prometheus alert in
|
||
`infra/stacks/broker-sync/main.tf` — pending first successful manual run.
|