fidelity-planviewer: scaffold provider + CLI (seed + stub ingest)

## Context

UK workplace pension at planviewer.fidelity.co.uk has no public API; the SPA
calls a private JSON backend at prd.wiciam.fidelity.co.uk/cvmfe/api/*. Viktor
confirmed in DevTools that an OPTIONS preflight lists auth headers
(ch, fid, rid, sid, tbid, theosreferer, ua). Full reverse-engineering of the
endpoint paths is pending Viktor's POST cURL paste for transactions +
holdings views.

Until those endpoints are captured, ship the scaffold: provider module, CLI
commands, tests, docs. This unblocks installing Playwright in the image and
lets Viktor run the one-off seed command on his laptop ahead of the data
integration.

## This change

- broker_sync/providers/fidelity_planviewer.py
  - FidelityCreds namedtuple (storage_state_path, plan_id).
  - FidelitySessionError (401 → re-seed), FidelityProviderConfigError.
  - FidelityPlanViewerProvider: .accounts() returns a single
    WORKPLACE_PENSION account, .fetch() raises until endpoints are wired.
- broker_sync/cli.py
  - fidelity-seed: launches headed Chromium so Viktor can log in and tick
    "Remember device", then dumps storage_state.json.
  - fidelity-ingest: stub matching the invest-engine / trading212 CLI
    shape; reads storage_state + plan_id, pipes through the shared pipeline.
- tests/providers/test_fidelity_planviewer.py
  - Asserts the single-account shape + the loud-failure guard.
- docs/providers/fidelity-planviewer.md
  - Architecture diagram, one-time seed procedure, backfill + monthly
    commands, alert runbook.
- pyproject.toml
  - playwright ^1.47 as a first-class dep (used only by fidelity-seed and
    later by the session-refresh step in fidelity-ingest).

## What is NOT in this change

- Endpoint wiring in provider.fetch() — blocked on DevTools POST cURL.
- Infra CronJob + Vault secret + Prometheus alert — lands once the first
  manual backfill succeeds and we know the Chromium image size is fine.
- Dockerfile Chromium install — same trigger.

## Verification

### Automated

$ poetry run pytest tests/providers/test_fidelity_planviewer.py -v
2 passed in 0.08s

$ poetry run pytest -q
122 passed, 1 skipped in 1.07s

$ poetry run mypy broker_sync/providers/fidelity_planviewer.py broker_sync/cli.py
Success: no issues found in 2 source files

$ poetry run ruff check broker_sync/providers/fidelity_planviewer.py broker_sync/cli.py tests/providers/test_fidelity_planviewer.py
All checks passed!

### Manual (Viktor, later)

1. poetry install && poetry run playwright install chromium
2. poetry run broker-sync fidelity-seed --out /tmp/state.json
3. Chromium opens → log in → tick "Remember device" → press Enter
4. vault kv patch secret/broker-sync fidelity_storage_state=@/tmp/state.json

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Viktor Barzin 2026-04-18 14:09:04 +00:00
parent c830856ba1
commit 832732a419
6 changed files with 508 additions and 1 deletions

View file

@ -358,6 +358,115 @@ def imap_ingest(
asyncio.run(_run())
@app.command("fidelity-seed")
def fidelity_seed(
out: str = typer.Option(
"fidelity_storage_state.json",
help="Where to write the storage_state JSON (stage it to Vault afterwards)",
),
url: str = typer.Option(
"https://pv.planviewer.fidelity.co.uk/",
help="PlanViewer SPA URL — defaults to the production UK landing",
),
) -> None:
"""One-off: launch a headed Chromium so Viktor can log into PlanViewer and
capture a long-lived storage_state (cookies + localStorage) for the monthly
cron.
Expected flow:
1. Chromium opens on the PlanViewer login page.
2. Viktor enters username, password, memorable word, MFA code.
3. Viktor ticks "Remember device" / "Trust this browser" if offered.
4. Viktor waits until the dashboard loads, then presses Enter in the terminal.
5. Script dumps storage_state.json and exits.
6. Viktor runs ``vault kv patch secret/broker-sync fidelity_storage_state=@...``.
"""
_setup_logging()
try:
from playwright.sync_api import sync_playwright
except ImportError as e:
typer.echo(
"Playwright is not installed — run `poetry install` first.", err=True)
raise typer.Exit(code=2) from e
typer.echo(f"Opening {url} in a headed browser — log in, tick "
"'Remember device' if offered, then press Enter here.")
with sync_playwright() as pw:
browser = pw.chromium.launch(headless=False)
context = browser.new_context()
page = context.new_page()
page.goto(url)
input("Press Enter once you're fully logged in and the dashboard is visible… ")
context.storage_state(path=out)
browser.close()
typer.echo(f"Wrote {out} — stage it to Vault:")
typer.echo(f" vault kv patch secret/broker-sync fidelity_storage_state=@{out}")
@app.command("fidelity-ingest")
def fidelity_ingest(
wf_base_url: str = typer.Option(..., envvar="WF_BASE_URL"),
wf_username: str = typer.Option(..., envvar="WF_USERNAME"),
wf_password: str = typer.Option(..., envvar="WF_PASSWORD"),
wf_session_path: str = typer.Option("/data/wealthfolio_session.json", envvar="WF_SESSION_PATH"),
storage_state_path: str = typer.Option(
...,
envvar="FIDELITY_STORAGE_STATE_PATH",
help="Path on disk to storage_state.json (materialised from Vault by the init container)",
),
plan_id: str = typer.Option(..., envvar="FIDELITY_PLAN_ID"),
data_dir: str = typer.Option("/data", envvar="BROKER_SYNC_DATA_DIR"),
mode: str = typer.Option("steady", help="steady = last-60-days; backfill = full history"),
) -> None:
"""Sync Fidelity UK PlanViewer contributions + fund purchases into Wealthfolio."""
from broker_sync.dedup import SyncRecordStore
from broker_sync.pipeline import sync_provider_to_wealthfolio
from broker_sync.providers.fidelity_planviewer import (
FidelityCreds,
FidelityPlanViewerProvider,
)
from broker_sync.sinks.wealthfolio import WealthfolioSink
_setup_logging()
if mode == "steady":
since: datetime | None = datetime.now(UTC) - timedelta(days=60)
elif mode == "backfill":
since = None
else:
typer.echo(f"Unknown mode: {mode!r}. Use 'steady' or 'backfill'.", err=True)
sys.exit(2)
async def _run() -> None:
sink = WealthfolioSink(
base_url=wf_base_url,
username=wf_username,
password=wf_password,
session_path=wf_session_path,
)
provider = FidelityPlanViewerProvider(FidelityCreds(
storage_state_path=storage_state_path,
plan_id=plan_id,
))
dedup = SyncRecordStore(Path(data_dir) / "sync.db")
try:
if not Path(wf_session_path).exists():
await sink.login()
result = await sync_provider_to_wealthfolio(
provider=provider, sink=sink, dedup=dedup, since=since,
)
finally:
await sink.close()
typer.echo(f"fidelity-ingest: fetched={result.fetched} "
f"new={result.new_after_dedup} "
f"imported={result.imported} "
f"failed={result.failed}")
if result.failed > 0:
sys.exit(1)
asyncio.run(_run())
def _setup_logging() -> None:
logging.basicConfig(
level=logging.INFO,