fidelity: replace snapshot-push with delta gains-offset DEPOSITs
Some checks failed
ci/woodpecker/push/build Pipeline failed
CI / test (push) Has been cancelled
CI / build (push) Has been cancelled
CI / deploy (push) Has been cancelled

Per-fund snapshot import landed quantities but dropped cost basis +
needed a separate quote-push path we never identified. Snapshotting
also collided with WF's own TOTAL aggregation and ZEROED the Fidelity
cash balance.

Simpler plan: each monthly scrape emits a single DEPOSIT (or
WITHDRAWAL on a market drop) sized to the delta between the live
PlanViewer pot value and Wealthfolio's running total. dav_corrected
PG view continues to subtract these offsets from net_contribution so
the dashboard Growth/ROI math stays right.

- New gains_offset_delta_activity() — current_gain - prior_offset.
- New WealthfolioSink.cumulative_amount_with_notes_prefix() — sums
  the existing fidelity-planviewer:unrealised-gains-offset DEPOSITs
  in WF so we know what's already been emitted.
- CLI runs sync_provider_to_wealthfolio first (cash flows), then
  computes + emits the delta via import_activities.
- 4 new provider tests for the delta logic; full suite (144 + 1
  skipped) green; mypy + ruff clean.

The old fidelity_holdings_to_snapshot helper + push_manual_snapshots
sink method stay for future use but are no longer called.
This commit is contained in:
Viktor Barzin 2026-05-17 00:35:17 +00:00
parent c9c0310733
commit 98c4729622
4 changed files with 183 additions and 35 deletions

View file

@ -103,10 +103,10 @@ class FidelityPlanViewerProvider:
storage_state, navigates to the transaction-history page with a wide
date range, scrapes the table, and intercepts the valuation XHR.
- After ``fetch()`` completes, ``last_holdings`` holds the per-fund
unit positions captured in the same scrape used by the
``fidelity-ingest`` CLI to push a manual snapshot to Wealthfolio
so per-fund quantities + cost basis land in the Positions table
(the activity stream alone only carries cash flows).
unit positions and ``last_total_contribution`` the cumulative cash
contribution used by the ``fidelity-ingest`` CLI to emit a
delta-shaped DEPOSIT that nudges WF's net worth to match the
PlanViewer reported pot value (see ``gains_offset_delta_activity``).
"""
name = "fidelity-planviewer"
@ -162,12 +162,53 @@ class FidelityPlanViewerProvider:
if before is not None and tx.date >= before:
continue
yield _tx_to_activity(tx)
# NB: the gains-offset DEPOSIT we used to emit here is superseded
# by the manual snapshot push the CLI does after fetch() drains.
# The snapshot sets per-fund quantity + cost basis directly, so
# Wealthfolio computes growth from positions instead of needing a
# fake cash entry. Old offset rows still in WF are corrected at
# the dashboard layer by the dav_corrected view.
# Gains-offset DEPOSITs are emitted by the CLI (which has the
# prior cumulative offset from WF). See `gains_offset_delta_activity`.
def gains_offset_delta_activity(
holdings: list[FidelityHolding],
total_real_contribution: Decimal,
prior_offset_cumulative: Decimal,
as_of: datetime,
min_delta: Decimal = Decimal("0.5"),
) -> Activity | None:
"""Compute the gains-offset DELTA since the last scrape and shape it
as a DEPOSIT (or WITHDRAWAL on a market drop).
The pension's per-fund prices aren't trackable in WF directly (no
public quote feed for these institutional life-fund share classes).
Instead, each monthly scrape emits a single small DEPOSIT/WITHDRAWAL
sized to ``(current_pot - real_contributions) - prior_cumulative_offset``
i.e., the growth (or loss) accrued since the last run.
Wealthfolio's net_contribution then incorrectly includes all these
offsets; the ``dav_corrected`` PG view subtracts them back out so the
dashboard's Growth/ROI panels remain accurate. The deterministic
external_id (per scrape date) lets re-runs of the same day overwrite
rather than stack duplicates.
"""
if not holdings:
return None
current_pot = sum((h.total_value for h in holdings), Decimal(0))
current_gain = current_pot - total_real_contribution
delta = current_gain - prior_offset_cumulative
if abs(delta) < min_delta:
return None
return Activity(
external_id=f"fidelity:gains-delta:{as_of.date().isoformat()}",
account_id=ACCOUNT_ID,
account_type=AccountType.WORKPLACE_PENSION,
date=as_of,
activity_type=ActivityType.DEPOSIT if delta > 0 else ActivityType.WITHDRAWAL,
currency=_CCY,
amount=abs(delta),
notes=(
f"fidelity-planviewer:unrealised-gains-offset delta=£{delta} "
f"(pot=£{current_pot}, contrib=£{total_real_contribution}, "
f"prior_offset=£{prior_offset_cumulative})"
),
)
def fidelity_holdings_to_snapshot(