fire-planner: lazy-refresh /networth from wf_sync (default TTL 1d)
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
The account_snapshot cache fed /networth, /networth/history, and
/scenarios/{id}/progress. No CronJob populated it, so the cache had
drifted ~18 days behind the wealthfolio_sync mirror (last refresh
2026-05-09 via manual kubectl exec; Grafana reads wf_sync directly
and stayed fresh).
Switch to lazy refresh on read: each request to those endpoints now
checks MAX(account_snapshot.snapshot_date) — if it's older than
NETWORTH_CACHE_TTL_DAYS (default 1), pull fresh rows from wf_sync via
read_account_snapshots_from_pg and upsert. Idempotent under
concurrency (existing ON CONFLICT DO UPDATE).
Plumbing:
- Add get_wf_sync_session dependency that yields None when the wf_sync
factory isn't wired (keeps existing tests' behaviour: no refresh
attempted, they continue to seed account_snapshot directly).
- Wire wf_sync engine + session_factory in app.lifespan when
WEALTHFOLIO_SYNC_DB_CONNECTION_STRING is set.
- Centralise the staleness check in refresh_account_snapshots_if_stale.
Tests:
- 271 existing tests still green.
- Three new tests in test_api_networth_refresh.py covering: empty cache
triggers refresh, stale cache triggers refresh, fresh cache skips
refresh (asserts the wf_sync value is NOT served).
This commit is contained in:
parent
e72fd22a17
commit
4da58fe56e
6 changed files with 317 additions and 9 deletions
|
|
@ -53,6 +53,7 @@ from fire_planner.api.spending import router as spending_router
|
|||
from fire_planner.api.spending_profile import router as spending_profile_router
|
||||
from fire_planner.api.year_stats import router as year_stats_router
|
||||
from fire_planner.db import create_engine_from_env, make_session_factory
|
||||
from fire_planner.ingest.wealthfolio_pg import create_wf_sync_engine_from_env
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
|
@ -77,6 +78,17 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
|||
# Tests inject these via dependency_overrides; nothing to wire.
|
||||
log.warning("DB_CONNECTION_STRING unset; skipping engine init")
|
||||
|
||||
# wealthfolio_sync engine — powers the lazy refresh of /networth's
|
||||
# account_snapshot cache. Optional: when unset we serve whatever is
|
||||
# already cached in account_snapshot (still useful for tests + dev).
|
||||
if os.environ.get("WEALTHFOLIO_SYNC_DB_CONNECTION_STRING"):
|
||||
wf_engine = create_wf_sync_engine_from_env()
|
||||
app.state.wf_sync_engine = wf_engine
|
||||
app.state.wf_sync_session_factory = make_session_factory(wf_engine)
|
||||
else:
|
||||
log.info("WEALTHFOLIO_SYNC_DB_CONNECTION_STRING unset; "
|
||||
"lazy refresh of /networth will no-op")
|
||||
|
||||
worker = asyncio.create_task(_drain_queue(app))
|
||||
app.state._worker = worker
|
||||
try:
|
||||
|
|
@ -88,6 +100,9 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
|||
eng = getattr(app.state, "engine", None)
|
||||
if eng is not None:
|
||||
await eng.dispose()
|
||||
wf_eng = getattr(app.state, "wf_sync_engine", None)
|
||||
if wf_eng is not None:
|
||||
await wf_eng.dispose()
|
||||
|
||||
|
||||
async def _drain_queue(app: FastAPI) -> None:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue