broker-sync/broker_sync
Viktor Barzin 0d23487608
Some checks failed
ci/woodpecker/push/build Pipeline was successful
CI / test (push) Has been cancelled
CI / build (push) Has been cancelled
CI / deploy (push) Has been cancelled
imap: skip InvestEngine by default; opt back in via INCLUDE env
Post-mortem 2026-05-27: 39 IMAP-source IE BUYs + their cash-flow
DEPOSITs were re-inserted into Wealthfolio at 09:22:18 UTC, exactly
the rows the £252k dedup removed the previous day. The cron's
BROKER_SYNC_IMAP_EXCLUDE_PROVIDERS=invest-engine env var did its job
(cron logged ie_skipped=53), but some other entry point — kubectl run,
poetry run on the devvm, or a sibling agent session — ran the IMAP
ingest WITHOUT that env. The opt-out was a foot-gun.

This change makes the IE-via-IMAP safety STRUCTURAL: `invest-engine`
is in the default exclude set inside _resolve_excluded_providers().
Any code path now skips IE unless the caller explicitly sets
`BROKER_SYNC_IMAP_INCLUDE_PROVIDERS=invest-engine`. The
`BROKER_SYNC_IMAP_EXCLUDE_PROVIDERS` env still works (additive) for
forward-compat in case Schwab etc. ever need similar treatment.

INCLUDE wins over both the default exclude set and EXCLUDE env.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 17:24:54 +00:00
..
providers imap: skip InvestEngine by default; opt back in via INCLUDE env 2026-05-27 17:24:54 +00:00
sinks sinks/wealthfolio: treat duplicates as success in import-summary check 2026-05-26 22:52:11 +00:00
__init__.py Initial scaffold + canonical Activity model 2026-04-17 19:16:11 +00:00
cli.py ibkr: emit ibkr_cash_balance{currency, account} per CashReport row 2026-05-27 12:03:41 +00:00
dedup.py Add SyncRecordStore for authoritative dedup 2026-04-17 19:17:12 +00:00
fx.py Add FxCache and convert_to_gbp core 2026-04-17 19:18:41 +00:00
fx_ecb.py Add ECB FX fetcher + cache population 2026-04-17 19:32:23 +00:00
metrics.py metrics: add Pushgateway client for broker-sync providers 2026-05-26 22:24:55 +00:00
models.py schwab: detect vest-confirmation emails + emit VestEvent 2026-04-19 18:27:58 +00:00
normaliser.py Add Provider protocol and normaliser 2026-04-17 19:20:12 +00:00
pipeline.py pipeline: emit matching DEPOSIT/WITHDRAWAL for every BUY/SELL 2026-04-18 19:12:49 +00:00