From 2fb1fbbdd8b38e9fe01ff4121786b9580052efb1 Mon Sep 17 00:00:00 2001 From: Viktor Barzin Date: Tue, 26 May 2026 22:34:46 +0000 Subject: [PATCH] docs: add IBKR provider runbook --- docs/providers/ibkr.md | 124 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 124 insertions(+) create mode 100644 docs/providers/ibkr.md diff --git a/docs/providers/ibkr.md b/docs/providers/ibkr.md new file mode 100644 index 0000000..a21df5b --- /dev/null +++ b/docs/providers/ibkr.md @@ -0,0 +1,124 @@ +# Provider: Interactive Brokers (IBKR Flex Web Service) + +Pulls a daily Activity Flex Query via the [`ibflex`](https://github.com/csingley/ibflex) +library, maps Trades + CashTransactions to broker-sync Activities, and +reconciles broker-side OpenPositions against WF-computed quantities. + +## When this runs + +- K8s CronJob `broker-sync-ibkr` in the `broker-sync` namespace, daily 02:00 UK. +- Manual trigger: + ```bash + kubectl -n broker-sync create job --from=cronjob/broker-sync-ibkr broker-sync-ibkr-manual-$(date +%s) + ``` + +## Vault secrets — `secret/broker-sync` + +| Key | Description | +|---|---| +| `ibkr_flex_token` | Flex Web Service token (1-year validity, rotate via IBKR Client Portal). | +| `ibkr_flex_query_id` | Activity Flex Query ID (5–7 digit number). | +| `ibkr_account_id` | Wealthfolio account UUID for "Interactive Brokers (UK)". | +| `ibkr_account_id_upstream` | IBKR-side account number (e.g. `U12345678`) — guards against wrong-account ingestion. | + +ExternalSecret `broker-sync-secrets` syncs all keys from `secret/broker-sync` +to a K8s secret of the same name. New keys take ~15 min to propagate. + +## IBKR Flex Query design + +In IBKR Client Portal → Reports → Flex Queries → Activity Flex Query, create +a new query named `broker-sync-activity` with: + +| Section | Required fields | +|---|---| +| Account Information | accountId | +| Trades | tradeID, tradeDate, tradeTime, symbol, buySell, quantity, tradePrice, currency, ibCommission, assetCategory, exchange | +| Cash Transactions | transactionID, dateTime, type, amount, currency, description | +| Open Positions | symbol, position, markPrice, currency, assetCategory, exchange | +| Securities Information | symbol, description, conid | + +**Date Format:** `yyyy-MM-dd`. **Time Format:** `HH:mm:ss` (no timezone +suffix — ibflex 1.1 rejects timezone abbreviations in the time field). +**Date Range:** `Last Business Day` for daily incremental. Switch to +`Year to Date` only for one-off backfills. + +## Cash type mapping + +| IBKR Flex `CashTransaction.type` | broker-sync `ActivityType` | +|---|---| +| Dividends | DIVIDEND | +| Withholding Tax | TAX | +| Broker Interest Received | INTEREST | +| Broker Interest Paid | FEE | +| Commission Adjustments | FEE | +| Other Fees | FEE | +| Deposits & Withdrawals | DEPOSIT (amount > 0) / WITHDRAWAL (amount < 0) | +| anything else | skipped + WARNING logged (refuse to guess) | + +## Dedup keys + +- Trades: `external_id = "ibkr:trade:" + tradeID` +- Cash: `external_id = "ibkr:cash:" + transactionID` + +Both are stable across re-runs; `dedup.SyncRecordStore` rejects already- +synced IDs. + +## Symbol canonicalisation + +LSE-listed GBP instruments get a `.L` suffix (Wealthfolio convention). +US instruments and anything already suffixed pass through unchanged. + +The heuristic: `exchange in {LSE, LSEETF, LSEIOB1}` OR +`(exchange is None AND currency == GBP)` → suffix with `.L`. Edge cases +not yet covered (Euronext, XETRA) — extend `canonical_symbol` when those +holdings exist. + +## Position reconciliation + +Each run pushes to Pushgateway under job `broker-sync-ibkr`: +- `ibkr_position_drift_shares{symbol, account="ibkr-uk"}` — + broker_qty − wf_qty per asset. +- `ibkr_sync_last_success_timestamp_seconds` — unix timestamp. + +Alerts (TODO, will be added to the monitoring stack on first +non-zero drift): +- `IBKRPositionDrift{symbol}` — `|drift| > 0.01` for >24h, Slack `#security`. +- `IBKRSyncStale` — timestamp > 36h old. +- `IBKRFlexTokenExpired` — Loki rule on the "code 1003" log line. + +## Account guard + +Before yielding any activities, the provider checks +`flex.accountId == IBKR_ACCOUNT_ID_UPSTREAM`. Mismatch → raises +`IBKRAccountMismatchError` and writes nothing. Prevents wrong-account +ingestion from a misconfigured query (e.g., someone replaced the token +with another user's by mistake). + +## Token rotation + +Flex tokens expire after 1 year. When the cron starts failing with +`ResponseCodeError(code=1003)`: + +1. Sign in to IBKR Client Portal → Reports → Settings → Flex Web Service + → regenerate token. +2. `vault kv patch secret/broker-sync ibkr_flex_token=''` +3. ExternalSecrets controller picks up the new value within ~15 min; no + manual pod restart needed. + +## Troubleshooting + +| Symptom | Likely cause | Fix | +|---|---|---| +| `IBKR_FLEX_TOKEN not provided` exit 2 | Vault has placeholder value or key missing | `vault kv patch secret/broker-sync ibkr_flex_token=''` | +| `IBKRAccountMismatchError` | `ibkr_account_id_upstream` doesn't match the account in the Flex query | Re-check IBKR account number; fix the Vault value | +| `ResponseCodeError(code=1003)` | Flex token expired | See "Token rotation" above | +| `StatementGenerationTimeout` | IBKR side slow | Single retry built in; if it persists, try a smaller date range | +| `Can't convert '... TZ' to time` parser error | Flex query has Time Format with timezone suffix | Switch to `HH:mm:ss` (no TZ) in Flex query settings | +| `'ETF' is not a valid AssetClass` | ETF set in fixture not in ibflex enum | Use `STK` in fixtures (IBKR Flex categorises ETFs under STK) | + +## References + +- Spec: [`docs/specs/2026-05-26-ibkr-ingest-design.md`](../specs/2026-05-26-ibkr-ingest-design.md) +- Plan: [`docs/plans/2026-05-26-ibkr-flex-ingestion.md`](../plans/2026-05-26-ibkr-flex-ingestion.md) +- Library: +- IBKR Flex Web Service docs: