Context: InvestEngine has no public API. The web app uses an undocumented
Django REST backend at /api/v0.3X/*, which requires a Bearer token and
rolls its minor every 4-6 weeks. MFA (push-approval) is mandatory on
every login, so we do NOT automate login — Viktor logs in manually in
a browser, copies the Bearer out of devtools, and pastes it into Vault.
This provider consumes that token.
The response shape is UNVERIFIED (MFA blocks an unauthed probe, so the
research leading into Phase 2b could only confirm endpoint existence via
401 responses on v0.31 and v0.32). `_transaction_to_activity` is written
defensively:
- accepts both `results`/`data` list wrappers and `next`/`meta.next_page`
cursor fields for pagination;
- accepts `symbol`/`ticker`, `price`/`unit_price`, `amount`/`value`,
`date`/`created_at`/`timestamp` field-name variants;
- maps exact type strings (BUY, SELL, DIVIDEND, INTEREST, DEPOSIT,
WITHDRAWAL, FEE, TAX) and substring-matches DEPOSIT/WITHDRAWAL for
variants like "CASH_DEPOSIT"; refuses to guess on anything else —
unknown types log WARNING and return None (silent misclassification
would corrupt tax reporting).
Version probe:
_START_VERSION_MINOR=32 (research: v0.31/v0.32 live, v0.30 Gone)
GET /api/v0.{n}/ → 410 ? advance : done
cap at v0.60 so a misconfigured backend doesn't infinite-loop.
A 410 response on a data endpoint triggers exactly one re-probe + retry
against the newer version; the new version is cached on the instance for
the rest of the process.
Token expiry is tracked at the Python layer:
- constructor takes token_expires_at (set by Viktor when he pastes);
- fetch() fails fast with InvestEngineTokenExpiredError if the clock
says the token is already dead — cheaper than burning a request for
a known 401;
- a real 401 response also raises InvestEngineTokenExpiredError so the
CLI/pipeline can alert Viktor to paste a new token.
Vault schema expected (consumed by the CLI in the follow-up commit):
secret/broker-sync
investengine_bearer_token <devtools-captured Bearer>
investengine_token_expires_at <ISO-8601 set at paste time>
investengine_refresh_token <optional, not used yet>
This module does NOT read Vault — the caller hands values in, keeping
the provider testable.
This change:
- New `broker_sync/providers/invest_engine.py`:
* InvestEngineProvider with .accounts(), .fetch(), .close()
* _probe_version / _active_version with 410-retry + cache
* _transaction_to_activity with defensive type + field-name mapping
* InvestEngineError / InvestEngineTokenExpiredError / InvestEngineVersionError
- New `tests/providers/test_invest_engine.py`: 22 tests covering version
probe, expiry fail-fast, 401→TokenExpired, 410→reprobe, header
shape, pagination variants, and the full txn→activity mapping. One
@pytest.mark.skip integration stub for when Viktor has a live token.
Assumptions flagged for verification with a live token:
- IE id field is castable to str (int or string)
- Type strings match or fuzz-contain: BUY, SELL, DIVIDEND, INTEREST,
DEPOSIT, WITHDRAWAL, FEE, TAX
- Transactions carry numeric quantity/price/amount (Decimal-convertible)
- Date field is one of: date / created_at / timestamp
- Pagination shape is {results, next} OR {data, meta.next_page}
- /transactions/ accepts ?portfolio=<id>&start=YYYY-MM-DD&end=YYYY-MM-DD
## Automated
poetry run pytest tests/providers/test_invest_engine.py -v
======================== 22 passed, 1 skipped in 0.26s =========================
poetry run pytest -q
95 passed, 1 skipped in 0.84s
poetry run mypy --strict .
Success: no issues found in 34 source files
poetry run ruff check .
All checks passed!
poetry run yapf --diff broker_sync/providers/invest_engine.py tests/providers/test_invest_engine.py
(clean)
## Manual Verification
Once Viktor pastes a live token:
1. Export:
export IE_BEARER_TOKEN='<paste>'
export IE_TOKEN_EXPIRES_AT='2026-05-17T00:00:00+00:00'
2. Unmark the @pytest.mark.skip on test_live_integration_smoke
3. poetry run pytest tests/providers/test_invest_engine.py::test_live_integration_smoke -v
Expected: a successful round-trip that returns an empty-or-populated
list of Activity objects — prove the version probe + auth header +
portfolio enumeration actually work against the real IE backend.
4. Validate the Assumptions list above against the real transaction JSON.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dc4d3f889d