viktor/broker-sync
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
broker-sync/tests/fixtures/invest_engine/csv_attachment.eml

23 lines
616 B
Text
Raw Normal View History

Add CSV attachment fallback for InvestEngine email parser Context: IE has not (yet) sent CSV-attached statements in production, but the upstream parser had _extract_positions_csv as a third fallback for exactly this case. Keeping the fallback preserves behaviour-parity with the legacy parser and makes future statement support one fixture away — the shape is documented by column set, not scraped live. Unlike the upstream which split the body on whitespace and broke on any embedded commas in names, this port walks real MIME attachments using Python's csv.DictReader. A part qualifies as CSV if: - its Content-Type is text/csv / application/csv / application/vnd.ms-excel, OR - its filename ends in .csv (defence against IE mis-labelling the part) Rows missing required columns or containing unparseable numbers/dates are skipped silently — consistent with the "partial match" contract: a half-corrupt CSV yields whatever rows were intact. Required columns: ticker, unit_price, quantity, date (YYYY-MM-DD), currency. Non-GBP rows are filtered because the IE ISA is strictly sterling — flagging this assumption in the review notes. This change: - Adds `_parse_csv_attachment(raw_email)` as the third strategy after text/plain and text/html; it re-parses the raw email bytes so we can inspect Content-Type/filename on each part. - Flags symbols/currencies, filters non-GBP, and runs each row through the shared `_build_activity` so external_id formation matches every other strategy (dedup stays consistent across strategies). - Fixture `csv_attachment.eml` has three rows (VUAG, SWDA, VUSA) in a `text/csv` part with a `.csv` filename — covers both detection paths. Test plan: poetry run pytest tests/providers/parsers/ -q → 6 passed in 0.15s poetry run mypy broker_sync/providers/parsers/invest_engine.py tests/providers/parsers/test_invest_engine.py → clean poetry run ruff check broker_sync/providers/parsers/invest_engine.py tests/providers/parsers/test_invest_engine.py → All checks passed! poetry run yapf --diff → clean (no diff) Manual verification: load csv_attachment.eml, call parse_invest_engine_email, assert 3 activities each with symbol in {VUAG,SWDA,VUSA}, currency=GBP, notes containing "csv".
2026-04-17 22:01:46 +00:00
From: InvestEngine <no-reply@investengine.com>
To: viktorbarzin@example.com
Subject: Your InvestEngine statement
Date: Mon, 07 Apr 2025 09:00:00 +0000
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----=_MIXED_1"
------=_MIXED_1
Content-Type: text/plain; charset=UTF-8
Your monthly statement is attached as a CSV.
------=_MIXED_1
Content-Type: text/csv; charset=UTF-8; name="statement.csv"
Content-Disposition: attachment; filename="statement.csv"
ticker,unit_price,quantity,date,currency
VUAG,63.21,12.5,2025-04-02,GBP
SWDA,86.40,4.75,2025-04-03,GBP
VUSA,90.10,1.0,2025-04-04,GBP
------=_MIXED_1--
Reference in a new issue Copy permalink