sinks: detect silent Wealthfolio /import drops

After the check step returns isValid=true + no errors, a row can still
be silently dropped by /import (response returns activities=[] on
200 OK). Root-cause is usually a field that check hydrates but
/import re-normalises differently (date string form, asset_id resolution).

When we send N valid rows and get back 0, raise ImportValidationError
with a snippet of the check output + first warning — gives the
operator a concrete hint to fix the producer instead of silently
growing dedup against activities that never landed.

poetry run pytest -q   →  109 passed, 1 skipped
poetry run mypy        →  clean
poetry run ruff check  →  clean
This commit is contained in:
Viktor Barzin 2026-04-17 22:24:36 +00:00
parent 6efd03570a
commit 4e2da87637

View file

@ -224,7 +224,24 @@ class WealthfolioSink:
if isinstance(raw, dict) and "activities" in raw:
got = raw["activities"]
assert isinstance(got, list)
return got
if isinstance(raw, list):
return raw
return []
elif isinstance(raw, list):
got = raw
else:
got = []
# Silent-drop detection: if we sent N valid rows but got 0 back, something
# is silently rejecting them (usually a date-format or asset-resolution
# quirk that check() didn't catch). Raise so the pipeline records failure
# instead of marking the rows as synced when they never landed.
if valid_rows and not got:
# Also surface any per-row `errors` or `warnings` from the check step
# — those are often the best hint about why /import dropped them.
first_warn = next(
(r.get("warnings") for r in checked if isinstance(r, dict) and r.get("warnings")),
None,
)
raise ImportValidationError(
f"Wealthfolio /import silently dropped all {len(valid_rows)} rows. "
f"First checked row: {checked[0] if checked else 'none'}. "
f"First warning (if any): {first_warn}"
)
return got