`compute_aggregate_snapshot` returns ORM rows that were created inside
a `with Session(engine)` block — by the time the Celery task tries to
serialise their attributes into the result dict the session has closed,
triggering SQLAlchemy's DetachedInstanceError. Combined with acks_late
this caused the task to be redelivered repeatedly (4× in the first
manual trigger).
Fix: drop the per-row dict-serialisation in the task return — keep just
`aggregates_written: int`. The per-band stats are already logged by the
aggregator's own info-level lines, so no observability is lost.
Caught when manually firing the task on prod to seed today's snapshot
before the 04:00 UTC daily fire. Aggregator itself ran fine (the rows
were written before the session closed); only the post-return access
was broken.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
CI test-unit failed on pipeline #49 because the three TestSetupPeriodicTasks
cases asserted exact call counts on `sender.add_periodic_task` and the
new unconditional `daily-market-aggregator` registration bumped each by
one. Fix:
- `tasks/listing_tasks.py`: lifted the market-aggregator registration
out of the SchedulesConfig try-block — it's now independent of the
user's SCRAPE_SCHEDULES (a malformed scrape config no longer takes
the aggregator down with it).
- `tests/unit/test_listing_tasks.py`: updated the four cases to account
for the +1 unconditional aggregator call. `test_handles_config_error_
gracefully` now asserts the aggregator still registers when
SchedulesConfig.from_env raises (regression coverage for the
independence guarantee).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two surfaces wired up so the user can "get a vibe of the market":
**Per-listing** — each PropertyCard now shows a small pill next to the
price when the listing's total_price moved >=1% over a 14-day lookback
(e.g. "↓ £200 (-4%) in 14d"). Drops render green, rises render red.
Computed from `price_history_json` by the daily aggregator and
denormalised onto the listing row so the streaming endpoint just
passes it through.
**Macro** — new always-visible inline strip above the chip strip
showing today's median total price, median £/m², and listing count
for the current filter's bedroom band, each with a 30-day % delta:
"Rent · 1-2 bed · 30d: Median £2,500 ↓ -4% · £/m² £50 ↓ -2% · Listings 4,200 ↑ +5%".
Both data sources are populated daily at 04:00 UTC by a new Celery
beat task that fires 1h after the 03:00 RENT scrape and feeds two
sinks: a per-listing update pass and an upsert to a new
`dailylistingaggregate` table keyed on
(snapshot_date, listing_type, min_bedrooms, max_bedrooms).
## Backend
- `models/listing.py`: Listing parent gains `price_14d_ago` + `price_
change_pct_14d` nullable floats (inherited by RentListing/BuyListing).
New `DailyListingAggregate` table model with unique constraint on
(date, type, min_bed, max_bed).
- Alembic `a8b9c0d1e2f3`: adds the two columns to both listing tables
and creates the aggregate table + date index.
- `services/market_aggregator.py` (new): `compute_trend_for_listing`,
`update_per_listing_trend` (batched, idempotent), `_stats` (median
+ mean filtered to positive finite values), `compute_aggregate_
snapshot` (dialect-aware MySQL / SQLite upsert), `fetch_trend_
series` (range query for the API).
- `tasks/market_tasks.py` (new): `compute_daily_market_aggregates_task`
Celery task wrapping both stages.
- `tasks/listing_tasks.py:setup_periodic_tasks`: registers the daily
task at 04:00 UTC alongside the existing scrape schedules.
- `celery_app.py`: includes the new tasks module.
- `api/app.py`: new `GET /api/market_trend?listing_type=&min_bedrooms=&
max_bedrooms=&days=` endpoint returning the daily series.
- `ui_exporter.py`: GeoJSON feature properties now carry
`price_14d_ago` and `price_change_pct_14d` so the frontend can
render the badge without an extra round-trip.
## Frontend
- `types/index.ts`: new `MarketTrendPoint`; `PropertyProperties` gains
the two optional trend fields.
- `components/PropertyCard.tsx`: derived `trendBadge` (>=1% threshold,
null-safe) rendered as a small pill on both card variants.
- `hooks/useMarketTrend.ts` (new): fetches the trend series, derives
current-vs-oldest deltas per metric (% change rounded to 1dp).
- `components/MarketTrendStrip.tsx` (new): compact inline strip with
three metric cells. Hidden when the aggregator hasn't produced any
rows yet (graceful start during the first week post-launch).
- `App.tsx`: renders the strip above the chip strip whenever the
active queryParameters are known.
## Tests
- pytest: 10 new (trend math edge cases including null history,
malformed JSON, only-recent entries, drops, rises, zero current
price; _stats empty / nonpositive filtering; upsert idempotency on
an in-memory SQLite seed). 34 decision + aggregator tests pass.
- vitest: 8 new (useMarketTrend fetch URL, two-point delta,
single-point null delta, empty series; PropertyCard trend badge
arrow direction + sign for drops/rises, noise threshold, null
guard). 229 tests pass total, tsc clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Opening a listing's detail sheet for >3s now passively marks it `seen`
and snapshots its current `total_price`. Seen listings are hidden from
the main list by default but automatically resurface when the price
changes (any direction). Distinct from `disliked` — explicit
like/dislike always overrides the passive seen state.
New "Hidden (N)" toggle on the FilterBar appears whenever at least one
listing is currently being hidden by the seen filter. Toggling it
reveals those rows in-place (without unmarking them) so the user can
review or explicitly clear via the existing Like/Dislike/Clear flow.
## Backend
- Alembic f7a8b9c0d1e2: `ALTER TABLE listingdecision ADD COLUMN
price_at_decision FLOAT NULL`.
- `models/decision.py`: ListingDecision gains nullable
`price_at_decision: float | None`.
- `services/decision_service.py`: adds `seen` to VALID_DECISIONS;
set_decision accepts an optional `price_at_decision`; it's only
forwarded to the repo for decision='seen' (other types null-out the
column to keep semantics clean).
- `repositories/decision_repository.py`: upsert_decision now carries
price_at_decision through the MySQL + SQLite upsert paths.
- `api/decision_routes.py`: SetDecisionRequest + DecisionResponse
expose the new field.
## Frontend
- `types/index.ts`: DecisionType = 'liked' | 'disliked' | 'seen';
ListingDecision gains `price_at_decision?: number | null`.
- `services/decisionService.ts`: setDecision sends the price only for
decision='seen' (and only when it's a finite number).
- `hooks/useDecisions.ts`: rewritten to store `Map<key, DecisionEntry>`
(decision + price snapshot). New `markSeen(id, price, type)` short-
circuits on existing liked/disliked. New `getDecisionEntry`,
`seenCount`.
- `App.tsx`: 3s `setTimeout` dwell timer fires markSeen when the
detail sheet stays open. Filter logic in `processedListingData`
hides `seen` rows whose `total_price === price_at_decision`, with
`showHidden` bypass. Computes `hiddenCount` to drive the toggle.
- `components/FilterBar.tsx`: new conditional "Hidden (N)" / "Showing
hidden (N)" toggle button (Eye / EyeOff lucide icons), surfaces only
when hiddenCount > 0.
## Tests
- pytest: 2 new (test_set_seen_carries_price, test_liked_drops_price_
even_if_supplied) + 1 updated to assert the new 5-arg repo
signature. 24 passed.
- vitest: 6 new for useDecisions (markSeen liked/disliked skip, price
snapshot, re-mark, null price, seenCount) + 5 new for decisionService
payload shape. 221 total passed, tsc clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
When the user clicks a heatmap hex that contains multiple properties,
each property card in the resulting popup now renders a left-edge
color stripe matching the heatmap gradient for that property's
individual value of the active metric (Price/m², Total Price, Size,
Bedrooms, …). The "color code" carries from the map into the popup
instead of dying at the hex boundary.
Plumbing:
- `colorSchemes.ts` gains `interpolateMetricColor(value, min, max, stops)`
that walks the color-stop ramp and returns `rgb(R, G, B)`.
- `Map.tsx` stashes the latest `{min, max}` from `computeColorScale` in
a ref so `getListingDialog` can compute per-property colors without
re-running the worker.
- `PropertyCard` accepts an optional `metricColor` prop and applies it
as a 4px `border-left`. Compact variant unchanged (no stripe).
Also resolves the Round-3 Fix-4 follow-up: `CardCarousel` (inside
PropertyCard.tsx) now has clickable prev/next chevron buttons in
addition to drag + keyboard navigation. Buttons fade in on hover
(group-hover) and are always focus-visible for keyboard users; clicks
stop propagation so the parent card click handler doesn't fire.
Tests: 9 new (4 covering interpolateMetricColor edge cases —
null/NaN/clamp — and 4 covering metricColor stripe + carousel
buttons present/absent). Full suite 210/210.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The previous attempt passed the Mapbox token via `--build-arg`, but
the docker-buildx plugin's KEY=VALUE list-parser mangled the value
(the rendered command was `--build-arg *=VITE_MAPBOX_TOKEN=********`,
key got lost). Inspecting `viktorbarzin/immoweb:45` confirmed
`pk.eyJ...` was nowhere in the bundle.
Switching to the idiomatic Vite path: a new `prepare-frontend-env`
commands step writes `frontend/.env.production` from the
`wrongmove-mapbox-token` Woodpecker secret. `COPY . .` in the
Dockerfile pulls the file into the build context, and Vite
auto-loads `.env.production` during `npx vite build`.
Net diff:
- `.woodpecker/frontend.yml`: new prepare step, build step now
depends on it, dropped the build_args line.
- `frontend/Dockerfile`: dropped the ARG/ENV lines (no longer needed,
also silences `SecretsUsedInArgOrEnv` linter warning).
- `frontend/.gitignore`: ignore `.env.production` / `.env.local` so
the CI-written file never gets accidentally committed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The previous attempt used a step-level `environment:` block with
`from_secret:`, which the Woodpecker linter rejected on plugin steps
("Should not configure both `environment` and `settings`"). Net effect
was build-and-push-frontend reverted to a commands step and the
docker daemon never started.
The Mapbox `pk.*` token ends up baked into the public bundle anyway —
its security model is domain restrictions in the Mapbox dashboard, not
build-time secrecy. Inlining the value in `build_args` is the simplest
working path and avoids the secret-indirection footgun. The token also
still lives in Vault at `secret/ci/global/wrongmove-mapbox-token` for
the day we adopt a private style URL or replace this with a different
provider.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a build-arg path so the Mapbox public token is injected at
`vite build` time instead of being hardcoded in the bundle:
- `frontend/Dockerfile` declares `ARG VITE_MAPBOX_TOKEN` in the
builder stage and re-exports it via `ENV` so Vite picks it up.
- `.woodpecker/frontend.yml` maps the global `wrongmove-mapbox-token`
Woodpecker secret into a step-level `VITE_MAPBOX_TOKEN` env var,
then forwards it via `build_args_from_env`.
Token is a domain-restricted `pk.*` public token (Mapbox), so bundle
exposure is the intended threat model. Vault-stored at
`secret/ci/global/wrongmove-mapbox-token`; synced to Woodpecker by
the existing vault-woodpecker-sync CronJob every 6h.
Replaces the post-Fix-4 "Map unavailable — set VITE_MAPBOX_TOKEN"
banner with a working basemap.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
BUY listings can come back from the API with null `total_price`, `qm`,
`qmprice`, `rooms`, and `last_seen` even though the TypeScript types
declare them non-nullable. The cards called `.toLocaleString()` and
`.split('T')` on those values, throwing TypeError and tripping the
ErrorBoundary — which is the "BUY part of the page crashes" symptom.
Coerce numeric fields via a `safeNum` helper (0 fallback), gate the
"Nd ago" line on a finite last_seen, and apply the same guards in
PropertyCard, PropertyCardCompact, SwipeCard, and the price-history
section of ListingDetail. Added regression tests asserting both card
variants render with all-null backend fields without throwing.
The 4 vitest shards were getting SIGKILL (exit code 137) because the
container memory limit was tighter than NODE_OPTIONS=--max-old-space-size=1024,
and 1024 wasn't enough headroom for the test workers either.
Set explicit kubernetes resources (request 1Gi / limit 2Gi) and bump
the V8 heap to 1.5Gi on install-frontend-deps and all 4 test shards.
Confirmed-by: pipeline 2081 step states (test-shard-1..4 all
state=failure exit_code=137; build/deploy steps then skipped).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Surgical UX/a11y fixes layered on top of the team's UI redesign,
keeping the teal palette and FilterBar architecture intact.
- TaskIndicator: NaN-safe progressPercentage (Number.isFinite +
clamp 0..100). No more "NaN%" when a task posts undefined progress.
- StreamingProgressBar: same NaN guard on the inline width calc.
- StatsBar: pluralize listings ("1 listing" / "30 listings"), drop
the duplicated "Avg:" label (now "Avg price" / "<n>/m²" / "Size"),
tabular-nums on every numeric.
- FilterPanel furnishing pills: aria-pressed + data-state for
screen readers and tests.
- ListView sort buttons: aria-pressed + aria-sort
(ascending/descending/none); listing count pluralizes
("1 property" / "N properties") with tabular-nums.
- Map: only fitBounds the FIRST time data loads (hasFittedOnceRef).
The previous lastDataLengthRef-based gate refit when results
went N → 0 → M, blowing away the user's pan/zoom.
Verified: tsc --noEmit clean, 125/125 vitest specs pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the old Docker-in-Docker build approach (plugins/docker) with
the modern buildx plugin that natively supports BuildKit. This fixes:
- "Cannot connect to the Docker daemon" errors (no DinD needed)
- "the --mount option requires BuildKit" errors (buildx = BuildKit)
- OOM in publish step (skopeo no longer needed, buildx pushes directly)
Also removes the intermediate build-tag/skopeo-copy/publish dance —
buildx pushes both versioned and latest tags in a single step.
Replace picsum.photos placeholders with actual Rightmove listing data
fetched from the API - real property photos, prices, addresses, and
coordinates. Skip auto-load API call in dev bypass mode.
Guarded by VITE_DEV_BYPASS_AUTH env var + import.meta.env.DEV check.
Vite tree-shakes the DEV branch in production builds.
The .env.development.local file is gitignored (**.env pattern).
Includes mock listing data to preview property cards without API.
Replace the fixed w-80 sidebar with a horizontal FilterBar below the
header, giving the map full viewport width. Key changes:
- App.tsx: Remove sidebar layout, add FilterBar + FilterChips + inline
StreamingProgressBar between header and main content area
- Header.tsx: Add Rent/Buy listing type toggle (compact Tabs) after logo
- StatsBar.tsx: Add "Color by" metric selector (moved from
VisualizationCard) as a compact Select alongside view mode toggles
- Mobile: Replace Sheet-based filter panel with full-screen Dialog
Add a horizontal FilterBar component with popover-based dropdowns for
Price, Beds, Size, and a "More Filters" panel with advanced options
(price/m2, furnishing, district, date, POI travel filters). Action
buttons (Show Listings / Scrape New) are aligned to the right.
Add FilterChips component that renders active (non-default) filter
values as removable pills below the filter bar.
- Wrap App in BrowserRouter in main.tsx
- Create useFilterParams hook that syncs filter state with URL search params
and derives viewMode from the URL pathname
- Replace window.location.pathname callback check with React Router Routes
- Split App into AppContent (main UI) and App (route definitions)
- Primary/accent changed from achromatic black to teal (oklch 0.55 0.14 175)
- Background/foreground given subtle cool slate tint
- Added --deal-good (emerald) and --deal-above (amber) custom properties
- Ring color updated to teal for focus states
- Dark mode updated to match teal theme
Approach C: Map-First with Modern Chrome
- Horizontal filter bar replacing sidebar
- React Router for URL state / deep linking
- Redesigned property cards with better visual hierarchy
- Refined color palette and typography
- Tabbed listing detail sheet
Replace .drone.yml with .woodpecker/ pipeline configs (frontend.yml, api.yml).
Convert Drone env vars to Woodpecker equivalents (CI_PIPELINE_NUMBER, CI_COMMIT_SHA),
use woodpeckerci/plugin-git for clone with retry, woodpeckerci/plugin-slack for
notifications, and plugins/docker for image builds. Update all docs and skills.
Instrument every API endpoint with Server-Timing headers so sub-operation
durations are visible in browser DevTools Network tab. Also adds Grafana
dashboard panels for per-endpoint latency comparison (p50/p95 timeseries
and p95 ranking bar gauge).
- Increase Redis cache TTL from 30 minutes to 24 hours
- Add stale-while-revalidate: serve stale cache (>4h) immediately while
repopulating in background with SETNX lock to prevent concurrent rebuilds
- Add in-memory frontend LRU cache (5 entries) so repeat filter visits
are instant without network requests
- Invalidate frontend cache on listing refresh and task completion
- Add unit tests for get_cache_age, is_cache_stale, acquire_repopulation_lock
