49e3514780
Two surfaces wired up so the user can "get a vibe of the market": **Per-listing** — each PropertyCard now shows a small pill next to the price when the listing's total_price moved >=1% over a 14-day lookback (e.g. "↓ £200 (-4%) in 14d"). Drops render green, rises render red. Computed from `price_history_json` by the daily aggregator and denormalised onto the listing row so the streaming endpoint just passes it through. **Macro** — new always-visible inline strip above the chip strip showing today's median total price, median £/m², and listing count for the current filter's bedroom band, each with a 30-day % delta: "Rent · 1-2 bed · 30d: Median £2,500 ↓ -4% · £/m² £50 ↓ -2% · Listings 4,200 ↑ +5%". Both data sources are populated daily at 04:00 UTC by a new Celery beat task that fires 1h after the 03:00 RENT scrape and feeds two sinks: a per-listing update pass and an upsert to a new `dailylistingaggregate` table keyed on (snapshot_date, listing_type, min_bedrooms, max_bedrooms). ## Backend - `models/listing.py`: Listing parent gains `price_14d_ago` + `price_ change_pct_14d` nullable floats (inherited by RentListing/BuyListing). New `DailyListingAggregate` table model with unique constraint on (date, type, min_bed, max_bed). - Alembic `a8b9c0d1e2f3`: adds the two columns to both listing tables and creates the aggregate table + date index. - `services/market_aggregator.py` (new): `compute_trend_for_listing`, `update_per_listing_trend` (batched, idempotent), `_stats` (median + mean filtered to positive finite values), `compute_aggregate_ snapshot` (dialect-aware MySQL / SQLite upsert), `fetch_trend_ series` (range query for the API). - `tasks/market_tasks.py` (new): `compute_daily_market_aggregates_task` Celery task wrapping both stages. - `tasks/listing_tasks.py:setup_periodic_tasks`: registers the daily task at 04:00 UTC alongside the existing scrape schedules. - `celery_app.py`: includes the new tasks module. - `api/app.py`: new `GET /api/market_trend?listing_type=&min_bedrooms=& max_bedrooms=&days=` endpoint returning the daily series. - `ui_exporter.py`: GeoJSON feature properties now carry `price_14d_ago` and `price_change_pct_14d` so the frontend can render the badge without an extra round-trip. ## Frontend - `types/index.ts`: new `MarketTrendPoint`; `PropertyProperties` gains the two optional trend fields. - `components/PropertyCard.tsx`: derived `trendBadge` (>=1% threshold, null-safe) rendered as a small pill on both card variants. - `hooks/useMarketTrend.ts` (new): fetches the trend series, derives current-vs-oldest deltas per metric (% change rounded to 1dp). - `components/MarketTrendStrip.tsx` (new): compact inline strip with three metric cells. Hidden when the aggregator hasn't produced any rows yet (graceful start during the first week post-launch). - `App.tsx`: renders the strip above the chip strip whenever the active queryParameters are known. ## Tests - pytest: 10 new (trend math edge cases including null history, malformed JSON, only-recent entries, drops, rises, zero current price; _stats empty / nonpositive filtering; upsert idempotency on an in-memory SQLite seed). 34 decision + aggregator tests pass. - vitest: 8 new (useMarketTrend fetch URL, two-point delta, single-point null delta, empty series; PropertyCard trend badge arrow direction + sign for drops/rises, noise threshold, null guard). 229 tests pass total, tsc clean. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
57 lines
1.9 KiB
Python
57 lines
1.9 KiB
Python
"""Daily market-trend aggregator Celery task.
|
|
|
|
Fires daily at 04:00 UTC — one hour after the 03:00 RENT scrape so the
|
|
data is fresh. Calls into `services.market_aggregator` to:
|
|
1. Recompute per-listing `price_14d_ago` / `price_change_pct_14d`.
|
|
2. Upsert the per-(listing_type, bedroom-band) row in
|
|
`dailylistingaggregate` for today's snapshot.
|
|
|
|
Idempotent: re-running on the same day refreshes both surfaces in place.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
from typing import Any
|
|
|
|
from celery_app import app
|
|
from database import engine
|
|
from services import market_aggregator
|
|
|
|
celery_logger = logging.getLogger("celery_app")
|
|
|
|
|
|
@app.task(
|
|
bind=True,
|
|
name="tasks.market_tasks.compute_daily_market_aggregates_task",
|
|
time_limit=3600,
|
|
soft_time_limit=3500,
|
|
acks_late=True,
|
|
)
|
|
def compute_daily_market_aggregates_task(self: Any) -> dict[str, Any]:
|
|
"""Run both stages of the daily market aggregator."""
|
|
celery_logger.info("Starting daily market aggregator (task=%s)", self.request.id)
|
|
per_listing = market_aggregator.update_per_listing_trend(engine)
|
|
aggregates = market_aggregator.compute_aggregate_snapshot(engine)
|
|
result = {
|
|
"status": "ok",
|
|
"per_listing": per_listing,
|
|
"aggregates": [
|
|
{
|
|
"snapshot_date": a.snapshot_date.isoformat(),
|
|
"listing_type": a.listing_type,
|
|
"min_bedrooms": a.min_bedrooms,
|
|
"max_bedrooms": a.max_bedrooms,
|
|
"listing_count": a.listing_count,
|
|
"median_total_price": a.median_total_price,
|
|
"median_qmprice": a.median_qmprice,
|
|
}
|
|
for a in aggregates
|
|
],
|
|
}
|
|
celery_logger.info(
|
|
"Daily market aggregator complete: rent_updated=%s buy_updated=%s aggregates=%s",
|
|
per_listing.get("rent_updated"),
|
|
per_listing.get("buy_updated"),
|
|
len(aggregates),
|
|
)
|
|
return result
|