wrongmove: daily price-trend monitoring (per-listing badge + macro strip)
Two surfaces wired up so the user can "get a vibe of the market":
**Per-listing** — each PropertyCard now shows a small pill next to the
price when the listing's total_price moved >=1% over a 14-day lookback
(e.g. "↓ £200 (-4%) in 14d"). Drops render green, rises render red.
Computed from `price_history_json` by the daily aggregator and
denormalised onto the listing row so the streaming endpoint just
passes it through.
**Macro** — new always-visible inline strip above the chip strip
showing today's median total price, median £/m², and listing count
for the current filter's bedroom band, each with a 30-day % delta:
"Rent · 1-2 bed · 30d: Median £2,500 ↓ -4% · £/m² £50 ↓ -2% · Listings 4,200 ↑ +5%".
Both data sources are populated daily at 04:00 UTC by a new Celery
beat task that fires 1h after the 03:00 RENT scrape and feeds two
sinks: a per-listing update pass and an upsert to a new
`dailylistingaggregate` table keyed on
(snapshot_date, listing_type, min_bedrooms, max_bedrooms).
## Backend
- `models/listing.py`: Listing parent gains `price_14d_ago` + `price_
change_pct_14d` nullable floats (inherited by RentListing/BuyListing).
New `DailyListingAggregate` table model with unique constraint on
(date, type, min_bed, max_bed).
- Alembic `a8b9c0d1e2f3`: adds the two columns to both listing tables
and creates the aggregate table + date index.
- `services/market_aggregator.py` (new): `compute_trend_for_listing`,
`update_per_listing_trend` (batched, idempotent), `_stats` (median
+ mean filtered to positive finite values), `compute_aggregate_
snapshot` (dialect-aware MySQL / SQLite upsert), `fetch_trend_
series` (range query for the API).
- `tasks/market_tasks.py` (new): `compute_daily_market_aggregates_task`
Celery task wrapping both stages.
- `tasks/listing_tasks.py:setup_periodic_tasks`: registers the daily
task at 04:00 UTC alongside the existing scrape schedules.
- `celery_app.py`: includes the new tasks module.
- `api/app.py`: new `GET /api/market_trend?listing_type=&min_bedrooms=&
max_bedrooms=&days=` endpoint returning the daily series.
- `ui_exporter.py`: GeoJSON feature properties now carry
`price_14d_ago` and `price_change_pct_14d` so the frontend can
render the badge without an extra round-trip.
## Frontend
- `types/index.ts`: new `MarketTrendPoint`; `PropertyProperties` gains
the two optional trend fields.
- `components/PropertyCard.tsx`: derived `trendBadge` (>=1% threshold,
null-safe) rendered as a small pill on both card variants.
- `hooks/useMarketTrend.ts` (new): fetches the trend series, derives
current-vs-oldest deltas per metric (% change rounded to 1dp).
- `components/MarketTrendStrip.tsx` (new): compact inline strip with
three metric cells. Hidden when the aggregator hasn't produced any
rows yet (graceful start during the first week post-launch).
- `App.tsx`: renders the strip above the chip strip whenever the
active queryParameters are known.
## Tests
- pytest: 10 new (trend math edge cases including null history,
malformed JSON, only-recent entries, drops, rises, zero current
price; _stats empty / nonpositive filtering; upsert idempotency on
an in-memory SQLite seed). 34 decision + aggregator tests pass.
- vitest: 8 new (useMarketTrend fetch URL, two-point delta,
single-point null delta, empty series; PropertyCard trend badge
arrow direction + sign for drops/rises, noise threshold, null
guard). 229 tests pass total, tsc clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 12:02:25 +00:00
|
|
|
"""Daily market-trend aggregator Celery task.
|
|
|
|
|
|
|
|
|
|
Fires daily at 04:00 UTC — one hour after the 03:00 RENT scrape so the
|
|
|
|
|
data is fresh. Calls into `services.market_aggregator` to:
|
|
|
|
|
1. Recompute per-listing `price_14d_ago` / `price_change_pct_14d`.
|
|
|
|
|
2. Upsert the per-(listing_type, bedroom-band) row in
|
|
|
|
|
`dailylistingaggregate` for today's snapshot.
|
|
|
|
|
|
|
|
|
|
Idempotent: re-running on the same day refreshes both surfaces in place.
|
|
|
|
|
"""
|
|
|
|
|
from __future__ import annotations
|
|
|
|
|
|
|
|
|
|
import logging
|
|
|
|
|
from typing import Any
|
|
|
|
|
|
|
|
|
|
from celery_app import app
|
|
|
|
|
from database import engine
|
|
|
|
|
from services import market_aggregator
|
|
|
|
|
|
|
|
|
|
celery_logger = logging.getLogger("celery_app")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@app.task(
|
|
|
|
|
bind=True,
|
|
|
|
|
name="tasks.market_tasks.compute_daily_market_aggregates_task",
|
|
|
|
|
time_limit=3600,
|
|
|
|
|
soft_time_limit=3500,
|
|
|
|
|
acks_late=True,
|
|
|
|
|
)
|
|
|
|
|
def compute_daily_market_aggregates_task(self: Any) -> dict[str, Any]:
|
|
|
|
|
"""Run both stages of the daily market aggregator."""
|
|
|
|
|
celery_logger.info("Starting daily market aggregator (task=%s)", self.request.id)
|
|
|
|
|
per_listing = market_aggregator.update_per_listing_trend(engine)
|
|
|
|
|
aggregates = market_aggregator.compute_aggregate_snapshot(engine)
|
2026-05-16 12:53:26 +00:00
|
|
|
# Materialise only the count — the row objects came from a session
|
|
|
|
|
# that's already closed, so accessing any lazy-loaded attribute would
|
|
|
|
|
# raise DetachedInstanceError. The aggregator's own logger lines have
|
|
|
|
|
# already printed the per-band stats.
|
|
|
|
|
aggregates_count = len(aggregates)
|
wrongmove: daily price-trend monitoring (per-listing badge + macro strip)
Two surfaces wired up so the user can "get a vibe of the market":
**Per-listing** — each PropertyCard now shows a small pill next to the
price when the listing's total_price moved >=1% over a 14-day lookback
(e.g. "↓ £200 (-4%) in 14d"). Drops render green, rises render red.
Computed from `price_history_json` by the daily aggregator and
denormalised onto the listing row so the streaming endpoint just
passes it through.
**Macro** — new always-visible inline strip above the chip strip
showing today's median total price, median £/m², and listing count
for the current filter's bedroom band, each with a 30-day % delta:
"Rent · 1-2 bed · 30d: Median £2,500 ↓ -4% · £/m² £50 ↓ -2% · Listings 4,200 ↑ +5%".
Both data sources are populated daily at 04:00 UTC by a new Celery
beat task that fires 1h after the 03:00 RENT scrape and feeds two
sinks: a per-listing update pass and an upsert to a new
`dailylistingaggregate` table keyed on
(snapshot_date, listing_type, min_bedrooms, max_bedrooms).
## Backend
- `models/listing.py`: Listing parent gains `price_14d_ago` + `price_
change_pct_14d` nullable floats (inherited by RentListing/BuyListing).
New `DailyListingAggregate` table model with unique constraint on
(date, type, min_bed, max_bed).
- Alembic `a8b9c0d1e2f3`: adds the two columns to both listing tables
and creates the aggregate table + date index.
- `services/market_aggregator.py` (new): `compute_trend_for_listing`,
`update_per_listing_trend` (batched, idempotent), `_stats` (median
+ mean filtered to positive finite values), `compute_aggregate_
snapshot` (dialect-aware MySQL / SQLite upsert), `fetch_trend_
series` (range query for the API).
- `tasks/market_tasks.py` (new): `compute_daily_market_aggregates_task`
Celery task wrapping both stages.
- `tasks/listing_tasks.py:setup_periodic_tasks`: registers the daily
task at 04:00 UTC alongside the existing scrape schedules.
- `celery_app.py`: includes the new tasks module.
- `api/app.py`: new `GET /api/market_trend?listing_type=&min_bedrooms=&
max_bedrooms=&days=` endpoint returning the daily series.
- `ui_exporter.py`: GeoJSON feature properties now carry
`price_14d_ago` and `price_change_pct_14d` so the frontend can
render the badge without an extra round-trip.
## Frontend
- `types/index.ts`: new `MarketTrendPoint`; `PropertyProperties` gains
the two optional trend fields.
- `components/PropertyCard.tsx`: derived `trendBadge` (>=1% threshold,
null-safe) rendered as a small pill on both card variants.
- `hooks/useMarketTrend.ts` (new): fetches the trend series, derives
current-vs-oldest deltas per metric (% change rounded to 1dp).
- `components/MarketTrendStrip.tsx` (new): compact inline strip with
three metric cells. Hidden when the aggregator hasn't produced any
rows yet (graceful start during the first week post-launch).
- `App.tsx`: renders the strip above the chip strip whenever the
active queryParameters are known.
## Tests
- pytest: 10 new (trend math edge cases including null history,
malformed JSON, only-recent entries, drops, rises, zero current
price; _stats empty / nonpositive filtering; upsert idempotency on
an in-memory SQLite seed). 34 decision + aggregator tests pass.
- vitest: 8 new (useMarketTrend fetch URL, two-point delta,
single-point null delta, empty series; PropertyCard trend badge
arrow direction + sign for drops/rises, noise threshold, null
guard). 229 tests pass total, tsc clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 12:02:25 +00:00
|
|
|
result = {
|
|
|
|
|
"status": "ok",
|
|
|
|
|
"per_listing": per_listing,
|
2026-05-16 12:53:26 +00:00
|
|
|
"aggregates_written": aggregates_count,
|
wrongmove: daily price-trend monitoring (per-listing badge + macro strip)
Two surfaces wired up so the user can "get a vibe of the market":
**Per-listing** — each PropertyCard now shows a small pill next to the
price when the listing's total_price moved >=1% over a 14-day lookback
(e.g. "↓ £200 (-4%) in 14d"). Drops render green, rises render red.
Computed from `price_history_json` by the daily aggregator and
denormalised onto the listing row so the streaming endpoint just
passes it through.
**Macro** — new always-visible inline strip above the chip strip
showing today's median total price, median £/m², and listing count
for the current filter's bedroom band, each with a 30-day % delta:
"Rent · 1-2 bed · 30d: Median £2,500 ↓ -4% · £/m² £50 ↓ -2% · Listings 4,200 ↑ +5%".
Both data sources are populated daily at 04:00 UTC by a new Celery
beat task that fires 1h after the 03:00 RENT scrape and feeds two
sinks: a per-listing update pass and an upsert to a new
`dailylistingaggregate` table keyed on
(snapshot_date, listing_type, min_bedrooms, max_bedrooms).
## Backend
- `models/listing.py`: Listing parent gains `price_14d_ago` + `price_
change_pct_14d` nullable floats (inherited by RentListing/BuyListing).
New `DailyListingAggregate` table model with unique constraint on
(date, type, min_bed, max_bed).
- Alembic `a8b9c0d1e2f3`: adds the two columns to both listing tables
and creates the aggregate table + date index.
- `services/market_aggregator.py` (new): `compute_trend_for_listing`,
`update_per_listing_trend` (batched, idempotent), `_stats` (median
+ mean filtered to positive finite values), `compute_aggregate_
snapshot` (dialect-aware MySQL / SQLite upsert), `fetch_trend_
series` (range query for the API).
- `tasks/market_tasks.py` (new): `compute_daily_market_aggregates_task`
Celery task wrapping both stages.
- `tasks/listing_tasks.py:setup_periodic_tasks`: registers the daily
task at 04:00 UTC alongside the existing scrape schedules.
- `celery_app.py`: includes the new tasks module.
- `api/app.py`: new `GET /api/market_trend?listing_type=&min_bedrooms=&
max_bedrooms=&days=` endpoint returning the daily series.
- `ui_exporter.py`: GeoJSON feature properties now carry
`price_14d_ago` and `price_change_pct_14d` so the frontend can
render the badge without an extra round-trip.
## Frontend
- `types/index.ts`: new `MarketTrendPoint`; `PropertyProperties` gains
the two optional trend fields.
- `components/PropertyCard.tsx`: derived `trendBadge` (>=1% threshold,
null-safe) rendered as a small pill on both card variants.
- `hooks/useMarketTrend.ts` (new): fetches the trend series, derives
current-vs-oldest deltas per metric (% change rounded to 1dp).
- `components/MarketTrendStrip.tsx` (new): compact inline strip with
three metric cells. Hidden when the aggregator hasn't produced any
rows yet (graceful start during the first week post-launch).
- `App.tsx`: renders the strip above the chip strip whenever the
active queryParameters are known.
## Tests
- pytest: 10 new (trend math edge cases including null history,
malformed JSON, only-recent entries, drops, rises, zero current
price; _stats empty / nonpositive filtering; upsert idempotency on
an in-memory SQLite seed). 34 decision + aggregator tests pass.
- vitest: 8 new (useMarketTrend fetch URL, two-point delta,
single-point null delta, empty series; PropertyCard trend badge
arrow direction + sign for drops/rises, noise threshold, null
guard). 229 tests pass total, tsc clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 12:02:25 +00:00
|
|
|
}
|
|
|
|
|
celery_logger.info(
|
2026-05-16 12:53:26 +00:00
|
|
|
"Daily market aggregator complete: rent_updated=%s buy_updated=%s aggregates=%d",
|
wrongmove: daily price-trend monitoring (per-listing badge + macro strip)
Two surfaces wired up so the user can "get a vibe of the market":
**Per-listing** — each PropertyCard now shows a small pill next to the
price when the listing's total_price moved >=1% over a 14-day lookback
(e.g. "↓ £200 (-4%) in 14d"). Drops render green, rises render red.
Computed from `price_history_json` by the daily aggregator and
denormalised onto the listing row so the streaming endpoint just
passes it through.
**Macro** — new always-visible inline strip above the chip strip
showing today's median total price, median £/m², and listing count
for the current filter's bedroom band, each with a 30-day % delta:
"Rent · 1-2 bed · 30d: Median £2,500 ↓ -4% · £/m² £50 ↓ -2% · Listings 4,200 ↑ +5%".
Both data sources are populated daily at 04:00 UTC by a new Celery
beat task that fires 1h after the 03:00 RENT scrape and feeds two
sinks: a per-listing update pass and an upsert to a new
`dailylistingaggregate` table keyed on
(snapshot_date, listing_type, min_bedrooms, max_bedrooms).
## Backend
- `models/listing.py`: Listing parent gains `price_14d_ago` + `price_
change_pct_14d` nullable floats (inherited by RentListing/BuyListing).
New `DailyListingAggregate` table model with unique constraint on
(date, type, min_bed, max_bed).
- Alembic `a8b9c0d1e2f3`: adds the two columns to both listing tables
and creates the aggregate table + date index.
- `services/market_aggregator.py` (new): `compute_trend_for_listing`,
`update_per_listing_trend` (batched, idempotent), `_stats` (median
+ mean filtered to positive finite values), `compute_aggregate_
snapshot` (dialect-aware MySQL / SQLite upsert), `fetch_trend_
series` (range query for the API).
- `tasks/market_tasks.py` (new): `compute_daily_market_aggregates_task`
Celery task wrapping both stages.
- `tasks/listing_tasks.py:setup_periodic_tasks`: registers the daily
task at 04:00 UTC alongside the existing scrape schedules.
- `celery_app.py`: includes the new tasks module.
- `api/app.py`: new `GET /api/market_trend?listing_type=&min_bedrooms=&
max_bedrooms=&days=` endpoint returning the daily series.
- `ui_exporter.py`: GeoJSON feature properties now carry
`price_14d_ago` and `price_change_pct_14d` so the frontend can
render the badge without an extra round-trip.
## Frontend
- `types/index.ts`: new `MarketTrendPoint`; `PropertyProperties` gains
the two optional trend fields.
- `components/PropertyCard.tsx`: derived `trendBadge` (>=1% threshold,
null-safe) rendered as a small pill on both card variants.
- `hooks/useMarketTrend.ts` (new): fetches the trend series, derives
current-vs-oldest deltas per metric (% change rounded to 1dp).
- `components/MarketTrendStrip.tsx` (new): compact inline strip with
three metric cells. Hidden when the aggregator hasn't produced any
rows yet (graceful start during the first week post-launch).
- `App.tsx`: renders the strip above the chip strip whenever the
active queryParameters are known.
## Tests
- pytest: 10 new (trend math edge cases including null history,
malformed JSON, only-recent entries, drops, rises, zero current
price; _stats empty / nonpositive filtering; upsert idempotency on
an in-memory SQLite seed). 34 decision + aggregator tests pass.
- vitest: 8 new (useMarketTrend fetch URL, two-point delta,
single-point null delta, empty series; PropertyCard trend badge
arrow direction + sign for drops/rises, noise threshold, null
guard). 229 tests pass total, tsc clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 12:02:25 +00:00
|
|
|
per_listing.get("rent_updated"),
|
|
|
|
|
per_listing.get("buy_updated"),
|
2026-05-16 12:53:26 +00:00
|
|
|
aggregates_count,
|
wrongmove: daily price-trend monitoring (per-listing badge + macro strip)
Two surfaces wired up so the user can "get a vibe of the market":
**Per-listing** — each PropertyCard now shows a small pill next to the
price when the listing's total_price moved >=1% over a 14-day lookback
(e.g. "↓ £200 (-4%) in 14d"). Drops render green, rises render red.
Computed from `price_history_json` by the daily aggregator and
denormalised onto the listing row so the streaming endpoint just
passes it through.
**Macro** — new always-visible inline strip above the chip strip
showing today's median total price, median £/m², and listing count
for the current filter's bedroom band, each with a 30-day % delta:
"Rent · 1-2 bed · 30d: Median £2,500 ↓ -4% · £/m² £50 ↓ -2% · Listings 4,200 ↑ +5%".
Both data sources are populated daily at 04:00 UTC by a new Celery
beat task that fires 1h after the 03:00 RENT scrape and feeds two
sinks: a per-listing update pass and an upsert to a new
`dailylistingaggregate` table keyed on
(snapshot_date, listing_type, min_bedrooms, max_bedrooms).
## Backend
- `models/listing.py`: Listing parent gains `price_14d_ago` + `price_
change_pct_14d` nullable floats (inherited by RentListing/BuyListing).
New `DailyListingAggregate` table model with unique constraint on
(date, type, min_bed, max_bed).
- Alembic `a8b9c0d1e2f3`: adds the two columns to both listing tables
and creates the aggregate table + date index.
- `services/market_aggregator.py` (new): `compute_trend_for_listing`,
`update_per_listing_trend` (batched, idempotent), `_stats` (median
+ mean filtered to positive finite values), `compute_aggregate_
snapshot` (dialect-aware MySQL / SQLite upsert), `fetch_trend_
series` (range query for the API).
- `tasks/market_tasks.py` (new): `compute_daily_market_aggregates_task`
Celery task wrapping both stages.
- `tasks/listing_tasks.py:setup_periodic_tasks`: registers the daily
task at 04:00 UTC alongside the existing scrape schedules.
- `celery_app.py`: includes the new tasks module.
- `api/app.py`: new `GET /api/market_trend?listing_type=&min_bedrooms=&
max_bedrooms=&days=` endpoint returning the daily series.
- `ui_exporter.py`: GeoJSON feature properties now carry
`price_14d_ago` and `price_change_pct_14d` so the frontend can
render the badge without an extra round-trip.
## Frontend
- `types/index.ts`: new `MarketTrendPoint`; `PropertyProperties` gains
the two optional trend fields.
- `components/PropertyCard.tsx`: derived `trendBadge` (>=1% threshold,
null-safe) rendered as a small pill on both card variants.
- `hooks/useMarketTrend.ts` (new): fetches the trend series, derives
current-vs-oldest deltas per metric (% change rounded to 1dp).
- `components/MarketTrendStrip.tsx` (new): compact inline strip with
three metric cells. Hidden when the aggregator hasn't produced any
rows yet (graceful start during the first week post-launch).
- `App.tsx`: renders the strip above the chip strip whenever the
active queryParameters are known.
## Tests
- pytest: 10 new (trend math edge cases including null history,
malformed JSON, only-recent entries, drops, rises, zero current
price; _stats empty / nonpositive filtering; upsert idempotency on
an in-memory SQLite seed). 34 decision + aggregator tests pass.
- vitest: 8 new (useMarketTrend fetch URL, two-point delta,
single-point null delta, empty series; PropertyCard trend badge
arrow direction + sign for drops/rises, noise threshold, null
guard). 229 tests pass total, tsc clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 12:02:25 +00:00
|
|
|
)
|
|
|
|
|
return result
|