No description
49e3514780
Two surfaces wired up so the user can "get a vibe of the market": **Per-listing** — each PropertyCard now shows a small pill next to the price when the listing's total_price moved >=1% over a 14-day lookback (e.g. "↓ £200 (-4%) in 14d"). Drops render green, rises render red. Computed from `price_history_json` by the daily aggregator and denormalised onto the listing row so the streaming endpoint just passes it through. **Macro** — new always-visible inline strip above the chip strip showing today's median total price, median £/m², and listing count for the current filter's bedroom band, each with a 30-day % delta: "Rent · 1-2 bed · 30d: Median £2,500 ↓ -4% · £/m² £50 ↓ -2% · Listings 4,200 ↑ +5%". Both data sources are populated daily at 04:00 UTC by a new Celery beat task that fires 1h after the 03:00 RENT scrape and feeds two sinks: a per-listing update pass and an upsert to a new `dailylistingaggregate` table keyed on (snapshot_date, listing_type, min_bedrooms, max_bedrooms). ## Backend - `models/listing.py`: Listing parent gains `price_14d_ago` + `price_ change_pct_14d` nullable floats (inherited by RentListing/BuyListing). New `DailyListingAggregate` table model with unique constraint on (date, type, min_bed, max_bed). - Alembic `a8b9c0d1e2f3`: adds the two columns to both listing tables and creates the aggregate table + date index. - `services/market_aggregator.py` (new): `compute_trend_for_listing`, `update_per_listing_trend` (batched, idempotent), `_stats` (median + mean filtered to positive finite values), `compute_aggregate_ snapshot` (dialect-aware MySQL / SQLite upsert), `fetch_trend_ series` (range query for the API). - `tasks/market_tasks.py` (new): `compute_daily_market_aggregates_task` Celery task wrapping both stages. - `tasks/listing_tasks.py:setup_periodic_tasks`: registers the daily task at 04:00 UTC alongside the existing scrape schedules. - `celery_app.py`: includes the new tasks module. - `api/app.py`: new `GET /api/market_trend?listing_type=&min_bedrooms=& max_bedrooms=&days=` endpoint returning the daily series. - `ui_exporter.py`: GeoJSON feature properties now carry `price_14d_ago` and `price_change_pct_14d` so the frontend can render the badge without an extra round-trip. ## Frontend - `types/index.ts`: new `MarketTrendPoint`; `PropertyProperties` gains the two optional trend fields. - `components/PropertyCard.tsx`: derived `trendBadge` (>=1% threshold, null-safe) rendered as a small pill on both card variants. - `hooks/useMarketTrend.ts` (new): fetches the trend series, derives current-vs-oldest deltas per metric (% change rounded to 1dp). - `components/MarketTrendStrip.tsx` (new): compact inline strip with three metric cells. Hidden when the aggregator hasn't produced any rows yet (graceful start during the first week post-launch). - `App.tsx`: renders the strip above the chip strip whenever the active queryParameters are known. ## Tests - pytest: 10 new (trend math edge cases including null history, malformed JSON, only-recent entries, drops, rises, zero current price; _stats empty / nonpositive filtering; upsert idempotency on an in-memory SQLite seed). 34 decision + aggregator tests pass. - vitest: 8 new (useMarketTrend fetch URL, two-point delta, single-point null delta, empty series; PropertyCard trend badge arrow direction + sign for drops/rises, noise threshold, null guard). 229 tests pass total, tsc clean. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|---|---|---|
| .claude | ||
| .github/workflows | ||
| .woodpecker | ||
| alembic | ||
| api | ||
| cli | ||
| config | ||
| docs | ||
| frontend | ||
| grafana | ||
| k8s | ||
| models | ||
| rec | ||
| repositories | ||
| scripts | ||
| services | ||
| tasks | ||
| tests | ||
| utils | ||
| .dockerignore | ||
| .env.sample | ||
| .gitignore | ||
| .style.yapf | ||
| alembic.ini | ||
| celery_app.py | ||
| CLAUDE.md | ||
| csv_exporter.py | ||
| data_access.py | ||
| database.py | ||
| docker-compose.yml | ||
| Dockerfile | ||
| GUIDE | ||
| listing_processor.py | ||
| logging_config.py | ||
| main.py | ||
| notifications.py | ||
| podman-compose.yml | ||
| poetry.lock | ||
| pyproject.toml | ||
| README.md | ||
| redis_repository.py | ||
| requirements.txt | ||
| runall.sh | ||
| start.sh | ||
| TASKS.md | ||
| ui_exporter.py | ||
Setup
- Instal deps:
poetry install && cp .env.sample .env
- Check
.envif you want to customize settings for broker and db - run
./start.sh
This starts the backend
To start the fronend:
cd frontend && cp .env.sample .env
Change the DEV_HOST to any name you want to use to access the web interface.
Next, setup the DNS record (e.g in your /etc/hosts) file. This is important as auth is done via external [authentik] service that needs to redirect to a name.
Run ./start.sh
This starts a Caddy proxy with correct certificates, and npm dev server.
All requests going to the frontend are forwarded to the npm server and the ones for the backed (that go to /api/*) are forwarded to the backend service.
Lastly, reachout to Viktor to allowlist your DEV_HOST so that authentik can authorize callbacks to your host.
Formatting
yapf --style .style.yapf --recursive .
For VSCode - install yapf extension. Enable formatting using yap and the style file in this repo (there may be an easier way; I put this in my user settings json):
{
"[python]": {
"editor.formatOnSaveMode": "file",
"editor.formatOnSave": true,
"editor.defaultFormatter": "eeyore.yapf",
"editor.formatOnType": false
},
"yapf.args": ["--style", "/home/wizard/code/realestate-crawler/crawler/.style.yapf"]
}
ADB commands (from /Applications/BlueStacks.app/Contents/MacOS):
Set proxy
./hd-adb shell settings put global http_proxy 192.168.9.110:8080
Disable proxy:
/hd-adb shell settings put global http_proxy :0
Connect adb
./hd-adb connect 127.0.0.1:5555
Disconnect adb
/hd-adb disconnect