wrongmove/crawler/services
Viktor Barzin 5514fa6381 Remove 1000-result limit, add Redis caching and virtual scrolling
- Remove hard-coded limit=1000 default from listing_geojson and streaming
  endpoints, allowing all matching results to be returned
- Add Redis caching service (db=2, 30min TTL) that caches query results
  as Redis Lists for fast re-queries with reduced DB load
- Integrate cache into streaming endpoint: serve from cache on hit,
  populate cache on miss during DB streaming
- Invalidate cache after scrape completes (both success and no-new-listings)
- Replace ScrollArea with react-virtuoso in ListView for virtual scrolling,
  keeping only ~20-30 DOM nodes regardless of list size
- Handle metadata streaming message to show "0 / N" progress from start
- Throttle frontend state updates with requestAnimationFrame to prevent
  UI jank from rapid re-renders during cached response streaming
2026-02-06 20:47:36 +00:00
..
listing_cache.py Remove 1000-result limit, add Redis caching and virtual scrolling 2026-02-06 20:47:36 +00:00
listing_fetcher.py Add throttling detection and circuit breaker for Rightmove scraper 2026-02-06 20:47:36 +00:00
query_splitter.py Add throttling detection and circuit breaker for Rightmove scraper 2026-02-06 20:47:36 +00:00
task_service.py Fix stuck Celery tasks and add purge all tasks functionality 2026-02-01 20:40:07 +00:00