No description

Viktor Barzin b9f576ae2b Stream-process listings as IDs arrive via asyncio.Queue ... Replace the sequential fetch-all-then-process pipeline with a streaming architecture where listing processing starts as soon as IDs become available from each subquery. A producer task fetches pages and enqueues new IDs (filtered inline against DB), while 20 consumer workers process listings concurrently from the queue. - Add ListingRepository.get_listing_ids() for fast ID-only projection - Refactor listing_tasks.py: remove get_ids_to_process/dump_listings_and_monitor, replace with unified producer/worker/monitor pipeline - Apply same pattern to CLI path in listing_fetcher.py - Remove 'filtering' phase from frontend, show combined fetch+process metrics - Add fetching_done flag to TaskResult for phase transition tracking		2026-02-06 23:43:54 +00:00
.github/workflows	adding ruff auto check for pull requests as well as fixing all ruff errors (#1)	2025-09-14 19:40:18 +01:00
crawler	Stream-process listings as IDs arrive via asyncio.Queue	2026-02-06 23:43:54 +00:00
immoweb	reduce size of json by pulling interesting fields above instead of relying on the entire thing	2025-06-09 21:46:33 +00:00
vqa	adding ruff auto check for pull requests as well as fixing all ruff errors (#1)	2025-09-14 19:40:18 +01:00
.drone.yml	update drone.yaml to build on push	2026-02-01 21:53:18 +00:00
.gitignore	Add runtime artifacts to .gitignore	2026-02-06 21:04:08 +00:00