Commit graph

11 commits

Author SHA1 Message Date
Viktor Barzin
b9f576ae2b
Stream-process listings as IDs arrive via asyncio.Queue
Replace the sequential fetch-all-then-process pipeline with a streaming
architecture where listing processing starts as soon as IDs become
available from each subquery. A producer task fetches pages and enqueues
new IDs (filtered inline against DB), while 20 consumer workers process
listings concurrently from the queue.

- Add ListingRepository.get_listing_ids() for fast ID-only projection
- Refactor listing_tasks.py: remove get_ids_to_process/dump_listings_and_monitor,
  replace with unified producer/worker/monitor pipeline
- Apply same pattern to CLI path in listing_fetcher.py
- Remove 'filtering' phase from frontend, show combined fetch+process metrics
- Add fetching_done flag to TaskResult for phase transition tracking
2026-02-06 23:43:54 +00:00
Viktor Barzin
6d8f69610f Add proper buy listing support with type-aware UI filters and display 2026-02-01 19:13:29 +00:00
Kadir
0801aaf200
More ruff fixes (#2)
* adding ruff auto check for pull requests as well as fixing all ruff errors

* More ruff fixes: forgot half of the ruff checks

Forgot to do a git add all :D

---------

Co-authored-by: Kadir <git@k8n.dev>
2025-09-14 19:44:03 +01:00
Kadir
b1e0a414cf Used ruff to cleanup
I hope it just works right as I cannot test things if they work
2025-09-14 19:02:30 +01:00
Viktor Barzin
526f4fc0c3
update last seen property when processing listings to refresh data 2025-08-23 22:36:37 +00:00
Viktor Barzin
762408e054
add apprise and send notification when refreshing listings 2025-07-25 21:32:06 +00:00
Viktor Barzin
c98f6ff97f
hack - add available from parameter only for rent listings 2025-07-06 12:18:32 +00:00
Viktor Barzin
d4b22deda0
save user queries in redis so that user can refresh the page and still come back to their latest task 2025-07-06 12:02:25 +00:00
Viktor Barzin
69d15e9a16
add support for querying buying listings as well as by max price 2025-06-18 20:38:50 +00:00
Viktor Barzin
ff2f141b75
allow limiting the number of listings fetched 2025-06-14 13:39:56 +00:00
Viktor Barzin
4ad04775c9
add user auth boilerplate 2025-06-11 21:08:48 +00:00