Commit graph

7 commits

Author SHA1 Message Date
Viktor Barzin
b4837e1603
Add crawl job progress drawer with phase tracking and live logs
- Add phase-aware progress reporting across all crawl phases (splitting,
  fetching, filtering, processing) with per-step counters
- Add TaskProgressDrawer component with phase timeline stepper, detail
  counters, progress bar with ETA, and live worker log viewer
- Add on_step_complete callback to ListingProcessor for granular tracking
  of details/images/OCR steps
- Extend QuerySplitter on_progress callback with structured counter data
- Capture celery worker logs via ring buffer handler and inject into task
  state updates for frontend display
- Guard taskResult updates with phase presence check to prevent drawer
  from blanking during state transitions
2026-02-06 22:37:53 +00:00
Viktor Barzin
c4b11ccfe9 Add comprehensive logging to Celery tasks and listing processor 2026-02-06 20:47:36 +00:00
Kadir
0801aaf200
More ruff fixes (#2)
* adding ruff auto check for pull requests as well as fixing all ruff errors

* More ruff fixes: forgot half of the ruff checks

Forgot to do a git add all :D

---------

Co-authored-by: Kadir <git@k8n.dev>
2025-09-14 19:44:03 +01:00
Viktor Barzin
526f4fc0c3
update last seen property when processing listings to refresh data 2025-08-23 22:36:37 +00:00
Viktor Barzin
480957dc72
add timeout when fetching details and use new entrypoint for task processing 2025-08-23 22:20:42 +00:00
Viktor Barzin
d1cef99c5a
make task processing a bit better. still doing 1 query to check if needs processing; will fix later 2025-07-27 20:09:41 +00:00
Viktor Barzin
91a0436f7f
migrate processing to a pipeline approach where each listing is processed in a pipeline in parallel and status reported back to track progress 2025-07-27 18:33:39 +00:00