Viktor Barzin
5514fa6381
Remove 1000-result limit, add Redis caching and virtual scrolling
...
- Remove hard-coded limit=1000 default from listing_geojson and streaming
endpoints, allowing all matching results to be returned
- Add Redis caching service (db=2, 30min TTL) that caches query results
as Redis Lists for fast re-queries with reduced DB load
- Integrate cache into streaming endpoint: serve from cache on hit,
populate cache on miss during DB streaming
- Invalidate cache after scrape completes (both success and no-new-listings)
- Replace ScrollArea with react-virtuoso in ListView for virtual scrolling,
keeping only ~20-30 DOM nodes regardless of list size
- Handle metadata streaming message to show "0 / N" progress from start
- Throttle frontend state updates with requestAnimationFrame to prevent
UI jank from rapid re-renders during cached response streaming
2026-02-06 20:47:36 +00:00
Viktor Barzin
835a2a9d53
Fix stuck Celery tasks and add purge all tasks functionality
2026-02-01 20:40:07 +00:00
Viktor Barzin
6d8f69610f
Add proper buy listing support with type-aware UI filters and display
2026-02-01 19:13:29 +00:00
Viktor Barzin
c7ac448f15
Add configurable scheduling, UI health/task indicators, and auto-load map with default filters
2026-02-01 17:28:37 +00:00
Viktor Barzin
ced9a153bd
replace pymysql with mysqlclient as it is "better"; also fix an issue in the ui exporter that had wrong imports
2025-10-18 09:58:55 +00:00
Kadir
0801aaf200
More ruff fixes ( #2 )
...
* adding ruff auto check for pull requests as well as fixing all ruff errors
* More ruff fixes: forgot half of the ruff checks
Forgot to do a git add all :D
---------
Co-authored-by: Kadir <git@k8n.dev>
2025-09-14 19:44:03 +01:00
Kadir
b1e0a414cf
Used ruff to cleanup
...
I hope it just works right as I cannot test things if they work
2025-09-14 19:02:30 +01:00
Viktor Barzin
91a0436f7f
migrate processing to a pipeline approach where each listing is processed in a pipeline in parallel and status reported back to track progress
2025-07-27 18:33:39 +00:00
Viktor Barzin
4e7734d327
add healtcheck api endpoint
2025-07-26 13:06:28 +00:00
Viktor Barzin
762408e054
add apprise and send notification when refreshing listings
2025-07-25 21:32:06 +00:00
Viktor Barzin
d4b22deda0
save user queries in redis so that user can refresh the page and still come back to their latest task
2025-07-06 12:02:25 +00:00
Viktor Barzin
20ff91d663
reduce concurrency when fetching images + add retries
2025-07-01 16:12:06 +00:00
Viktor Barzin
a526f81517
add api endpoint for getting all available districts
2025-06-23 21:09:03 +00:00
Viktor Barzin
5a9a837b27
handle serialization of result more generically - if we cannot json dumps, then return string representation
2025-06-23 19:45:31 +00:00
Viktor Barzin
a0d099e62b
expiry tasks after 10 minutes; also handle expired tasks
2025-06-23 19:43:54 +00:00
Viktor Barzin
5610f1d868
remove deleted import for worker
2025-06-22 22:39:51 +00:00
Viktor Barzin
93129333e6
migrate background tasks to celery
2025-06-22 21:18:52 +00:00
Viktor Barzin
744fa7b8a7
use the uvicorn logger; also update dockerfile to run migrations as parto f startup instead of when starting uvicorn
2025-06-22 14:00:47 +00:00
Viktor Barzin
8c65aa0916
add logger for more strucutred logging in python
2025-06-21 22:38:30 +00:00
Viktor Barzin
a73eea4d9e
run alembic migrations on startup
2025-06-21 21:52:51 +00:00
Viktor Barzin
5bd4562205
setup boilerplate for creating background workers and add 1 for dumping listings
2025-06-21 17:25:56 +00:00
Viktor Barzin
4e4a5ece15
start coroutine from the thread instead of the wrapper normal func
2025-06-21 13:48:28 +00:00
Viktor Barzin
a7e0773c0a
add worker api to refresh data in the background
2025-06-21 12:49:04 +00:00
Viktor Barzin
69d15e9a16
add support for querying buying listings as well as by max price
2025-06-18 20:38:50 +00:00
Viktor Barzin
8e41032c6c
setup map component to render correctly and visualize all listings in scrollable components
2025-06-15 21:06:10 +00:00
Viktor Barzin
b995bc2286
add api endpoint for geojson data fetching
2025-06-15 12:42:56 +00:00
Viktor Barzin
aca76111f0
add react vite app with oidc support and configured api calls to api
2025-06-14 15:36:38 +00:00
Viktor Barzin
9b03ab83d2
setup fastapi auth using authentik instance
2025-06-14 13:39:37 +00:00
Viktor Barzin
4ad04775c9
add user auth boilerplate
2025-06-11 21:08:48 +00:00
Viktor Barzin
4a65664f4a
add fastapi endpoint + deps + update dockerfile to run service instead
2025-06-11 20:05:26 +00:00
