Viktor Barzin
d205d15c74
Add services layer, tests, streaming UI, and cleanup legacy code
2026-02-06 20:55:10 +00:00
Viktor Barzin
e8293c6042
Add intelligent query splitting to maximize Rightmove data extraction
2026-02-06 20:47:36 +00:00
Viktor Barzin
ced9a153bd
replace pymysql with mysqlclient as it is "better"; also fix an issue in the ui exporter that had wrong imports
2025-10-18 09:58:55 +00:00
Kadir
8d11e4a81c
Fix deps and move to a better local environment
...
- Cleaned up some deps and moved them to the dev section
- Moved from mysqlclient to pymysql which is a python native one which does not require the OS to have the correct mysql lib
- Added a podman compose file so we can have all dependencies in one place easily without the need to install redis or a database locally
For podman install
- podman
- podman-compose (do a poetry sync I think?)
- podman-compose up to start the containers
2025-09-14 19:00:26 +01:00
Viktor Barzin
e5c68f6bb7
add opentelemetry
2025-08-02 17:25:56 +00:00
Viktor Barzin
762408e054
add apprise and send notification when refreshing listings
2025-07-25 21:32:06 +00:00
Viktor Barzin
efe3248c07
migrate to using db connection string from env
2025-06-22 21:16:55 +00:00
Viktor Barzin
519e5f384a
upgrade some poetry files
2025-06-21 20:10:30 +00:00
Viktor Barzin
9b03ab83d2
setup fastapi auth using authentik instance
2025-06-14 13:39:37 +00:00
Viktor Barzin
4a65664f4a
add fastapi endpoint + deps + update dockerfile to run service instead
2025-06-11 20:05:26 +00:00
Viktor Barzin
9b2653ce91
add tenacity to retry transient blockouts by rightmove
2025-06-08 20:59:04 +00:00
Viktor Barzin
0d3393ed94
add sqlmodel + alembic + setup models skeleton to slowly enable transition towards a db
2025-06-03 20:01:30 +00:00
Viktor Barzin
16b88c4aac
remove unused torch pkg as that was consuming 12gb of the container img
2025-06-01 20:31:50 +00:00
Viktor Barzin
01ac24b4b7
use aiohttp to fetch details concurrently
2025-05-17 22:34:27 +00:00
Viktor Barzin
61b8c82592
add ipdb to the dev deps
2025-05-17 22:11:51 +00:00
Kadir
bb2488a63b
Removing sqlalchemy and the db part as it was never used
2025-05-12 01:17:45 +01:00
Viktor Barzin
90b531f5d9
[1/n] click-ify - add entrypoint for click script and add
...
1_dump_listings command
run via:
poetry run python main.py --step dump_listings
2025-05-11 18:59:41 +00:00
Kadir
1b69fd4305
adding new dependencies
2025-03-02 17:11:48 +00:00
Kadir
c8c53b8696
Reduced need for routing by limiting to a radius of 7 miles
2024-04-01 15:22:08 +02:00
Kadir
de2639f9c3
fixing bugs, adding properties for querying and analysis§
2024-03-11 09:44:37 +00:00
Kadir
d108bf11ee
adding tesseract OCR for floorplan detection
2024-03-10 22:32:34 +00:00
Kadir
508aa02812
Real crawling scripts and floorplan detection
...
1. get all listings
2. get all detail jsons
3. get all images
4. get all floorplans
5. detecting floorplans
Also updating dependencies for huggingface etc.
2024-03-10 18:49:39 +00:00
Kadir
e2f7998ee9
merging the visual query answering with the crawler. Monorepo go!
2024-03-01 16:42:48 +01:00
