Commit graph

14 commits

Author SHA1 Message Date
Viktor Barzin
9b2653ce91
add tenacity to retry transient blockouts by rightmove 2025-06-08 20:59:04 +00:00
Viktor Barzin
0d3393ed94
add sqlmodel + alembic + setup models skeleton to slowly enable transition towards a db 2025-06-03 20:01:30 +00:00
Viktor Barzin
16b88c4aac
remove unused torch pkg as that was consuming 12gb of the container img 2025-06-01 20:31:50 +00:00
Viktor Barzin
01ac24b4b7
use aiohttp to fetch details concurrently 2025-05-17 22:34:27 +00:00
Viktor Barzin
61b8c82592
add ipdb to the dev deps 2025-05-17 22:11:51 +00:00
Kadir
bb2488a63b Removing sqlalchemy and the db part as it was never used 2025-05-12 01:17:45 +01:00
Viktor Barzin
90b531f5d9
[1/n] click-ify - add entrypoint for click script and add
1_dump_listings command

run via:
poetry run python main.py --step dump_listings
2025-05-11 18:59:41 +00:00
Viktor Barzin
0a66efa48a
update pyproject.toml to use latest version of dev dependency section 2025-05-07 21:56:12 +00:00
Kadir
1b69fd4305 adding new dependencies 2025-03-02 17:11:48 +00:00
Kadir
c8c53b8696 Reduced need for routing by limiting to a radius of 7 miles 2024-04-01 15:22:08 +02:00
Kadir
de2639f9c3 fixing bugs, adding properties for querying and analysis§ 2024-03-11 09:44:37 +00:00
Kadir
d108bf11ee adding tesseract OCR for floorplan detection 2024-03-10 22:32:34 +00:00
Kadir
508aa02812 Real crawling scripts and floorplan detection
1. get all listings
2. get all detail jsons
3. get all images
4. get all floorplans
5. detecting floorplans

Also updating dependencies for huggingface etc.
2024-03-10 18:49:39 +00:00
Kadir
e2f7998ee9 merging the visual query answering with the crawler. Monorepo go! 2024-03-01 16:42:48 +01:00
Renamed from pyproject.toml (Browse further)