Commit graph

18 commits

Author SHA1 Message Date
Viktor Barzin
01ac24b4b7
use aiohttp to fetch details concurrently 2025-05-17 22:34:27 +00:00
Viktor Barzin
3e7a144fb4
make dumping details async 2025-05-17 22:11:33 +00:00
Viktor Barzin
ad879f2d4f
convert listings dump to asyncio 2025-05-17 21:55:42 +00:00
Viktor Barzin
df24c2c1b7
add cli param for querying properties to rent
example:
python main.py --data-dir data/rs2 dump-listings --max-price 3500 --min-bedrooms 2 --max-bedrooms 4 --district islington -t rent
2025-05-17 21:22:39 +00:00
Viktor Barzin
bb9afc76fe
expose rightmove query parameters as cli options 2025-05-17 20:40:44 +00:00
Viktor Barzin
96562c0895
add csv exporter command 2025-05-17 20:13:28 +00:00
Viktor Barzin
ca5619976f
parameterize routing step to work with custom data paths 2025-05-14 21:08:03 +00:00
Viktor Barzin
e3abf4b373
parameterize detect floorplan step to work with custom data paths 2025-05-14 21:05:59 +00:00
Viktor Barzin
07fef7fbab
parameterize dump images step to work with custom data paths 2025-05-14 21:02:32 +00:00
Viktor Barzin
e424361ed9
parameterize dump_detail to use a custom data dir and also move data dir param as part of the click context 2025-05-14 20:32:37 +00:00
Viktor Barzin
48d379567b
parameterize data path when fetching listings 2025-05-14 20:19:08 +00:00
Viktor Barzin
ea56555884 refactor main.py click to use click commands to allow passing parameters to commands and enable fetching districts by district name 2025-05-14 19:42:08 +00:00
Viktor Barzin
962c9a2f38
print all steps that we are running with 2025-05-11 19:13:19 +00:00
Viktor Barzin
9134145e02
[5/n] click-ify add routing command
run with:
poetry run python main.py --step routing
2025-05-11 19:11:23 +00:00
Viktor Barzin
48f694e002
[4/n] click-ify add detect floorplan command
run with
poetry run python main.py --step detect_floorplan
2025-05-11 19:06:08 +00:00
Viktor Barzin
70e8ef9f95
[3/n] click-ify add dump images command
run with
poetry run python main.py --step dump_images
2025-05-11 19:04:19 +00:00
Viktor Barzin
c2196c15c1
[2/n] click-ify - add 2_dump_detail command
run with
poetry run python main.py --step dump_detail
2025-05-11 19:02:23 +00:00
Viktor Barzin
90b531f5d9
[1/n] click-ify - add entrypoint for click script and add
1_dump_listings command

run via:
poetry run python main.py --step dump_listings
2025-05-11 18:59:41 +00:00