Viktor Barzin
|
482fff689b
|
parameterize routing logic - extract api key as env var; allow searching dest by address; limit the number of listings to process to prevent accidental api key usage
|
2025-05-18 21:13:50 +00:00 |
|
Viktor Barzin
|
57f477c54d
|
add .env.sample env with api key for routing service exported
|
2025-05-18 19:52:49 +00:00 |
|
Viktor Barzin
|
ea24847cae
|
add stratford region
|
2025-05-18 18:02:32 +00:00 |
|
Viktor Barzin
|
9067d45327
|
add council tax band when exporting
|
2025-05-18 18:02:19 +00:00 |
|
Viktor Barzin
|
5192b1955c
|
export agency
|
2025-05-18 17:41:50 +00:00 |
|
Viktor Barzin
|
9f3e466b23
|
add filter for furnished/unfurnished type for rented listings
|
2025-05-18 17:22:48 +00:00 |
|
Viktor Barzin
|
b873eaf203
|
fix types and format
|
2025-05-18 12:30:49 +00:00 |
|
Viktor Barzin
|
91d3237516
|
remove some empty fields in exploration notebook
|
2025-05-17 23:24:40 +00:00 |
|
Viktor Barzin
|
0b3c644ea3
|
set the min price param in runall
|
2025-05-17 23:18:27 +00:00 |
|
Viktor Barzin
|
2741a77560
|
sort output dataframe by price per sqm
|
2025-05-17 23:14:18 +00:00 |
|
Viktor Barzin
|
47347543d2
|
update runall script to use the click entrypoint
|
2025-05-17 23:14:00 +00:00 |
|
Viktor Barzin
|
b1e0ed170b
|
detect floorplan using asyncio
|
2025-05-17 22:58:35 +00:00 |
|
Viktor Barzin
|
68cc70bd11
|
dump images using aiohttp and concurrently
|
2025-05-17 22:43:05 +00:00 |
|
Viktor Barzin
|
01ac24b4b7
|
use aiohttp to fetch details concurrently
|
2025-05-17 22:34:27 +00:00 |
|
Viktor Barzin
|
61b8c82592
|
add ipdb to the dev deps
|
2025-05-17 22:11:51 +00:00 |
|
Viktor Barzin
|
3e7a144fb4
|
make dumping details async
|
2025-05-17 22:11:33 +00:00 |
|
Viktor Barzin
|
ad879f2d4f
|
convert listings dump to asyncio
|
2025-05-17 21:55:42 +00:00 |
|
Viktor Barzin
|
df24c2c1b7
|
add cli param for querying properties to rent
example:
python main.py --data-dir data/rs2 dump-listings --max-price 3500 --min-bedrooms 2 --max-bedrooms 4 --district islington -t rent
|
2025-05-17 21:22:39 +00:00 |
|
Viktor Barzin
|
bb9afc76fe
|
expose rightmove query parameters as cli options
|
2025-05-17 20:40:44 +00:00 |
|
Viktor Barzin
|
96562c0895
|
add csv exporter command
|
2025-05-17 20:13:28 +00:00 |
|
Viktor Barzin
|
ca5619976f
|
parameterize routing step to work with custom data paths
|
2025-05-14 21:08:03 +00:00 |
|
Viktor Barzin
|
e3abf4b373
|
parameterize detect floorplan step to work with custom data paths
|
2025-05-14 21:05:59 +00:00 |
|
Viktor Barzin
|
07fef7fbab
|
parameterize dump images step to work with custom data paths
|
2025-05-14 21:02:32 +00:00 |
|
Viktor Barzin
|
e424361ed9
|
parameterize dump_detail to use a custom data dir and also move data dir param as part of the click context
|
2025-05-14 20:32:37 +00:00 |
|
Viktor Barzin
|
48d379567b
|
parameterize data path when fetching listings
|
2025-05-14 20:19:08 +00:00 |
|
Viktor Barzin
|
ea56555884
|
refactor main.py click to use click commands to allow passing parameters to commands and enable fetching districts by district name
|
2025-05-14 19:42:08 +00:00 |
|
Kadir
|
bb2488a63b
|
Removing sqlalchemy and the db part as it was never used
|
2025-05-12 01:17:45 +01:00 |
|
Kadir
|
3f4be8b7ff
|
decisions + logger
|
2025-05-12 01:01:48 +01:00 |
|
Viktor Barzin
|
962c9a2f38
|
print all steps that we are running with
|
2025-05-11 19:13:19 +00:00 |
|
Viktor Barzin
|
9134145e02
|
[5/n] click-ify add routing command
run with:
poetry run python main.py --step routing
|
2025-05-11 19:11:23 +00:00 |
|
Viktor Barzin
|
48f694e002
|
[4/n] click-ify add detect floorplan command
run with
poetry run python main.py --step detect_floorplan
|
2025-05-11 19:06:08 +00:00 |
|
Viktor Barzin
|
70e8ef9f95
|
[3/n] click-ify add dump images command
run with
poetry run python main.py --step dump_images
|
2025-05-11 19:04:19 +00:00 |
|
Viktor Barzin
|
c2196c15c1
|
[2/n] click-ify - add 2_dump_detail command
run with
poetry run python main.py --step dump_detail
|
2025-05-11 19:02:23 +00:00 |
|
Viktor Barzin
|
90b531f5d9
|
[1/n] click-ify - add entrypoint for click script and add
1_dump_listings command
run via:
poetry run python main.py --step dump_listings
|
2025-05-11 18:59:41 +00:00 |
|
Viktor Barzin
|
0a66efa48a
|
update pyproject.toml to use latest version of dev dependency section
|
2025-05-07 21:56:12 +00:00 |
|
Viktor Barzin
|
835494d29f
|
reformat most things
|
2025-05-07 21:25:40 +00:00 |
|
Viktor Barzin
|
bd7c781adb
|
remove requirements.txt as we are using poetry...im blind..
|
2025-05-07 21:06:43 +00:00 |
|
Viktor Barzin
|
028eef63a8
|
add readme and .style.yapf for formatting
|
2025-05-07 20:58:22 +00:00 |
|
Viktor Barzin
|
d24b667e73
|
add sqlalchemy to requirements.txt
|
2025-05-07 20:57:28 +00:00 |
|
Viktor Barzin
|
dd2de477bc
|
Add requirements.txt
|
2025-05-07 17:31:29 +00:00 |
|
Kadir
|
8f8956818e
|
refactor routing executor to:
- be nicer
- include checks on floorplan
- include checks on if listing disabled
|
2025-03-31 03:06:41 +01:00 |
|
Kadir
|
cd21bd0bb6
|
notebooks
|
2025-03-30 23:42:29 +01:00 |
|
Kadir
|
2c2adcfa7c
|
improving OCR
|
2025-03-30 23:41:52 +01:00 |
|
Kadir
|
1b69fd4305
|
adding new dependencies
|
2025-03-02 17:11:48 +00:00 |
|
Kadir
|
244de72877
|
reverting isRemoved function to fix
|
2025-02-16 04:44:42 +00:00 |
|
Kadir
|
302ca95cfb
|
fixing full detail dumping
|
2025-02-16 03:02:21 +00:00 |
|
Kadir
|
e0e7853c8c
|
updating districts of relevance to me
|
2025-02-14 21:35:44 +00:00 |
|
Kadir
|
bb765a9312
|
making last_seen to a relative date
|
2025-02-14 21:21:50 +00:00 |
|
Kadir
|
29c8d1960b
|
adding last seen date into the listing
|
2025-01-26 21:41:18 +00:00 |
|
Kadir
|
4b6b8628c2
|
add runall script, update parameters to 4 bed etc and allow incremental updating
|
2024-11-23 22:57:22 +00:00 |
|