Kadir
c8c53b8696
Reduced need for routing by limiting to a radius of 7 miles
2024-04-01 15:22:08 +02:00
Kadir
6a43a7f485
adding tasks and updating exploration notebook
2024-04-01 15:10:15 +02:00
Kadir
a7a4f88a39
sort districts
2024-03-30 19:24:35 +01:00
Kadir
acc67192c9
add counter to 2_dump_details to understand how many new are crawled
2024-03-30 19:24:03 +01:00
Kadir
5720e68547
Rewriting 1_dump to enabling crawling of more real estate
2024-03-30 19:23:19 +01:00
Kadir
40285245d5
remove debug print statements
2024-03-30 18:35:32 +01:00
Kadir
ec86b572b3
adding districts and location identifier to search
2024-03-30 18:31:49 +01:00
Kadir
4c40462bb8
adding property type
2024-03-25 20:58:35 +00:00
Kadir
d777558b34
ruff format
2024-03-25 20:48:48 +00:00
Kadir
37e3e8ad6f
using ruff
2024-03-25 20:48:27 +00:00
Kadir
d5a0964b12
exploration notebook
2024-03-25 20:47:50 +00:00
Kadir
47f7b2b672
Adding initial walking time and identifier to the information
2024-03-25 20:47:31 +00:00
Kadir
ce632c795d
fixing gitignores
2024-03-25 20:47:15 +00:00
Kadir
e79e24ff98
fix: remove debug statement
2024-03-18 01:02:06 +00:00
Kadir
4dea766a12
fixing floorplan detection and adding recalculation method
2024-03-18 00:56:39 +00:00
Kadir
335adc0856
add routing, incremental crawling, travel time, lease and development
2024-03-13 16:24:57 +00:00
Kadir
d4f87bed76
add routing and util to the right places
2024-03-13 16:22:53 +00:00
Kadir
a9b8d4d630
adding musthave and lastXdays
2024-03-13 16:21:54 +00:00
Kadir
36258d877f
crawling for 3 and refactoring to allow incremental crawls
2024-03-11 14:43:53 +00:00
Kadir
de2639f9c3
fixing bugs, adding properties for querying and analysis§
2024-03-11 09:44:37 +00:00
Kadir
d108bf11ee
adding tesseract OCR for floorplan detection
2024-03-10 22:32:34 +00:00
Kadir
508aa02812
Real crawling scripts and floorplan detection
...
1. get all listings
2. get all detail jsons
3. get all images
4. get all floorplans
5. detecting floorplans
Also updating dependencies for huggingface etc.
2024-03-10 18:49:39 +00:00
Kadir
46bb641026
adding floorplans, detail json, refactored the folders
2024-03-07 22:02:09 +00:00
Kadir
e2f7998ee9
merging the visual query answering with the crawler. Monorepo go!
2024-03-01 16:42:48 +01:00
Kadir
85686a8b24
update tasks
2024-02-29 20:00:18 +01:00
Kadir
83b3f1d3fd
Adding the duration and distance aggregation from the routes api
2024-02-29 19:58:54 +01:00
Kadir
ce9956ea52
Routes API implemented for travel time
2024-02-29 18:14:42 +01:00
Kadir
5349088ba0
routing temp
2024-01-29 01:09:56 +00:00
Kadir Tugan
f30115eecd
Fixing __repr__ for RightmoveListing and adding a small testing script
2023-11-18 13:25:00 +02:00
Kadir Tugan
8698b41e0e
adding lat/lon to the db
2023-11-18 13:09:03 +02:00
Kadir Tugan
72ff6a7202
adding query to fetch a single detailpage
2023-11-18 12:56:15 +02:00
Kadir Tugan
cfd98ed40c
adding tasks file
2023-11-18 12:46:11 +02:00
Kadir Tugan
31d760cf30
adding more parameters to the function
2023-11-18 12:38:54 +02:00
Kadir Tugan
d49b11e28b
ruff format
2023-11-18 12:30:04 +02:00
Kadir Tugan
65fc4cdda4
updating poetry
2023-11-18 12:20:22 +02:00
Kadir Tugan
200caadabd
Makefile to remake db
2023-11-06 00:33:49 +00:00
Kadir Tugan
4ee7ae16c4
Initial commit: Rightmove fetch, database, caching, poetry etc.
...
- Fetching rightmove listing api
- Memoizing query
- Writing to sqlite database with sqlalchemy
- Poetry dependencies
2023-11-06 00:31:58 +00:00
