Viktor Barzin
|
4c7fe8927b
|
add minimal working version where we fetch data and visualize it corectly
|
2025-06-15 12:48:52 +00:00 |
|
Viktor Barzin
|
b995bc2286
|
add api endpoint for geojson data fetching
|
2025-06-15 12:42:56 +00:00 |
|
Viktor Barzin
|
7602c3762b
|
properly ignore caddy dev dir
|
2025-06-15 12:42:23 +00:00 |
|
Viktor Barzin
|
0dc6a30dbd
|
fail start script if caddy is not installed
|
2025-06-14 15:52:27 +00:00 |
|
Viktor Barzin
|
1b97cf0dc3
|
exclude caddy files
|
2025-06-14 15:47:36 +00:00 |
|
Viktor Barzin
|
c7d996dbeb
|
rename frontend directory to frontend
|
2025-06-14 15:43:14 +00:00 |
|
Viktor Barzin
|
71a9b69e04
|
do not include caddy dev dir in repo as that is generated
|
2025-06-14 15:40:52 +00:00 |
|
Viktor Barzin
|
aca76111f0
|
add react vite app with oidc support and configured api calls to api
|
2025-06-14 15:36:38 +00:00 |
|
Viktor Barzin
|
ff2f141b75
|
allow limiting the number of listings fetched
|
2025-06-14 13:39:56 +00:00 |
|
Viktor Barzin
|
9b03ab83d2
|
setup fastapi auth using authentik instance
|
2025-06-14 13:39:37 +00:00 |
|
Viktor Barzin
|
4ad04775c9
|
add user auth boilerplate
|
2025-06-11 21:08:48 +00:00 |
|
Viktor Barzin
|
4a65664f4a
|
add fastapi endpoint + deps + update dockerfile to run service instead
|
2025-06-11 20:05:26 +00:00 |
|
Viktor Barzin
|
45af54e4e4
|
do not ignore data directory, just the contents; allows creating the sqlite db inside
|
2025-06-09 22:08:01 +00:00 |
|
Viktor Barzin
|
073020c6d2
|
reduce size of json by pulling interesting fields above instead of relying on the entire thing
|
2025-06-09 21:46:33 +00:00 |
|
Viktor Barzin
|
296a4e7603
|
bugfix fetching transactions to make use of db cache
|
2025-06-09 21:45:29 +00:00 |
|
Viktor Barzin
|
9b2653ce91
|
add tenacity to retry transient blockouts by rightmove
|
2025-06-08 20:59:04 +00:00 |
|
Viktor Barzin
|
289206afc0
|
some cleanups
|
2025-06-08 20:58:28 +00:00 |
|
Viktor Barzin
|
831c45e1f5
|
use cached data when fetching details
|
2025-06-08 18:23:25 +00:00 |
|
Viktor Barzin
|
3785d01009
|
migrate immoweb exporter to use models
|
2025-06-08 18:18:38 +00:00 |
|
Viktor Barzin
|
e317d2ec54
|
use query params to filter out models; also make csv exporter work with models
|
2025-06-08 17:01:33 +00:00 |
|
Viktor Barzin
|
80c335ba04
|
migrate routing command to use the models and store data there
|
2025-06-08 11:45:05 +00:00 |
|
Viktor Barzin
|
325823e631
|
refactor detect floorplan to use model listings
|
2025-06-07 14:30:32 +00:00 |
|
Viktor Barzin
|
ba87d07cd2
|
migrate dump images command to use model listings
|
2025-06-07 13:56:00 +00:00 |
|
Viktor Barzin
|
4f5a934fa9
|
refactor dump listings to start using model instead of the data_access object
|
2025-06-07 12:46:53 +00:00 |
|
Viktor Barzin
|
842f7cefbe
|
merge dump listings and dump details commands - fetch both details and listings in the same command
|
2025-06-07 12:00:23 +00:00 |
|
Viktor Barzin
|
29213f3d26
|
refactor the semaphore when dumping listings
|
2025-06-06 20:09:16 +00:00 |
|
Viktor Barzin
|
b7a2ea75aa
|
add json field to store any additional blob of data that may be missing; also populate db when dumping listings
|
2025-06-06 19:57:50 +00:00 |
|
Viktor Barzin
|
8b2025e700
|
add command to dump existing listing from fs to db
|
2025-06-04 22:03:55 +00:00 |
|
Viktor Barzin
|
f7fb891648
|
add models
|
2025-06-04 21:09:29 +00:00 |
|
Viktor Barzin
|
bc5c258170
|
add robots.txt
|
2025-06-04 20:50:50 +00:00 |
|
Viktor Barzin
|
e50f33ed14
|
filter out for furnished properties only
|
2025-06-03 20:02:29 +00:00 |
|
Viktor Barzin
|
0d3393ed94
|
add sqlmodel + alembic + setup models skeleton to slowly enable transition towards a db
|
2025-06-03 20:01:30 +00:00 |
|
Viktor Barzin
|
8c646a5322
|
touch geojson file to ensure it exists before opening to write
|
2025-06-01 22:13:22 +00:00 |
|
Viktor Barzin
|
16b88c4aac
|
remove unused torch pkg as that was consuming 12gb of the container img
|
2025-06-01 20:31:50 +00:00 |
|
Viktor Barzin
|
0bdfeec195
|
make the csv exporter use the filtering params to allow exporting customizable reports
|
2025-06-01 20:11:00 +00:00 |
|
Viktor Barzin
|
0acd417d34
|
add option to filter for min sqm per listing
|
2025-06-01 19:26:24 +00:00 |
|
Viktor Barzin
|
1c77c78518
|
add price history in the ui if theres more than 1 price history
|
2025-06-01 18:13:29 +00:00 |
|
Viktor Barzin
|
ab5c4ce0a6
|
refactor runall script to share filter options when dumping listings and when exporting to immoweb
|
2025-06-01 18:10:09 +00:00 |
|
Viktor Barzin
|
2e3e900ec3
|
store price history when fetching details. this will be helpful later on as we start periodically scraping to store prices over time
|
2025-06-01 18:09:27 +00:00 |
|
Viktor Barzin
|
8b90ecde11
|
add filter for last seen days
|
2025-06-01 15:26:38 +00:00 |
|
Viktor Barzin
|
11315359d2
|
reuse query params when exporting to immoweb and allow filtering from available date
|
2025-06-01 15:17:14 +00:00 |
|
Viktor Barzin
|
a23a5ae192
|
extract common listing filtering options into a decorator to enable reuse between commands
|
2025-06-01 12:11:15 +00:00 |
|
Viktor Barzin
|
c543be7ff6
|
add let date available from parameter
|
2025-06-01 00:51:44 +00:00 |
|
Viktor Barzin
|
9735db72a0
|
limit the number of concurrenct requests when dumping listings as right move blocks us
|
2025-06-01 00:27:12 +00:00 |
|
Viktor Barzin
|
24bf44caf9
|
trust env when fetching details and listings. this allows setting proxy env vars. I use this in conjunction with a tor proxy to avoid ip-bans from rightmove
|
2025-06-01 00:12:31 +00:00 |
|
Viktor Barzin
|
0b9d50af47
|
reformat with black; looks better
|
2025-05-31 23:50:43 +00:00 |
|
Viktor Barzin
|
1122f5a96f
|
use aiohttp for dumping listing query
|
2025-05-31 23:48:45 +00:00 |
|
Viktor Barzin
|
de1416de7e
|
add last seen in immoweb
|
2025-05-31 23:32:59 +00:00 |
|
Viktor Barzin
|
b312a53526
|
remove exporting to csv in the runall script - we now use immoweb as a frontend
|
2025-05-31 23:31:41 +00:00 |
|
Viktor Barzin
|
f9a7620bf0
|
recalculate floorplans when theres only an empty object
|
2025-05-31 23:31:06 +00:00 |
|