History

Viktor Barzin 0dc6a30dbd fail start script if caddy is not installed		2025-06-14 15:52:27 +00:00
..
alembic	add user auth boilerplate	2025-06-11 21:08:48 +00:00
api	add react vite app with oidc support and configured api calls to api	2025-06-14 15:36:38 +00:00
data	decisions + logger	2025-05-12 01:01:48 +01:00
frontend	fail start script if caddy is not installed	2025-06-14 15:52:27 +00:00
models	add user auth boilerplate	2025-06-11 21:08:48 +00:00
proof_of_concept	ruff format	2024-03-25 20:48:48 +00:00
rec	add tenacity to retry transient blockouts by rightmove	2025-06-08 20:59:04 +00:00
repositories	allow limiting the number of listings fetched	2025-06-14 13:39:56 +00:00
.dockerignore	do not ignore data directory, just the contents; allows creating the sqlite db inside	2025-06-09 22:08:01 +00:00
.env.sample	parameterize routing logic - extract api key as env var; allow searching dest by address; limit the number of listings to process to prevent accidental api key usage	2025-05-18 21:13:50 +00:00
.style.yapf	add readme and .style.yapf for formatting	2025-05-07 20:58:22 +00:00
1_dump_listings.py	bugfix fetching transactions to make use of db cache	2025-06-09 21:45:29 +00:00
3_dump_images.py	some cleanups	2025-06-08 20:58:28 +00:00
4_detect_floorplan.py	refactor detect floorplan to use model listings	2025-06-07 14:30:32 +00:00
5_routing.py	migrate routing command to use the models and store data there	2025-06-08 11:45:05 +00:00
9_recalculate_regex_squaremeter.py	ruff format	2024-03-25 20:48:48 +00:00
91_recalculate_floorplan.py	reformat most things	2025-05-07 21:25:40 +00:00
alembic.ini	add sqlmodel + alembic + setup models skeleton to slowly enable transition towards a db	2025-06-03 20:01:30 +00:00
csv_exporter.py	use query params to filter out models; also make csv exporter work with models	2025-06-08 17:01:33 +00:00
data_access.py	bugfix fetching transactions to make use of db cache	2025-06-09 21:45:29 +00:00
database.py	add command to dump existing listing from fs to db	2025-06-04 22:03:55 +00:00
Dockerfile	add fastapi endpoint + deps + update dockerfile to run service instead	2025-06-11 20:05:26 +00:00
exploration.ipynb	remove some empty fields in exploration notebook	2025-05-17 23:24:40 +00:00
GUIDE	merging the visual query answering with the crawler. Monorepo go!	2024-03-01 16:42:48 +01:00
logger.py	reformat with black; looks better	2025-05-31 23:50:43 +00:00
main.py	some cleanups	2025-06-08 20:58:28 +00:00
main_tmp.py	ruff format	2024-03-25 20:48:48 +00:00
ocr.ipynb	notebooks	2025-03-30 23:42:29 +01:00
poetry.lock	setup fastapi auth using authentik instance	2025-06-14 13:39:37 +00:00
pyproject.toml	setup fastapi auth using authentik instance	2025-06-14 13:39:37 +00:00
README.md	add readme and .style.yapf for formatting	2025-05-07 20:58:22 +00:00
runall.sh	some cleanups	2025-06-08 20:58:28 +00:00
TASKS.md	adding tasks and updating exploration notebook	2024-04-01 15:10:15 +02:00
ui_exporter.py	reduce size of json by pulling interesting fields above instead of relying on the entire thing	2025-06-09 21:46:33 +00:00

README.md

Setup

pip install -r requirements.txt

Formatting

yapf --style .style.yapf --recursive .

For VSCode - install yapf extension. Enable formatting using yap and the style file in this repo (there may be an easier way; I put this in my user settings json):

{
    "[python]": {
        "editor.formatOnSaveMode": "file",
        "editor.formatOnSave": true,
        "editor.defaultFormatter": "eeyore.yapf",
        "editor.formatOnType": false
      },
      "yapf.args": ["--style", "/home/wizard/code/realestate-crawler/crawler/.style.yapf"]
}