wrongmove/crawler
2025-05-14 21:05:59 +00:00
..
data decisions + logger 2025-05-12 01:01:48 +01:00
proof_of_concept ruff format 2024-03-25 20:48:48 +00:00
rec refactor main.py click to use click commands to allow passing parameters to commands and enable fetching districts by district name 2025-05-14 19:42:08 +00:00
.style.yapf add readme and .style.yapf for formatting 2025-05-07 20:58:22 +00:00
1_dump_listings.py parameterize data path when fetching listings 2025-05-14 20:19:08 +00:00
2_dump_detail.py parameterize dump_detail to use a custom data dir and also move data dir param as part of the click context 2025-05-14 20:32:37 +00:00
3_dump_images.py parameterize dump images step to work with custom data paths 2025-05-14 21:02:32 +00:00
4_detect_floorplan.py parameterize detect floorplan step to work with custom data paths 2025-05-14 21:05:59 +00:00
5_routing.py [5/n] click-ify add routing command 2025-05-11 19:11:23 +00:00
9_recalculate_regex_squaremeter.py ruff format 2024-03-25 20:48:48 +00:00
91_recalculate_floorplan.py reformat most things 2025-05-07 21:25:40 +00:00
data_access.py parameterize dump images step to work with custom data paths 2025-05-14 21:02:32 +00:00
exploration.ipynb decisions + logger 2025-05-12 01:01:48 +01:00
GUIDE merging the visual query answering with the crawler. Monorepo go! 2024-03-01 16:42:48 +01:00
logger.py decisions + logger 2025-05-12 01:01:48 +01:00
main.py parameterize detect floorplan step to work with custom data paths 2025-05-14 21:05:59 +00:00
main_tmp.py ruff format 2024-03-25 20:48:48 +00:00
ocr.ipynb notebooks 2025-03-30 23:42:29 +01:00
poetry.lock Removing sqlalchemy and the db part as it was never used 2025-05-12 01:17:45 +01:00
pyproject.toml Removing sqlalchemy and the db part as it was never used 2025-05-12 01:17:45 +01:00
README.md add readme and .style.yapf for formatting 2025-05-07 20:58:22 +00:00
runall.sh add runall script, update parameters to 4 bed etc and allow incremental updating 2024-11-23 22:57:22 +00:00
TASKS.md adding tasks and updating exploration notebook 2024-04-01 15:10:15 +02:00

Setup

pip install -r requirements.txt

Formatting

yapf --style .style.yapf --recursive .

For VSCode - install yapf extension. Enable formatting using yap and the style file in this repo (there may be an easier way; I put this in my user settings json):

{
    "[python]": {
        "editor.formatOnSaveMode": "file",
        "editor.formatOnSave": true,
        "editor.defaultFormatter": "eeyore.yapf",
        "editor.formatOnType": false
      },
      "yapf.args": ["--style", "/home/wizard/code/realestate-crawler/crawler/.style.yapf"]
}