No description
Find a file
Viktor Barzin 692d68c270
Unpause deployments and add ReplicaSet/status debug output
Add spec.paused: null to the strategic merge patch to clear any
paused state that might be blocking rollouts. Also add detailed
deployment status debug output (conditions, replica counts,
paused state) and list ReplicaSets to diagnose why new pods
aren't being created despite spec changes.
2026-02-22 19:40:44 +00:00
.claude Add Claude agent definitions [ci skip] 2026-02-09 23:02:30 +00:00
.github/workflows adding ruff auto check for pull requests as well as fixing all ruff errors (#1) 2025-09-14 19:40:18 +01:00
alembic fix: remove duplicate alembic migration causing multiple heads on startup 2026-02-21 16:08:02 +00:00
api Add frontend performance metrics pipeline to Prometheus 2026-02-22 17:30:29 +00:00
cli Add debug CLI listings subcommand (list, detail, stream, refresh) 2026-02-22 15:17:56 +00:00
config Add configurable request timeout and retry on TimeoutError 2026-02-21 17:50:36 +00:00
docs Add debug CLI implementation plan 2026-02-22 15:08:34 +00:00
frontend Improve frontend UI/UX: accessibility, discoverability, and cleanup 2026-02-22 18:47:09 +00:00
grafana Fix celery metrics not reaching Prometheus and update Grafana dashboard 2026-02-22 17:58:20 +00:00
k8s Set explicit resource limits for API and celery-beat pods 2026-02-22 13:20:56 +00:00
models fix: remove duplicate alembic migration causing multiple heads on startup 2026-02-21 16:08:02 +00:00
rec Add configurable request timeout and retry on TimeoutError 2026-02-21 17:50:36 +00:00
repositories Remove watchdog and tqdm dependencies, replace with logging 2026-02-21 19:39:49 +00:00
scripts Add setup-test-venv.sh for running tests without Docker [ci skip] 2026-02-14 11:44:49 +00:00
services Remove watchdog and tqdm dependencies, replace with logging 2026-02-21 19:39:49 +00:00
tasks Fix metric imports: use module-level access instead of name imports 2026-02-14 11:21:49 +00:00
tests Add debug CLI decisions, POIs, and tasks subcommands 2026-02-22 15:15:30 +00:00
utils Flatten repo structure: move crawler/ to root, remove vqa/ and immoweb/ 2026-02-07 23:01:20 +00:00
.dockerignore Flatten repo structure: move crawler/ to root, remove vqa/ and immoweb/ 2026-02-07 23:01:20 +00:00
.drone.yml Unpause deployments and add ReplicaSet/status debug output 2026-02-22 19:40:44 +00:00
.env.sample Add OSRM and OTP Docker services with setup scripts 2026-02-08 13:16:10 +00:00
.gitignore Add .worktrees/ to .gitignore for git worktree isolation 2026-02-21 15:48:34 +00:00
.style.yapf Flatten repo structure: move crawler/ to root, remove vqa/ and immoweb/ 2026-02-07 23:01:20 +00:00
alembic.ini Flatten repo structure: move crawler/ to root, remove vqa/ and immoweb/ 2026-02-07 23:01:20 +00:00
celery_app.py Fix celery metrics not reaching Prometheus and update Grafana dashboard 2026-02-22 17:58:20 +00:00
CLAUDE.md Add setup-test-venv.sh for running tests without Docker [ci skip] 2026-02-14 11:44:49 +00:00
csv_exporter.py Replace pandas with stdlib csv, apprise with direct Slack webhook, switch to opencv-headless 2026-02-21 19:47:10 +00:00
data_access.py Flatten repo structure: move crawler/ to root, remove vqa/ and immoweb/ 2026-02-07 23:01:20 +00:00
database.py Fall back to SQLite default when DB_CONNECTION_STRING is unset 2026-02-10 22:38:52 +00:00
docker-compose.yml Add structured JSON logging, OTel business metrics, and Grafana dashboard 2026-02-14 10:59:12 +00:00
Dockerfile Parallelize API tests and build, gate publish on test pass 2026-02-21 23:46:16 +00:00
GUIDE Flatten repo structure: move crawler/ to root, remove vqa/ and immoweb/ 2026-02-07 23:01:20 +00:00
listing_processor.py Fix metric imports: use module-level access instead of name imports 2026-02-14 11:21:49 +00:00
logging_config.py Add structured JSON logging, OTel business metrics, and Grafana dashboard 2026-02-14 10:59:12 +00:00
main.py Add debug CLI listings subcommand (list, detail, stream, refresh) 2026-02-22 15:17:56 +00:00
notifications.py Replace pandas with stdlib csv, apprise with direct Slack webhook, switch to opencv-headless 2026-02-21 19:47:10 +00:00
podman-compose.yml Flatten repo structure: move crawler/ to root, remove vqa/ and immoweb/ 2026-02-07 23:01:20 +00:00
poetry.lock Regenerate requirements.txt after dependency cleanup 2026-02-21 19:47:15 +00:00
pyproject.toml Replace pandas with stdlib csv, apprise with direct Slack webhook, switch to opencv-headless 2026-02-21 19:47:10 +00:00
README.md Flatten repo structure: move crawler/ to root, remove vqa/ and immoweb/ 2026-02-07 23:01:20 +00:00
redis_repository.py Flatten repo structure: move crawler/ to root, remove vqa/ and immoweb/ 2026-02-07 23:01:20 +00:00
requirements.txt Regenerate requirements.txt after dependency cleanup 2026-02-21 19:47:15 +00:00
runall.sh Flatten repo structure: move crawler/ to root, remove vqa/ and immoweb/ 2026-02-07 23:01:20 +00:00
start.sh Add shared pre-push hook to run tests before pushing 2026-02-14 11:36:26 +00:00
TASKS.md Flatten repo structure: move crawler/ to root, remove vqa/ and immoweb/ 2026-02-07 23:01:20 +00:00
ui_exporter.py Fix photo extraction: look for both 'photos' and 'images' keys 2026-02-22 01:21:50 +00:00

Setup

  1. Instal deps:
poetry install && cp .env.sample .env
  1. Check .env if you want to customize settings for broker and db
  2. run ./start.sh

This starts the backend

To start the fronend:

cd frontend && cp .env.sample .env

Change the DEV_HOST to any name you want to use to access the web interface.

Next, setup the DNS record (e.g in your /etc/hosts) file. This is important as auth is done via external [authentik] service that needs to redirect to a name.

Run ./start.sh

This starts a Caddy proxy with correct certificates, and npm dev server. All requests going to the frontend are forwarded to the npm server and the ones for the backed (that go to /api/*) are forwarded to the backend service.

Lastly, reachout to Viktor to allowlist your DEV_HOST so that authentik can authorize callbacks to your host.

Formatting

yapf --style .style.yapf --recursive .

For VSCode - install yapf extension. Enable formatting using yap and the style file in this repo (there may be an easier way; I put this in my user settings json):

{
    "[python]": {
        "editor.formatOnSaveMode": "file",
        "editor.formatOnSave": true,
        "editor.defaultFormatter": "eeyore.yapf",
        "editor.formatOnType": false
      },
      "yapf.args": ["--style", "/home/wizard/code/realestate-crawler/crawler/.style.yapf"]
}

ADB commands (from /Applications/BlueStacks.app/Contents/MacOS):

Set proxy

./hd-adb shell settings put global http_proxy 192.168.9.110:8080

Disable proxy:

/hd-adb shell settings put global http_proxy :0

Connect adb

./hd-adb connect 127.0.0.1:5555

Disconnect adb

/hd-adb disconnect