Flatten repo structure: move crawler/ to root, remove vqa/ and immoweb/
The crawler subdirectory was the only active project. Moving it to the repo root simplifies paths and removes the unnecessary nesting. The vqa/ and immoweb/ directories were legacy/unused and have been removed. Updated .drone.yml, .gitignore, .claude/ docs, and skills to reflect the new flat structure.
This commit is contained in:
parent
e2247be700
commit
eafbc1ac52
221 changed files with 70 additions and 146140 deletions
|
|
@ -2,11 +2,7 @@
|
|||
|
||||
## Project Overview
|
||||
|
||||
A real estate listing aggregation platform that scrapes Rightmove UK listings, extracts square meter data from floorplan images via OCR, calculates transit routes, and serves an interactive map-based web UI. The repo contains three sub-projects:
|
||||
|
||||
- **`crawler/`** — Main application (Python backend + React frontend). Has its own `CLAUDE.md` with detailed architecture docs.
|
||||
- **`immoweb/`** — Separate scraper (Node.js, legacy/reference).
|
||||
- **`vqa/`** — Visual QA / testing tooling.
|
||||
A real estate listing aggregation platform that scrapes Rightmove UK listings, extracts square meter data from floorplan images via OCR, calculates transit routes, and serves an interactive map-based web UI.
|
||||
|
||||
## Command Execution
|
||||
|
||||
|
|
@ -19,18 +15,18 @@ See `.claude/skills/` for detailed skills on dev environment, building, and depl
|
|||
|
||||
## Quick Reference
|
||||
|
||||
| Action | Command | Where |
|
||||
|--------|---------|-------|
|
||||
| Start all services | `./start.sh` | `crawler/` |
|
||||
| Rebuild & start | `./start.sh --build` | `crawler/` |
|
||||
| Stop services | `./start.sh --down` | `crawler/` |
|
||||
| Run tests | `pytest tests/ -v --cov=. --cov-report=term-missing` | `crawler/` |
|
||||
| Type check | `mypy .` | `crawler/` |
|
||||
| Format code | `yapf --style .style.yapf --recursive .` | `crawler/` |
|
||||
| Lint (CI runs Ruff) | `ruff check .` | `crawler/` |
|
||||
| DB migration | `alembic upgrade head` | `crawler/` |
|
||||
| New migration | `alembic revision -m "description"` | `crawler/` |
|
||||
| Frontend dev | `cd frontend && ./start.sh` | `crawler/` |
|
||||
| Action | Command |
|
||||
|--------|---------|
|
||||
| Start all services | `./start.sh` |
|
||||
| Rebuild & start | `./start.sh --build` |
|
||||
| Stop services | `./start.sh --down` |
|
||||
| Run tests | `pytest tests/ -v --cov=. --cov-report=term-missing` |
|
||||
| Type check | `mypy .` |
|
||||
| Format code | `yapf --style .style.yapf --recursive .` |
|
||||
| Lint (CI runs Ruff) | `ruff check .` |
|
||||
| DB migration | `alembic upgrade head` |
|
||||
| New migration | `alembic revision -m "description"` |
|
||||
| Frontend dev | `cd frontend && ./start.sh` |
|
||||
|
||||
## Tech Stack
|
||||
|
||||
|
|
@ -49,7 +45,7 @@ See `.claude/skills/` for detailed skills on dev environment, building, and depl
|
|||
- **Repository pattern:** `repositories/` for database queries. Don't put raw SQL in services or API.
|
||||
- **Tests:** pytest with `pytest-asyncio` (auto mode). Unit tests in `tests/unit/`, integration in `tests/integration/`.
|
||||
|
||||
## Architecture Layers (in `crawler/`)
|
||||
## Architecture Layers
|
||||
|
||||
```
|
||||
API (api/app.py) ←→ Services (services/) ←→ Repositories (repositories/)
|
||||
|
|
@ -75,7 +71,7 @@ API (api/app.py) ←→ Services (services/) ←→ Repositories (repositori
|
|||
|
||||
## Environment Variables
|
||||
|
||||
See `crawler/.env.sample` for the full list. Key ones:
|
||||
See `.env.sample` for the full list. Key ones:
|
||||
|
||||
- `DB_CONNECTION_STRING` — Database URL
|
||||
- `CELERY_BROKER_URL` / `CELERY_RESULT_BACKEND` — Redis URLs
|
||||
|
|
@ -92,4 +88,4 @@ See `crawler/.env.sample` for the full list. Key ones:
|
|||
|
||||
## Directories to Ignore
|
||||
|
||||
- `node_modules/`, `__pycache__/`, `.idea/`, `crawler/data/`, `venv/`, `_cache/`
|
||||
- `node_modules/`, `__pycache__/`, `.idea/`, `data/`, `venv/`, `_cache/`
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue