Commit graph

450 commits

Author SHA1 Message Date
Viktor Barzin
7a35489f35
Add debug CLI districts subcommand 2026-02-22 15:12:02 +00:00
Viktor Barzin
df0fa41586
Add cli/ package with shared debug context 2026-02-22 15:11:58 +00:00
Viktor Barzin
9f38b1ea9c
Add debug CLI implementation plan 2026-02-22 15:08:34 +00:00
Viktor Barzin
01e928151a
Add debug CLI design doc 2026-02-22 15:05:18 +00:00
Viktor Barzin
81ff9d9e41
Move hexgrid heatmap computation to Web Worker
R-tree building, hex grid generation, and percentile sorting now run
off the main thread, eliminating 20s+ UI freezes on large datasets.
The old bundled HexgridHeatmap.js is replaced by a typed worker +
main-thread client with dual R-trees (worker for grid gen, main
thread for synchronous click queries).
2026-02-22 15:04:37 +00:00
Viktor Barzin
c6f7b47446
Fix e2e test for adaptive first batch size
test_large_batch_streaming expected 4 batches (200/50) but the
adaptive first batch splits the first batch into 5+45, producing
5 batches total.
2026-02-22 13:48:36 +00:00
Viktor Barzin
8ef6868881
Eliminate frontend POI waterfall for faster initial load
Listing stream fires immediately on auth without waiting for POI
fetch. POI distances are not needed for initial rendering and are
only computed when user selects POI metric or sets travel filters.
This saves ~200-500ms on initial load and keeps the stream on the
cached Redis path.
2026-02-22 13:31:10 +00:00
Viktor Barzin
3885fd52fe
Add bulk POI distances endpoint for decoupled loading
New GET /api/poi/distances/bulk returns all POI distances keyed by
listing ID, allowing the frontend to fetch distances separately
from the listing stream and keep the stream on the cached path.
2026-02-22 13:29:35 +00:00
Viktor Barzin
2f3d2dc480
Add Server-Timing header to streaming endpoint
Reports cache_check latency in the response header, visible in
browser DevTools Network tab for ongoing performance monitoring.
2026-02-22 13:27:11 +00:00
Viktor Barzin
e53b1e120a
Defer decision ID fetch to after metadata message
Decision IDs are now loaded inside the streaming generators after
the metadata message is yielded, eliminating a blocking DB query
from the pre-stream path (~100-200ms improvement to TTFB).
2026-02-22 13:26:17 +00:00
Viktor Barzin
e99006e2f9
Send smaller first batch (5 features) for faster first paint
Subsequent batches use the normal batch_size (default 50). This
reduces server-side time-to-first-property by ~10x since only 5
features need to be serialized before the first yield.
2026-02-22 13:24:01 +00:00
Viktor Barzin
9179456bf7
Set explicit resource limits for API and celery-beat pods
Overrides LimitRange defaults (500m CPU) which caused kernel CPU
throttling during streaming requests. API gets 2000m CPU limit,
celery-beat gets 200m.
2026-02-22 13:20:56 +00:00
Viktor Barzin
8db7b60493
Add time-to-first-property implementation plan
7 tasks: K8s resource limits, adaptive first batch, deferred
decision fetch, Server-Timing headers, bulk POI distances endpoint,
frontend waterfall elimination, and end-to-end verification.
2026-02-22 13:16:42 +00:00
Viktor Barzin
6d653dba63
Add time-to-first-property performance optimization design
Measured baseline: 877ms TTFP cold, 334ms warm, with CPU throttling
confirmed (500m limit, 10-12 throttle events per streaming request).
Plan covers: CPU limit bump, frontend waterfall elimination, adaptive
first batch, deferred decision filtering, POI decoupling, and
Server-Timing headers.
2026-02-22 13:14:01 +00:00
Viktor Barzin
9dc011754b
Add cancel button to streaming progress bar
The X button aborts the in-flight fetch via AbortController,
which was already wired up but had no UI trigger. Works for
both desktop and mobile views.
2026-02-22 02:04:56 +00:00
Viktor Barzin
301579c255
Separate photo swipe from card swipe gestures
Move drag handler from outer card to details section only.
Swiping on photos now navigates the carousel, swiping on
the details area below triggers like/dislike/skip actions.
2026-02-22 02:02:10 +00:00
Viktor Barzin
a2c1f81644
Fix photo extraction: look for both 'photos' and 'images' keys
Rightmove API stores photos under the 'photos' key in the response,
but the GeoJSON export and detail API were only checking 'images'.
This key mismatch caused all listings to fall back to the single
photo_thumbnail. Now checks both keys with fallback.
2026-02-22 01:21:50 +00:00
Viktor Barzin
d50d1c07f6
Use high-res images and return all photos in GeoJSON
- Use maxSizeUrl instead of url for photo URLs (highest available
  resolution from Rightmove)
- Remove 5-photo cap in GeoJSON export — return all available photos
- Apply same fix to both streaming and model-based export paths,
  and to the listing detail API endpoint
2026-02-22 00:54:58 +00:00
Viktor Barzin
e2c22f025f
Expand swipe card to 50/50 photo/details split with all info
- Card now fills available height with photo carousel in top half
  and property details in bottom half
- Details section shows: price, beds/sqm/price-per-sqm, agency,
  available date, all POI distances, and price history summary
- Fix DialogTitle accessibility warning in ListingDetailSheet and
  MobileBottomSheet (add sr-only Drawer.Title)
2026-02-22 00:49:32 +00:00
Viktor Barzin
611449d328
Add photo carousel to swipe cards
SwipeCard now shows all available photos (up to 5) using
embla-carousel instead of just the thumbnail. Includes prev/next
arrow buttons and dot indicators. The photo area uses touch-action:
pan-x so carousel swipes don't trigger card swipes.
2026-02-21 23:57:26 +00:00
Viktor Barzin
a742f9bb65
Fix frontend verify-deploy: remove stale 60s age filter
The age filter rejected pods older than 60s, but by the time the
verify step runs after build+publish+deploy, the new pods are often
already past that window. The image tag check is sufficient to
identify the correct deployment.
2026-02-21 23:50:20 +00:00
Viktor Barzin
0bb7e4c723
Parallelize API tests and build, gate publish on test pass
Same pattern as the frontend pipeline: build pushes to a staging
tag (build-N) in parallel with tests. Tests split into unit and
integration shards using a shared venv from the workspace. Publish
step uses skopeo to copy the manifest to final tags (:N, :latest,
:builder) only after all tests and the build succeed. Named the
final Dockerfile stage 'production' so target skips the test stage.
2026-02-21 23:46:16 +00:00
Viktor Barzin
68e7f2a334
Separate frontend image build from publish, gate publish on tests
Build step now pushes to a staging tag (build-N) in parallel with
test shards. A new publish step uses skopeo to copy the manifest to
the final tags (:N and :latest) only after all tests pass. This is
a server-side manifest copy with no layer re-upload.
2026-02-21 21:16:37 +00:00
Viktor Barzin
eacdf24621
Add tap-to-detail on swipe cards and fix color overlay alignment
- Add onTap callback to SwipeCard using useDrag's tap detection
- Wire through SwipeReviewMode to open ListingDetailSheet on tap
- Fix color overlay misalignment: add relative to card container so
  the absolute overlay positions within the rounded card, not the
  full-width outer wrapper
2026-02-21 21:13:32 +00:00
Viktor Barzin
9c954c0e43
Add prev/next arrow buttons to photo carousel 2026-02-21 21:07:17 +00:00
Viktor Barzin
15dbcfa332
Parallelize frontend tests in Drone pipeline with 4 shards
Split the monolithic "build and test" kaniko step into a DAG:
tests run in 4 parallel shards (vitest --shard) alongside the
Docker image build, gated by a shared npm ci step. The kaniko
build now targets the named 'production' stage to skip the
in-Dockerfile test stage.
2026-02-21 21:02:22 +00:00
Viktor Barzin
a36c0dfdf7
Enable BuildKit in Drone API pipeline 2026-02-21 20:51:30 +00:00
Viktor Barzin
aea4b3c008
Expand frontend .dockerignore to exclude build artifacts 2026-02-21 19:51:04 +00:00
Viktor Barzin
231552c5e0
Remove unused frontend deps, move @types to devDependencies 2026-02-21 19:50:54 +00:00
Viktor Barzin
a122a7983a
Optimize Dockerfile: uv, BuildKit cache mounts, non-root user, healthcheck
- Replace pip with uv for 10-25x faster dependency installation
- Add BuildKit cache mounts for apt and uv caches across builds
- Add non-root appuser for improved container security
- Add HEALTHCHECK directive for container orchestration
- Add curl to runtime for healthcheck support
- Remove libgl1 (unused), add syntax directive for BuildKit
2026-02-21 19:49:11 +00:00
Viktor Barzin
c762d5a0a6
Optimize frontend Dockerfile: npm cache mount, non-root nginx, healthcheck 2026-02-21 19:49:06 +00:00
Viktor Barzin
795e1f7b57
Regenerate requirements.txt after dependency cleanup
- Remove pandas, apprise and their transitive dependencies
- Add opencv-python-headless, httpx to prod requirements
2026-02-21 19:47:15 +00:00
Viktor Barzin
3d9550c7f1
Replace pandas with stdlib csv, apprise with direct Slack webhook, switch to opencv-headless
- Rewrite csv_exporter.py to use stdlib csv.DictWriter instead of pandas DataFrame
- Rewrite notifications.py to use aiohttp direct Slack webhook instead of apprise
- Switch opencv-python to opencv-python-headless in pyproject.toml
- Move httpx from dev to prod dependencies
- Remove pandas and apprise from mypy ignore_missing_imports
2026-02-21 19:47:10 +00:00
Viktor Barzin
cde3540a1e
Remove watchdog and tqdm dependencies, replace with logging
- Remove watchdog (unused) and tqdm from pyproject.toml dependencies
- Replace tqdm.gather() with asyncio.gather() + logger.info() in
  image_fetcher, floorplan_detector, and route_calculator services
- Replace tqdm progress bar with logger.info() in listing_repository
- Remove tqdm from mypy ignore_missing_imports overrides
2026-02-21 19:39:49 +00:00
Viktor Barzin
d488208a26
Add build optimization implementation plan 2026-02-21 19:36:33 +00:00
Viktor Barzin
54c0999b94
Add build optimization design doc 2026-02-21 19:33:26 +00:00
Viktor Barzin
4deed9911c
Add photo carousel to listing cards and fix tap-to-detail
Backend: include first 5 photo URLs from additional_info in GeoJSON
streaming response, with fallback to photo_thumbnail.

Frontend: replace single thumbnail with swipeable embla-carousel on
compact cards. Remove window.open on card tap so clicking opens the
detail bottom sheet instead of navigating to Rightmove.
2026-02-21 19:19:32 +00:00
Viktor Barzin
f2e8d7d9f9
docs: add card carousel and tap-to-detail implementation plan [ci skip] 2026-02-21 19:10:06 +00:00
Viktor Barzin
089ee88728
docs: add card carousel and tap-to-detail design [ci skip] 2026-02-21 19:08:12 +00:00
Viktor Barzin
a153f64af4
Format available_from dates as human-readable on listing cards
Display dates like "22 Jun 2025" instead of raw ISO timestamps.
2026-02-21 18:02:14 +00:00
Viktor Barzin
578b97b0c5
Add configurable request timeout and retry on TimeoutError
Requests to Rightmove API previously had no explicit timeout, causing
hung connections to block workers indefinitely. Add a configurable
request_timeout (default 30s) to ScraperConfig and apply it to all
aiohttp sessions. Also retry on TimeoutError in addition to
ThrottlingError for all API query functions.
2026-02-21 17:50:36 +00:00
Viktor Barzin
7a1042741e
fix: remove duplicate alembic migration causing multiple heads on startup 2026-02-21 16:08:02 +00:00
Viktor Barzin
1536e98c8b
fix: skip frontend config tests when frontend/ dir absent (Docker build) 2026-02-21 15:52:43 +00:00
Viktor Barzin
43f9d210fb
Fix tests to match decision service API and add filtering to non-streaming endpoint
- Update test mocks from _get_disliked_ids to _get_user_id_safe
- Fix decision service test method names (clear_decision -> remove_decision, etc.)
- Fix positional vs keyword arg assertion in set_decision test
- Add decision_filter param to non-streaming listing_geojson endpoint
2026-02-21 15:52:31 +00:00
Viktor Barzin
a2745c1478
Add tappable cards, detail bottom sheet, swipe gestures, and favorites view
- Decision types, services (decisionService, listingDetailService), and index exports
- useDecisions hook with optimistic updates and Map-based state
- useListingDetail hook with session-level caching
- PhotoCarousel component using embla-carousel-react
- ListingDetail component with full property info, like/dislike buttons
- ListingDetailSheet using vaul Drawer (slide-up bottom sheet)
- SwipeablePropertyCard with @use-gesture/react and @react-spring/web
- SwipeReviewMode for mobile full-screen swipe review
- FavoritesView with virtualized liked listings and remove button
- App.tsx integration: decision state, client-side disliked filtering, detail sheet, swipe handlers
- ListView conditionally renders SwipeablePropertyCard when handlers provided
- StatsBar adds 'saved' view mode with heart icon
- Header adds liked count indicator
- New deps: vaul, embla-carousel-react, @use-gesture/react, @react-spring/web
2026-02-21 15:49:15 +00:00
Viktor Barzin
9e1beb7495
Add listing decisions (like/dislike) backend with detail endpoint
- ListingDecision model with unique constraint on (user_id, listing_id, listing_type)
- Alembic migration for listingdecision table
- DecisionRepository with dialect-aware upsert (MySQL/SQLite)
- DecisionService with input validation
- Decision API routes: PUT/GET/DELETE on /api/decisions
- GET /api/listing/{id}/detail endpoint extracting full property info from additional_info
- Add listing ID to GeoJSON feature properties
- Decision filtering on GeoJSON stream endpoint (decision_filter param)
2026-02-21 15:49:10 +00:00
Viktor Barzin
a2e7d59af2
Add .worktrees/ to .gitignore for git worktree isolation 2026-02-21 15:48:34 +00:00
Viktor Barzin
318385035e
fix: revert CI to use Docker Hub for caching — local registry is read-only pull-through cache 2026-02-21 15:25:36 +00:00
Viktor Barzin
b1be4d4170
perf: optimize CI pipeline — eliminate double dependency installs, use local registry cache
- Frontend Dockerfile: split into deps/test/builder/nginx stages so npm ci
  runs once (cached when package-lock.json unchanged), tests run in build
- Backend Dockerfile: add test stage that runs pytest inside the build,
  eliminating separate test image build
- .drone.yml: remove separate test steps (now inside Dockerfile builds),
  point cache_from/cache_repo at local registry (10.0.20.10:5000) instead
  of Docker Hub for faster layer cache pulls
2026-02-21 15:10:55 +00:00
Viktor Barzin
68859ae577
docs: add CI pipeline optimization design document
Merge tests into Dockerfiles to eliminate double dependency installs,
switch Docker layer caching to local registry at 10.0.20.10:5000.
2026-02-21 15:07:50 +00:00