Commit graph

50 commits

Author SHA1 Message Date
Viktor Barzin
42d34fd21a
Fix map UX: instant load, stable hex grid, and clickable hexagons
- Skip fly-to animation on initial data load (fitBounds duration: 0)
- Replace turf.hexGrid with origin-anchored manual hex generation so
  hexagons stay stable across pan/zoom instead of flickering
- Quantize zoom to integers so hex pattern only changes at discrete levels
- Scale search radius with cell size so low-zoom hexagons find their data
- Make hexagons clickable with pointer cursor; each hex shows the exact
  listings it aggregated via stored search bounds
- Remove unused SEARCH_BUFFER constant
2026-02-07 21:59:02 +00:00
Viktor Barzin
150342bb9e
Refactor codebase following Clean Code principles and add 229 tests
- Extract helpers to reduce function sizes (listing_tasks, app.py, query.py, listing_fetcher)
  - Replace nonlocal mutations with _PipelineState dataclass in listing_tasks
  - Fix bugs: isinstance→equality check in repository, verify_exp for OIDC tokens
  - Consolidate duplicate filter methods in listing_repository
  - Move hardcoded config to env vars with backward-compatible defaults
  - Simplify CLI decorator to auto-build QueryParameters
  - Add deprecation docstring to data_access.py
  - Test count: 158 → 387 (all passing)
2026-02-07 20:19:57 +00:00
Viktor Barzin
c76a7ac028
Add post-logout redirect URI to OIDC config
Redirects users back to the app home page after SSO logout instead of
leaving them stranded on the Authentik end-session page.
2026-02-07 10:19:08 +00:00
Viktor Barzin
a8b7eace48
Add passkey (WebAuthn) authentication with self-registration
Enable users to sign up and sign in using passkeys (biometrics/security
keys) without needing a manually-created Authentik account. The existing
SSO login remains as an alternative.

Backend:
- Add WebAuthn registration/authentication endpoints via py-webauthn
- Issue HS256 JWTs for passkey users, with Redis-backed challenge storage
- Dual JWT verification in auth middleware (issuer-based routing: passkey
  HS256 vs Authentik RS256)
- PasskeyCredential model + migration making user.password nullable
- UserRepository with full CRUD for users and credentials

Frontend:
- AuthUser type abstraction unifying OIDC and passkey users
- Passkey service using @simplewebauthn/browser for WebAuthn ceremonies
- LoginModal redesigned with Sign In / Sign Up tabs
- Type migration from oidc-client-ts User to AuthUser across all services
  and components
2026-02-07 00:34:47 +00:00
Viktor Barzin
e55902d813
Add frontend and Caddy to Docker Compose dev environment
Containerize the frontend dev server (Vite) and add a Caddy reverse
proxy for HTTPS termination, replacing the manual local setup. The
Caddy config proxies /api/* to the backend and everything else to the
frontend dev server.

Also simplify start.sh: remove --local Poetry mode, extract
get_compose_cmd helper, and document new services and DEV_HOST env var.
2026-02-06 23:45:29 +00:00
Viktor Barzin
2626870396
Use localhost as dev origin instead of devvm.viktorbarzin.lan
DEV_HOST env var now controls the hostname, defaulting to localhost.
This makes the dev setup work on any machine without requiring a
specific DNS entry.
2026-02-06 23:44:50 +00:00
Viktor Barzin
b9f576ae2b
Stream-process listings as IDs arrive via asyncio.Queue
Replace the sequential fetch-all-then-process pipeline with a streaming
architecture where listing processing starts as soon as IDs become
available from each subquery. A producer task fetches pages and enqueues
new IDs (filtered inline against DB), while 20 consumer workers process
listings concurrently from the queue.

- Add ListingRepository.get_listing_ids() for fast ID-only projection
- Refactor listing_tasks.py: remove get_ids_to_process/dump_listings_and_monitor,
  replace with unified producer/worker/monitor pipeline
- Apply same pattern to CLI path in listing_fetcher.py
- Remove 'filtering' phase from frontend, show combined fetch+process metrics
- Add fetching_done flag to TaskResult for phase transition tracking
2026-02-06 23:43:54 +00:00
Viktor Barzin
b4837e1603
Add crawl job progress drawer with phase tracking and live logs
- Add phase-aware progress reporting across all crawl phases (splitting,
  fetching, filtering, processing) with per-step counters
- Add TaskProgressDrawer component with phase timeline stepper, detail
  counters, progress bar with ETA, and live worker log viewer
- Add on_step_complete callback to ListingProcessor for granular tracking
  of details/images/OCR steps
- Extend QuerySplitter on_progress callback with structured counter data
- Capture celery worker logs via ring buffer handler and inject into task
  state updates for frontend display
- Guard taskResult updates with phase presence check to prevent drawer
  from blanking during state transitions
2026-02-06 22:37:53 +00:00
Viktor Barzin
d205d15c74 Add services layer, tests, streaming UI, and cleanup legacy code 2026-02-06 20:55:10 +00:00
Viktor Barzin
5514fa6381 Remove 1000-result limit, add Redis caching and virtual scrolling
- Remove hard-coded limit=1000 default from listing_geojson and streaming
  endpoints, allowing all matching results to be returned
- Add Redis caching service (db=2, 30min TTL) that caches query results
  as Redis Lists for fast re-queries with reduced DB load
- Integrate cache into streaming endpoint: serve from cache on hit,
  populate cache on miss during DB streaming
- Invalidate cache after scrape completes (both success and no-new-listings)
- Replace ScrollArea with react-virtuoso in ListView for virtual scrolling,
  keeping only ~20-30 DOM nodes regardless of list size
- Handle metadata streaming message to show "0 / N" progress from start
- Throttle frontend state updates with requestAnimationFrame to prevent
  UI jank from rapid re-renders during cached response streaming
2026-02-06 20:47:36 +00:00
Viktor Barzin
29ba739063
Improve login UI with error handling and callback page 2026-02-02 20:33:40 +00:00
Viktor Barzin
835a2a9d53 Fix stuck Celery tasks and add purge all tasks functionality 2026-02-01 20:40:07 +00:00
Viktor Barzin
218781e038 Add visual progress bar to task indicator 2026-02-01 19:23:27 +00:00
Viktor Barzin
0ca955796e Show processed/total counts in task progress indicator 2026-02-01 19:19:59 +00:00
Viktor Barzin
6d8f69610f Add proper buy listing support with type-aware UI filters and display 2026-02-01 19:13:29 +00:00
Viktor Barzin
c7ac448f15 Add configurable scheduling, UI health/task indicators, and auto-load map with default filters 2026-02-01 17:28:37 +00:00
Kadir
0801aaf200
More ruff fixes (#2)
* adding ruff auto check for pull requests as well as fixing all ruff errors

* More ruff fixes: forgot half of the ruff checks

Forgot to do a git add all :D

---------

Co-authored-by: Kadir <git@k8n.dev>
2025-09-14 19:44:03 +01:00
Viktor Barzin
4fa09e31c8
make property image clickable 2025-07-27 11:11:04 +00:00
Viktor Barzin
ce386e748d
default to last seen 28 days from 7 2025-07-15 18:52:25 +00:00
Viktor Barzin
d4b22deda0
save user queries in redis so that user can refresh the page and still come back to their latest task 2025-07-06 12:02:25 +00:00
Viktor Barzin
db85cf2d3a
be more explicit with what the parameter submit buttons do 2025-07-04 23:10:43 +00:00
Viktor Barzin
7c43907c2f
welcome email as not all users have given names 2025-07-01 16:28:56 +00:00
Viktor Barzin
92c8403157
add subtitle to the login modal with instructions on how to get invited 2025-06-30 22:39:37 +00:00
Viktor Barzin
8ab79b7c72
squash parametrs form to take less space 2025-06-23 21:16:07 +00:00
Viktor Barzin
e05b4788e0
show spinner while running backgroun task 2025-06-23 19:45:53 +00:00
Viktor Barzin
9a164ddfdc
migrate frontend to use new celery api and improve ux around spinners whilst loading 2025-06-22 21:32:46 +00:00
Viktor Barzin
fbd39bb67f
replace available from component with a nicer search which takes human input 2025-06-21 23:46:19 +00:00
Viktor Barzin
ce199152e2
add available from filter 2025-06-21 23:22:11 +00:00
Viktor Barzin
170ab33eee
add ts ignore for errors 2025-06-21 18:51:06 +00:00
Viktor Barzin
8d32c2cbd7
add filter for last seen days 2025-06-21 17:39:10 +00:00
Viktor Barzin
b1cbe8505a
handle errors better in the ui with an alert component 2025-06-21 17:26:45 +00:00
Viktor Barzin
a7e0773c0a
add worker api to refresh data in the background 2025-06-21 12:49:04 +00:00
Viktor Barzin
3e1be6750c
add filter for sqm 2025-06-18 21:06:02 +00:00
Viktor Barzin
70460bee5d
add filter for min and max price 2025-06-18 20:57:20 +00:00
Viktor Barzin
dadb56aa16
add filter for min and max number of bedrooms 2025-06-18 20:54:47 +00:00
Viktor Barzin
69d15e9a16
add support for querying buying listings as well as by max price 2025-06-18 20:38:50 +00:00
Viktor Barzin
ba4a95825b
reset the map when new parameters are chosen 2025-06-18 19:12:00 +00:00
Viktor Barzin
7c70da4bed hook up query parameters form with rest query 2025-06-18 18:56:11 +00:00
Viktor Barzin
1caaa247d8
paameterize form using zod to get a type checked fields 2025-06-17 20:42:41 +00:00
Viktor Barzin
a092b4e9c8
disable login button whilst logging in 2025-06-17 19:15:18 +00:00
Viktor Barzin
b72569b6b9
make the app buildable for prod 2025-06-16 22:43:46 +00:00
Viktor Barzin
1e868f1b0d
do not typescript check map.tsx as that has a lot of the legacy js code that is not typed 2025-06-16 22:42:32 +00:00
Viktor Barzin
10a997a8a1
make oauth redirect url depends on env mode 2025-06-16 22:42:06 +00:00
Viktor Barzin
e598a02670
add crossfilter2 do deps as aparently it is not part of types/crossfilter2 2025-06-16 22:41:11 +00:00
Viktor Barzin
8e41032c6c
setup map component to render correctly and visualize all listings in scrollable components 2025-06-15 21:06:10 +00:00
Viktor Barzin
a8ee95b9d9
install shadcdn and move login dialog to a separate component 2025-06-15 13:49:34 +00:00
Viktor Barzin
4c7fe8927b
add minimal working version where we fetch data and visualize it corectly 2025-06-15 12:48:52 +00:00
Viktor Barzin
0dc6a30dbd
fail start script if caddy is not installed 2025-06-14 15:52:27 +00:00
Viktor Barzin
1b97cf0dc3
exclude caddy files 2025-06-14 15:47:36 +00:00
Viktor Barzin
c7d996dbeb
rename frontend directory to frontend 2025-06-14 15:43:14 +00:00