Add structured JSON logging, OTel business metrics, and Grafana dashboard

Structured logging via JsonFormatter replaces uvicorn's default format so
Loki can parse timestamps and fields.  14 business metrics (scrape stats,
throttle events, circuit breaker state, cache hit rate, OCR success rate,
Celery task lifecycle) are defined in a shared metrics module and
instrumented across the scraper pipeline, API, and workers.  Celery
workers expose a Prometheus HTTP endpoint on configurable ports.
This commit is contained in:
Viktor Barzin 2026-02-14 10:59:12 +00:00
parent a1829957c1
commit d6edb747d2
No known key found for this signature in database
GPG key ID: 0EB088298288D958
12 changed files with 742 additions and 49 deletions

View file

@ -324,6 +324,15 @@ class DetectFloorplanStep(Step):
listing.square_meters = max_sqm
await self.listing_repository.upsert_listings([listing])
# Record OCR metrics
try:
from api.metrics import ocr_attempts, ocr_successes
ocr_attempts.add(1)
if max_sqm > 0:
ocr_successes.add(1)
except Exception:
pass # Metrics not initialised
if max_sqm > 0:
logger.info(f"[{listing_id}] OCR detected {max_sqm} sqm")
else: