viktor/fire-planner
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
fire-planner/fire_planner/col/baseline.py
Viktor Barzin e72fd22a17 col: simulator auto-adjusts spending to local prices via Numbeo+Expatistan 
The Monte Carlo used to compare jurisdictions at a flat London-equivalent
spend, which silently overstated the cost-of-living for any move to a
cheaper region. Now every cross-jurisdiction simulation auto-scales
spending_gbp by the real Numbeo/Expatistan ratio between the user's
baseline city and the target city.

Architecture:
- fire_planner/col/baseline.py — 22 cities with headline Numbeo data
  (source URLs + snapshot dates embedded) — fallback when scraper fails
- col/numbeo.py + col/expatistan.py — httpx async scrapers, regex-parsed,
  polite 1.1s rate-limit, EUR/USD anchored
- col/cache.py — PG-backed cache (col_snapshot table, 1-year TTL)
- col/service.py — sync compute_col_ratio() for the simulator; async
  lookup_city_cached() with source reconciliation for the refresh CronJob
- alembic 0005 — col_snapshot table, UNIQUE(city_slug, source_name)

Simulator wiring:
- SimulateRequest gains col_auto_adjust=True (default), col_baseline_city,
  col_target_city. Defaults pick the jurisdiction's representative city.
- _resolve_col_adjustment scales spending_gbp before path-building.
- SimulateResult surfaces col_multiplier_applied + col_adjusted_spending_gbp.

CLIs:
- python -m fire_planner col-seed — loads BASELINES into col_snapshot
  (post-migration seed step)
- python -m fire_planner col-refresh-stale --within-days 7 — used by the
  weekly fire-planner-col-refresh CronJob

268 tests pass. Mypy strict + ruff clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 14:14:57 +00:00

342 lines
16 KiB
Python
Raw Blame History

"""Hand-curated baselines from Numbeo public pages.
All figures are GBP/month for a single person. Source URLs and snapshot
dates are embedded so we can re-validate. Refresh by re-running the
WebFetch prompts that built this file (see `docs/col-baseline-refresh.md`
or the conversation in 2026-05-21).
Adding a new city: pull the Numbeo page, find "Estimated monthly costs
for a single person without rent" (the headline), then the rent + per-
category breakdowns. Add an entry below — the simulator picks it up
automatically via `lookup_city()`.
Currency conversion uses the rate visible on Numbeo at fetch time —
re-fetch when sterling moves >5% against the local currency.
"""
from __future__ import annotations
from datetime import date
from decimal import Decimal
from fire_planner.col.models import CategoryBreakdown, CityCostIndex, ColSource
def _src(url: str, snap: str, ccy: str, gbp_per_unit: Decimal | float) -> ColSource:
return ColSource(
name="numbeo",
url=url,
snapshot_date=date.fromisoformat(snap),
raw_currency=ccy,
gbp_per_unit=Decimal(str(gbp_per_unit)),
)
BASELINES: dict[str, CityCostIndex] = {
"london": CityCostIndex(
city="London",
city_slug="london",
country="United Kingdom",
total_single_no_rent_gbp=Decimal("1092.40"),
total_single_with_rent_gbp=Decimal("3409.59"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("2317.19"),
rent_1bed_outside=Decimal("1728.85"),
groceries=Decimal("420.00"),
restaurants=Decimal("285.00"),
transport=Decimal("190.00"),
utilities=Decimal("327.18"),
leisure=Decimal("127.40"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/London", "2026-05-20", "GBP", 1.0),
),
"sofia": CityCostIndex(
city="Sofia",
city_slug="sofia",
country="Bulgaria",
total_single_no_rent_gbp=Decimal("712.54"),
total_single_with_rent_gbp=Decimal("1391.71"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("679.17"),
rent_1bed_outside=Decimal("520.26"),
groceries=Decimal("280.00"), # per-category figures sanity-checked
restaurants=Decimal("199.27"), # vs Numbeo summary; LLM extraction
transport=Decimal("28.50"), # of detail rows is noisy — headline
utilities=Decimal("130.00"), # totals (no_rent + with_rent) are
leisure=Decimal("75.00"), # the canonical anchors for ratios
),
source=_src("https://www.numbeo.com/cost-of-living/in/Sofia", "2026-05-20", "BGN", 0.435),
),
"limassol": CityCostIndex(
city="Limassol",
city_slug="limassol",
country="Cyprus",
total_single_no_rent_gbp=Decimal("932.30"),
total_single_with_rent_gbp=Decimal("2282.30"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("1350.00"),
rent_1bed_outside=Decimal("1162.94"),
groceries=Decimal("350.00"),
restaurants=Decimal("240.00"),
transport=Decimal("40.00"),
utilities=Decimal("233.43"),
leisure=Decimal("104.44"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Limassol", "2026-05-18",
"EUR", 0.862),
),
"dubai": CityCostIndex(
city="Dubai",
city_slug="dubai",
country="United Arab Emirates",
total_single_no_rent_gbp=Decimal("911.83"),
total_single_with_rent_gbp=Decimal("2768.31"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("1856.48"),
rent_1bed_outside=Decimal("1139.98"),
groceries=Decimal("96.77"), # Dubai groceries unusually low —
restaurants=Decimal("86.02"), # subsidised + lots of cheap labour
transport=Decimal("21.51"), # Metro pass AED 100. Sanity check
utilities=Decimal("188.24"), # in next refresh — could be Numbeo
leisure=Decimal("64.52"), # contributor undercounting
),
source=_src("https://www.numbeo.com/cost-of-living/in/Dubai", "2026-05-19", "AED", 0.21505),
),
"kuala-lumpur": CityCostIndex(
city="Kuala Lumpur",
city_slug="kuala-lumpur",
country="Malaysia",
total_single_no_rent_gbp=Decimal("420.64"),
total_single_with_rent_gbp=Decimal("865.08"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("444.44"),
rent_1bed_outside=Decimal("263.89"),
groceries=Decimal("76.95"),
restaurants=Decimal("145.35"),
transport=Decimal("17.10"),
utilities=Decimal("45.18"),
leisure=Decimal("42.75"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Kuala-Lumpur", "2026-05-17",
"MYR", 0.171),
),
"bangkok": CityCostIndex(
city="Bangkok",
city_slug="bangkok",
country="Thailand",
total_single_no_rent_gbp=Decimal("491.21"),
total_single_with_rent_gbp=Decimal("970.57"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("479.36"),
rent_1bed_outside=Decimal("233.76"),
groceries=Decimal("97.25"),
restaurants=Decimal("119.34"),
transport=Decimal("43.21"),
utilities=Decimal("69.04"),
leisure=Decimal("65.29"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Bangkok", "2026-05-20",
"THB", 0.02198),
),
# ── Expansion batch — fetched 2026-05-21, headline totals only ──
# Per-category breakdowns set to 0 where Numbeo LLM extraction was
# unreliable. Only `total_single_no_rent_gbp` / `total_single_with_rent_gbp`
# are used by the simulator's COL ratio; the breakdowns are for the
# UI / playbook. Refresh in Phase 3 (live scraper with HTML parsing).
"lisbon": CityCostIndex(
city="Lisbon", city_slug="lisbon", country="Portugal",
total_single_no_rent_gbp=Decimal("647.97"),
total_single_with_rent_gbp=Decimal("1856.03"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("1208.06"), rent_1bed_outside=Decimal("923.14"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Lisbon", "2026-05-21",
"EUR", 0.862),
),
"porto": CityCostIndex(
city="Porto", city_slug="porto", country="Portugal",
total_single_no_rent_gbp=Decimal("609.07"),
total_single_with_rent_gbp=Decimal("1562.50"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("953.43"), rent_1bed_outside=Decimal("726.19"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Porto", "2026-05-16",
"EUR", 0.862),
),
"madrid": CityCostIndex(
city="Madrid", city_slug="madrid", country="Spain",
total_single_no_rent_gbp=Decimal("706.87"),
total_single_with_rent_gbp=Decimal("1825.72"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("1118.85"), rent_1bed_outside=Decimal("873.06"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Madrid", "2026-05-21",
"EUR", 0.862),
),
"valencia": CityCostIndex(
city="Valencia", city_slug="valencia", country="Spain",
total_single_no_rent_gbp=Decimal("614.71"),
total_single_with_rent_gbp=Decimal("1663.97"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("1049.26"), rent_1bed_outside=Decimal("779.35"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Valencia", "2026-05-15",
"EUR", 0.862),
),
"athens": CityCostIndex(
city="Athens", city_slug="athens", country="Greece",
total_single_no_rent_gbp=Decimal("711.46"),
total_single_with_rent_gbp=Decimal("1245.89"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("534.43"), rent_1bed_outside=Decimal("453.23"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Athens", "2026-05-21",
"EUR", 0.862),
),
"bucharest": CityCostIndex(
city="Bucharest", city_slug="bucharest", country="Romania",
total_single_no_rent_gbp=Decimal("572.13"),
total_single_with_rent_gbp=Decimal("1102.46"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("530.33"), rent_1bed_outside=Decimal("363.06"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Bucharest", "2026-05-21",
"EUR", 0.862),
),
"tbilisi": CityCostIndex(
city="Tbilisi", city_slug="tbilisi", country="Georgia",
# LLM extraction unreliable; manual estimate of headline from
# secondary sources puts ex-rent ~€420-500 → £400.
total_single_no_rent_gbp=Decimal("400.00"),
total_single_with_rent_gbp=Decimal("941.43"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("541.43"), rent_1bed_outside=Decimal("350.82"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Tbilisi", "2026-05-18",
"GEL", 0.295),
),
"tallinn": CityCostIndex(
city="Tallinn", city_slug="tallinn", country="Estonia",
total_single_no_rent_gbp=Decimal("837.63"),
total_single_with_rent_gbp=Decimal("1441.06"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("603.43"), rent_1bed_outside=Decimal("434.23"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Tallinn", "2026-05-21",
"EUR", 0.862),
),
"penang": CityCostIndex(
city="Penang", city_slug="penang", country="Malaysia",
total_single_no_rent_gbp=Decimal("361.66"),
total_single_with_rent_gbp=Decimal("643.39"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("281.73"), rent_1bed_outside=Decimal("160.61"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Penang", "2026-05-18",
"MYR", 0.171),
),
"chiang-mai": CityCostIndex(
city="Chiang Mai", city_slug="chiang-mai", country="Thailand",
total_single_no_rent_gbp=Decimal("412.36"),
total_single_with_rent_gbp=Decimal("775.43"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("363.07"), rent_1bed_outside=Decimal("205.95"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Chiang-Mai", "2026-05-06",
"THB", 0.02198),
),
"bali": CityCostIndex(
city="Bali", city_slug="bali", country="Indonesia",
# Bali Numbeo conflates Ubud/Canggu/Denpasar; rent figures are
# manual estimates (Numbeo's £915 was implausibly high).
total_single_no_rent_gbp=Decimal("433.24"),
total_single_with_rent_gbp=Decimal("883.24"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("450.00"), rent_1bed_outside=Decimal("350.00"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Bali", "2026-05-15",
"IDR", 0.0000485),
),
"singapore": CityCostIndex(
city="Singapore", city_slug="singapore", country="Singapore",
total_single_no_rent_gbp=Decimal("579.63"),
total_single_with_rent_gbp=Decimal("2661.63"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("2082.00"), rent_1bed_outside=Decimal("1556.00"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Singapore", "2026-05-21",
"SGD", 0.585),
),
"taipei": CityCostIndex(
city="Taipei", city_slug="taipei", country="Taiwan",
total_single_no_rent_gbp=Decimal("646.50"),
total_single_with_rent_gbp=Decimal("1223.06"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("576.56"), rent_1bed_outside=Decimal("373.77"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Taipei", "2026-05-18",
"TWD", 0.0246),
),
"ho-chi-minh-city": CityCostIndex(
city="Ho Chi Minh City", city_slug="ho-chi-minh-city", country="Vietnam",
total_single_no_rent_gbp=Decimal("348.85"),
total_single_with_rent_gbp=Decimal("828.77"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("479.92"), rent_1bed_outside=Decimal("223.06"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Ho-Chi-Minh-City",
"2026-05-16", "VND", 0.0000316),
),
"mexico-city": CityCostIndex(
city="Mexico City", city_slug="mexico-city", country="Mexico",
total_single_no_rent_gbp=Decimal("600.47"),
total_single_with_rent_gbp=Decimal("1390.42"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("789.95"), rent_1bed_outside=Decimal("513.96"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Mexico-City", "2026-05-19",
"MXN", 0.0394),
),
"medellin": CityCostIndex(
city="Medellin", city_slug="medellin", country="Colombia",
# LLM extraction gave £105 — too low. Manual estimate ~£400.
total_single_no_rent_gbp=Decimal("400.00"),
total_single_with_rent_gbp=Decimal("902.13"),
breakdown=CategoryBreakdown(
rent_1bed_center=Decimal("502.13"), rent_1bed_outside=Decimal("373.02"),
groceries=Decimal("0"), restaurants=Decimal("0"), transport=Decimal("0"),
utilities=Decimal("0"), leisure=Decimal("0"),
),
source=_src("https://www.numbeo.com/cost-of-living/in/Medellin", "2026-05-21",
"COP", 0.000195),
),
}
Reference in a new issue View git blame Copy permalink