wealth: SQLite→PG ETL sidecar + new Grafana dashboard

Mirrors Wealthfolio's daily_account_valuation / accounts / activities
from SQLite into a new PG database (wealthfolio_sync) every hour, so
Grafana can chart net worth, contributions, and growth over time.

Components:
- dbaas: null_resource creates wealthfolio_sync DB + role on the CNPG
  cluster (dynamic primary lookup so it survives failover).
- vault: pg-wealthfolio-sync static role rotates the password every 7d.
- wealthfolio: ExternalSecret pulls the rotated password into the WF
  namespace; new pg-sync sidecar (alpine + sqlite + postgresql-client +
  busybox crond) does sqlite3 .backup → TSV dump → truncate-and-reload
  psql, hourly at :07. Plus a grafana-wealth-datasource ConfigMap in
  the monitoring namespace (uid: wealth-pg).
- monitoring: new Wealth dashboard (wealth.json, 10 panels) — current
  net worth / contribution / growth / ROI% stats, then time-series
  for net worth, contribution-vs-market, growth area, per-account
  stacked area, cash-vs-invested, and a 100-row activity log.

Initial sync: 6 accounts, 10,798 daily valuations, 518 activities.
Verified PG totals match SQLite latest snapshot exactly.
This commit is contained in:
Viktor Barzin 2026-04-25 17:07:33 +00:00
parent 7dd580972a
commit bf4c7618d8
6 changed files with 762 additions and 1 deletions

View file

@ -1209,6 +1209,33 @@ resource "null_resource" "pg_job_hunter_db" {
}
}
# Create wealthfolio_sync database for the SQLitePG ETL sidecar that mirrors
# Wealthfolio's daily_account_valuation/accounts/activities into PG so Grafana
# can chart net worth, contributions, and growth.
# Role password is managed by Vault Database Secrets Engine (static role `pg-wealthfolio-sync`, 7d rotation).
resource "null_resource" "pg_wealthfolio_sync_db" {
depends_on = [null_resource.pg_cluster]
triggers = {
db_name = "wealthfolio_sync"
username = "wealthfolio_sync"
}
provisioner "local-exec" {
command = <<-EOT
PRIMARY=$(kubectl --kubeconfig ${var.kube_config_path} get cluster -n dbaas pg-cluster -o jsonpath='{.status.currentPrimary}')
kubectl --kubeconfig ${var.kube_config_path} exec -n dbaas $PRIMARY -c postgres -- \
bash -c '
psql -U postgres -tc "SELECT 1 FROM pg_catalog.pg_roles WHERE rolname = '"'"'wealthfolio_sync'"'"'" | grep -q 1 || \
psql -U postgres -c "CREATE ROLE wealthfolio_sync WITH LOGIN PASSWORD '"'"'changeme-vault-will-rotate'"'"'"
psql -U postgres -tc "SELECT 1 FROM pg_catalog.pg_database WHERE datname = '"'"'wealthfolio_sync'"'"'" | grep -q 1 || \
psql -U postgres -c "CREATE DATABASE wealthfolio_sync OWNER wealthfolio_sync"
psql -U postgres -c "GRANT ALL PRIVILEGES ON DATABASE wealthfolio_sync TO wealthfolio_sync"
'
EOT
}
}
# Old PostgreSQL deployment kept commented for rollback reference
# resource "kubernetes_deployment" "postgres" {
# metadata {