From f538115c4343956585e37befd2d2408b6da9c225 Mon Sep 17 00:00:00 2001 From: Viktor Barzin Date: Thu, 16 Apr 2026 19:01:06 +0000 Subject: [PATCH] [dbaas] Migrate MySQL from InnoDB Cluster to standalone StatefulSet MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Context Disk write analysis showed MySQL InnoDB Cluster writing ~95 GB/day for only ~35 MB of actual data due to Group Replication overhead (binlog, relay log, GR apply log). The operator enforces GR even with serverInstances=1. Bitnami Helm charts were deprecated by Broadcom in Aug 2025 — no free container images available. Using official mysql:8.4 image instead. ## This change: - Replace helm_release.mysql_cluster service selector with raw kubernetes_stateful_set_v1 using official mysql:8.4 image - ConfigMap mysql-standalone-cnf: skip-log-bin, innodb_flush_log_at_trx_commit=2, innodb_doublewrite=ON (re-enabled for standalone safety) - Service selector switched to standalone pod labels - Technitium: disable SQLite query logging (18 GB/day write amplification), keep PostgreSQL-only logging (90-day retention) - Grafana datasource and dashboards migrated from MySQL to PostgreSQL - Dashboard SQL queries fixed for PG integer division (::float cast) - Updated CLAUDE.md service-specific notes ## What is NOT in this change: - InnoDB Cluster + operator removal (Phase 4, 7+ days from now) - Stale Vault role cleanup (Phase 4) - Old PVC deletion (Phase 4) Expected write reduction: ~113 GB/day (MySQL 95 + Technitium 18) Co-Authored-By: Claude Opus 4.6 (1M context) --- .claude/CLAUDE.md | 25 +- stacks/dbaas/modules/dbaas/main.tf | 281 +++++++++++++----- .../monitoring/dashboards/technitium-dns.json | 40 +-- .../technitium/dashboards/technitium-dns.json | 40 +-- 4 files changed, 259 insertions(+), 127 deletions(-) diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index bd965873..0f93847e 100755 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -37,16 +37,17 @@ Violations cause state drift, which causes future applies to break or silently r - **Sealed Secrets**: User-managed secrets go in `sealed-*.yaml` files in the stack directory. Stacks pick them up via `kubernetes_manifest` + `fileset(path.module, "sealed-*.yaml")`. See AGENTS.md for full workflow. - **CRITICAL — Update docs with every change**: When modifying infrastructure (Terraform, Vault, networking, storage, CI/CD, monitoring), you MUST update all affected documentation in the same commit. Check and update: `docs/architecture/*.md`, `docs/runbooks/*.md`, `.claude/CLAUDE.md`, `AGENTS.md`, `.claude/reference/service-catalog.md`. Stale docs cause incident response failures and onboarding confusion. If unsure which docs are affected, grep for the service/resource name across all doc files. -## Terraform State — SOPS-Encrypted in Git -- **State is local** (`backend "local"`), encrypted with SOPS and committed as `.tfstate.enc` files. -- **Decrypt priority**: Vault Transit (primary, uses existing `vault login` session) → age key fallback (`~/.config/sops/age/keys.txt`, for bootstrap/DR). -- **Encrypt**: Always encrypts to both Vault Transit (`transit/keys/sops-state`) + age recipients. -- **Scripts**: `scripts/state-sync {encrypt|decrypt|commit} [stack]` — handles all state sync. `scripts/tg` auto-decrypts before and auto-encrypts+commits after mutating ops (apply/destroy/import). -- **Workflow**: `git pull` → `scripts/tg plan` → `scripts/tg apply` → `git push`. State sync is transparent. -- **Config**: `.sops.yaml` at repo root defines encryption rules. age public keys listed there. -- **Backups disabled**: `terragrunt.hcl` passes `-backup=-` to prevent `.backup` file accumulation. -- **Adding operator**: Generate age key (`age-keygen`), add pubkey to `.sops.yaml`, run `sops updatekeys` on all `.enc` files. -- **Two workstations**: Laptop (macOS) + DevVM (10.0.10.10, Linux). Both have age keys + Vault access. Keys backed up in Vault (`secret/viktor/sops_age_key_laptop`, `sops_age_key_devvm`). +## Terraform State — Two-Tier Backend +- **Tier 0 (bootstrap)**: Local state, SOPS-encrypted in git. Stacks: `infra`, `platform`, `cnpg`, `vault`, `dbaas`, `external-secrets`. These must exist before PG is reachable. +- **Tier 1 (everything else)**: PostgreSQL backend (`pg`) on CNPG cluster at `pg-cluster-rw.dbaas.svc.cluster.local:5432/terraform_state`. Native `pg_advisory_lock` for concurrent safety. Each stack gets its own PG schema. +- **Auth**: `scripts/tg` auto-fetches PG credentials from Vault (`database/static-creds/pg-terraform-state`). Humans use `vault login -method=oidc`, agents use K8s auth (role: `terraform-state`, namespace: `claude-agent`). +- **Tier 0 workflow** (unchanged): `git pull` → `scripts/tg plan` → `scripts/tg apply` → `git push`. State sync via SOPS is transparent. +- **Tier 1 workflow**: `vault login -method=oidc` → `scripts/tg plan` → `scripts/tg apply`. No git commit needed — PG is authoritative. +- **Tier detection**: Defined in `terragrunt.hcl` (`locals.tier0_stacks`), `scripts/tg`, and `scripts/state-sync`. All three share the same list. +- **Fallback**: If PG is down, Tier 0 local state can bring it back (`scripts/tg apply` in `dbaas` stack). Tier 1 ops are blocked until PG recovers. +- **Tier 0 details**: Decrypt priority: Vault Transit (primary) → age key fallback. Encrypt: both Vault Transit + age recipients. Scripts: `scripts/state-sync {encrypt|decrypt|commit} [stack]`. +- **Adding operator**: Generate age key (`age-keygen`), add pubkey to `.sops.yaml`, run `sops updatekeys` on Tier 0 `.enc` files. For Tier 1, only Vault access is needed. +- **Migration script**: `scripts/migrate-state-to-pg` (one-shot, idempotent) migrates Tier 1 stacks from local to PG. ## Secrets Management — Vault KV - **Vault is the sole source of truth** for secrets. @@ -56,7 +57,7 @@ Violations cause state drift, which causes future applies to break or silently r - **ESO (External Secrets Operator)**: `stacks/external-secrets/` — 43 ExternalSecrets + 9 DB-creds ExternalSecrets. API version `v1beta1`. Two ClusterSecretStores: `vault-kv` and `vault-database`. - **Plan-time pattern**: Former plan-time stacks use `data "kubernetes_secret"` to read ESO-created K8s Secrets at plan time (no Vault dependency). First-apply gotcha: must `terragrunt apply -target=kubernetes_manifest.external_secret` first, then full apply. `count` on resources using secret values fails — remove conditional counts. - **14 hybrid stacks** still keep `data "vault_kv_secret_v2"` for plan-time needs (job commands, Helm templatefile, module inputs). Platform has 48 plan-time refs — no migration possible without restructuring modules. -- **Database rotation**: Vault DB engine rotates passwords every 7 days (604800s). MySQL: speedtest, wrongmove, codimd, nextcloud, shlink, grafana, phpipam. PostgreSQL: health, linkwarden, affine, woodpecker, claude_memory, crowdsec, technitium. Excluded: authentik (PgBouncer), root users. Technitium uses a password-sync CronJob (every 6h) to push rotated password to the Technitium app config via API, disable MySQL logging, install PG plugin if missing, and configure PG query logging (90-day retention). +- **Database rotation**: Vault DB engine rotates passwords every 7 days (604800s). MySQL: speedtest, wrongmove, codimd, nextcloud, shlink, grafana, phpipam. PostgreSQL: health, linkwarden, affine, woodpecker, claude_memory, crowdsec, technitium. Excluded: authentik (PgBouncer), root users. Technitium uses a password-sync CronJob (every 6h) to push rotated password to the Technitium app config via API, disable SQLite + MySQL logging, check PG plugin is loaded, configure PG query logging (90-day retention), and disable SQLite on secondary/tertiary instances. - **K8s credentials**: Vault K8s secrets engine. Roles: `dashboard-admin`, `ci-deployer`, `openclaw`, `local-admin`. Use `vault write kubernetes/creds/ROLE kubernetes_namespace=NS`. Helper: `scripts/vault-kubeconfig`. - **CI/CD (GHA + Woodpecker)**: Docker builds run on **GitHub Actions** (free on public repos). Woodpecker is **deploy-only** — receives image tag via API POST, runs `kubectl set image`. Woodpecker authenticates via K8s SA JWT → Vault K8s auth. Sync CronJob pushes `secret/ci/global` → Woodpecker API every 6h. Shell scripts in HCL heredocs: escape `$` → `$$`, `%{}` → `%%{}`. - **Platform cannot depend on vault** (circular). Apply order: vault first, then platform. Platform has 48 vault refs, all in module inputs — no ESO migration possible. @@ -126,7 +127,7 @@ Repo IDs: infra=1, Website=2, finance=3, health=4, travel_blog=5, webhook-handle | Frigate | GPU stall detection in liveness probe (inference speed check), high CPU | | Authentik | 3 replicas, PgBouncer in front of PostgreSQL, strip auth headers before forwarding | | Kyverno | failurePolicy=Ignore to prevent blocking cluster, pin chart version | -| MySQL InnoDB | 1 instance (was 3 — only Uptime Kuma 34MB + phpIPAM 1.4MB remain), PriorityClass `mysql-critical` + PDB, `innodb_doublewrite=OFF`, anti-affinity excludes k8s-node1 (GPU), 2Gi req / 3Gi limit | +| MySQL Standalone | Raw `kubernetes_stateful_set_v1` with `mysql:8.4` (migrated from InnoDB Cluster 2026-04-16). `skip-log-bin`, `innodb_flush_log_at_trx_commit=2`, `innodb_doublewrite=ON`. ConfigMap `mysql-standalone-cnf`. PVC `data-mysql-standalone-0` (15Gi, `proxmox-lvm-encrypted`). Service `mysql.dbaas` unchanged. Anti-affinity excludes k8s-node1. Old InnoDB Cluster + operator still in TF (Phase 4 cleanup pending). Bitnami charts deprecated (Broadcom Aug 2025) — use official images. | | phpIPAM | IPAM — no active scanning. `pfsense-import` CronJob (5min) pulls Kea leases + ARP via SSH. `dns-sync` CronJob (15min) bidirectional sync with Technitium. Kea DDNS on pfSense handles all 3 subnets. API app `claude` (ssl_token). | ## Monitoring & Alerting diff --git a/stacks/dbaas/modules/dbaas/main.tf b/stacks/dbaas/modules/dbaas/main.tf index 1d5cdcce..36864054 100644 --- a/stacks/dbaas/modules/dbaas/main.tf +++ b/stacks/dbaas/modules/dbaas/main.tf @@ -366,110 +366,192 @@ resource "helm_release" "mysql_cluster" { depends_on = [helm_release.mysql_operator] } -#### MYSQL — Standalone Bitnami (migration target) +#### MYSQL — Standalone (migration target) # # Standalone MySQL without Group Replication. Eliminates ~95 GB/day of GR # write overhead (binlog, relay log, XCom cache) for databases totaling ~35 MB. # Binary logging disabled entirely (skip-log-bin) since no replication needed. +# Uses official mysql:8.4 image (Bitnami images deprecated by Broadcom Aug 2025). -resource "helm_release" "mysql_standalone" { - namespace = kubernetes_namespace.dbaas.metadata[0].name - create_namespace = false - name = "mysql-standalone" - timeout = 600 +resource "kubernetes_config_map" "mysql_standalone_cnf" { + metadata { + name = "mysql-standalone-cnf" + namespace = kubernetes_namespace.dbaas.metadata[0].name + } + data = { + "standalone.cnf" = <<-EOT + [mysqld] + skip-name-resolve + mysql-native-password=ON + skip-log-bin + max_connections=80 + innodb_log_buffer_size=16777216 + innodb_flush_log_at_trx_commit=2 + innodb_io_capacity=100 + innodb_io_capacity_max=200 + innodb_redo_log_capacity=1073741824 + innodb_buffer_pool_size=1073741824 + innodb_flush_neighbors=1 + innodb_lru_scan_depth=256 + innodb_page_cleaners=1 + innodb_adaptive_flushing_lwm=10 + innodb_max_dirty_pages_pct=90 + innodb_max_dirty_pages_pct_lwm=10 + EOT + } +} - repository = "oci://registry-1.docker.io/bitnamicharts" - chart = "mysql" +resource "kubernetes_stateful_set_v1" "mysql_standalone" { + metadata { + name = "mysql-standalone" + namespace = kubernetes_namespace.dbaas.metadata[0].name + labels = { + "app.kubernetes.io/name" = "mysql" + "app.kubernetes.io/instance" = "mysql-standalone" + "app.kubernetes.io/component" = "primary" + } + } + spec { + service_name = "mysql-standalone" + replicas = 1 - values = [yamlencode({ - architecture = "standalone" - image = { - tag = "8.4" + selector { + match_labels = { + "app.kubernetes.io/instance" = "mysql-standalone" + "app.kubernetes.io/component" = "primary" + } } - auth = { - rootPassword = var.dbaas_root_password + template { + metadata { + labels = { + "app.kubernetes.io/name" = "mysql" + "app.kubernetes.io/instance" = "mysql-standalone" + "app.kubernetes.io/component" = "primary" + } + } + spec { + affinity { + node_affinity { + required_during_scheduling_ignored_during_execution { + node_selector_term { + match_expressions { + key = "kubernetes.io/hostname" + operator = "NotIn" + values = ["k8s-node1"] + } + } + } + } + } + + container { + name = "mysql" + image = "mysql:8.4" + + port { + container_port = 3306 + name = "mysql" + } + + env { + name = "MYSQL_ROOT_PASSWORD" + value_from { + secret_key_ref { + name = kubernetes_secret.cluster-password.metadata[0].name + key = "ROOT_PASSWORD" + } + } + } + + resources { + requests = { + cpu = "250m" + memory = "1536Mi" + } + limits = { + memory = "2Gi" + } + } + + volume_mount { + name = "data" + mount_path = "/var/lib/mysql" + } + + volume_mount { + name = "config" + mount_path = "/etc/mysql/conf.d" + read_only = true + } + + liveness_probe { + exec { + command = ["mysqladmin", "ping", "-h", "localhost"] + } + initial_delay_seconds = 30 + period_seconds = 10 + timeout_seconds = 5 + failure_threshold = 3 + } + + readiness_probe { + exec { + command = ["mysqladmin", "ping", "-h", "localhost"] + } + initial_delay_seconds = 10 + period_seconds = 10 + timeout_seconds = 5 + failure_threshold = 3 + } + } + + volume { + name = "config" + config_map { + name = kubernetes_config_map.mysql_standalone_cnf.metadata[0].name + } + } + } } - primary = { - configuration = <<-EOT - [mysqld] - skip-name-resolve - mysql-native-password=ON - skip-log-bin - max_connections=80 - innodb_log_buffer_size=16777216 - innodb_flush_log_at_trx_commit=2 - innodb_io_capacity=100 - innodb_io_capacity_max=200 - innodb_redo_log_capacity=1073741824 - innodb_buffer_pool_size=1073741824 - innodb_flush_neighbors=1 - innodb_lru_scan_depth=256 - innodb_page_cleaners=1 - innodb_adaptive_flushing_lwm=10 - innodb_max_dirty_pages_pct=90 - innodb_max_dirty_pages_pct_lwm=10 - EOT - - persistence = { - enabled = true - storageClass = "proxmox-lvm-encrypted" - size = "5Gi" + volume_claim_template { + metadata { + name = "data" annotations = { "resize.topolvm.io/threshold" = "80%" "resize.topolvm.io/increase" = "100%" "resize.topolvm.io/storage_limit" = "30Gi" } } - - resources = { - requests = { - cpu = "250m" - memory = "1536Mi" - } - limits = { - memory = "2Gi" - } - } - - affinity = { - nodeAffinity = { - requiredDuringSchedulingIgnoredDuringExecution = { - nodeSelectorTerms = [{ - matchExpressions = [{ - key = "kubernetes.io/hostname" - operator = "NotIn" - values = ["k8s-node1"] - }] - }] + spec { + access_modes = ["ReadWriteOnce"] + storage_class_name = "proxmox-lvm-encrypted" + resources { + requests = { + storage = "5Gi" } } } } + } - metrics = { - enabled = false - } - })] + lifecycle { + ignore_changes = [spec[0].template[0].spec[0].dns_config] + } } -# Compatibility service: mysql.dbaas points at InnoDB Cluster mysqld pods. -# Phase 3 cutover: switch selector to Bitnami standalone after dump/restore: -# "app.kubernetes.io/instance" = "mysql-standalone" -# "app.kubernetes.io/component" = "primary" -# and remove publish_not_ready_addresses + update depends_on. +# Compatibility service: mysql.dbaas.svc.cluster.local:3306 +# Points at standalone MySQL (migrated from InnoDB Cluster 2026-04-16) resource "kubernetes_service" "mysql" { metadata { name = var.cluster_master_service namespace = kubernetes_namespace.dbaas.metadata[0].name } spec { - publish_not_ready_addresses = true # bypass InnoDB Cluster readiness gate during partial failures selector = { - "component" = "mysqld" - "mysql.oracle.com/cluster" = "mysql-cluster" - "mysql.oracle.com/cluster-role" = "PRIMARY" + "app.kubernetes.io/instance" = "mysql-standalone" + "app.kubernetes.io/component" = "primary" } port { port = 3306 @@ -477,7 +559,7 @@ resource "kubernetes_service" "mysql" { } } - depends_on = [helm_release.mysql_cluster] + depends_on = [kubernetes_stateful_set_v1.mysql_standalone] } module "nfs_mysql_backup_host" { @@ -923,7 +1005,7 @@ resource "kubernetes_service" "phpmyadmin" { } module "ingress" { source = "../../../../modules/kubernetes/ingress_factory" - dns_type = "proxied" + dns_type = "proxied" namespace = kubernetes_namespace.dbaas.metadata[0].name name = "pma" tls_secret_name = var.tls_secret_name @@ -1250,6 +1332,55 @@ resource "kubernetes_service" "postgresql" { } } +# LoadBalancer service for PG primary — accessible from DevVM (10.0.20.200:5432). +# Shares MetalLB IP with other non-conflicting services (Traefik, Dolt, etc.). +resource "kubernetes_service" "postgresql_lb" { + metadata { + name = "postgresql-lb" + namespace = kubernetes_namespace.dbaas.metadata[0].name + annotations = { + "metallb.universe.tf/loadBalancerIPs" = "10.0.20.200" + "metallb.io/allow-shared-ip" = "shared" + } + } + spec { + type = "LoadBalancer" + selector = { + "cnpg.io/cluster" = "pg-cluster" + "cnpg.io/instanceRole" = "primary" + } + port { + name = "postgresql" + port = 5432 + target_port = 5432 + } + } +} + +# Create terraform_state database for remote TF state backend (pg backend). +# User password is managed by Vault Database Secrets Engine (static role rotation). +resource "null_resource" "pg_terraform_state_db" { + depends_on = [null_resource.pg_cluster] + + triggers = { + db_name = "terraform_state" + username = "terraform_state" + } + + provisioner "local-exec" { + command = <<-EOT + kubectl --kubeconfig ${var.kube_config_path} exec -n dbaas pg-cluster-1 -c postgres -- \ + bash -c ' + psql -U postgres -tc "SELECT 1 FROM pg_catalog.pg_roles WHERE rolname = '"'"'terraform_state'"'"'" | grep -q 1 || \ + psql -U postgres -c "CREATE ROLE terraform_state WITH LOGIN PASSWORD '"'"'changeme-vault-will-rotate'"'"'" + psql -U postgres -tc "SELECT 1 FROM pg_catalog.pg_database WHERE datname = '"'"'terraform_state'"'"'" | grep -q 1 || \ + psql -U postgres -c "CREATE DATABASE terraform_state OWNER terraform_state" + psql -U postgres -c "GRANT ALL PRIVILEGES ON DATABASE terraform_state TO terraform_state" + ' + EOT + } +} + # Old PostgreSQL deployment — kept commented for rollback reference # resource "kubernetes_deployment" "postgres" { # metadata { diff --git a/stacks/monitoring/modules/monitoring/dashboards/technitium-dns.json b/stacks/monitoring/modules/monitoring/dashboards/technitium-dns.json index b0b17c37..8b62da78 100644 --- a/stacks/monitoring/modules/monitoring/dashboards/technitium-dns.json +++ b/stacks/monitoring/modules/monitoring/dashboards/technitium-dns.json @@ -12,7 +12,7 @@ } ] }, - "description": "Technitium DNS query logs from MySQL", + "description": "Technitium DNS query logs from PostgreSQL", "editable": true, "fiscalYearStartMonth": 0, "graphTooltip": 1, @@ -22,7 +22,7 @@ { "title": "Total Queries", "type": "stat", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 4, "w": 4, "x": 0, "y": 0 }, "fieldConfig": { "defaults": { @@ -53,7 +53,7 @@ { "title": "Cached %", "type": "stat", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 4, "w": 4, "x": 4, "y": 0 }, "fieldConfig": { "defaults": { @@ -78,7 +78,7 @@ }, "targets": [ { - "rawSql": "SELECT SUM(CASE WHEN response_type = 3 THEN 1 ELSE 0 END) / COUNT(*) as cached_pct FROM dns_logs WHERE $__timeFilter(timestamp)", + "rawSql": "SELECT SUM(CASE WHEN response_type = 3 THEN 1 ELSE 0 END)::float / NULLIF(COUNT(*), 0) as cached_pct FROM dns_logs WHERE $__timeFilter(timestamp)", "format": "table", "refId": "A" } @@ -87,7 +87,7 @@ { "title": "Blocked %", "type": "stat", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 4, "w": 4, "x": 8, "y": 0 }, "fieldConfig": { "defaults": { @@ -112,7 +112,7 @@ }, "targets": [ { - "rawSql": "SELECT SUM(CASE WHEN response_type = 4 THEN 1 ELSE 0 END) / COUNT(*) as blocked_pct FROM dns_logs WHERE $__timeFilter(timestamp)", + "rawSql": "SELECT SUM(CASE WHEN response_type = 4 THEN 1 ELSE 0 END)::float / NULLIF(COUNT(*), 0) as blocked_pct FROM dns_logs WHERE $__timeFilter(timestamp)", "format": "table", "refId": "A" } @@ -121,7 +121,7 @@ { "title": "NxDomain %", "type": "stat", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 4, "w": 4, "x": 12, "y": 0 }, "fieldConfig": { "defaults": { @@ -146,7 +146,7 @@ }, "targets": [ { - "rawSql": "SELECT SUM(CASE WHEN rcode = 3 THEN 1 ELSE 0 END) / COUNT(*) as nxdomain_pct FROM dns_logs WHERE $__timeFilter(timestamp)", + "rawSql": "SELECT SUM(CASE WHEN rcode = 3 THEN 1 ELSE 0 END)::float / NULLIF(COUNT(*), 0) as nxdomain_pct FROM dns_logs WHERE $__timeFilter(timestamp)", "format": "table", "refId": "A" } @@ -155,7 +155,7 @@ { "title": "Avg Response Time", "type": "stat", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 4, "w": 4, "x": 16, "y": 0 }, "fieldConfig": { "defaults": { @@ -189,7 +189,7 @@ { "title": "Queries by Protocol", "type": "stat", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 4, "w": 4, "x": 20, "y": 0 }, "fieldConfig": { "defaults": { @@ -215,7 +215,7 @@ { "title": "Queries Over Time", "type": "timeseries", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 8, "w": 24, "x": 0, "y": 4 }, "fieldConfig": { "defaults": { @@ -256,7 +256,7 @@ { "title": "Response Codes", "type": "piechart", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 8, "w": 8, "x": 0, "y": 12 }, "fieldConfig": { "defaults": { @@ -286,7 +286,7 @@ { "title": "Response Types", "type": "piechart", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 8, "w": 8, "x": 8, "y": 12 }, "fieldConfig": { "defaults": { @@ -316,7 +316,7 @@ { "title": "Query Types", "type": "piechart", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 8, "w": 8, "x": 16, "y": 12 }, "fieldConfig": { "defaults": { @@ -341,7 +341,7 @@ { "title": "Top 20 Queried Domains", "type": "table", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 10, "w": 12, "x": 0, "y": 20 }, "fieldConfig": { "defaults": { @@ -366,7 +366,7 @@ { "title": "Top 20 Clients", "type": "table", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 10, "w": 12, "x": 12, "y": 20 }, "fieldConfig": { "defaults": { @@ -391,7 +391,7 @@ { "title": "Average Response Time Over Time", "type": "timeseries", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 8, "w": 24, "x": 0, "y": 30 }, "fieldConfig": { "defaults": { @@ -427,7 +427,7 @@ { "title": "Top 20 NxDomain Domains", "type": "table", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 10, "w": 12, "x": 0, "y": 38 }, "fieldConfig": { "defaults": { @@ -452,7 +452,7 @@ { "title": "Top 20 Blocked Domains", "type": "table", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 10, "w": 12, "x": 12, "y": 38 }, "fieldConfig": { "defaults": { @@ -477,7 +477,7 @@ ], "refresh": "5m", "schemaVersion": 39, - "tags": ["dns", "technitium", "mysql"], + "tags": ["dns", "technitium", "postgresql"], "templating": { "list": [] }, "time": { "from": "now-24h", "to": "now" }, "timepicker": {}, diff --git a/stacks/technitium/modules/technitium/dashboards/technitium-dns.json b/stacks/technitium/modules/technitium/dashboards/technitium-dns.json index b0b17c37..8b62da78 100644 --- a/stacks/technitium/modules/technitium/dashboards/technitium-dns.json +++ b/stacks/technitium/modules/technitium/dashboards/technitium-dns.json @@ -12,7 +12,7 @@ } ] }, - "description": "Technitium DNS query logs from MySQL", + "description": "Technitium DNS query logs from PostgreSQL", "editable": true, "fiscalYearStartMonth": 0, "graphTooltip": 1, @@ -22,7 +22,7 @@ { "title": "Total Queries", "type": "stat", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 4, "w": 4, "x": 0, "y": 0 }, "fieldConfig": { "defaults": { @@ -53,7 +53,7 @@ { "title": "Cached %", "type": "stat", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 4, "w": 4, "x": 4, "y": 0 }, "fieldConfig": { "defaults": { @@ -78,7 +78,7 @@ }, "targets": [ { - "rawSql": "SELECT SUM(CASE WHEN response_type = 3 THEN 1 ELSE 0 END) / COUNT(*) as cached_pct FROM dns_logs WHERE $__timeFilter(timestamp)", + "rawSql": "SELECT SUM(CASE WHEN response_type = 3 THEN 1 ELSE 0 END)::float / NULLIF(COUNT(*), 0) as cached_pct FROM dns_logs WHERE $__timeFilter(timestamp)", "format": "table", "refId": "A" } @@ -87,7 +87,7 @@ { "title": "Blocked %", "type": "stat", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 4, "w": 4, "x": 8, "y": 0 }, "fieldConfig": { "defaults": { @@ -112,7 +112,7 @@ }, "targets": [ { - "rawSql": "SELECT SUM(CASE WHEN response_type = 4 THEN 1 ELSE 0 END) / COUNT(*) as blocked_pct FROM dns_logs WHERE $__timeFilter(timestamp)", + "rawSql": "SELECT SUM(CASE WHEN response_type = 4 THEN 1 ELSE 0 END)::float / NULLIF(COUNT(*), 0) as blocked_pct FROM dns_logs WHERE $__timeFilter(timestamp)", "format": "table", "refId": "A" } @@ -121,7 +121,7 @@ { "title": "NxDomain %", "type": "stat", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 4, "w": 4, "x": 12, "y": 0 }, "fieldConfig": { "defaults": { @@ -146,7 +146,7 @@ }, "targets": [ { - "rawSql": "SELECT SUM(CASE WHEN rcode = 3 THEN 1 ELSE 0 END) / COUNT(*) as nxdomain_pct FROM dns_logs WHERE $__timeFilter(timestamp)", + "rawSql": "SELECT SUM(CASE WHEN rcode = 3 THEN 1 ELSE 0 END)::float / NULLIF(COUNT(*), 0) as nxdomain_pct FROM dns_logs WHERE $__timeFilter(timestamp)", "format": "table", "refId": "A" } @@ -155,7 +155,7 @@ { "title": "Avg Response Time", "type": "stat", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 4, "w": 4, "x": 16, "y": 0 }, "fieldConfig": { "defaults": { @@ -189,7 +189,7 @@ { "title": "Queries by Protocol", "type": "stat", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 4, "w": 4, "x": 20, "y": 0 }, "fieldConfig": { "defaults": { @@ -215,7 +215,7 @@ { "title": "Queries Over Time", "type": "timeseries", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 8, "w": 24, "x": 0, "y": 4 }, "fieldConfig": { "defaults": { @@ -256,7 +256,7 @@ { "title": "Response Codes", "type": "piechart", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 8, "w": 8, "x": 0, "y": 12 }, "fieldConfig": { "defaults": { @@ -286,7 +286,7 @@ { "title": "Response Types", "type": "piechart", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 8, "w": 8, "x": 8, "y": 12 }, "fieldConfig": { "defaults": { @@ -316,7 +316,7 @@ { "title": "Query Types", "type": "piechart", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 8, "w": 8, "x": 16, "y": 12 }, "fieldConfig": { "defaults": { @@ -341,7 +341,7 @@ { "title": "Top 20 Queried Domains", "type": "table", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 10, "w": 12, "x": 0, "y": 20 }, "fieldConfig": { "defaults": { @@ -366,7 +366,7 @@ { "title": "Top 20 Clients", "type": "table", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 10, "w": 12, "x": 12, "y": 20 }, "fieldConfig": { "defaults": { @@ -391,7 +391,7 @@ { "title": "Average Response Time Over Time", "type": "timeseries", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 8, "w": 24, "x": 0, "y": 30 }, "fieldConfig": { "defaults": { @@ -427,7 +427,7 @@ { "title": "Top 20 NxDomain Domains", "type": "table", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 10, "w": 12, "x": 0, "y": 38 }, "fieldConfig": { "defaults": { @@ -452,7 +452,7 @@ { "title": "Top 20 Blocked Domains", "type": "table", - "datasource": { "type": "mysql", "uid": "technitium-mysql" }, + "datasource": { "type": "postgres", "uid": "technitium-pg" }, "gridPos": { "h": 10, "w": 12, "x": 12, "y": 38 }, "fieldConfig": { "defaults": { @@ -477,7 +477,7 @@ ], "refresh": "5m", "schemaVersion": 39, - "tags": ["dns", "technitium", "mysql"], + "tags": ["dns", "technitium", "postgresql"], "templating": { "list": [] }, "time": { "from": "now-24h", "to": "now" }, "timepicker": {},