viktor/infra
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

mam-farming: migrate data volume proxmox-lvm → NFS

Browse source 
The grabber + bp-spender shared a 1Gi proxmox-lvm RWO PVC holding two
plain-text files (mam_id cookie + grabbed_ids.txt dedup list — no embedded
DB). On 2026-06-04 a grabber pod wedged in ContainerCreating for >1h because
its proxmox-lvm disk couldn't hot-plug onto a SCSI-LUN-saturated node VM
(k8s-node3, QEMU `query-pci` QMP timeout); `concurrencyPolicy: Forbid` then
blocked every run → 0 grabs → MAMFarmingStuck.

NFS (nfs_volume module, matching the other 9 servarr apps) removes this
volume from the per-VM SCSI hotplug path entirely: it mounts over the
network, consumes zero LUN slots, and is RWX so the grabber + bp-spender can
co-schedule on any node. Data (mam_id + grabbed_ids.txt) was copied across
before the switch; verified a grabber run Succeeds on NFS on node4 with the
preserved dedup list (tracked IDs carried over). Lever #1 from
docs/architecture/storage.md "Per-VM SCSI-LUN cap".
This commit is contained in:
Viktor Barzin 2026-06-04 03:32:39 +00:00
parent cb96d5d590
commit 55a8b238a0
1 changed files with 18 additions and 36 deletions
Download patch file Download diff file Expand all files Collapse all files

54
stacks/servarr/mam-farming/main.tf
View file

@ -6,7 +6,6 @@ variable "namespace" {
locals { locals {
python_image = "docker.io/library/python:3.12-alpine" python_image = "docker.io/library/python:3.12-alpine"
pip_prefix = "pip install -q requests > /dev/null 2>&1; python3 /tmp/script.py" pip_prefix = "pip install -q requests > /dev/null 2>&1; python3 /tmp/script.py"
data_pvc = "mam-farming-data-proxmox"
# Dry-run window was satisfied by a one-shot test on 2026-04-19 that # Dry-run window was satisfied by a one-shot test on 2026-04-19 that
# produced 466 `never_started` candidates and 0 matches in any other # produced 466 `never_started` candidates and 0 matches in any other
@ -15,39 +14,22 @@ locals {
janitor_dry_run = "0" janitor_dry_run = "0"
} }
# ------------------------------- PVC ------------------------------- # ----------------------------- NFS data volume -----------------------
# Shared scratch volume for cookie + grabbed-ID dedup list. The existing # Migrated off proxmox-lvm (2026-06-04): the cookie + grabbed-ID dedup list
# in-cluster PVC (kubectl-applied 2026-04-14) is adopted via an `import {}` # are two plain-text files (no embedded DB), so NFS is safe and removes this
# block declared in the root module (servarr/main.tf) — Terraform 1.5+ # volume from the per-VM SCSI-LUN hotplug path entirely — a stuck `query-pci`
# rejects imports inside child modules. # on a disk-heavy node VM used to wedge the grabber in ContainerCreating (the
# disk never enumerated, Forbid blocked every run → MAMFarmingStuck). NFS
resource "kubernetes_persistent_volume_claim" "mam_data" { # mounts over the network, consumes zero LUN slots, and is RWX so the grabber
wait_until_bound = false # and bp-spender can co-schedule on any node. See docs/architecture/storage.md
metadata { # "Per-VM SCSI-LUN cap" lever #1.
name = local.data_pvc module "mam_data_nfs" {
namespace = var.namespace source = "../../../modules/kubernetes/nfs_volume"
annotations = { name = "servarr-mam-farming-data"
"resize.topolvm.io/threshold" = "10%" namespace = var.namespace
"resize.topolvm.io/increase" = "100%" nfs_server = "192.168.1.127"
"resize.topolvm.io/storage_limit" = "5Gi" nfs_path = "/srv/nfs/servarr/mam-farming"
} storage = "1Gi"
}
spec {
access_modes = ["ReadWriteOnce"]
storage_class_name = "proxmox-lvm"
resources {
requests = {
storage = "1Gi"
}
}
}
lifecycle {
# The autoresizer expands requests.storage up to storage_limit and
# PVCs can't shrink. Without this, every TF apply tries to revert
# to the spec value, K8s rejects the shrink, and the PVC ends up
# in Terminating-but-in-use limbo.
ignore_changes = [spec[0].resources[0].requests]
}
} }
# --------------------------- Grabber --------------------------------- # --------------------------- Grabber ---------------------------------
@ -120,7 +102,7 @@ resource "kubernetes_cron_job_v1" "grabber" {
volume { volume {
name = "data" name = "data"
persistent_volume_claim { persistent_volume_claim {
claim_name = kubernetes_persistent_volume_claim.mam_data.metadata[0].name claim_name = module.mam_data_nfs.claim_name
} }
} }
} }
@ -204,7 +186,7 @@ resource "kubernetes_cron_job_v1" "bp_spender" {
volume { volume {
name = "data" name = "data"
persistent_volume_claim { persistent_volume_claim {
claim_name = kubernetes_persistent_volume_claim.mam_data.metadata[0].name claim_name = module.mam_data_nfs.claim_name
} }
} }
} }