## Context The @viktorbarzin.me catch-all routes to spam@viktorbarzin.me. The mailbox had no retention policy. On 2026-04-18 it held 519 messages consuming 43 MiB. Without a policy, the only brake on growth was manual deletion, which has not been happening - hence the bd task. Viktor's explicit constraint when filing code-oy4: DO NOT blind age-expunge. We need targeted retention that keeps genuine forwarded human mail for a long time while shedding the recurring-newsletter cruft that dominates the byte count. ## Profile findings (2026-04-18, verified on the live pod) Total: 519 messages, 43 MiB, 0 in new/, 0 in tmp/. Top senders by volume: 138 dan@tldrnewsletter.com 51 hi@ratepunk.com 40 uber@uber.com 35 truenas@viktorbarzin.me 19 ubereats@uber.com 15 hello@travel.jacksflightclub.com 12 chris@chriswillx.com 10 me@viktorbarzin.me Top senders by storage bytes: 8,176,481 dan@tldrnewsletter.com (19 % of 43 MiB alone) 2,866,104 uber@uber.com 2,207,458 noreply@mail.selfh.st 2,066,094 hi@ratepunk.com 1,675,435 ubereats@uber.com Age distribution: 97 % older than 14 days (502 / 519) 23 % older than 90 days (121 / 519) Automated-sender markers: 66 % carry List-Unsubscribe: (342 / 519) 4 % carry Precedence: bulk|list|junk ( 21 / 519) 34 % carry neither marker (= human-ish tail) (177 / 519) Combined "automated AND >14d": 328 messages -> target of rule 1. ## Retention strategy Signed off by Viktor 2026-04-18. Two rules, both delete-leaf: 1. Older than 14 days AND header matches one of: - `^List-Unsubscribe:` - `^Precedence:\s*(bulk|list|junk)` - `^Auto-Submitted:\s*auto-` -> DELETE. Rationale: these markers are the RFC-agreed indicators of bulk / robotic senders. A 14-day window still lets genuine subscription alerts (delivery, flight, calendar invite) come to attention. 2. Older than 90 days AND no automated marker at all -> DELETE. Rationale: these are long-tail forwards from real people to the catch-all. 90 days is deliberately generous - I would rather leak bytes than lose Viktor's personal correspondence. 3. Everything else -> KEEP (recent traffic, or aged human tail younger than 90d). ## Implementation A `kubernetes_cron_job_v1.spam_retention` running every 4h (at :17 past) that `kubectl exec`s a Python retention script into the mailserver pod. Why kubectl exec and not a sibling CronJob with the Maildir mounted: mailserver-data-encrypted is a RWO volume held by the mailserver pod. A sibling would fail to attach. The nextcloud-watchdog pattern in stacks/nextcloud/main.tf already solves this for a similar "interact with the live pod on a schedule" shape. Mirrored here with its own SA + Role + RoleBinding scoped to list/get pods and create pods/exec in the mailserver namespace only. Why Python and not pure shell: POSIX `find + stat + awk` struggles with the header-scan-up-to-blank-line rule, and `stat -c` is Linux- GNU-specific anyway. The script reads each message's first 64 KiB, stops at the first blank line, scans headers only, then checks mtime. The CronJob streams the Python source via `kubectl exec -i ... -- python3 - <<PYEOF`. After the retention pass, `doveadm force-resync -u spam@viktorbarzin.me INBOX/spam` refreshes Dovecot's cached index so the deletions appear in IMAP immediately instead of after the next pod restart. Includes the standard KYVERNO_LIFECYCLE_V1 marker on the CronJob so Kyverno ndots mutation does not cause perpetual drift. ## What is NOT in this change - Dovecot sieve rules (no sieve infrastructure exists in the module; the plan file's fallback option was precisely this CronJob path). - Push of retention metrics to Pushgateway - the script prints them to the job log for now; plumbing Pushgateway is a follow-up if Viktor wants alerts. - Any touch of other mailboxes - only `/var/mail/viktorbarzin.me/spam/cur` is walked. - Any mailserver pod restart or config reload. ## Test plan ### Automated `terraform fmt` + `terragrunt hclfmt` pass. `scripts/tg plan` on the mailserver stack shows: Plan: 7 to add, 3 to change, 0 to destroy. Of the 7 adds, 4 are mine (SA + Role + RoleBinding + CronJob). The other 3 adds belong to the concurrent roundcube-backup CronJob + nfs_roundcube_backup_host PV + PVC already on master in parallel. The 3 in-place updates are pre-existing drift on the mailserver Deployment, Service and email_roundtrip_monitor CronJob, not introduced by this change. ### Manual Verification After `scripts/tg apply` lands the CronJob: 1. Trigger an immediate run: `kubectl -n mailserver create job --from=cronjob/spam-retention manual-1` 2. Wait for completion, read the log: `kubectl -n mailserver logs job/manual-1` -> expected tail: spam_retention_scanned_total <N> spam_retention_auto_deleted_total <M> spam_retention_human_deleted_total <H> spam_retention_kept_total <K> spam_retention_errors_total 0 Retention pass complete 3. Confirm mailbox shrunk: `kubectl -n mailserver exec deploy/mailserver -c docker-mailserver \ -- du -sh /var/mail/viktorbarzin.me/spam/` -> expected: well below 43 MiB within one run (bulk rule alone purges ~328 messages per the profile numbers above). 4. Confirm IMAP reflects the deletions: `kubectl -n mailserver exec deploy/mailserver -c docker-mailserver \ -- doveadm mailbox status -u spam@viktorbarzin.me messages INBOX/spam` -> expected: message count dropped accordingly. 5. 4 hours later, confirm the next scheduled run logs a much smaller scan count and 0 deletions (nothing new crossed the threshold). Closes: code-oy4 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| modules/mailserver | ||
| main.tf | ||
| secrets | ||
| terragrunt.hcl | ||