infra/docs/runbooks/paperless-mail-ingest.md

Paperless-ngx Mail Ingest (docs@viktorbarzin.me)

Last updated: 2026-07-03 (initial build)

Forward any email with document attachments to docs@viktorbarzin.me and paperless-ngx ingests the attachments, owned by the paperless account mapped from the sender (From) address. Built entirely from existing parts: a docker-mailserver mailbox + Dovecot sieve, and paperless-ngx's native mail consumer (the same machinery as the utility: rules).

Flow

family member forwards email ──> MX ──> docker-mailserver
    │  postfix virtual: docs@ has an explicit self-alias (extra/aliases.txt),
    │  so the @domain catch-all (→ spam@, swept by TripIt) does NOT apply
    ▼
Dovecot LMTP delivery to docs@
    │  per-user sieve (docs@viktorbarzin.me.dovecot.sieve): sender NOT in
    │  allowlist → discard (decision 2026-07-03: unmatched = ignore & delete)
    ▼
docs@ INBOX ── paperless-ngx mail task (every 10 min, PAPERLESS_EMAIL_TASK_CRON
    │          default) applies mail rules in order: filter_from = <sender>
    │          → consume attachments (attachments-only: inline images like
    │          signature logos are skipped), owner = mapped user,
    │          tag = email-ingest, title = mail subject
    ▼
consumed mail is MOVED to the "Processed" IMAP folder (audit trail);
INBOX stays empty in steady state

Sender → paperless account map (as built)

Sender (From)	Paperless user	Rule
me@viktorbarzin.me	root (id 3)	forward: Viktor (me@)
vbarzin@gmail.com	root (id 3)	forward: Viktor (gmail)
viktorbarzin@meta.com	root (id 3)	forward: Viktor (meta)
ancaelena98@gmail.com	anca (id 4)	forward: Anca
emil.barzin@gmail.com	emo (id 7)	forward: Emo

The map lives in two places by design — keep them in sync:

1. Delivery gate (infra, Terraform): stacks/mailserver/modules/mailserver/extra/docs-at-viktorbarzin.me.dovecot.sieve — senders not listed here are discarded at delivery (spam control + the "ignore and delete unmatched" behaviour; paperless cannot express "delete without ingesting", so this must happen before the mailbox).
2. Owner map (paperless DB, via API/UI): one mail rule per sender on the docs@viktorbarzin.me mail account. DB-state like workflows — NOT Terraform.

Add a family member / sender

1. Add the address to the sieve allowlist file above; commit; apply the mailserver stack (normal apply is enough — the sieve CM key is not under ignore_changes; Reloader restarts the pod).
2. Clone an existing forward: mail rule in the paperless admin UI (Mail → Rules) or via API, changing filter_from and the rule owner (documents are owned by the rule owner — assign_owner_from_rule=true). Keep: action = Move to Processed, attachment type = attachments-only, consumption scope = attachments only, tag email-ingest, order after the existing rules.

Operations

• Trigger a fetch immediately (instead of waiting ≤10 min): kubectl -n paperless-ngx exec deploy/paperless-ngx -c paperless-ngx -- s6-setuidgid paperless python3 manage.py mail_fetcher The s6-setuidgid paperless is required: kubectl exec runs as root, and a root-run fetcher downloads attachments root-owned into the scratch dir, which the celery consumer (uid 1000) then can't read — PermissionError on /tmp/paperless/paperless-mail-*/..., consume task FAILURE (hit during the 2026-07-03 build E2E). The mail correctly stays in INBOX for retry (the move action is a chord callback on successful consumption). Recover: rm -rf /tmp/paperless/paperless-mail-* (as root) and let the next scheduled fetch re-process.
• Mailbox credentials: Vault secret/platform → mailserver_accounts JSON, key docs@viktorbarzin.me (also used by the paperless mail account).
• Inspect the mailbox: python3 -c IMAP to mailserver.mailserver.svc.cluster.local:993 (in-cluster, from a pod) or mail.viktorbarzin.me:993 (externally / devvm).
• Paperless-side logs: kubectl -n paperless-ngx logs deploy/paperless-ngx | grep -i mail (also Loki, ns paperless-ngx). Rule/account state: GET /api/mail_rules/, GET /api/mail_accounts/ with the admin token (k8s secret paperless-ngx-secrets, field api_token).
• Account/mailbox provisioning: adding/rotating anything in mailserver_accounts requires the ConfigMap replace workaround — scripts/tg apply mailserver -- -replace=module.mailserver.kubernetes_config_map.mailserver_config — because postfix-accounts.cf is under ignore_changes (non-deterministic bcrypt; see the module comment).

Design notes / caveats

• Why not the catch-all? Mail to unknown @viktorbarzin.me addresses lands in spam@, which the TripIt ingest-plans CronJob sweeps every 15 min: it marks everything \Seen, LLM-parses mail from linked senders and replies with ack/failure emails. Forwarded bank statements would get "couldn't parse a trip" replies. docs@ being a real mailbox bypasses that path entirely; TripIt, the smoke-test@ roundtrip probe, and dmarc@ are untouched.
• Spoofing: the sender match is on the From header. Rspamd verifies SPF/DKIM/DMARC on inbound mail, but gmail.com publishes p=none, so a crafted spoof could ingest documents into a family member's account. Accepted risk (worst case: unwanted documents appear, visible + deletable in paperless).
• Not PDF-only: any real attachment type paperless supports is consumed (PDF, images, Office via the existing tika+gotenberg pipeline). Inline images are excluded by attachment_type=1.
• No dedicated alerting (deliberate, 2026-07-03): mail-task errors surface in paperless logs; the mailserver inbound path is covered by email-roundtrip-monitor. Revisit if forwards start silently failing.
• Workflows: the global payslip-webhook + claude-mcp-readers auto-permission workflows fire for mail-ingested docs like any other consumption source (verified pre-build; payslip receiver does its own filtering).

Rollback

1. Disable/delete the 5 forward: mail rules + the docs@ mail account (paperless admin UI or API).
2. Revert the infra commit (aliases.txt entry, sieve file, CM key + mount).
3. Remove docs@viktorbarzin.me from Vault mailserver_accounts, then apply with the -replace workaround above. Mail to docs@ then falls back to the catch-all (spam@) like any unknown address.