## Context Dovecot auth logs have been steadily spamming `passwd-file /etc/dovecot/userdb: User r730-idrac@viktorbarzin.me exists more than once` (and the same for vaultwarden@) at ~31 occurrences per 500 log lines. Under load this flakes IMAP auth for the e2e email-roundtrip probe (spam@viktorbarzin.me uses the catch-all), which was masquerading as "Brevo or probe timing" noise. ## Root cause docker-mailserver builds Dovecot's `/etc/dovecot/userdb` from two sources: real accounts (`postfix-accounts.cf`) AND virtual-alias entries whose *target* resolves to a local mailbox (`postfix-virtual.cf`). When the same address appears as BOTH a real mailbox AND an alias whose target is another local mailbox, the generated userdb has two lines for that username pointing to different home directories — e.g.: r730-idrac@viktorbarzin.me:...:/var/mail/.../r730-idrac/home r730-idrac@viktorbarzin.me:...:/var/mail/.../spam/home ← from alias Dovecot's passwd-file driver rejects the duplicate, and every subsequent auth lookup logs the error. This affected exactly two addresses: - r730-idrac@viktorbarzin.me (real account + alias → spam@) - vaultwarden@viktorbarzin.me (real account + alias → me@) Other aliases are fine: they either forward to external addresses (gmail etc.) — no local userdb entry generated — or map an address to itself (me@ → me@) which docker-mailserver dedups internally. Note: removing the real accounts is not an option because Vaultwarden uses `vaultwarden@viktorbarzin.me` as its live SMTP_USERNAME (stacks/vaultwarden/modules/vaultwarden/main.tf:121). ## This change Introduces a `local.postfix_virtual` that concatenates the Vault-sourced aliases with `extra/aliases.txt`, then filters out any line matching the exact "LHS RHS" shape where both sides are in `var.mailserver_accounts` and LHS != RHS. That is, only the pure local→local redundant entries are dropped; all forwarding aliases and the catch-all are preserved. The filter is self-healing: if a future alias ever collides with a real account, it gets silently suppressed instead of breaking Dovecot auth. ``` Vault mailserver_aliases ─┐ ├─ concat ─ split \n ─ filter ─ join \n ─► postfix-virtual.cf extra/aliases.txt ─────────┘ │ └── drop if LHS+RHS both in mailserver_accounts and LHS != RHS ``` Filtered entries (confirmed via locally-simulated filter on live data): - r730-idrac@viktorbarzin.me spam@viktorbarzin.me - vaultwarden@viktorbarzin.me me@viktorbarzin.me Preserved (sample): postmaster→me, contact→me, alarm-valchedrym→self+3 ext, lubohristov→gmail, yoana→gmail, @viktorbarzin.me→spam (catch-all), all four disposable `*-generated@` aliases. ## What is NOT in this change - Real accounts in Vault (`secret/platform.mailserver_accounts`) are untouched — vaultwarden SMTP auth keeps working. - Postfix postscreen btree lock contention (separate commit). - Email-roundtrip probe IMAP window (separate commit). ## Test Plan ### Automated `terraform validate` — passes (docker-mailserver module): ``` Success! The configuration is valid, but there were some validation warnings as shown above. ``` `scripts/tg plan -target=module.mailserver.kubernetes_config_map.mailserver_config`: ``` # module.mailserver.kubernetes_config_map.mailserver_config will be updated in-place ~ resource "kubernetes_config_map" "mailserver_config" { ~ data = { ~ "postfix-virtual.cf" = (sensitive value) # (9 unchanged elements hidden) } id = "mailserver/mailserver.config" } Plan: 0 to add, 1 to change, 0 to destroy. ``` `scripts/tg apply` — applied: ``` Apply complete! Resources: 0 added, 1 changed, 0 destroyed. ``` ### Manual Verification Post-apply configmap content (the two lines are gone): ``` $ kubectl -n mailserver get cm mailserver.config -o jsonpath='{.data.postfix-virtual\.cf}' postmaster@viktorbarzin.me me@viktorbarzin.me contact@viktorbarzin.me me@viktorbarzin.me me@viktorbarzin.me me@viktorbarzin.me lubohristov@viktorbarzin.me lyubomir.hristov3@gmail.com alarm-valchedrym@viktorbarzin.me alarm-valchedrym@...,vbarzin@...,emil.barzin@...,me@... yoana@viktorbarzin.me divcheva.yoana@gmail.com @viktorbarzin.me spam@viktorbarzin.me firmly-gerardo-generated@viktorbarzin.me me@viktorbarzin.me closely-keith-generated@viktorbarzin.me vbarzin@gmail.com literally-paolo-generated@viktorbarzin.me viktorbarzin@fb.com hastily-stefanie-generated@viktorbarzin.me elliestamenova@gmail.com ``` Reloader triggers a pod rollout; once new pod is Ready: - `kubectl -n mailserver exec <pod> -c docker-mailserver -- cut -d: -f1 /etc/dovecot/userdb | sort | uniq -d` expected output: empty (no duplicate usernames) - `kubectl -n mailserver logs <pod> -c docker-mailserver --tail=500 | grep -c "exists more than once"` expected output: 0 (baseline was 31/500 lines) ## Reproduce locally 1. `kubectl -n mailserver get cm mailserver.config -o jsonpath='{.data.postfix-virtual\.cf}'` 2. Expect: no `r730-idrac@viktorbarzin.me spam@viktorbarzin.me` line and no `vaultwarden@viktorbarzin.me me@viktorbarzin.me` line. 3. After pod restart: `kubectl -n mailserver logs -l app=mailserver -c docker-mailserver --tail=500 | grep -c "exists more than once"` → 0. Closes: code-27l Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .beads | ||
| .claude | ||
| .git-crypt | ||
| .github | ||
| .planning | ||
| .woodpecker | ||
| ci | ||
| cli | ||
| diagram | ||
| docs | ||
| modules | ||
| playbooks | ||
| scripts | ||
| secrets | ||
| stacks | ||
| state/stacks | ||
| .gitattributes | ||
| .gitignore | ||
| .sops.yaml | ||
| AGENTS.md | ||
| config.tfvars | ||
| CONTRIBUTING.md | ||
| LICENSE.txt | ||
| MEMORY.md | ||
| README.md | ||
| setup-monitoring.sh | ||
| terragrunt.hcl | ||
| tiers.tf | ||
This repo contains my infra-as-code sources.
My infrastructure is built using Terraform, Kubernetes and CI/CD is done using Woodpecker CI.
Read more by visiting my website: https://viktorbarzin.me
Documentation
Full architecture documentation is available in docs/ — covering networking, storage, security, monitoring, secrets, CI/CD, databases, and more.
Adding a New User (Admin)
Adding a new namespace-owner to the cluster requires three steps — no code changes needed.
1. Authentik Group Assignment
In the Authentik admin UI, add the user to:
kubernetes-namespace-ownersgroup (grants OIDC group claim for K8s RBAC)Headscale Usersgroup (if they need VPN access)
2. Vault KV Entry
Add a JSON entry to secret/platform → k8s_users key in Vault:
"username": {
"role": "namespace-owner",
"email": "user@example.com",
"namespaces": ["username"],
"domains": ["myapp"],
"quota": {
"cpu_requests": "2",
"memory_requests": "4Gi",
"memory_limits": "8Gi",
"pods": "20"
}
}
usernamekey must match the user's Forgejo username (for Woodpecker admin access)namespaces— K8s namespaces to create and grant admin access todomains— subdomains underviktorbarzin.mefor Cloudflare DNS recordsquota— resource limits per namespace (defaults shown above)
3. Apply Stacks
vault login -method=oidc
cd stacks/vault && terragrunt apply --non-interactive
# Creates: namespace, Vault policy, identity entity, K8s deployer role
cd ../platform && terragrunt apply --non-interactive
# Creates: RBAC bindings, ResourceQuota, TLS secret, DNS records
cd ../woodpecker && terragrunt apply --non-interactive
# Adds user to Woodpecker admin list
What Gets Auto-Generated
| Resource | Stack |
|---|---|
| Kubernetes namespace | vault |
Vault policy (namespace-owner-{user}) |
vault |
| Vault identity entity + OIDC alias | vault |
| K8s deployer Role + Vault K8s role | vault |
| RBAC RoleBinding (namespace admin) | platform |
| RBAC ClusterRoleBinding (cluster read-only) | platform |
| ResourceQuota | platform |
| TLS secret in namespace | platform |
| Cloudflare DNS records | platform |
| Woodpecker admin access | woodpecker |
New User Onboarding
If you've been added as a namespace-owner, follow these steps to get started.
1. Join the VPN
# Install Tailscale: https://tailscale.com/download
tailscale login --login-server https://headscale.viktorbarzin.me
# Send the registration URL to Viktor, wait for approval
ping 10.0.20.100 # verify connectivity
2. Install Tools
Run the setup script to install kubectl, kubelogin, Vault CLI, Terraform, and Terragrunt:
# macOS
bash <(curl -fsSL https://k8s-portal.viktorbarzin.me/setup/script?os=mac)
# Linux
bash <(curl -fsSL https://k8s-portal.viktorbarzin.me/setup/script?os=linux)
3. Authenticate
# Log into Vault (opens browser for SSO)
vault login -method=oidc
# Test kubectl (opens browser for OIDC login)
kubectl get pods -n YOUR_NAMESPACE
4. Deploy Your First App
# Clone the infra repo
git clone https://github.com/ViktorBarzin/infra.git && cd infra
# Copy the stack template
cp -r stacks/_template stacks/myapp
mv stacks/myapp/main.tf.example stacks/myapp/main.tf
# Edit main.tf — replace all <placeholders>
# Store secrets in Vault
vault kv put secret/YOUR_USERNAME/myapp DB_PASSWORD=secret123
# Submit a PR
git checkout -b feat/myapp
git add stacks/myapp/
git commit -m "add myapp stack"
git push -u origin feat/myapp
After review and merge, an admin runs cd stacks/myapp && terragrunt apply.
5. Set Up CI/CD (Optional)
Create .woodpecker.yml in your app's Forgejo repo:
steps:
- name: build
image: woodpeckerci/plugin-docker-buildx
settings:
repo: YOUR_DOCKERHUB_USER/myapp
tag: ["${CI_PIPELINE_NUMBER}", "latest"]
username:
from_secret: dockerhub-username
password:
from_secret: dockerhub-token
platforms: linux/amd64
- name: deploy
image: hashicorp/vault:1.18.1
commands:
- export VAULT_ADDR=http://vault-active.vault.svc.cluster.local:8200
- export VAULT_TOKEN=$(vault write -field=token auth/kubernetes/login
role=ci jwt=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token))
- KUBE_TOKEN=$(vault write -field=service_account_token
kubernetes/creds/YOUR_NAMESPACE-deployer
kubernetes_namespace=YOUR_NAMESPACE)
- kubectl --server=https://kubernetes.default.svc
--token=$KUBE_TOKEN
--certificate-authority=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
-n YOUR_NAMESPACE set image deployment/myapp
myapp=YOUR_DOCKERHUB_USER/myapp:${CI_PIPELINE_NUMBER}
Useful Commands
# Check your pods
kubectl get pods -n YOUR_NAMESPACE
# View quota usage
kubectl describe resourcequota -n YOUR_NAMESPACE
# Store/read secrets
vault kv put secret/YOUR_USERNAME/myapp KEY=value
vault kv get secret/YOUR_USERNAME/myapp
# Get a short-lived K8s deploy token
vault write kubernetes/creds/YOUR_NAMESPACE-deployer \
kubernetes_namespace=YOUR_NAMESPACE
Important Rules
- All changes go through Terraform — never
kubectl apply/edit/patchdirectly - Never put secrets in code — use Vault:
vault kv put secret/YOUR_USERNAME/... - Always use a PR — never push directly to master
- Docker images: build for
linux/amd64, use versioned tags (not:latest)
git-crypt setup
To decrypt the secrets, you need to setup git-crypt.
- Install git-crypt.
- Setup gpg keys on the machine
git-crypt unlock
This will unlock the secrets and will lock them on commit