infra/scripts/pfsense-nat-mailserver-haproxy-unflip.php

49 lines
1.4 KiB
PHP
Raw Permalink Normal View History

[mailserver] Phase 4+5 — pfSense HAProxy cutover for all 4 mail ports [ci skip] ## Context (bd code-yiu) Cutover of external mail traffic from the MetalLB LB IP path (ETP:Local, pod-speaker colocation) to pfSense HAProxy + PROXY v2 (ETP:Cluster). Real client IP now preserved end-to-end on ports 25/465/587/993, both for postscreen anti-spam scoring and CrowdSec auth-failure bans. ## This change ### k8s (stacks/mailserver/modules/mailserver/main.tf) - `mailserver-user-patches` ConfigMap's `user-patches.sh` now appends 3 alt PROXY-speaking services to master.cf: - `:2525` postscreen (alt :25) - `:4465` smtpd (alt :465 SMTPS, wrappermode TLS) - `:5587` smtpd (alt :587 submission) All with `postscreen_upstream_proxy_protocol=haproxy` / `smtpd_upstream_proxy_protocol=haproxy`. Mirror stock submission/submissions options (SASL via Dovecot, TLS, client restrictions, mua_sender_restrictions). chroot=n so the SASL socket path `/dev/shm/sasl-auth.sock` resolves outside the chroot. - `dovecot.cf` ConfigMap adds: ``` haproxy_trusted_networks = 10.0.20.0/24 service imap-login { inet_listener imaps_proxy { port=10993; ssl=yes; haproxy=yes } } ``` Stock :993 stays PROXY-free for internal Roundcube/probe clients. - Container ports: 4 new (4465, 5587, 10993, 2525 already there). - `mailserver-proxy` NodePort Service now exposes all 4 ports: 25→2525→30125, 465→4465→30126, 587→5587→30127, 993→10993→30128 (ETP:Cluster). ### pfSense (scripts/pfsense-haproxy-bootstrap.php) Rebuilt to declare 4 backend pools (one per NodePort) and 4 production frontends on `10.0.20.1:{25,465,587,993}` TCP mode, plus the legacy `:2525` test frontend. All pools: `send-proxy-v2 check inter 120000`. Idempotent — re-runs converge on declared state. ### pfSense (scripts/pfsense-nat-mailserver-haproxy-{flip,unflip}.php) Flip script: updates `<nat><rule>` entries for mail ports from target `<mailserver>` alias (10.0.20.202 MetalLB) → `10.0.20.1` (pfSense HAProxy). Runs `filter_configure()` to rebuild pf rules. Unflip is the rollback. Both scripts are idempotent. ## What is NOT in this change - Phase 6 (decommission MetalLB LB path, downgrade mailserver Service from LoadBalancer to ClusterIP, free 10.0.20.202) — USER-GATED. Do NOT run until explicit approval. - Legacy MetalLB `mailserver` LB still live on 10.0.20.202 with stock ETP:Local ports — functional backup path + consumed by internal clients that hit `mailserver.mailserver.svc.cluster.local` (routes via ClusterIP layer of the LB Service, bypassing ETP). - Port :143 (plain IMAP) — no HAProxy frontend; stays on MetalLB via unchanged NAT rule. ## Test Plan ### Automated (verified pre-commit 2026-04-19) ``` # k8s container listens on all 8 ports $ kubectl exec -c docker-mailserver deployment/mailserver -n mailserver \ -- ss -ltn | grep -E ':(25|2525|465|4465|587|5587|993|10993)\b' ... all 8 listening ... # pfSense HAProxy listens on all 5 (production + legacy test) $ ssh admin@10.0.20.1 'sockstat -l | grep haproxy' www haproxy 49418 5 tcp4 *:25 www haproxy 49418 6 tcp4 *:2525 www haproxy 49418 10 tcp4 *:465 www haproxy 49418 11 tcp4 *:587 www haproxy 49418 12 tcp4 *:993 # Post-flip: pf rdr rules point at pfSense, not <mailserver> $ ssh admin@10.0.20.1 'pfctl -sn' | grep 'smtp\|sub\|imap\|:25' rdr on vtnet0 ... port = submission -> 10.0.20.1 rdr on vtnet0 ... port = imaps -> 10.0.20.1 rdr on vtnet0 ... port = smtps -> 10.0.20.1 rdr on vtnet0 ... port = 25 -> 10.0.20.1 # 4 HAProxy frontends reachable + SMTP/IMAP banners $ python3 <test script> → SMTP/SMTPS/Sub/IMAPS all respond correctly # Real client IP in maillog for external delivery via Brevo → MX postfix/smtpd-proxy25/postscreen: CONNECT from [77.32.148.26]:36334 to [10.0.20.1]:25 postfix/smtpd-proxy25/postscreen: PASS NEW [77.32.148.26]:36334 # E2E probe (Brevo HTTP → external SMTP delivery → IMAP fetch) succeeds $ kubectl create job --from=cronjob/email-roundtrip-monitor probe-yiu-flip -n mailserver ... Round-trip SUCCESS in 20.3s ... # Internal Roundcube path unchanged $ curl -sI https://mail.viktorbarzin.me/ → 302 (Authentik gate intact) # No mail alerts firing $ kubectl exec prometheus-server ... /api/v1/alerts | grep Email → (empty) ``` ### Rollback ``` scp infra/scripts/pfsense-nat-mailserver-haproxy-unflip.php admin@10.0.20.1:/tmp/ ssh admin@10.0.20.1 'php /tmp/pfsense-nat-mailserver-haproxy-unflip.php' ``` Immediate (<2s). Flips all 4 NAT rdrs back to `<mailserver>` alias. Pre-flip config snapshot also saved at `/tmp/config.xml.pre-yiu-flip.20260419-1222` on pfSense. ## Phase roadmap (bd code-yiu) | Phase | Status | |---|---| | 1a | ✅ commit ef75c02f — alt :2525 listener + NodePort | | 2 | ✅ 2026-04-19 — HAProxy pkg installed on pfSense | | 3 | ✅ commit ba697b02 — HAProxy config persisted in pfSense XML | | 4+5| ✅ **this commit** — 4-port alt listeners + HAProxy frontends + NAT flip | | 6 | ⏸ USER-GATED — MetalLB LB decommission after 48h observation |
2026-04-19 12:24:50 +00:00
<?php
// REVERT of pfsense-nat-mailserver-haproxy-flip.php.
// Moves mail-port NAT rdr target from 10.0.20.1 (pfSense HAProxy) back to
// <mailserver> alias (10.0.20.202 MetalLB LB IP). bd code-yiu rollback.
//
// USE THIS IF: external mail breaks after the flip, any postscreen
// PROXY timeouts show up in logs, or you need to back out before Phase 6.
require_once('/etc/inc/config.inc');
require_once('/etc/inc/filter.inc');
global $config;
parse_config(true);
$PORTS_TO_REVERT = ['25', '465', '587', '993'];
$OLD_TARGET = '10.0.20.1';
$NEW_TARGET = 'mailserver';
$changed = 0;
foreach ($config['nat']['rule'] as $i => &$r) {
$iface = $r['interface'] ?? '';
$lport = $r['local-port'] ?? '';
$tgt = $r['target'] ?? '';
if ($iface !== 'wan') continue;
if (!in_array($lport, $PORTS_TO_REVERT, true)) continue;
if ($tgt !== $OLD_TARGET) {
printf("rule %d (dport=%s) target=%s — not reverting (already %s or unexpected)\n",
$i, $lport, $tgt, $NEW_TARGET);
continue;
}
$r['target'] = $NEW_TARGET;
$changed++;
printf("rule %d (dport=%s): target %s → %s\n", $i, $lport, $OLD_TARGET, $NEW_TARGET);
}
unset($r);
if ($changed === 0) {
echo "No changes. (Already reverted.)\n";
exit(0);
}
write_config("code-yiu: NAT rdr — mail ports {$changed} reverted to <mailserver> alias");
$rc = filter_configure();
printf("filter_configure rc=%s\n", var_export($rc, true));
echo "done.\n";