viktor/infra
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

[ci skip] Add skills: pfsense-nat-rule-creation, coturn-k8s-without-hostnetwork

Browse source
This commit is contained in:
Viktor Barzin 2026-02-21 18:29:32 +00:00
parent de1a43a3c7
commit 9b2ec7716e
2 changed files with 250 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

145
.claude/skills/coturn-k8s-without-hostnetwork/SKILL.md Normal file
View file

@ -0,0 +1,145 @@
---
name: coturn-k8s-without-hostnetwork
description: |
Deploy coturn (TURN/STUN server) on Kubernetes without hostNetwork by using a
narrow relay port range and MetalLB LoadBalancer service. Use when: (1) deploying
a WebRTC relay server on k8s, (2) want coturn to run on any node (not pinned),
(3) avoiding hostNetwork for better pod scheduling and multi-replica support,
(4) need TURN for NAT traversal in WebRTC apps (video streaming, conferencing).
Covers relay port range sizing, MetalLB IP sharing, ephemeral TURN credentials
via HMAC-SHA1, and pfSense port forwarding.
author: Claude Code
version: 1.0.0
date: 2026-02-21
---
# coturn on Kubernetes Without hostNetwork
## Problem
TURN servers traditionally require hostNetwork because they relay media over a wide
UDP port range (49152-65535). This pins the server to a single node, prevents rolling
updates, and wastes cluster flexibility.
## Context / Trigger Conditions
- Deploying a TURN/STUN server for WebRTC applications on Kubernetes
- Want the TURN pod to be schedulable on any node
- Need to avoid hostNetwork for better availability and scheduling
## Solution
### Key insight: Narrow the relay port range
A home lab with ~20 concurrent WebRTC viewers needs ~40 relay ports (2 per viewer).
Use 100 ports (49152-49252) instead of 16K. This makes it practical to expose via
a K8s LoadBalancer service.
### Terraform module structure
```hcl
locals {
turn_port = 3478
min_port = 49152
max_port = 49252 # 100 ports — enough for ~50 concurrent streams
}
resource "kubernetes_deployment" "coturn" {
spec {
# No hostNetwork, no nodeSelector — runs anywhere
template {
spec {
container {
image = "coturn/coturn:latest"
args = ["-c", "/etc/turnserver/turnserver.conf"]
port {
container_port = 3478
protocol = "UDP"
}
}
}
}
}
}
resource "kubernetes_service" "coturn" {
metadata {
annotations = {
# Share an existing MetalLB IP to avoid consuming a new one
"metallb.universe.tf/loadBalancerIPs" = "10.0.20.200"
"metallb.universe.tf/allow-shared-ip" = "shared"
}
}
spec {
type = "LoadBalancer"
# Signaling port
port {
name = "turn-udp"
port = 3478
protocol = "UDP"
}
# Relay ports — dynamic block generates 100 port definitions
dynamic "port" {
for_each = range(49152, 49253)
content {
name = "relay-${port.value}"
port = port.value
target_port = port.value
protocol = "UDP"
}
}
}
}
```
### coturn config (turnserver.conf)
```
listening-port=3478
fingerprint
lt-cred-mech
use-auth-secret
static-auth-secret=YOUR_SECRET_HERE
realm=yourdomain.com
listening-ip=0.0.0.0
min-port=49152
max-port=49252
no-multicast-peers
no-cli
```
### MetalLB IP sharing
To reuse an existing MetalLB IP (e.g., the WireGuard/Shadowsocks shared IP):
1. Add `metallb.universe.tf/allow-shared-ip: shared` to the coturn service
2. The same annotation must exist on all other services sharing that IP
3. **Port conflicts are not allowed** — verify no other service uses 3478 or 49152-49252
4. After changing the IP annotation, **delete and recreate** the service — MetalLB won't reassign IPs on annotation changes alone
### Ephemeral TURN credentials
coturn's `use-auth-secret` mode generates time-limited credentials via HMAC-SHA1:
```javascript
const crypto = require('crypto');
const TURN_SECRET = 'your-shared-secret';
function getTurnCredentials(name = 'user', ttl = 86400) {
const timestamp = Math.floor(Date.now() / 1000) + ttl;
const username = `${timestamp}:${name}`;
const credential = crypto.createHmac('sha1', TURN_SECRET)
.update(username).digest('base64');
return { username, credential };
}
```
## Verification
```bash
# STUN binding request (raw UDP probe)
echo -ne '\x00\x01\x00\x00\x21\x12\xa4\x42\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' \
| nc -u -w2 <METALLB_IP> 3478 | xxd | head -3
# Response starting with 0101 = successful STUN binding response
```
## Notes
- 100 relay ports supports ~50 concurrent streams (2 ports per stream)
- If you need more, increase `max_port` and add more ports to the service
- coturn auto-detects pod IP — no need to set `relay-ip` or `external-ip` explicitly
- For public access, add NAT port forwards on pfSense for UDP 3478 + 49152-49252
- See also: `pfsense-nat-rule-creation` skill for adding the port forwards

105
.claude/skills/pfsense-nat-rule-creation/SKILL.md Normal file
View file

@ -0,0 +1,105 @@
---
name: pfsense-nat-rule-creation
description: |
Create NAT port forward rules on pfSense programmatically via PHP/SSH.
Use when: (1) adding port forwards for new K8s services, (2) NAT rules
added via PHP don't appear in pfctl output, (3) config_read_array() throws
"undefined function" error, (4) destination "wanip" not working in NAT rules,
(5) rules saved to config.xml but not loaded into pfctl. Covers the correct
PHP array structure, config API differences between pfSense versions, and
the required pfctl reload step.
author: Claude Code
version: 1.0.0
date: 2026-02-21
---
# pfSense NAT Rule Creation via PHP
## Problem
Creating NAT port forward rules on pfSense programmatically via SSH/PHP has
multiple gotchas around the config API, rule structure, and rule loading.
## Context / Trigger Conditions
- Adding a port forward for a new Kubernetes service (e.g., TURN, game server)
- Using `ssh admin@10.0.20.1` + PHP to automate pfSense config
- NAT rules don't appear in `pfctl -sn` after `write_config()` + `filter_configure()`
- `config_read_array()` throws "Call to undefined function"
- Rules saved to config.xml but pfctl doesn't have them
## Solution
### Correct PHP for adding NAT rules
```php
<?php
require_once("config.inc");
require_once("filter.inc");
global $config; // NOT config_read_array() — that doesn't exist in pfSense 2.7.x
$config["nat"]["rule"][] = array(
"interface" => "wan",
"ipprotocol" => "inet", // Required! Must be "inet" for IPv4
"protocol" => "tcp/udp", // Or "udp" or "tcp"
"source" => array("any" => ""),
"destination" => array(
"network" => "wanip", // Use "network" => "wanip", NOT "address" => "wanip"
"port" => "3478" // Single port or "start:end" for range
),
"target" => "10.0.20.200", // Internal destination IP
"local-port" => "3478", // Internal port (for ranges, just the start port)
"descr" => "My port forward",
"associated-rule-id" => "pass" // Auto-create firewall pass rule
);
write_config("Description for config history");
filter_configure();
```
### Key gotchas
1. **`config_read_array()` doesn't exist** in pfSense 2.7.x. Use `global $config` instead.
2. **Destination format**: Use `"network" => "wanip"`, NOT `"address" => "wanip"` or `"address" => "192.168.1.2"`. The `"network"` key with `"wanip"` tells pfSense to resolve the WAN IP dynamically.
3. **`ipprotocol` is required**: Must include `"ipprotocol" => "inet"` or rules won't generate in `/tmp/rules.debug`.
4. **Port ranges**: Use `"port" => "49152:49252"` for ranges. The `"local-port"` should be just the start port — pfSense maps the range automatically.
5. **Rules may not load immediately**: After `write_config()` + `filter_configure()`, rules appear in `/tmp/rules.debug` but may not be in pfctl until the next filter reload. Force with:
```bash
pfctl -f /tmp/rules.debug
```
6. **SSH quoting**: The pfsense.py `php` command breaks on `\n` in strings. For multi-line PHP, write a `.php` file, `scp` it, and execute:
```bash
scp script.php admin@10.0.20.1:/tmp/
ssh admin@10.0.20.1 "php /tmp/script.php"
```
### Execution via pfsense.py
For simple single-line PHP (no newlines or backslashes):
```bash
python3 .claude/pfsense.py php 'require_once("config.inc"); ...; echo "Done";'
```
For complex scripts, use scp + ssh as above.
## Verification
```bash
# Check rules in config
ssh admin@10.0.20.1 "grep 'YOUR_PORT' /cf/conf/config.xml"
# Check generated pf rules
ssh admin@10.0.20.1 "grep 'YOUR_PORT' /tmp/rules.debug"
# Check active pfctl rules
python3 .claude/pfsense.py pfctl "-sn" | grep YOUR_PORT
```
## Notes
- Existing working NAT rules on this pfSense use the same structure (check WireGuard port 51820 as reference)
- The `associated-rule-id: pass` auto-creates a WAN firewall rule to allow the forwarded traffic
- pfSense applies NAT rules across ALL interfaces when using the web UI, but PHP-created rules only apply to the specified interface
- See also: `pfsense` skill for general pfSense management