[ci skip] Add skills: pfsense-nat-rule-creation, coturn-k8s-without-hostnetwork
This commit is contained in:
Viktor Barzin
parent
a4817e8192
commit
3739906747
2 changed files with 250 additions and 0 deletions
145
.claude/skills/coturn-k8s-without-hostnetwork/SKILL.md
Normal file
145
.claude/skills/coturn-k8s-without-hostnetwork/SKILL.md
Normal file
|
|
@ -0,0 +1,145 @@
|
|||
---
|
||||
name: coturn-k8s-without-hostnetwork
|
||||
description: |
|
||||
Deploy coturn (TURN/STUN server) on Kubernetes without hostNetwork by using a
|
||||
narrow relay port range and MetalLB LoadBalancer service. Use when: (1) deploying
|
||||
a WebRTC relay server on k8s, (2) want coturn to run on any node (not pinned),
|
||||
(3) avoiding hostNetwork for better pod scheduling and multi-replica support,
|
||||
(4) need TURN for NAT traversal in WebRTC apps (video streaming, conferencing).
|
||||
Covers relay port range sizing, MetalLB IP sharing, ephemeral TURN credentials
|
||||
via HMAC-SHA1, and pfSense port forwarding.
|
||||
author: Claude Code
|
||||
version: 1.0.0
|
||||
date: 2026-02-21
|
||||
---
|
||||
|
||||
# coturn on Kubernetes Without hostNetwork
|
||||
|
||||
## Problem
|
||||
TURN servers traditionally require hostNetwork because they relay media over a wide
|
||||
UDP port range (49152-65535). This pins the server to a single node, prevents rolling
|
||||
updates, and wastes cluster flexibility.
|
||||
|
||||
## Context / Trigger Conditions
|
||||
- Deploying a TURN/STUN server for WebRTC applications on Kubernetes
|
||||
- Want the TURN pod to be schedulable on any node
|
||||
- Need to avoid hostNetwork for better availability and scheduling
|
||||
|
||||
## Solution
|
||||
|
||||
### Key insight: Narrow the relay port range
|
||||
A home lab with ~20 concurrent WebRTC viewers needs ~40 relay ports (2 per viewer).
|
||||
Use 100 ports (49152-49252) instead of 16K. This makes it practical to expose via
|
||||
a K8s LoadBalancer service.
|
||||
|
||||
### Terraform module structure
|
||||
|
||||
```hcl
|
||||
locals {
|
||||
turn_port = 3478
|
||||
min_port = 49152
|
||||
max_port = 49252 # 100 ports — enough for ~50 concurrent streams
|
||||
}
|
||||
|
||||
resource "kubernetes_deployment" "coturn" {
|
||||
spec {
|
||||
# No hostNetwork, no nodeSelector — runs anywhere
|
||||
template {
|
||||
spec {
|
||||
container {
|
||||
image = "coturn/coturn:latest"
|
||||
args = ["-c", "/etc/turnserver/turnserver.conf"]
|
||||
port {
|
||||
container_port = 3478
|
||||
protocol = "UDP"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "kubernetes_service" "coturn" {
|
||||
metadata {
|
||||
annotations = {
|
||||
# Share an existing MetalLB IP to avoid consuming a new one
|
||||
"metallb.universe.tf/loadBalancerIPs" = "10.0.20.200"
|
||||
"metallb.universe.tf/allow-shared-ip" = "shared"
|
||||
}
|
||||
}
|
||||
spec {
|
||||
type = "LoadBalancer"
|
||||
# Signaling port
|
||||
port {
|
||||
name = "turn-udp"
|
||||
port = 3478
|
||||
protocol = "UDP"
|
||||
}
|
||||
# Relay ports — dynamic block generates 100 port definitions
|
||||
dynamic "port" {
|
||||
for_each = range(49152, 49253)
|
||||
content {
|
||||
name = "relay-${port.value}"
|
||||
port = port.value
|
||||
target_port = port.value
|
||||
protocol = "UDP"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### coturn config (turnserver.conf)
|
||||
|
||||
```
|
||||
listening-port=3478
|
||||
fingerprint
|
||||
lt-cred-mech
|
||||
use-auth-secret
|
||||
static-auth-secret=YOUR_SECRET_HERE
|
||||
realm=yourdomain.com
|
||||
listening-ip=0.0.0.0
|
||||
min-port=49152
|
||||
max-port=49252
|
||||
no-multicast-peers
|
||||
no-cli
|
||||
```
|
||||
|
||||
### MetalLB IP sharing
|
||||
To reuse an existing MetalLB IP (e.g., the WireGuard/Shadowsocks shared IP):
|
||||
1. Add `metallb.universe.tf/allow-shared-ip: shared` to the coturn service
|
||||
2. The same annotation must exist on all other services sharing that IP
|
||||
3. **Port conflicts are not allowed** — verify no other service uses 3478 or 49152-49252
|
||||
4. After changing the IP annotation, **delete and recreate** the service — MetalLB won't reassign IPs on annotation changes alone
|
||||
|
||||
### Ephemeral TURN credentials
|
||||
coturn's `use-auth-secret` mode generates time-limited credentials via HMAC-SHA1:
|
||||
|
||||
```javascript
|
||||
const crypto = require('crypto');
|
||||
const TURN_SECRET = 'your-shared-secret';
|
||||
|
||||
function getTurnCredentials(name = 'user', ttl = 86400) {
|
||||
const timestamp = Math.floor(Date.now() / 1000) + ttl;
|
||||
const username = `${timestamp}:${name}`;
|
||||
const credential = crypto.createHmac('sha1', TURN_SECRET)
|
||||
.update(username).digest('base64');
|
||||
return { username, credential };
|
||||
}
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# STUN binding request (raw UDP probe)
|
||||
echo -ne '\x00\x01\x00\x00\x21\x12\xa4\x42\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' \
|
||||
| nc -u -w2 <METALLB_IP> 3478 | xxd | head -3
|
||||
# Response starting with 0101 = successful STUN binding response
|
||||
```
|
||||
|
||||
## Notes
|
||||
- 100 relay ports supports ~50 concurrent streams (2 ports per stream)
|
||||
- If you need more, increase `max_port` and add more ports to the service
|
||||
- coturn auto-detects pod IP — no need to set `relay-ip` or `external-ip` explicitly
|
||||
- For public access, add NAT port forwards on pfSense for UDP 3478 + 49152-49252
|
||||
- See also: `pfsense-nat-rule-creation` skill for adding the port forwards
|
||||
105
.claude/skills/pfsense-nat-rule-creation/SKILL.md
Normal file
105
.claude/skills/pfsense-nat-rule-creation/SKILL.md
Normal file
|
|
@ -0,0 +1,105 @@
|
|||
---
|
||||
name: pfsense-nat-rule-creation
|
||||
description: |
|
||||
Create NAT port forward rules on pfSense programmatically via PHP/SSH.
|
||||
Use when: (1) adding port forwards for new K8s services, (2) NAT rules
|
||||
added via PHP don't appear in pfctl output, (3) config_read_array() throws
|
||||
"undefined function" error, (4) destination "wanip" not working in NAT rules,
|
||||
(5) rules saved to config.xml but not loaded into pfctl. Covers the correct
|
||||
PHP array structure, config API differences between pfSense versions, and
|
||||
the required pfctl reload step.
|
||||
author: Claude Code
|
||||
version: 1.0.0
|
||||
date: 2026-02-21
|
||||
---
|
||||
|
||||
# pfSense NAT Rule Creation via PHP
|
||||
|
||||
## Problem
|
||||
Creating NAT port forward rules on pfSense programmatically via SSH/PHP has
|
||||
multiple gotchas around the config API, rule structure, and rule loading.
|
||||
|
||||
## Context / Trigger Conditions
|
||||
- Adding a port forward for a new Kubernetes service (e.g., TURN, game server)
|
||||
- Using `ssh admin@10.0.20.1` + PHP to automate pfSense config
|
||||
- NAT rules don't appear in `pfctl -sn` after `write_config()` + `filter_configure()`
|
||||
- `config_read_array()` throws "Call to undefined function"
|
||||
- Rules saved to config.xml but pfctl doesn't have them
|
||||
|
||||
## Solution
|
||||
|
||||
### Correct PHP for adding NAT rules
|
||||
|
||||
```php
|
||||
<?php
|
||||
require_once("config.inc");
|
||||
require_once("filter.inc");
|
||||
global $config; // NOT config_read_array() — that doesn't exist in pfSense 2.7.x
|
||||
|
||||
$config["nat"]["rule"][] = array(
|
||||
"interface" => "wan",
|
||||
"ipprotocol" => "inet", // Required! Must be "inet" for IPv4
|
||||
"protocol" => "tcp/udp", // Or "udp" or "tcp"
|
||||
"source" => array("any" => ""),
|
||||
"destination" => array(
|
||||
"network" => "wanip", // Use "network" => "wanip", NOT "address" => "wanip"
|
||||
"port" => "3478" // Single port or "start:end" for range
|
||||
),
|
||||
"target" => "10.0.20.200", // Internal destination IP
|
||||
"local-port" => "3478", // Internal port (for ranges, just the start port)
|
||||
"descr" => "My port forward",
|
||||
"associated-rule-id" => "pass" // Auto-create firewall pass rule
|
||||
);
|
||||
|
||||
write_config("Description for config history");
|
||||
filter_configure();
|
||||
```
|
||||
|
||||
### Key gotchas
|
||||
|
||||
1. **`config_read_array()` doesn't exist** in pfSense 2.7.x. Use `global $config` instead.
|
||||
|
||||
2. **Destination format**: Use `"network" => "wanip"`, NOT `"address" => "wanip"` or `"address" => "192.168.1.2"`. The `"network"` key with `"wanip"` tells pfSense to resolve the WAN IP dynamically.
|
||||
|
||||
3. **`ipprotocol` is required**: Must include `"ipprotocol" => "inet"` or rules won't generate in `/tmp/rules.debug`.
|
||||
|
||||
4. **Port ranges**: Use `"port" => "49152:49252"` for ranges. The `"local-port"` should be just the start port — pfSense maps the range automatically.
|
||||
|
||||
5. **Rules may not load immediately**: After `write_config()` + `filter_configure()`, rules appear in `/tmp/rules.debug` but may not be in pfctl until the next filter reload. Force with:
|
||||
```bash
|
||||
pfctl -f /tmp/rules.debug
|
||||
```
|
||||
|
||||
6. **SSH quoting**: The pfsense.py `php` command breaks on `\n` in strings. For multi-line PHP, write a `.php` file, `scp` it, and execute:
|
||||
```bash
|
||||
scp script.php admin@10.0.20.1:/tmp/
|
||||
ssh admin@10.0.20.1 "php /tmp/script.php"
|
||||
```
|
||||
|
||||
### Execution via pfsense.py
|
||||
|
||||
For simple single-line PHP (no newlines or backslashes):
|
||||
```bash
|
||||
python3 .claude/pfsense.py php 'require_once("config.inc"); ...; echo "Done";'
|
||||
```
|
||||
|
||||
For complex scripts, use scp + ssh as above.
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# Check rules in config
|
||||
ssh admin@10.0.20.1 "grep 'YOUR_PORT' /cf/conf/config.xml"
|
||||
|
||||
# Check generated pf rules
|
||||
ssh admin@10.0.20.1 "grep 'YOUR_PORT' /tmp/rules.debug"
|
||||
|
||||
# Check active pfctl rules
|
||||
python3 .claude/pfsense.py pfctl "-sn" | grep YOUR_PORT
|
||||
```
|
||||
|
||||
## Notes
|
||||
- Existing working NAT rules on this pfSense use the same structure (check WireGuard port 51820 as reference)
|
||||
- The `associated-rule-id: pass` auto-creates a WAN firewall rule to allow the forwarded traffic
|
||||
- pfSense applies NAT rules across ALL interfaces when using the web UI, but PHP-created rules only apply to the specified interface
|
||||
- See also: `pfsense` skill for general pfSense management
|
||||
Loading…
Add table
Add a link
Reference in a new issue