[ci skip] Add rewrite-body Accept header skill, update NFS skill
New skill: traefik-rewrite-body-accept-header — rewrite-body plugin silently skips injection when request Accept header doesn't contain text/html (curl default Accept: */* doesn't match). Updated: k8s-nfs-mount-troubleshooting v1.1.0 — added variant for non-root container UID permission denied on NFS writes.
This commit is contained in:
parent
e5729c68b8
commit
7557c8ca4a
2 changed files with 163 additions and 6 deletions
|
|
@ -1,14 +1,16 @@
|
||||||
---
|
---
|
||||||
name: k8s-nfs-mount-troubleshooting
|
name: k8s-nfs-mount-troubleshooting
|
||||||
description: |
|
description: |
|
||||||
Debug Kubernetes NFS volume mount failures. Use when: (1) Pod stuck in ContainerCreating
|
Debug Kubernetes NFS volume mount failures. Use when: (1) Pod stuck in ContainerCreating
|
||||||
for extended time, (2) kubectl describe shows "MountVolume.SetUp failed" with NFS errors,
|
for extended time, (2) kubectl describe shows "MountVolume.SetUp failed" with NFS errors,
|
||||||
(3) Error message shows "Protocol not supported" or "mount.nfs: access denied",
|
(3) Error message shows "Protocol not supported" or "mount.nfs: access denied",
|
||||||
(4) NFS volume defined in pod spec but container won't start. Common root cause is
|
(4) NFS volume defined in pod spec but container won't start, (5) Container starts but
|
||||||
missing NFS export on the server, not a protocol issue.
|
gets "Permission denied" writing to NFS volume (non-root container UID mismatch),
|
||||||
|
(6) CronJob or init container fails silently when writing to NFS. Common root causes
|
||||||
|
are missing NFS export on the server and UID mismatch for non-root containers.
|
||||||
author: Claude Code
|
author: Claude Code
|
||||||
version: 1.0.0
|
version: 1.1.0
|
||||||
date: 2026-01-28
|
date: 2026-02-22
|
||||||
---
|
---
|
||||||
|
|
||||||
# Kubernetes NFS Mount Troubleshooting
|
# Kubernetes NFS Mount Troubleshooting
|
||||||
|
|
@ -91,6 +93,55 @@ ssh root@10.0.10.15 'mkdir -p /mnt/main/resume && chmod 777 /mnt/main/resume'
|
||||||
- NFSv3 vs NFSv4 issues are rare in modern setups; missing paths are more common
|
- NFSv3 vs NFSv4 issues are rare in modern setups; missing paths are more common
|
||||||
- Check that the NFS client packages are installed on Kubernetes nodes if this is a new cluster
|
- Check that the NFS client packages are installed on Kubernetes nodes if this is a new cluster
|
||||||
|
|
||||||
|
## Variant: Non-Root Container UID Permission Denied
|
||||||
|
|
||||||
|
### Problem
|
||||||
|
Container starts and mounts NFS successfully, but gets "Permission denied" when
|
||||||
|
writing files. The pod appears healthy but operations fail silently.
|
||||||
|
|
||||||
|
### Trigger Conditions
|
||||||
|
- Container logs show "Permission denied" or "client returned ERROR on write"
|
||||||
|
- Pod is Running (not stuck in ContainerCreating)
|
||||||
|
- NFS directory exists and is mounted, but owned by root (uid 0)
|
||||||
|
- Container image runs as a non-root user (e.g., `curlimages/curl` runs as uid 101)
|
||||||
|
- CronJobs or init containers that write to NFS fail with no obvious error
|
||||||
|
|
||||||
|
### Common Non-Root Container UIDs
|
||||||
|
| Image | UID | User |
|
||||||
|
|-------|-----|------|
|
||||||
|
| `curlimages/curl` | 101 | curl_user |
|
||||||
|
| `nginx` (unprivileged) | 101 | nginx |
|
||||||
|
| `node` | 1000 | node |
|
||||||
|
| `python` (slim) | 0 | root (safe) |
|
||||||
|
| `grafana/grafana` | 472 | grafana |
|
||||||
|
|
||||||
|
### Solution
|
||||||
|
Fix permissions on the NFS server:
|
||||||
|
```bash
|
||||||
|
# Option 1: World-writable (simplest, suitable for non-sensitive data)
|
||||||
|
ssh root@10.0.10.15 "chmod -R 777 /mnt/main/<service>/<subdir>"
|
||||||
|
|
||||||
|
# Option 2: Match container UID (more secure)
|
||||||
|
ssh root@10.0.10.15 "chown -R <uid>:<gid> /mnt/main/<service>/<subdir>"
|
||||||
|
|
||||||
|
# Option 3: Use securityContext in pod spec to run as root
|
||||||
|
spec:
|
||||||
|
securityContext:
|
||||||
|
runAsUser: 0
|
||||||
|
```
|
||||||
|
|
||||||
|
### Debugging
|
||||||
|
```bash
|
||||||
|
# Check what UID the container runs as
|
||||||
|
kubectl exec -n <namespace> <pod> -- id
|
||||||
|
|
||||||
|
# Test write access from inside container
|
||||||
|
kubectl exec -n <namespace> <pod> -- sh -c 'echo test > /path/to/nfs/testfile'
|
||||||
|
|
||||||
|
# Check NFS directory ownership on server
|
||||||
|
ssh root@10.0.10.15 "ls -la /mnt/main/<service>/"
|
||||||
|
```
|
||||||
|
|
||||||
## See Also
|
## See Also
|
||||||
- TrueNAS NFS configuration documentation
|
- TrueNAS NFS configuration documentation
|
||||||
- Kubernetes NFS volume documentation
|
- Kubernetes NFS volume documentation
|
||||||
|
|
|
||||||
106
.claude/skills/traefik-rewrite-body-accept-header/SKILL.md
Normal file
106
.claude/skills/traefik-rewrite-body-accept-header/SKILL.md
Normal file
|
|
@ -0,0 +1,106 @@
|
||||||
|
---
|
||||||
|
name: traefik-rewrite-body-accept-header
|
||||||
|
description: |
|
||||||
|
Fix for Traefik rewrite-body plugin (packruler/rewrite-body) silently skipping
|
||||||
|
content injection (rybbit analytics, trap links, or any HTML rewriting). Use when:
|
||||||
|
(1) rewrite-body middleware is enabled but injected content doesn't appear in responses,
|
||||||
|
(2) curl shows original HTML with no modifications but browser works fine,
|
||||||
|
(3) monitoring.types is set to ["text/html"] but responses aren't being rewritten,
|
||||||
|
(4) no errors in Traefik logs despite rewrite-body being in the middleware chain.
|
||||||
|
Root cause: the plugin checks the request Accept header (not response Content-Type)
|
||||||
|
against monitoring.types, and curl's default Accept: */* does not match text/html.
|
||||||
|
author: Claude Code
|
||||||
|
version: 1.0.0
|
||||||
|
date: 2026-02-22
|
||||||
|
---
|
||||||
|
|
||||||
|
# Traefik Rewrite-Body Accept Header Matching
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
The `packruler/rewrite-body` Traefik plugin silently skips HTML content injection
|
||||||
|
when the request's `Accept` header doesn't contain a value matching `monitoring.types`.
|
||||||
|
This means `curl` requests (which send `Accept: */*`) won't show injected content,
|
||||||
|
while browsers (which send `Accept: text/html,...`) will.
|
||||||
|
|
||||||
|
## Context / Trigger Conditions
|
||||||
|
- rewrite-body middleware is in the ingress middleware chain and shows status "enabled" in Traefik API
|
||||||
|
- `curl https://example.com/` returns original HTML with no injected content
|
||||||
|
- Browser shows injected content (rybbit script, trap links, etc.)
|
||||||
|
- No errors in Traefik logs — the plugin silently skips processing
|
||||||
|
- `monitoring.types = ["text/html"]` is configured in the middleware spec
|
||||||
|
- Middleware chain order is correct (strip-accept-encoding before rewrite-body)
|
||||||
|
|
||||||
|
## Misleading Symptoms
|
||||||
|
- Appears as if the middleware isn't working at all
|
||||||
|
- May look like a middleware ordering issue or configuration error
|
||||||
|
- `kubectl get middleware` shows the resource exists with correct spec
|
||||||
|
- Traefik API (`/api/http/middlewares/`) shows the middleware as "enabled"
|
||||||
|
- Checking the rewrite-body regex patterns seems pointless since nothing is being processed
|
||||||
|
|
||||||
|
## Root Cause
|
||||||
|
In the plugin source code, `SupportsProcessing()` checks:
|
||||||
|
```go
|
||||||
|
func (r *Rewriter) SupportsProcessing(req *http.Request) bool {
|
||||||
|
accept := req.Header.Get("Accept")
|
||||||
|
for _, monitoringType := range r.monitoring.Types {
|
||||||
|
if strings.Contains(accept, monitoringType) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
It uses `strings.Contains(accept, "text/html")`. The curl default `Accept: */*` does
|
||||||
|
NOT contain the substring `text/html`, so the plugin returns false and skips all
|
||||||
|
processing. Browser requests include `Accept: text/html,application/xhtml+xml,...`
|
||||||
|
which does match.
|
||||||
|
|
||||||
|
## Solution
|
||||||
|
This is **working as designed** — not a bug. The fix depends on context:
|
||||||
|
|
||||||
|
### For testing with curl
|
||||||
|
Add the `Accept` header to simulate a browser:
|
||||||
|
```bash
|
||||||
|
curl -s -H "Accept: text/html,application/xhtml+xml" https://example.com/
|
||||||
|
```
|
||||||
|
|
||||||
|
### For verifying injection is working
|
||||||
|
```bash
|
||||||
|
# Check for injected content (trap links, analytics, etc.)
|
||||||
|
curl -s -H "Accept: text/html,application/xhtml+xml" https://example.com/ \
|
||||||
|
| grep -oE 'href="https://poison[^"]*"'
|
||||||
|
|
||||||
|
# Check for rybbit analytics
|
||||||
|
curl -s -H "Accept: text/html,application/xhtml+xml" https://example.com/ \
|
||||||
|
| grep -oE 'src="https://rybbit[^"]*"'
|
||||||
|
```
|
||||||
|
|
||||||
|
### For programmatic clients that need injection
|
||||||
|
If a non-browser client needs to receive injected content, ensure it sends
|
||||||
|
`Accept: text/html` in its request headers.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
```bash
|
||||||
|
# Without Accept header — no injection (expected)
|
||||||
|
curl -s https://example.com/ | grep -c "rybbit"
|
||||||
|
# Output: 0
|
||||||
|
|
||||||
|
# With Accept header — injection works
|
||||||
|
curl -s -H "Accept: text/html" https://example.com/ | grep -c "rybbit"
|
||||||
|
# Output: 1
|
||||||
|
```
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
- This behavior is independent of the compression issue (see `traefik-rewrite-body-compression`)
|
||||||
|
- The check is on the **request** `Accept` header, not the **response** `Content-Type`
|
||||||
|
- `Accept: */*` does NOT match — `strings.Contains("*/*", "text/html")` is false
|
||||||
|
- Real AI scrapers typically send browser-like Accept headers, so trap links will be
|
||||||
|
injected for them correctly
|
||||||
|
- API calls (which typically send `Accept: application/json`) are correctly skipped
|
||||||
|
|
||||||
|
## See Also
|
||||||
|
- `traefik-rewrite-body-compression` — Different issue: plugin fails to decompress
|
||||||
|
gzip responses, corrupting content. That skill covers the compression fix
|
||||||
|
(strip-accept-encoding middleware).
|
||||||
|
- `ingress-factory-migration` — Covers the ingress factory module middleware chain
|
||||||
Loading…
Add table
Add a link
Reference in a new issue