[ci skip] Add rewrite-body Accept header skill, update NFS skill
New skill: traefik-rewrite-body-accept-header — rewrite-body plugin silently skips injection when request Accept header doesn't contain text/html (curl default Accept: */* doesn't match). Updated: k8s-nfs-mount-troubleshooting v1.1.0 — added variant for non-root container UID permission denied on NFS writes.
This commit is contained in:
Viktor Barzin
parent
e5729c68b8
commit
7557c8ca4a
2 changed files with 163 additions and 6 deletions
|
|
@ -1,14 +1,16 @@
|
|||
---
|
||||
name: k8s-nfs-mount-troubleshooting
|
||||
description: |
|
||||
Debug Kubernetes NFS volume mount failures. Use when: (1) Pod stuck in ContainerCreating
|
||||
Debug Kubernetes NFS volume mount failures. Use when: (1) Pod stuck in ContainerCreating
|
||||
for extended time, (2) kubectl describe shows "MountVolume.SetUp failed" with NFS errors,
|
||||
(3) Error message shows "Protocol not supported" or "mount.nfs: access denied",
|
||||
(4) NFS volume defined in pod spec but container won't start. Common root cause is
|
||||
missing NFS export on the server, not a protocol issue.
|
||||
(3) Error message shows "Protocol not supported" or "mount.nfs: access denied",
|
||||
(4) NFS volume defined in pod spec but container won't start, (5) Container starts but
|
||||
gets "Permission denied" writing to NFS volume (non-root container UID mismatch),
|
||||
(6) CronJob or init container fails silently when writing to NFS. Common root causes
|
||||
are missing NFS export on the server and UID mismatch for non-root containers.
|
||||
author: Claude Code
|
||||
version: 1.0.0
|
||||
date: 2026-01-28
|
||||
version: 1.1.0
|
||||
date: 2026-02-22
|
||||
---
|
||||
|
||||
# Kubernetes NFS Mount Troubleshooting
|
||||
|
|
@ -91,6 +93,55 @@ ssh root@10.0.10.15 'mkdir -p /mnt/main/resume && chmod 777 /mnt/main/resume'
|
|||
- NFSv3 vs NFSv4 issues are rare in modern setups; missing paths are more common
|
||||
- Check that the NFS client packages are installed on Kubernetes nodes if this is a new cluster
|
||||
|
||||
## Variant: Non-Root Container UID Permission Denied
|
||||
|
||||
### Problem
|
||||
Container starts and mounts NFS successfully, but gets "Permission denied" when
|
||||
writing files. The pod appears healthy but operations fail silently.
|
||||
|
||||
### Trigger Conditions
|
||||
- Container logs show "Permission denied" or "client returned ERROR on write"
|
||||
- Pod is Running (not stuck in ContainerCreating)
|
||||
- NFS directory exists and is mounted, but owned by root (uid 0)
|
||||
- Container image runs as a non-root user (e.g., `curlimages/curl` runs as uid 101)
|
||||
- CronJobs or init containers that write to NFS fail with no obvious error
|
||||
|
||||
### Common Non-Root Container UIDs
|
||||
| Image | UID | User |
|
||||
|-------|-----|------|
|
||||
| `curlimages/curl` | 101 | curl_user |
|
||||
| `nginx` (unprivileged) | 101 | nginx |
|
||||
| `node` | 1000 | node |
|
||||
| `python` (slim) | 0 | root (safe) |
|
||||
| `grafana/grafana` | 472 | grafana |
|
||||
|
||||
### Solution
|
||||
Fix permissions on the NFS server:
|
||||
```bash
|
||||
# Option 1: World-writable (simplest, suitable for non-sensitive data)
|
||||
ssh root@10.0.10.15 "chmod -R 777 /mnt/main/<service>/<subdir>"
|
||||
|
||||
# Option 2: Match container UID (more secure)
|
||||
ssh root@10.0.10.15 "chown -R <uid>:<gid> /mnt/main/<service>/<subdir>"
|
||||
|
||||
# Option 3: Use securityContext in pod spec to run as root
|
||||
spec:
|
||||
securityContext:
|
||||
runAsUser: 0
|
||||
```
|
||||
|
||||
### Debugging
|
||||
```bash
|
||||
# Check what UID the container runs as
|
||||
kubectl exec -n <namespace> <pod> -- id
|
||||
|
||||
# Test write access from inside container
|
||||
kubectl exec -n <namespace> <pod> -- sh -c 'echo test > /path/to/nfs/testfile'
|
||||
|
||||
# Check NFS directory ownership on server
|
||||
ssh root@10.0.10.15 "ls -la /mnt/main/<service>/"
|
||||
```
|
||||
|
||||
## See Also
|
||||
- TrueNAS NFS configuration documentation
|
||||
- Kubernetes NFS volume documentation
|
||||
|
|
|
|||
106
.claude/skills/traefik-rewrite-body-accept-header/SKILL.md
Normal file
106
.claude/skills/traefik-rewrite-body-accept-header/SKILL.md
Normal file
|
|
@ -0,0 +1,106 @@
|
|||
---
|
||||
name: traefik-rewrite-body-accept-header
|
||||
description: |
|
||||
Fix for Traefik rewrite-body plugin (packruler/rewrite-body) silently skipping
|
||||
content injection (rybbit analytics, trap links, or any HTML rewriting). Use when:
|
||||
(1) rewrite-body middleware is enabled but injected content doesn't appear in responses,
|
||||
(2) curl shows original HTML with no modifications but browser works fine,
|
||||
(3) monitoring.types is set to ["text/html"] but responses aren't being rewritten,
|
||||
(4) no errors in Traefik logs despite rewrite-body being in the middleware chain.
|
||||
Root cause: the plugin checks the request Accept header (not response Content-Type)
|
||||
against monitoring.types, and curl's default Accept: */* does not match text/html.
|
||||
author: Claude Code
|
||||
version: 1.0.0
|
||||
date: 2026-02-22
|
||||
---
|
||||
|
||||
# Traefik Rewrite-Body Accept Header Matching
|
||||
|
||||
## Problem
|
||||
The `packruler/rewrite-body` Traefik plugin silently skips HTML content injection
|
||||
when the request's `Accept` header doesn't contain a value matching `monitoring.types`.
|
||||
This means `curl` requests (which send `Accept: */*`) won't show injected content,
|
||||
while browsers (which send `Accept: text/html,...`) will.
|
||||
|
||||
## Context / Trigger Conditions
|
||||
- rewrite-body middleware is in the ingress middleware chain and shows status "enabled" in Traefik API
|
||||
- `curl https://example.com/` returns original HTML with no injected content
|
||||
- Browser shows injected content (rybbit script, trap links, etc.)
|
||||
- No errors in Traefik logs — the plugin silently skips processing
|
||||
- `monitoring.types = ["text/html"]` is configured in the middleware spec
|
||||
- Middleware chain order is correct (strip-accept-encoding before rewrite-body)
|
||||
|
||||
## Misleading Symptoms
|
||||
- Appears as if the middleware isn't working at all
|
||||
- May look like a middleware ordering issue or configuration error
|
||||
- `kubectl get middleware` shows the resource exists with correct spec
|
||||
- Traefik API (`/api/http/middlewares/`) shows the middleware as "enabled"
|
||||
- Checking the rewrite-body regex patterns seems pointless since nothing is being processed
|
||||
|
||||
## Root Cause
|
||||
In the plugin source code, `SupportsProcessing()` checks:
|
||||
```go
|
||||
func (r *Rewriter) SupportsProcessing(req *http.Request) bool {
|
||||
accept := req.Header.Get("Accept")
|
||||
for _, monitoringType := range r.monitoring.Types {
|
||||
if strings.Contains(accept, monitoringType) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
```
|
||||
|
||||
It uses `strings.Contains(accept, "text/html")`. The curl default `Accept: */*` does
|
||||
NOT contain the substring `text/html`, so the plugin returns false and skips all
|
||||
processing. Browser requests include `Accept: text/html,application/xhtml+xml,...`
|
||||
which does match.
|
||||
|
||||
## Solution
|
||||
This is **working as designed** — not a bug. The fix depends on context:
|
||||
|
||||
### For testing with curl
|
||||
Add the `Accept` header to simulate a browser:
|
||||
```bash
|
||||
curl -s -H "Accept: text/html,application/xhtml+xml" https://example.com/
|
||||
```
|
||||
|
||||
### For verifying injection is working
|
||||
```bash
|
||||
# Check for injected content (trap links, analytics, etc.)
|
||||
curl -s -H "Accept: text/html,application/xhtml+xml" https://example.com/ \
|
||||
| grep -oE 'href="https://poison[^"]*"'
|
||||
|
||||
# Check for rybbit analytics
|
||||
curl -s -H "Accept: text/html,application/xhtml+xml" https://example.com/ \
|
||||
| grep -oE 'src="https://rybbit[^"]*"'
|
||||
```
|
||||
|
||||
### For programmatic clients that need injection
|
||||
If a non-browser client needs to receive injected content, ensure it sends
|
||||
`Accept: text/html` in its request headers.
|
||||
|
||||
## Verification
|
||||
```bash
|
||||
# Without Accept header — no injection (expected)
|
||||
curl -s https://example.com/ | grep -c "rybbit"
|
||||
# Output: 0
|
||||
|
||||
# With Accept header — injection works
|
||||
curl -s -H "Accept: text/html" https://example.com/ | grep -c "rybbit"
|
||||
# Output: 1
|
||||
```
|
||||
|
||||
## Notes
|
||||
- This behavior is independent of the compression issue (see `traefik-rewrite-body-compression`)
|
||||
- The check is on the **request** `Accept` header, not the **response** `Content-Type`
|
||||
- `Accept: */*` does NOT match — `strings.Contains("*/*", "text/html")` is false
|
||||
- Real AI scrapers typically send browser-like Accept headers, so trap links will be
|
||||
injected for them correctly
|
||||
- API calls (which typically send `Accept: application/json`) are correctly skipped
|
||||
|
||||
## See Also
|
||||
- `traefik-rewrite-body-compression` — Different issue: plugin fails to decompress
|
||||
gzip responses, corrupting content. That skill covers the compression fix
|
||||
(strip-accept-encoding middleware).
|
||||
- `ingress-factory-migration` — Covers the ingress factory module middleware chain
|
||||
Loading…
Add table
Add a link
Reference in a new issue