docs+scripts: lock in nextcloud-as-PVE-NFS-browser surface
Some checks failed
ci/woodpecker/push/build-cli Pipeline failed
ci/woodpecker/push/default Pipeline was successful

- docs/architecture/storage.md: new "Nextcloud as PVE-NFS browser"
  section documenting mount-per-archive + applicable_users model,
  why mount-level ACL beats Files Access Control on NC 30/31, the
  manifest shape (with current applicableUsers + enableSharing
  fields), and the trade-off
- docs/runbooks/nextcloud-add-archive.md: 5-step runbook to surface
  a new directory under /srv/nfs/* to specific NC users via the
  bootstrap Job
- scripts/anca-elements-sync.sh: deployed at
  /usr/local/bin/anca-elements-sync.sh on the PVE host; fpsync from
  Synology Anca/Elements to /srv/nfs/anca-elements (idempotent +
  resumable). The PVE replica is what the NC /anca-elements mount
  serves; the offsite-sync pipeline excludes this path (committed
  earlier this session) so we don't write it back to Synology

NC usernames are admin/anca/emo (not display names — admin is
Viktor). Stale "viktor" references in the manifest example dropped.
This commit is contained in:
Viktor Barzin 2026-05-24 11:45:01 +00:00
parent c624caf65a
commit 34f8c0f537
3 changed files with 201 additions and 0 deletions

View file

@ -289,11 +289,43 @@ nfsiostat 5
1. Move hot data to SSD NFS: relocate from `/srv/nfs/<service>` to `/srv/nfs-ssd/<service>` and update PV path
2. Tune NFS mount: add `rsize=1048576,wsize=1048576` to StorageClass `mountOptions`
## Nextcloud as PVE-NFS browser
Both NFS export roots are mounted into the Nextcloud server pod — `/srv/nfs` at `/mnt/pve-nfs` and `/srv/nfs-ssd` at `/mnt/pve-nfs-ssd` — via standard NFS PVs (`nfs_volume` module). No host-level Unix user/group setup; Nextcloud is the sole household-facing surface.
**ACL model — two patterns:**
- **Root browser mounts** (`PVE NFS Pool`, `PVE NFS-SSD Pool`): scoped to NC group `admin`. Used by Viktor for ad-hoc browsing of any cluster NFS state. Other users never see these mounts.
- **Per-archive mounts** (e.g. `/anca-elements``/mnt/pve-nfs/anca-elements`): one NC External mount per archive, `applicable_users` set to the archive owners. Users see only the mounts assigned to them. Write/delete access is implicit at the OS level (NC pod writes via `no_root_squash`); deny semantics come from mount visibility — if the mount is not in your list, you cannot reach the path.
**Why mount-level ACL, not Files Access Control**: NC 30/31's workflow engine check classes are `FileName` (basename), `FileMimeType`, `FileSize`, `FileSystemTags`, and `UserGroupMembership`. There is no `FilePath` and no `UserId` check class. Per-(directory, user) rules are not expressible via FAC. Mount-level ACL via `occ files_external:applicable` is the supported primitive and maps cleanly onto the model.
**Manifest**: `kubernetes_config_map_v1.nextcloud_external_storage_manifest` in `stacks/nextcloud/external_storage.tf`. Mount entries reference NC usernames (`admin`, `anca`, `emo` — not display names; admin is Viktor). JSON shape:
```json
{
"rootMounts": [
{ "mountPoint": "/PVE NFS Pool", "dataDir": "/mnt/pve-nfs", "applicableGroup": "admin", "enableSharing": true },
{ "mountPoint": "/PVE NFS-SSD Pool", "dataDir": "/mnt/pve-nfs-ssd", "applicableGroup": "admin", "enableSharing": true }
],
"archiveMounts": [
{ "mountPoint": "/anca-elements", "dataDir": "/mnt/pve-nfs/anca-elements", "applicableUsers": ["anca", "admin"], "applicableGroups": [], "enableSharing": false }
]
}
```
A one-shot K8s bootstrap Job applies the manifest idempotently on every `tg apply` via `occ files_external:*`, `occ files_external:applicable`, and `occ files_external:option`. `enableSharing: true` lets admin re-share a subfolder of the mount with another NC user/group/public link; default is `false` (NC's local-backend default).
**Adding a new archive**: drop the directory under `/srv/nfs/<name>/` on PVE, append an `archiveMounts` entry to the manifest, then `scripts/tg apply` the nextcloud stack. See `docs/runbooks/nextcloud-add-archive.md` for the full step-by-step.
**Trade-off**: a compromised NC admin account has destructive reach over the cluster NFS roots (admin sees the root browser mounts). Accepted — Viktor's account is the single high-value target either way. No lateral movement to databases or block PVCs via this path (those are not NFS).
**Backup**: Synology retains a frozen copy of each archive (3-2-1 coverage); the existing `offsite-sync-backup` pipeline provides nightly delta sync from `/srv/nfs/<archive>` → Synology `nfs/`.
## Related
- **Runbooks**:
- `docs/runbooks/restore-postgresql.md`
- `docs/runbooks/restore-mysql.md`
- `docs/runbooks/recover-nfs-mount.md`
- `docs/runbooks/nextcloud-add-archive.md`
- **Architecture**: `docs/architecture/backup-dr.md` (backup strategy using LVM snapshots and Proxmox host scripts)
- **Reference**: `.claude/reference/service-catalog.md` (which services use NFS vs proxmox-lvm)

View file

@ -0,0 +1,57 @@
# Runbook: Add a new archive to Nextcloud / PVE NFS
Use this runbook when you need to surface a new directory under `/srv/nfs/` or `/srv/nfs-ssd/` to specific Nextcloud users as a dedicated External mount. Each archive gets its own NC mount; only the listed `applicableUsers` can see and access it.
## Steps
1. **Create the directory on PVE.**
```bash
ssh root@192.168.1.127
mkdir -p /srv/nfs/<archive-name>
# Use /srv/nfs-ssd/<archive-name> for the SSD pool instead.
```
2. **Populate the directory.**
Rsync from a remote source, copy from another NFS path, or let the granted user upload via the NC web UI after step 5. Example rsync:
```bash
rsync -avP --info=progress2 user@source:/path/ /srv/nfs/<archive-name>/
```
3. **Add a manifest entry.**
Edit `infra/stacks/nextcloud/external_storage.tf`. In the `kubernetes_config_map_v1.nextcloud_external_storage_manifest` resource, append a new entry to `archiveMounts`:
```json
{ "mountPoint": "/<archive-name>", "dataDir": "/mnt/pve-nfs/<archive-name>", "applicableUsers": ["<owner1>", "admin"], "applicableGroups": [], "enableSharing": false }
```
Use `/mnt/pve-nfs-ssd/<archive-name>` for the SSD pool. NC usernames are `admin`, `anca`, `emo` — not display names (`admin` is Viktor). `admin` is included so the owner of the homelab can always assist with the archive. Set `enableSharing: true` only if you want recipients to re-share subfolders.
4. **Plan and apply.**
```bash
cd infra/stacks/nextcloud
scripts/tg plan
scripts/tg apply
```
The bootstrap Job re-runs and applies the new mount plus `applicable_users` idempotently via `occ files_external:*` and `occ files_external:applicable`. No manual `occ` invocation needed.
5. **Verify.**
Log in as a granted user — `/<archive-name>` must appear in their NC sidebar; read, upload, and delete must all work. Log in as a non-granted user and confirm the mount is not visible at all.
## Backout
Remove the entry from `archiveMounts` in the manifest ConfigMap, then `scripts/tg apply`. The bootstrap Job re-runs and removes the mount. The root mounts (`PVE NFS Pool`, `PVE NFS-SSD Pool`, visible to group `admin` only) are unaffected throughout.
After the mount is gone there is no NC trash to clean. The directory on PVE (`/srv/nfs/<archive-name>`) can be `rmdir`'d once you have confirmed the data is safe elsewhere.
## Related
- Architecture: `docs/architecture/storage.md` — "Nextcloud as PVE-NFS browser" section
- Original design/plan: `infra/docs/plans/2026-05-23-anca-elements-{design,plan}.md` <!-- TODO: confirm path once orchestrator files the plan docs -->
- Manifest source: `infra/stacks/nextcloud/external_storage.tf` (`kubernetes_config_map_v1.nextcloud_external_storage_manifest`)

112
scripts/anca-elements-sync.sh Executable file
View file

@ -0,0 +1,112 @@
#!/usr/bin/env bash
# anca-elements-sync.sh — copy Anca's WD-Elements backup from Synology to PVE NFS
#
# Usage:
# /usr/local/bin/anca-elements-sync.sh
#
# Idempotent: re-running after a successful sync is a no-op (only the dry-run
# verification runs, which reports "sync verified clean" immediately).
#
# Resumable: if fpsync was interrupted, resume with:
# fpsync -r /var/tmp/fpsync \
# -n 4 -s 4G \
# -o "-lptgoD -H --no-perms --no-owner --no-group --exclude=@eaDir/ --exclude=*@synoeastream --exclude=.DS_Store --exclude=Thumbs.db" \
# /mnt/synology-backup/Anca/Elements/ /srv/nfs/anca-elements/
#
# NOTE: fpsync -o = rsync options override (what we want)
# fpsync -O = fpart partition options override (NOT rsync)
# NOTE: Do NOT use -a or -r in fpsync rsync options — fpsync handles
# recursion via fpart; -r causes fpsync to warn and skip the slab.
#
# Log: /var/log/anca-elements-sync.log
set -euo pipefail
LOG=/var/log/anca-elements-sync.log
SRC_HOST=192.168.1.13
SRC_EXPORT=/volume1/Backup
SRC_SUBPATH=Anca/Elements
MOUNT_POINT=/mnt/synology-backup
DEST=/srv/nfs/anca-elements
log() {
echo "[$(date -u '+%Y-%m-%dT%H:%M:%SZ')] $*" | tee -a "$LOG"
}
# ── 1. Ensure destination + mount-point directories exist ────────────────────
log "Step 1: ensuring directories"
mkdir -p "$DEST" "$MOUNT_POINT"
# ── 2. NFS-mount Synology read-only (skip if already mounted) ───────────────
MOUNTED_HERE=0
if mountpoint -q "$MOUNT_POINT"; then
log "Step 2: $MOUNT_POINT already mounted — skipping"
else
log "Step 2: mounting ${SRC_HOST}:${SRC_EXPORT} at $MOUNT_POINT (read-only)"
mount -t nfs \
-o ro,vers=4,nolock,soft,timeo=300,retrans=2 \
"${SRC_HOST}:${SRC_EXPORT}" \
"$MOUNT_POINT"
MOUNTED_HERE=1
log "Step 2: mount successful"
fi
# ── 3. Ensure fpsync (from fpart package) is available ──────────────────────
log "Step 3: checking for fpsync"
if ! command -v fpsync >/dev/null 2>&1; then
log "Step 3: fpsync not found — installing fpart"
apt-get install -y fpart
log "Step 3: fpart installed"
else
log "Step 3: fpsync already available"
fi
# ── 4. Run fpsync (4-way parallel, no compression — source is already-compressed media) ──
log "Step 4: starting fpsync"
log " source : ${MOUNT_POINT}/${SRC_SUBPATH}/"
log " dest : ${DEST}/"
log " workers: 4, slab: 4G"
fpsync \
-n 4 \
-s 4G \
-o "-lptgoD -H --no-perms --no-owner --no-group --exclude=@eaDir/ --exclude=*@synoeastream --exclude=.DS_Store --exclude=Thumbs.db" \
"${MOUNT_POINT}/${SRC_SUBPATH}/" \
"${DEST}/" \
2>&1 | tee -a "$LOG"
log "Step 4: fpsync completed"
# ── 5. Verification dry-run ──────────────────────────────────────────────────
log "Step 5: running dry-run verification rsync"
VERIFY_OUT=$(rsync \
-rlptgoD -H --no-perms --no-owner --no-group \
--exclude='@eaDir/' --exclude='*@synoeastream' \
--exclude='.DS_Store' --exclude='Thumbs.db' \
-n --delete \
--info=progress2 \
--out-format='%o %f' \
"${MOUNT_POINT}/${SRC_SUBPATH}/" \
"${DEST}/" \
2>&1 || true)
# Count lines that represent actual file changes (send / del. operations)
CHANGE_COUNT=$(echo "$VERIFY_OUT" | grep -cE '^(send|del\.)' || true)
if [ "$CHANGE_COUNT" -eq 0 ]; then
log "Step 5: sync verified clean — no pending changes"
else
log "Step 5: WARNING — verification found ${CHANGE_COUNT} pending change(s). First 50 lines:"
# Use printf to avoid SIGPIPE from head closing the pipe early (set -o pipefail)
{ echo "$VERIFY_OUT" | head -50; } >> "$LOG" 2>&1 || true
fi
# ── 6. Unmount (only if we mounted it) ──────────────────────────────────────
if [ "$MOUNTED_HERE" -eq 1 ]; then
log "Step 6: unmounting $MOUNT_POINT"
umount "$MOUNT_POINT"
rmdir "$MOUNT_POINT"
log "Step 6: unmounted"
else
log "Step 6: mount was pre-existing — leaving in place"
fi
log "Done. Final size: $(du -sh "${DEST}" | cut -f1)"