docs+scripts: lock in nextcloud-as-PVE-NFS-browser surface
- docs/architecture/storage.md: new "Nextcloud as PVE-NFS browser" section documenting mount-per-archive + applicable_users model, why mount-level ACL beats Files Access Control on NC 30/31, the manifest shape (with current applicableUsers + enableSharing fields), and the trade-off - docs/runbooks/nextcloud-add-archive.md: 5-step runbook to surface a new directory under /srv/nfs/* to specific NC users via the bootstrap Job - scripts/anca-elements-sync.sh: deployed at /usr/local/bin/anca-elements-sync.sh on the PVE host; fpsync from Synology Anca/Elements to /srv/nfs/anca-elements (idempotent + resumable). The PVE replica is what the NC /anca-elements mount serves; the offsite-sync pipeline excludes this path (committed earlier this session) so we don't write it back to Synology NC usernames are admin/anca/emo (not display names — admin is Viktor). Stale "viktor" references in the manifest example dropped.
This commit is contained in:
parent
c624caf65a
commit
34f8c0f537
3 changed files with 201 additions and 0 deletions
|
|
@ -289,11 +289,43 @@ nfsiostat 5
|
|||
1. Move hot data to SSD NFS: relocate from `/srv/nfs/<service>` to `/srv/nfs-ssd/<service>` and update PV path
|
||||
2. Tune NFS mount: add `rsize=1048576,wsize=1048576` to StorageClass `mountOptions`
|
||||
|
||||
## Nextcloud as PVE-NFS browser
|
||||
|
||||
Both NFS export roots are mounted into the Nextcloud server pod — `/srv/nfs` at `/mnt/pve-nfs` and `/srv/nfs-ssd` at `/mnt/pve-nfs-ssd` — via standard NFS PVs (`nfs_volume` module). No host-level Unix user/group setup; Nextcloud is the sole household-facing surface.
|
||||
|
||||
**ACL model — two patterns:**
|
||||
|
||||
- **Root browser mounts** (`PVE NFS Pool`, `PVE NFS-SSD Pool`): scoped to NC group `admin`. Used by Viktor for ad-hoc browsing of any cluster NFS state. Other users never see these mounts.
|
||||
- **Per-archive mounts** (e.g. `/anca-elements` → `/mnt/pve-nfs/anca-elements`): one NC External mount per archive, `applicable_users` set to the archive owners. Users see only the mounts assigned to them. Write/delete access is implicit at the OS level (NC pod writes via `no_root_squash`); deny semantics come from mount visibility — if the mount is not in your list, you cannot reach the path.
|
||||
|
||||
**Why mount-level ACL, not Files Access Control**: NC 30/31's workflow engine check classes are `FileName` (basename), `FileMimeType`, `FileSize`, `FileSystemTags`, and `UserGroupMembership`. There is no `FilePath` and no `UserId` check class. Per-(directory, user) rules are not expressible via FAC. Mount-level ACL via `occ files_external:applicable` is the supported primitive and maps cleanly onto the model.
|
||||
|
||||
**Manifest**: `kubernetes_config_map_v1.nextcloud_external_storage_manifest` in `stacks/nextcloud/external_storage.tf`. Mount entries reference NC usernames (`admin`, `anca`, `emo` — not display names; admin is Viktor). JSON shape:
|
||||
```json
|
||||
{
|
||||
"rootMounts": [
|
||||
{ "mountPoint": "/PVE NFS Pool", "dataDir": "/mnt/pve-nfs", "applicableGroup": "admin", "enableSharing": true },
|
||||
{ "mountPoint": "/PVE NFS-SSD Pool", "dataDir": "/mnt/pve-nfs-ssd", "applicableGroup": "admin", "enableSharing": true }
|
||||
],
|
||||
"archiveMounts": [
|
||||
{ "mountPoint": "/anca-elements", "dataDir": "/mnt/pve-nfs/anca-elements", "applicableUsers": ["anca", "admin"], "applicableGroups": [], "enableSharing": false }
|
||||
]
|
||||
}
|
||||
```
|
||||
A one-shot K8s bootstrap Job applies the manifest idempotently on every `tg apply` via `occ files_external:*`, `occ files_external:applicable`, and `occ files_external:option`. `enableSharing: true` lets admin re-share a subfolder of the mount with another NC user/group/public link; default is `false` (NC's local-backend default).
|
||||
|
||||
**Adding a new archive**: drop the directory under `/srv/nfs/<name>/` on PVE, append an `archiveMounts` entry to the manifest, then `scripts/tg apply` the nextcloud stack. See `docs/runbooks/nextcloud-add-archive.md` for the full step-by-step.
|
||||
|
||||
**Trade-off**: a compromised NC admin account has destructive reach over the cluster NFS roots (admin sees the root browser mounts). Accepted — Viktor's account is the single high-value target either way. No lateral movement to databases or block PVCs via this path (those are not NFS).
|
||||
|
||||
**Backup**: Synology retains a frozen copy of each archive (3-2-1 coverage); the existing `offsite-sync-backup` pipeline provides nightly delta sync from `/srv/nfs/<archive>` → Synology `nfs/`.
|
||||
|
||||
## Related
|
||||
|
||||
- **Runbooks**:
|
||||
- `docs/runbooks/restore-postgresql.md`
|
||||
- `docs/runbooks/restore-mysql.md`
|
||||
- `docs/runbooks/recover-nfs-mount.md`
|
||||
- `docs/runbooks/nextcloud-add-archive.md`
|
||||
- **Architecture**: `docs/architecture/backup-dr.md` (backup strategy using LVM snapshots and Proxmox host scripts)
|
||||
- **Reference**: `.claude/reference/service-catalog.md` (which services use NFS vs proxmox-lvm)
|
||||
|
|
|
|||
57
docs/runbooks/nextcloud-add-archive.md
Normal file
57
docs/runbooks/nextcloud-add-archive.md
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
# Runbook: Add a new archive to Nextcloud / PVE NFS
|
||||
|
||||
Use this runbook when you need to surface a new directory under `/srv/nfs/` or `/srv/nfs-ssd/` to specific Nextcloud users as a dedicated External mount. Each archive gets its own NC mount; only the listed `applicableUsers` can see and access it.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Create the directory on PVE.**
|
||||
|
||||
```bash
|
||||
ssh root@192.168.1.127
|
||||
mkdir -p /srv/nfs/<archive-name>
|
||||
# Use /srv/nfs-ssd/<archive-name> for the SSD pool instead.
|
||||
```
|
||||
|
||||
2. **Populate the directory.**
|
||||
|
||||
Rsync from a remote source, copy from another NFS path, or let the granted user upload via the NC web UI after step 5. Example rsync:
|
||||
|
||||
```bash
|
||||
rsync -avP --info=progress2 user@source:/path/ /srv/nfs/<archive-name>/
|
||||
```
|
||||
|
||||
3. **Add a manifest entry.**
|
||||
|
||||
Edit `infra/stacks/nextcloud/external_storage.tf`. In the `kubernetes_config_map_v1.nextcloud_external_storage_manifest` resource, append a new entry to `archiveMounts`:
|
||||
|
||||
```json
|
||||
{ "mountPoint": "/<archive-name>", "dataDir": "/mnt/pve-nfs/<archive-name>", "applicableUsers": ["<owner1>", "admin"], "applicableGroups": [], "enableSharing": false }
|
||||
```
|
||||
|
||||
Use `/mnt/pve-nfs-ssd/<archive-name>` for the SSD pool. NC usernames are `admin`, `anca`, `emo` — not display names (`admin` is Viktor). `admin` is included so the owner of the homelab can always assist with the archive. Set `enableSharing: true` only if you want recipients to re-share subfolders.
|
||||
|
||||
4. **Plan and apply.**
|
||||
|
||||
```bash
|
||||
cd infra/stacks/nextcloud
|
||||
scripts/tg plan
|
||||
scripts/tg apply
|
||||
```
|
||||
|
||||
The bootstrap Job re-runs and applies the new mount plus `applicable_users` idempotently via `occ files_external:*` and `occ files_external:applicable`. No manual `occ` invocation needed.
|
||||
|
||||
5. **Verify.**
|
||||
|
||||
Log in as a granted user — `/<archive-name>` must appear in their NC sidebar; read, upload, and delete must all work. Log in as a non-granted user and confirm the mount is not visible at all.
|
||||
|
||||
## Backout
|
||||
|
||||
Remove the entry from `archiveMounts` in the manifest ConfigMap, then `scripts/tg apply`. The bootstrap Job re-runs and removes the mount. The root mounts (`PVE NFS Pool`, `PVE NFS-SSD Pool`, visible to group `admin` only) are unaffected throughout.
|
||||
|
||||
After the mount is gone there is no NC trash to clean. The directory on PVE (`/srv/nfs/<archive-name>`) can be `rmdir`'d once you have confirmed the data is safe elsewhere.
|
||||
|
||||
## Related
|
||||
|
||||
- Architecture: `docs/architecture/storage.md` — "Nextcloud as PVE-NFS browser" section
|
||||
- Original design/plan: `infra/docs/plans/2026-05-23-anca-elements-{design,plan}.md` <!-- TODO: confirm path once orchestrator files the plan docs -->
|
||||
- Manifest source: `infra/stacks/nextcloud/external_storage.tf` (`kubernetes_config_map_v1.nextcloud_external_storage_manifest`)
|
||||
112
scripts/anca-elements-sync.sh
Executable file
112
scripts/anca-elements-sync.sh
Executable file
|
|
@ -0,0 +1,112 @@
|
|||
#!/usr/bin/env bash
|
||||
# anca-elements-sync.sh — copy Anca's WD-Elements backup from Synology to PVE NFS
|
||||
#
|
||||
# Usage:
|
||||
# /usr/local/bin/anca-elements-sync.sh
|
||||
#
|
||||
# Idempotent: re-running after a successful sync is a no-op (only the dry-run
|
||||
# verification runs, which reports "sync verified clean" immediately).
|
||||
#
|
||||
# Resumable: if fpsync was interrupted, resume with:
|
||||
# fpsync -r /var/tmp/fpsync \
|
||||
# -n 4 -s 4G \
|
||||
# -o "-lptgoD -H --no-perms --no-owner --no-group --exclude=@eaDir/ --exclude=*@synoeastream --exclude=.DS_Store --exclude=Thumbs.db" \
|
||||
# /mnt/synology-backup/Anca/Elements/ /srv/nfs/anca-elements/
|
||||
#
|
||||
# NOTE: fpsync -o = rsync options override (what we want)
|
||||
# fpsync -O = fpart partition options override (NOT rsync)
|
||||
# NOTE: Do NOT use -a or -r in fpsync rsync options — fpsync handles
|
||||
# recursion via fpart; -r causes fpsync to warn and skip the slab.
|
||||
#
|
||||
# Log: /var/log/anca-elements-sync.log
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
LOG=/var/log/anca-elements-sync.log
|
||||
SRC_HOST=192.168.1.13
|
||||
SRC_EXPORT=/volume1/Backup
|
||||
SRC_SUBPATH=Anca/Elements
|
||||
MOUNT_POINT=/mnt/synology-backup
|
||||
DEST=/srv/nfs/anca-elements
|
||||
|
||||
log() {
|
||||
echo "[$(date -u '+%Y-%m-%dT%H:%M:%SZ')] $*" | tee -a "$LOG"
|
||||
}
|
||||
|
||||
# ── 1. Ensure destination + mount-point directories exist ────────────────────
|
||||
log "Step 1: ensuring directories"
|
||||
mkdir -p "$DEST" "$MOUNT_POINT"
|
||||
|
||||
# ── 2. NFS-mount Synology read-only (skip if already mounted) ───────────────
|
||||
MOUNTED_HERE=0
|
||||
if mountpoint -q "$MOUNT_POINT"; then
|
||||
log "Step 2: $MOUNT_POINT already mounted — skipping"
|
||||
else
|
||||
log "Step 2: mounting ${SRC_HOST}:${SRC_EXPORT} at $MOUNT_POINT (read-only)"
|
||||
mount -t nfs \
|
||||
-o ro,vers=4,nolock,soft,timeo=300,retrans=2 \
|
||||
"${SRC_HOST}:${SRC_EXPORT}" \
|
||||
"$MOUNT_POINT"
|
||||
MOUNTED_HERE=1
|
||||
log "Step 2: mount successful"
|
||||
fi
|
||||
|
||||
# ── 3. Ensure fpsync (from fpart package) is available ──────────────────────
|
||||
log "Step 3: checking for fpsync"
|
||||
if ! command -v fpsync >/dev/null 2>&1; then
|
||||
log "Step 3: fpsync not found — installing fpart"
|
||||
apt-get install -y fpart
|
||||
log "Step 3: fpart installed"
|
||||
else
|
||||
log "Step 3: fpsync already available"
|
||||
fi
|
||||
|
||||
# ── 4. Run fpsync (4-way parallel, no compression — source is already-compressed media) ──
|
||||
log "Step 4: starting fpsync"
|
||||
log " source : ${MOUNT_POINT}/${SRC_SUBPATH}/"
|
||||
log " dest : ${DEST}/"
|
||||
log " workers: 4, slab: 4G"
|
||||
fpsync \
|
||||
-n 4 \
|
||||
-s 4G \
|
||||
-o "-lptgoD -H --no-perms --no-owner --no-group --exclude=@eaDir/ --exclude=*@synoeastream --exclude=.DS_Store --exclude=Thumbs.db" \
|
||||
"${MOUNT_POINT}/${SRC_SUBPATH}/" \
|
||||
"${DEST}/" \
|
||||
2>&1 | tee -a "$LOG"
|
||||
log "Step 4: fpsync completed"
|
||||
|
||||
# ── 5. Verification dry-run ──────────────────────────────────────────────────
|
||||
log "Step 5: running dry-run verification rsync"
|
||||
VERIFY_OUT=$(rsync \
|
||||
-rlptgoD -H --no-perms --no-owner --no-group \
|
||||
--exclude='@eaDir/' --exclude='*@synoeastream' \
|
||||
--exclude='.DS_Store' --exclude='Thumbs.db' \
|
||||
-n --delete \
|
||||
--info=progress2 \
|
||||
--out-format='%o %f' \
|
||||
"${MOUNT_POINT}/${SRC_SUBPATH}/" \
|
||||
"${DEST}/" \
|
||||
2>&1 || true)
|
||||
|
||||
# Count lines that represent actual file changes (send / del. operations)
|
||||
CHANGE_COUNT=$(echo "$VERIFY_OUT" | grep -cE '^(send|del\.)' || true)
|
||||
|
||||
if [ "$CHANGE_COUNT" -eq 0 ]; then
|
||||
log "Step 5: sync verified clean — no pending changes"
|
||||
else
|
||||
log "Step 5: WARNING — verification found ${CHANGE_COUNT} pending change(s). First 50 lines:"
|
||||
# Use printf to avoid SIGPIPE from head closing the pipe early (set -o pipefail)
|
||||
{ echo "$VERIFY_OUT" | head -50; } >> "$LOG" 2>&1 || true
|
||||
fi
|
||||
|
||||
# ── 6. Unmount (only if we mounted it) ──────────────────────────────────────
|
||||
if [ "$MOUNTED_HERE" -eq 1 ]; then
|
||||
log "Step 6: unmounting $MOUNT_POINT"
|
||||
umount "$MOUNT_POINT"
|
||||
rmdir "$MOUNT_POINT"
|
||||
log "Step 6: unmounted"
|
||||
else
|
||||
log "Step 6: mount was pre-existing — leaving in place"
|
||||
fi
|
||||
|
||||
log "Done. Final size: $(du -sh "${DEST}" | cut -f1)"
|
||||
Loading…
Add table
Add a link
Reference in a new issue