infra/docs/runbooks/homelab-vault-onboarding.md
Viktor Barzin 0525f0b12d homelab vault: self-default VAULT_ADDR + prefer scoped token over ~/.vault-token
Setting up emo's Bitwarden access via `homelab vault`, his one-time
`homelab vault setup` failed with an opaque "exit status 2". Two latent
CLI bugs, both of which any non-admin AFK invocation can hit:

1. The CLI set VAULT_TOKEN but never VAULT_ADDR, relying on the ambient
   value. It IS in /etc/environment (login shells), but emo runs his
   agents from long-lived tmux / non-login shells that never sourced it,
   so every `vault` child hit the 127.0.0.1:8200 default -> connection
   refused. claude-auth-sync already self-defaults VAULT_ADDR; the CLI
   now does the same.

2. Token precedence was env > ~/.vault-token > scoped. A power-user who
   ran `vault login -method=oidc` carries a read-only ~/.vault-token
   (policy `default`, capability `deny` on their workstation path), which
   shadowed the purpose-built scoped token -> 403 permission denied on
   the user's OWN path. This tool only ever touches
   secret/workstation/claude-users/<user>, which the scoped token covers
   exactly, so precedence is now env > scoped > ~/.vault-token. Verified
   the scoped tokens for both emo and wizard hold create/read/update on
   their own paths, so admins are unaffected.

Also stop swallowing the shelled `vault`/`bw` stderr: errors now carry
the real message (connection refused / permission denied) instead of a
bare "exit status N" — without that, (1) and (2) were indistinguishable.

Verified end-to-end as emo (VAULT_ADDR unset + his read-only
~/.vault-token present): writeCreds now succeeds.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 09:04:28 +00:00

7.2 KiB

homelab vault onboarding (per-user Vaultwarden access)

Scope

homelab vault gives each devvm roster user no-HITL access to their own Vaultwarden vault (and any Organization Collection shared with their account) from the command line. It shells out to the official bw CLI; the user's Vaultwarden credentials live only in their isolated Vault path secret/workstation/claude-users/<os-user> and are decrypted as that OS user — the admin never sees them.

homelab vault setup             one-time: store VW email + master password + API key
homelab vault status            configured / unlocked / reachable (no secrets)
homelab vault list [--search Q]  item names (no secrets)
homelab vault get <name> [--field password|username|uri|notes|totp] [--json]
homelab vault code <name>       current TOTP code
homelab vault lock              lock / log out the local bw session

How auth works (why a non-admin can use it)

homelab vault runs vault as the calling user. It resolves a Vault token in this order (ensureVaultToken, cli/cmd_vault.go):

  1. an explicit $VAULT_TOKEN (a deliberate override), then
  2. the per-user scoped token that claude-auth-sync maintains at ~/.config/claude-auth-sync/vault-token (policy workstation-claude-<user>), then
  3. a native ~/.vault-token (admins who carry one; non-admins usually don't).

The scoped token deliberately beats ~/.vault-token. This tool only touches your own secret/workstation/claude-users/<user> path, and a power-user who ran vault login -method=oidc carries a read-only ~/.vault-token (capability deny on that path); letting it win would shadow the scoped token and fail every op with 403 permission denied (this is exactly what bit emo, 2026-06-28). The CLI also self-defaults VAULT_ADDR to https://vault.viktorbarzin.me when unset, so it works from non-login shells (tmux panes, AFK agent subprocesses) that never sourced /etc/environment — otherwise every vault child hits the 127.0.0.1:8200 default and fails connection refused (exit 2).

That scoped policy grants exactly create/read/update on the user's own secret/workstation/claude-users/<user> path — no patch capability — so the tool writes with vault kv patch -method=rw (read-modify-write), falling back to kv put only when the path does not exist yet. This preserves the claude_ai_oauth_json key that claude-auth-sync co-locates there. (The admin-only bugs were fixed 2026-06-27; the VAULT_ADDR/token-precedence bugs above were fixed 2026-06-28.)

Prerequisites (per user)

  • The user is in scripts/workstation/roster.yaml and the vault stack has been applied → their workstation-claude-<user> policy exists.
  • The user's workstation was provisioned (setup-devvm.sh) → their scoped Vault token exists at ~/.config/claude-auth-sync/vault-token.
  • bw is installed system-wide at /usr/bin/bw (see below).
  • The user has a Vaultwarden account at https://vaultwarden.viktorbarzin.me (self-service signup is open; admin panel is disabled).

One-time admin steps (devvm)

bw must be system-wide so every user resolves it (it is a Node script, and node is already system-wide at /usr/bin/node). setup-devvm.sh installs it to the npm /usr prefix; the guard checks the system path, not command -v bw (an admin's own ~/.local/bin/bw used to mask the system install, leaving non-admins with no backend). To install on a running box:

sudo npm install -g --prefix /usr "@bitwarden/cli@^2024"
bw --version            # confirm /usr/bin/bw resolves

After landing a cli/ change, rebuild the binary so users pick it up:

sudo bash -c 'cd /home/wizard/code/infra/cli && \
  go build -ldflags "-X main.version=$(git -C /home/wizard/code/infra describe --tags --always 2>/dev/null || echo dev)" \
  -o /usr/local/bin/homelab .'

(or just re-run scripts/workstation/setup-devvm.sh as root, which rebuilds it.)

User onboarding

The user runs these as themselves. The master password / API key are entered interactively (never on the command line) and stored only in the user's Vault path.

  1. In the Vaultwarden web vault → Settings → Security → Keys → View API key, copy the client_id (user.xxxx) and client_secret.

  2. Configure:

    homelab vault setup        # prompts: VW email, API client_id/secret, master password
    homelab vault status       # → "vault: configured, unlocked, reachable ✓"
    homelab vault list         # item names (own vault + any shared Collections)
    

Shared-Collection access (sharing passwords with a user)

homelab vault surfaces Organization Collection items automatically once the user's Vaultwarden account is a confirmed member. These steps are done by the vault owner in the Vaultwarden web UI (they need the owner's master password — not an infra/Terraform operation):

  1. Create or reuse an Organization and a Collection of shared logins.
  2. Invite the user's Vaultwarden account to the Organization, granting "Can view" on that Collection (least privilege).
  3. The user accepts the email invite and confirms membership.
  4. The user runs homelab vault list — the shared items now appear alongside their own (a homelab vault status sync picks them up).

Security model (the no-HITL trade)

Identity is the kernel UID. Anything running as the user can decrypt the user's vault — this is the accepted trade for no-human-in-the-loop fetches. Secrets never appear in argv (passed via env or stdin), core dumps are disabled, TOTP fetches are logged to syslog/Loki, and on a TTY values go to the clipboard (auto-clearing) rather than scrollback. The admin's Vault token is never used by a non-admin: each user authenticates with their own scoped token.

Verification

# the scoped token carries the right policy
VAULT_TOKEN="$(sudo cat /home/<user>/.config/claude-auth-sync/vault-token)" \
  vault token lookup -format=json | jq '.data.display_name, .data.policies'
#   → "token-devvm-claude-auth-<user>", [..., "workstation-claude-<user>"]

sudo -u <user> -i bw --version        # /usr/bin/bw resolves for the user
sudo -u <user> -i homelab vault status

Troubleshooting

homelab vault setup (or any verb) fails with exit status 2 — older binaries swallowed the underlying vault error; the message now includes it. Two historical causes (both fixed in-CLI 2026-06-28, kept here for diagnosis):

  • ... connection refused to 127.0.0.1:8200VAULT_ADDR wasn't set in the caller's shell. The CLI now self-defaults it, but if you see this on an old binary: export VAULT_ADDR=https://vault.viktorbarzin.me.
  • 403 permission denied on PUT .../secret/data/workstation/claude-users/<user> → a stale read-only ~/.vault-token (e.g. from vault login -method=oidc, policy default, capability deny on that path) was shadowing the scoped token. The CLI now prefers the scoped token; on an old binary, rm ~/.vault-token (or unset VAULT_TOKEN) and retry. Confirm with VAULT_TOKEN="$(sudo cat /home/<user>/.config/claude-auth-sync/vault-token)" vault token capabilities secret/data/workstation/claude-users/<user> → must be create, read, update.