Records the design reached in a /grill-with-docs session: how to track which
Service talks to which as more Services are added, using k8s-native options.
Decision: service identity = the workload's namespace (primary) plus a
`service-identity` label only in the few multi-Service namespaces; east-west
observability = Calico 3.30 Goldmane/Whisker (already in our Calico v3.30.7,
currently disabled) emitting to Loki for a durable trail; enforcement reuses the
existing Wave 1 egress track. Dedicated per-Service ServiceAccounts deferred and
a service mesh / mTLS / SPIFFE rejected — the trust model needs attribution-grade
forensics on a trusted, etcd-constrained cluster, not cryptographic
non-repudiation. This is the service-mesh evaluation the 2026-04-20 infra audit
flagged as missing; rejected alternatives (Retina, Hubble, Kiali, a custom Alloy
enricher) are recorded with rationale.
Adds glossary terms (Service identity, Goldmane / Whisker) to CONTEXT.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add `homelab browser run|open` so agents can drive the cluster's headful
Chrome (chrome-service) over CDP from the devvm. The headless playwright/mcp
browser can load anti-bot sites and fill their forms, but the gated submit
silently fails — e.g. the Stirling Ackroyd Fixflo tenant portal returned
net::ERR_FILE_NOT_FOUND on its pre-submit check and hung, creating nothing.
Driving the real headful Chrome submits first try. That capability already
existed but was undiscoverable, so it cost ~40 min + redundant form re-runs to
find; now it is one command, versioned, test-covered, and `browser --help`
carries the when-to-use signature + an error-code cheat-sheet so the right tool
is reached at the right moment (the failure was judgment, not setup).
- port-forward svc/chrome-service:9222 (tunnels API-server->pod, so it bypasses
the :9222 NetworkPolicy), assert non-headless via /json/version,
connect_over_cdp, inject the same vendored stealth.js the in-cluster callers
use; the port-forward is always torn down, on success and on error.
- node CDP client pinned to playwright-core@1.48.2 to match the v1.48.0-noble
image (Chromium 130); self-provisioned lazily into ~/.cache/homelab, no
per-user setup.
- default is a fresh incognito context (safe for the shared browser + concurrent
callers); --shared-context reuses the warmed persistent profile.
- TDD: cmd_browser_test.go covers arg parsing, headless detection, the version
pin, the help cheat-sheet, and a stealth.js drift guard. Verified end-to-end
against bot.sannysoft.com (real Chrome UA, webdriver hidden, plugins/WebGL
spoofed) and `browser open`.
- docs: README v0.8 section, ADR-0013, and a chrome-service.md "driving from
outside the cluster" section.
Closes: code-nepg
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`ha token` originally read openclaw/openclaw-secrets -> skill_secrets, which only
cluster admins can read — so it hung/failed for the non-admin operator it was
built for (emo = emil.barzin@gmail.com, OIDC group "Home Server Admins", whose
identity is deliberately barred from secrets in the openclaw namespace).
Split the HA tokens into a dedicated secret openclaw/ha-tokens (keys sofia/london)
with a Role + RoleBinding granting `get` on JUST that secret to the Home Server
Admins group (k8s RBAC can't scope to a JSON sub-key, hence a separate object).
emo now resolves the HA token with their own identity, WITHOUT gaining the rest
of skill_secrets (slack_webhook, uptime_kuma_password). openclaw's own deployment
keeps reading openclaw-secrets — purely additive.
- stacks/openclaw/ha_tokens.tf: new secret + least-privilege Role/RoleBinding
- cli/cmd_ha.go: read openclaw/ha-tokens (raw base64 per-instance key); drop JSON parse
- README + ADR-0012 updated; VERSION -> v0.7.1
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mined another devvm user's Claude sessions for repeated, hand-rolled command
patterns worth absorbing into the shared CLI. The dominant signal was Home
Assistant "Sofia" work: a `kubectl | base64 | jq` token-extraction pipeline
re-derived ~420x, and a bespoke non-interactive `ssh -o …` invocation reinvented
~30x — every session. The existing `home-assistant-sofia.py` already covers the
API but goes unused from an arbitrary cwd (needs an env var set + a cwd-relative
path), so agents bypassed it and hand-rolled everything.
Add two verbs covering exactly the gaps the `ha` MCP can't (entity state/control
stays with the MCP):
- `ha token [--instance sofia|london]` (read): resolves the long-lived API token
live from k8s secret openclaw/openclaw-secrets via the ambient kubeconfig — no
pre-set env var. Composes as `curl -H "Authorization: Bearer $(homelab ha token)"`.
- `ha ssh [--instance sofia|london] -- <cmd>` (write): deterministic
non-interactive ssh to the HA host using the invoking user's key.
Also fix the root cause: `home-assistant-sofia.py` now falls back to
`homelab ha token` when its env var is unset (works from any directory), and the
home-assistant skill points agents at these verbs + `homelab metrics query`
instead of hand-rolled curls. README + ADR-0012 + AGENTS.md updated per the
per-verb-group convention.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Answers the question that drove the whole CLI — which verbs to add next — with
data instead of one maintainer's habits, and resolves the cross-user-usage ask
in-bounds (no reading anyone's home).
- emit on dispatch: every verb fire-and-forgets one Loki line {job,user,verb} +
"exit=N ver=X". ONLY the verb path + exit code — never args, paths, flags, or
secrets (the emit never sees arguments). Best-effort: 800ms timeout, errors
swallowed, never affects the command; opt-out HOMELAB_TELEMETRY=0. Discovery
verbs (manifest/version/help) and usage itself don't self-record.
- usage top [--since 30d] [--user U] [--json]: ranks verbs via
sum by (verb)(count_over_time({job="homelab-usage"}[…])) against the shared
Loki. Cross-user analytics WITHOUT touching ~/.claude — the privacy-preserving
answer to "what does the team use".
- Loki sink (zero new infra, dogfoods v0.5 logs path); push verified HTTP 204 no
auth. ADR docs/adr/0011.
Live-verified: ran 4 verbs, usage top ranked them correctly (metrics query=2).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The remaining verbs that pass the "saves reasoning, not just typing" test the
user posed mid-session: each encodes the non-obvious which-endpoint-reached-how
resolution otherwise re-derived every time. (Same test deprioritized node-ssh
and secret-get aliasing — thin wrappers over commands already known.)
- net check <host> [path]: two-legged reachability — external (public DNS→CF)
vs internal (Traefik LB) — so you see WHERE a break is, not just that one path
works. (live: surfaced the LB at 6ms vs CF 77ms.)
- dns lookup <name> [type]: Technitium (10.0.20.201) vs public (1.1.1.1) diff.
- metrics query "<promql>" / metrics alerts: Prometheus via the LB
(prometheus-query.viktorbarzin.lan); alerts uses the synthetic ALERTS series
since the query frontend has no /api/v1/alerts and Alertmanager has no ingress.
- logs query "<logql>" [--since 1h] [--limit N]: Loki range query via the LB.
All reach auth-free internal ingresses through the LB (Go form of
curl --resolve host:443:10.0.20.203) — no port-forward, no kubectl. In-cluster-
only endpoints (Alertmanager v2) deliberately out of scope. Verified live before
building; all five smoke-tested green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the verb-group that kills the single biggest reasoning sink in agent
sessions — watching a build/deploy to completion (proven the session that built
it: hours hand-rolling Woodpecker polling + DB-schema spelunking for one CI
incident).
- ci status/watch: Woodpecker REST API (version-stable, not its DB schema),
reached via the internal Traefik LB (dial 10.0.20.203, SNI=ci.viktorbarzin.me
so the cert verifies — the Go form of the house `curl --resolve` pattern),
token from WOODPECKER_TOKEN/Vault, repo id resolved from the cwd remote, with
retries that ride Woodpecker's intermittent empty responses. watch matches the
HEAD/given commit (avoids the post-push race) and exits non-zero on failure.
- deploy wait: image-sha match THEN rollout status (rollout status alone returns
success on the old ReplicaSet); kubectl-based.
- work land now auto-watches CI to green on the landed commit (--no-ci-watch to
skip), closing the v0.1 gap.
- ci logs deferred to v0.4.1 (Woodpecker detail/log endpoints were the least
reliable; status/watch use the working list endpoint).
Live-verified ci status/watch against the live API.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Lets agents search/navigate memory via the CLI, as the first step toward
deprecating the memory MCP. claude-memory is a FastAPI service (the MCP is just
one frontend); homelab memory is a thin Bearer-auth HTTP client over the same
API, using the env the hooks already set (CLAUDE_MEMORY_API_URL/KEY). It works
even when the MCP frontend is down — the recurring disconnect that took the MCP
offline for this whole session.
Verbs: recall (server-side semantic search), list, categories, tags, stats,
secret (read); store, update, delete (write). Validated against the live API
including a store→recall→delete round-trip — full data-plane parity with the MCP.
The deprecation itself (rewiring the per-prompt auto-recall + auto-learn hooks to
the CLI, then uninstalling the MCP) is a deliberate follow-up, sequenced after
the CLI is proven in the hooks — see docs/adr/0008.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bump cli/VERSION to v0.2.0; document the k8s verbs (README table + resolver
note), add docs/adr/0007 (resolver, read/write split, config-mutation stays
raw, db dbaas pattern), and extend the AGENTS.md discovery pointer with the
Kubernetes surface.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes v0.1: documentation, build/install path, and version stamping.
- cli/VERSION (v0.1.0) stamped into the binary via ldflags.
- cli/README.md rewritten as the homelab overview (verbs + tiers, manifest,
build, the preserved legacy webhook use-cases).
- docs/adr/0004-0006: why homelab exists (grown in place from infra/cli, not a
separate repo), v0.1 scope + everything-allowed/tiers-recorded, and the
work/tf behaviour (native worktree entry, verification-gated auto-land,
presence-coupled apply).
- setup-devvm.sh builds cli/ -> /usr/local/bin/homelab each provisioning run
(t3-dispatch pattern), so every devvm user gets the current binary.
- AGENTS.md: discovery pointer under Common Operations.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Grilled the 'swap Forgejo for GitHub' idea. Root cause of the divergence pain
is an incomplete push-mirror rollout (14 repos dual-pushed, push_mirrors=0),
not Forgejo itself — and CONTEXT.md already documents Forgejo-canonical +
one-way GitHub mirror. Decision: don't swap; finish the mirror, name the
GitHub-first exceptions, reconcile infra, enforce one-remote-per-clone. Adds
ADR-0003 + the GitHub-first repo glossary term + dual-push/force-overwrite
warnings on Canonical repo / GitHub mirror.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Viktor asked to evaluate fully external image builders because in-cluster
CI builds keep destabilising the homelab (Forgejo OOM under registry-push
load, hairpin push timeouts, build IO on the shared sdc HDD, registry PVC
at its 50Gi ceiling). The evaluation was grilled to a decision set:
- every owned image builds on GitHub Actions and lives on ghcr.io
(extends the 2026-06-09 tripit pilot to the whole fleet)
- per-repo visibility: 9 public mirrors + images (gated on a clean
gitleaks/PII history scan), the personal/finance/gray ones stay private
- clean cut: no in-cluster fallback build pipelines; existing
build-fallback.yml files are deleted
- Woodpecker becomes deploy-only; Forgejo registry freezes to one
last-known-good tag per Service after a manual cleanup pass
- dead builders (terminal-lobby, webhook-handler, hmrc-sync, trading-bot,
travel-agent, trip-planner) are decommissioned, not migrated;
travel_blog is decommissioned outright; manual images (x402-gateway,
chrome-service-novnc, chatterbox-tts, android-emulator) get formalized
GHA builds; infra-ci + CLI builds move to GHA on the public infra repo
CONTEXT.md: updated 'GHA build + Woodpecker deploy', added 'Canonical
repo', 'GitHub mirror', 'Forgejo registry' terms, image-path relationship,
and a 'registry' ambiguity entry.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Viktor is setting up an Android app development pipeline (tripit is the
first app) and wants agents to natively test changes on Android before
shipping. This adds the testing environment: an API-36 Google emulator
under KVM as a privileged pod (namespace joins the Kyverno exclude list),
SDK/system-image/AVD on a proxmox-lvm PVC, adb on the shared MetalLB IP
10.0.20.200:5555 (LAN only), noVNC screen view at
android-emulator.viktorbarzin.lan. Image is built manually from the
stack's docker/ dir (rare rebuilds; off-infra-CI rule targets repeated
builds). First infra ADR records the trade-offs (devvm/VM/redroid/budtmo
rejected).
