From 48b63ffa6f324d67392a7bc2599105c7d695af13 Mon Sep 17 00:00:00 2001 From: Viktor Barzin Date: Fri, 19 Jun 2026 05:56:25 +0000 Subject: [PATCH] =?UTF-8?q?homelab:=20add=20memory=20verb-group=20(v0.3.0)?= =?UTF-8?q?=20=E2=80=94=20direct=20claude-memory=20HTTP=20client?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lets agents search/navigate memory via the CLI, as the first step toward deprecating the memory MCP. claude-memory is a FastAPI service (the MCP is just one frontend); homelab memory is a thin Bearer-auth HTTP client over the same API, using the env the hooks already set (CLAUDE_MEMORY_API_URL/KEY). It works even when the MCP frontend is down — the recurring disconnect that took the MCP offline for this whole session. Verbs: recall (server-side semantic search), list, categories, tags, stats, secret (read); store, update, delete (write). Validated against the live API including a store→recall→delete round-trip — full data-plane parity with the MCP. The deprecation itself (rewiring the per-prompt auto-recall + auto-learn hooks to the CLI, then uninstalling the MCP) is a deliberate follow-up, sequenced after the CLI is proven in the hooks — see docs/adr/0008. Co-Authored-By: Claude Opus 4.8 --- AGENTS.md | 2 +- cli/README.md | 26 ++- cli/VERSION | 2 +- cli/cmd_memory.go | 302 ++++++++++++++++++++++++++ cli/homelab.go | 1 + cli/memory.go | 103 +++++++++ cli/memory_test.go | 51 +++++ docs/adr/0008-homelab-memory-verbs.md | 30 +++ 8 files changed, 514 insertions(+), 3 deletions(-) create mode 100644 cli/cmd_memory.go create mode 100644 cli/memory.go create mode 100644 cli/memory_test.go create mode 100644 docs/adr/0008-homelab-memory-verbs.md diff --git a/AGENTS.md b/AGENTS.md index 6947f27e..afd48a5d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -289,7 +289,7 @@ curl -X POST -H "Authorization: token $TOK" -H 'Content-Type: application/json' ``` ## Common Operations -- **`homelab` CLI** (`/usr/local/bin/homelab`, source `cli/`): unified infra-ops verbs — run `homelab manifest` to discover the surface (each verb tagged read/write). Infra loop: `homelab tf plan|fmt|apply ` (wraps `scripts/tg`; `apply` auto-claims presence + releases on exit, warns out-of-band), `homelab claim|release :`, `homelab work start|land|clean ` (worktree lifecycle; `land` gates on verification, `--verify-cmd`/`--no-verify`). Kubernetes (v0.2): `homelab k8s status|get|logs|describe|debug|pf|rollout-status ` (read; `` defaults to the namespace, target to `deploy/`), `homelab k8s db [--mysql] -- ""`, `k8s exec`, `k8s restart`, `k8s rm-pod` (pods/jobs only) — config-mutation kubectl verbs are intentionally absent (Terraform-only). Full docs: `cli/README.md`. +- **`homelab` CLI** (`/usr/local/bin/homelab`, source `cli/`): unified infra-ops verbs — run `homelab manifest` to discover the surface (each verb tagged read/write). Infra loop: `homelab tf plan|fmt|apply ` (wraps `scripts/tg`; `apply` auto-claims presence + releases on exit, warns out-of-band), `homelab claim|release :`, `homelab work start|land|clean ` (worktree lifecycle; `land` gates on verification, `--verify-cmd`/`--no-verify`). Kubernetes (v0.2): `homelab k8s status|get|logs|describe|debug|pf|rollout-status ` (read; `` defaults to the namespace, target to `deploy/`), `homelab k8s db [--mysql] -- ""`, `k8s exec`, `k8s restart`, `k8s rm-pod` (pods/jobs only) — config-mutation kubectl verbs are intentionally absent (Terraform-only). Memory (v0.3): `homelab memory recall ""` (semantic search), `memory list|categories|tags|stats|secret`, `memory store|update|delete` — a direct HTTP client to claude-memory that works even when the memory MCP is down. Full docs: `cli/README.md`. - **Deploy new service**: Use `stacks//` as template. Create stack, add DNS in tfvars, apply platform then service. - **Fix crashed pods**: Run healthcheck first. Safe to delete evicted/failed pods and CrashLoopBackOff pods with >10 restarts. - **OOMKilled**: Check `kubectl describe limitrange tier-defaults -n `. Increase `resources.limits.memory` in the stack's main.tf. diff --git a/cli/README.md b/cli/README.md index 8aeb2e85..65163ccb 100644 --- a/cli/README.md +++ b/cli/README.md @@ -70,6 +70,30 @@ Tiers are recorded per verb so a future PreToolUse classifier can auto-allow reads / prompt writes; v0.1 allows everything and relies on existing gates (permission mode, presence claims, plan approval). +### v0.3 verbs — memory + +A thin HTTP client over the **claude-memory** service (the same backend the +memory MCP wraps), authed with `CLAUDE_MEMORY_API_KEY` against +`CLAUDE_MEMORY_API_URL` (the env the hooks already set; defaults to the +ingress). Because it hits the HTTP API directly, it **works even when the MCP +frontend is down**. + +| Command | Tier | What it does | +|---|---|---| +| `memory recall "" [--query --category --sort --limit]` | read | semantic search (server-side ranking) — the navigate workhorse | +| `memory list [--category --tag --limit]` | read | recent memories | +| `memory categories` / `memory tags` / `memory stats` | read | enumerate the store | +| `memory secret ` | read | reveal a sensitive memory's content | +| `memory store "" [--category --tags --keywords --importance --sensitive]` | write | store a memory | +| `memory update [--content --tags --importance]` | write | edit a memory | +| `memory delete ` | write | delete a memory | + +All read/write paths are validated against the live API (incl. a +store→recall→delete round-trip). This gives full data-plane parity with the MCP; +the eventual deprecation (rewiring the per-prompt auto-recall + auto-learn hooks +to the CLI, then uninstalling the MCP) is a **separate, deliberate follow-up** — +see `docs/adr/0008`. + ## Build / install Built from source to `/usr/local/bin/homelab` during devvm provisioning @@ -89,4 +113,4 @@ original flag-based path unchanged, so the webhook handler is unaffected. ## Design -See `infra/docs/adr/0004`–`0007` for the architecture decisions. +See `infra/docs/adr/0004`–`0008` for the architecture decisions. diff --git a/cli/VERSION b/cli/VERSION index 1474d00f..268b0334 100644 --- a/cli/VERSION +++ b/cli/VERSION @@ -1 +1 @@ -v0.2.0 +v0.3.0 diff --git a/cli/cmd_memory.go b/cli/cmd_memory.go new file mode 100644 index 00000000..94f3a482 --- /dev/null +++ b/cli/cmd_memory.go @@ -0,0 +1,302 @@ +package main + +import ( + "encoding/json" + "fmt" + "net/url" + "strings" +) + +func memoryCommands() []Command { + return []Command{ + {Path: []string{"memory", "recall"}, Tier: TierRead, + Summary: `semantic search of memory: memory recall "" [--query …] [--category] [--sort] [--limit]`, Run: memoryRecall}, + {Path: []string{"memory", "list"}, Tier: TierRead, + Summary: "list recent memories [--category C] [--tag T] [--limit N]", Run: memoryList}, + {Path: []string{"memory", "categories"}, Tier: TierRead, + Summary: "list memory categories", Run: memorySimpleGet("/api/categories")}, + {Path: []string{"memory", "tags"}, Tier: TierRead, + Summary: "list memory tags", Run: memorySimpleGet("/api/tags")}, + {Path: []string{"memory", "stats"}, Tier: TierRead, + Summary: "memory store stats", Run: memorySimpleGet("/api/stats")}, + {Path: []string{"memory", "secret"}, Tier: TierRead, + Summary: "reveal a sensitive memory's content: memory secret ", Run: memorySecret}, + {Path: []string{"memory", "store"}, Tier: TierWrite, + Summary: `store a memory: memory store "" [--category --tags --keywords --importance --sensitive]`, Run: memoryStore}, + {Path: []string{"memory", "update"}, Tier: TierWrite, + Summary: "update a memory: memory update [--content --tags --importance --keywords]", Run: memoryUpdate}, + {Path: []string{"memory", "delete"}, Tier: TierWrite, + Summary: "delete a memory: memory delete ", Run: memoryDelete}, + } +} + +// printMemories renders a {memories:[…]} response as compact lines, or raw JSON. +func printMemories(raw []byte, jsonOut bool) error { + if jsonOut { + fmt.Println(string(raw)) + return nil + } + var r struct { + Memories []struct { + ID int `json:"id"` + Content string `json:"content"` + Category string `json:"category"` + Tags string `json:"tags"` + Importance float64 `json:"importance"` + } `json:"memories"` + } + if err := json.Unmarshal(raw, &r); err != nil { + fmt.Println(string(raw)) + return nil + } + if len(r.Memories) == 0 { + fmt.Println("(no memories)") + return nil + } + for _, m := range r.Memories { + c := strings.ReplaceAll(m.Content, "\n", " ") + if len(c) > 240 { + c = c[:240] + "…" + } + fmt.Printf("#%d [%s] (%.2f) %s\n", m.ID, m.Category, m.Importance, c) + if m.Tags != "" { + fmt.Printf(" tags: %s\n", m.Tags) + } + } + return nil +} + +func memoryRecall(args []string) error { + req := memRecallReq{} + jsonOut := false + var pos []string + for i := 0; i < len(args); i++ { + a := args[i] + switch { + case a == "--query": + if i+1 < len(args) { + req.ExpandedQuery = args[i+1] + i++ + } + case a == "--category": + if i+1 < len(args) { + req.Category = args[i+1] + i++ + } + case a == "--sort": + if i+1 < len(args) { + req.SortBy = args[i+1] + i++ + } + case a == "--limit": + if i+1 < len(args) { + fmt.Sscanf(args[i+1], "%d", &req.Limit) + i++ + } + case a == "--json": + jsonOut = true + case !strings.HasPrefix(a, "-"): + pos = append(pos, a) + } + } + req.Context = strings.Join(pos, " ") + if req.Context == "" { + return fmt.Errorf(`usage: homelab memory recall "" [--query …] [--category C] [--sort importance|relevance|recency] [--limit N]`) + } + c, err := newMemoryClient() + if err != nil { + return err + } + raw, err := c.do("POST", "/api/memories/recall", req) + if err != nil { + return err + } + return printMemories(raw, jsonOut) +} + +func memoryList(args []string) error { + q := url.Values{} + jsonOut := false + for i := 0; i < len(args); i++ { + a := args[i] + switch { + case a == "--category": + if i+1 < len(args) { + q.Set("category", args[i+1]) + i++ + } + case a == "--tag": + if i+1 < len(args) { + q.Set("tag", args[i+1]) + i++ + } + case a == "--limit": + if i+1 < len(args) { + q.Set("limit", args[i+1]) + i++ + } + case a == "--json": + jsonOut = true + } + } + c, err := newMemoryClient() + if err != nil { + return err + } + path := "/api/memories" + if len(q) > 0 { + path += "?" + q.Encode() + } + raw, err := c.do("GET", path, nil) + if err != nil { + return err + } + return printMemories(raw, jsonOut) +} + +func memorySimpleGet(path string) func([]string) error { + return func(args []string) error { + c, err := newMemoryClient() + if err != nil { + return err + } + raw, err := c.do("GET", path, nil) + if err != nil { + return err + } + fmt.Println(string(raw)) + return nil + } +} + +func memorySecret(args []string) error { + id, _ := firstPositional(args) + if id == "" { + return fmt.Errorf("usage: homelab memory secret ") + } + c, err := newMemoryClient() + if err != nil { + return err + } + raw, err := c.do("POST", "/api/memories/"+id+"/secret", nil) + if err != nil { + return err + } + fmt.Println(string(raw)) + return nil +} + +func memoryStore(args []string) error { + req := memStoreReq{Category: "facts", Importance: 0.5} + var pos []string + for i := 0; i < len(args); i++ { + a := args[i] + switch { + case a == "--category": + if i+1 < len(args) { + req.Category = args[i+1] + i++ + } + case a == "--tags": + if i+1 < len(args) { + req.Tags = args[i+1] + i++ + } + case a == "--keywords": + if i+1 < len(args) { + req.ExpandedKeywords = args[i+1] + i++ + } + case a == "--importance": + if i+1 < len(args) { + fmt.Sscanf(args[i+1], "%f", &req.Importance) + i++ + } + case a == "--sensitive": + req.ForceSensitive = true + case !strings.HasPrefix(a, "-"): + pos = append(pos, a) + } + } + req.Content = strings.Join(pos, " ") + if req.Content == "" { + return fmt.Errorf(`usage: homelab memory store "" [--category C] [--tags ...] [--keywords ...] [--importance 0.5] [--sensitive]`) + } + c, err := newMemoryClient() + if err != nil { + return err + } + raw, err := c.do("POST", "/api/memories", req) + if err != nil { + return err + } + fmt.Println(string(raw)) + return nil +} + +func memoryUpdate(args []string) error { + var id string + req := memUpdateReq{} + for i := 0; i < len(args); i++ { + a := args[i] + switch { + case a == "--content": + if i+1 < len(args) { + v := args[i+1] + req.Content = &v + i++ + } + case a == "--tags": + if i+1 < len(args) { + v := args[i+1] + req.Tags = &v + i++ + } + case a == "--keywords": + if i+1 < len(args) { + v := args[i+1] + req.ExpandedKeywords = &v + i++ + } + case a == "--importance": + if i+1 < len(args) { + var f float64 + fmt.Sscanf(args[i+1], "%f", &f) + req.Importance = &f + i++ + } + case !strings.HasPrefix(a, "-") && id == "": + id = a + } + } + if id == "" { + return fmt.Errorf("usage: homelab memory update [--content ...] [--tags ...] [--importance N] [--keywords ...]") + } + c, err := newMemoryClient() + if err != nil { + return err + } + raw, err := c.do("PUT", "/api/memories/"+id, req) + if err != nil { + return err + } + fmt.Println(string(raw)) + return nil +} + +func memoryDelete(args []string) error { + id, _ := firstPositional(args) + if id == "" { + return fmt.Errorf("usage: homelab memory delete ") + } + c, err := newMemoryClient() + if err != nil { + return err + } + raw, err := c.do("DELETE", "/api/memories/"+id, nil) + if err != nil { + return err + } + fmt.Println(string(raw)) + return nil +} diff --git a/cli/homelab.go b/cli/homelab.go index 239570b2..3f7ea174 100644 --- a/cli/homelab.go +++ b/cli/homelab.go @@ -15,6 +15,7 @@ func buildRegistry() []Command { reg = append(reg, tfCommands()...) reg = append(reg, workCommands()...) reg = append(reg, k8sCommands()...) + reg = append(reg, memoryCommands()...) return reg } diff --git a/cli/memory.go b/cli/memory.go new file mode 100644 index 00000000..286ee5bb --- /dev/null +++ b/cli/memory.go @@ -0,0 +1,103 @@ +package main + +import ( + "bytes" + "encoding/json" + "fmt" + "io" + "net/http" + "os" + "strings" + "time" +) + +// defaultMemoryURL is used when no env override is present (agents normally have +// CLAUDE_MEMORY_API_URL set by the memory hooks). +const defaultMemoryURL = "https://claude-memory.viktorbarzin.me" + +type memoryClient struct { + base string + key string + http *http.Client +} + +func firstEnv(keys ...string) string { + for _, k := range keys { + if v := os.Getenv(k); v != "" { + return v + } + } + return "" +} + +func resolveMemoryBase() string { + if b := firstEnv("CLAUDE_MEMORY_API_URL", "MEMORY_API_URL"); b != "" { + return strings.TrimRight(b, "/") + } + return defaultMemoryURL +} + +// newMemoryClient talks straight to the claude-memory HTTP API (the same backend +// the MCP wraps), so it works even when the MCP frontend is down. +func newMemoryClient() (*memoryClient, error) { + key := firstEnv("CLAUDE_MEMORY_API_KEY", "MEMORY_API_KEY") + if key == "" { + return nil, fmt.Errorf("no memory API key — set CLAUDE_MEMORY_API_KEY (or MEMORY_API_KEY)") + } + return &memoryClient{base: resolveMemoryBase(), key: key, http: &http.Client{Timeout: 30 * time.Second}}, nil +} + +func (c *memoryClient) do(method, path string, body interface{}) ([]byte, error) { + var r io.Reader + if body != nil { + b, err := json.Marshal(body) + if err != nil { + return nil, err + } + r = bytes.NewReader(b) + } + req, err := http.NewRequest(method, c.base+path, r) + if err != nil { + return nil, err + } + req.Header.Set("Authorization", "Bearer "+c.key) + if body != nil { + req.Header.Set("Content-Type", "application/json") + } + resp, err := c.http.Do(req) + if err != nil { + return nil, err + } + defer resp.Body.Close() + out, _ := io.ReadAll(resp.Body) + if resp.StatusCode >= 300 { + return nil, fmt.Errorf("memory API %s %s -> %d: %s", method, path, resp.StatusCode, strings.TrimSpace(string(out))) + } + return out, nil +} + +// Request bodies mirror src/claude_memory/api/models.py. + +type memRecallReq struct { + Context string `json:"context"` + ExpandedQuery string `json:"expanded_query,omitempty"` + Category string `json:"category,omitempty"` + SortBy string `json:"sort_by,omitempty"` + Limit int `json:"limit,omitempty"` +} + +type memStoreReq struct { + Content string `json:"content"` + Category string `json:"category,omitempty"` + Tags string `json:"tags,omitempty"` + ExpandedKeywords string `json:"expanded_keywords,omitempty"` + Importance float64 `json:"importance"` + ForceSensitive bool `json:"force_sensitive,omitempty"` +} + +type memUpdateReq struct { + Content *string `json:"content,omitempty"` + Tags *string `json:"tags,omitempty"` + Importance *float64 `json:"importance,omitempty"` + ExpandedKeywords *string `json:"expanded_keywords,omitempty"` +} diff --git a/cli/memory_test.go b/cli/memory_test.go new file mode 100644 index 00000000..7b14ef20 --- /dev/null +++ b/cli/memory_test.go @@ -0,0 +1,51 @@ +package main + +import ( + "encoding/json" + "os" + "strings" + "testing" +) + +func TestResolveMemoryBase(t *testing.T) { + old1, old2 := os.Getenv("CLAUDE_MEMORY_API_URL"), os.Getenv("MEMORY_API_URL") + defer func() { os.Setenv("CLAUDE_MEMORY_API_URL", old1); os.Setenv("MEMORY_API_URL", old2) }() + + os.Unsetenv("CLAUDE_MEMORY_API_URL") + os.Unsetenv("MEMORY_API_URL") + if got := resolveMemoryBase(); got != defaultMemoryURL { + t.Errorf("resolveMemoryBase() = %q, want default %q", got, defaultMemoryURL) + } + os.Setenv("CLAUDE_MEMORY_API_URL", "https://m.example/") // trailing slash trimmed + if got := resolveMemoryBase(); got != "https://m.example" { + t.Errorf("resolveMemoryBase() = %q, want https://m.example", got) + } +} + +func TestMemStoreReqAlwaysSendsImportance(t *testing.T) { + b, _ := json.Marshal(memStoreReq{Content: "x", Category: "facts", Importance: 0.5}) + s := string(b) + if !strings.Contains(s, `"content":"x"`) || !strings.Contains(s, `"importance":0.5`) { + t.Fatalf("memStoreReq JSON missing fields: %s", s) + } +} + +func TestMemUpdateReqOmitsUnsetFields(t *testing.T) { + tags := "a,b" + b, _ := json.Marshal(memUpdateReq{Tags: &tags}) + s := string(b) + if strings.Contains(s, "content") || strings.Contains(s, "importance") { + t.Fatalf("unset update fields must be omitted: %s", s) + } + if !strings.Contains(s, `"tags":"a,b"`) { + t.Fatalf("set field missing: %s", s) + } +} + +func TestMemRecallReqOmitsEmptyOptionals(t *testing.T) { + b, _ := json.Marshal(memRecallReq{Context: "hi"}) + s := string(b) + if strings.Contains(s, "expanded_query") || strings.Contains(s, "category") || strings.Contains(s, "limit") { + t.Fatalf("empty optionals must be omitted: %s", s) + } +} diff --git a/docs/adr/0008-homelab-memory-verbs.md b/docs/adr/0008-homelab-memory-verbs.md new file mode 100644 index 00000000..60f13850 --- /dev/null +++ b/docs/adr/0008-homelab-memory-verbs.md @@ -0,0 +1,30 @@ +# homelab memory verb-group: direct HTTP client to claude-memory; MCP deprecation path + +v0.3 adds the memory verb-group so agents can search and navigate memory from the +CLI. `claude-memory` is a FastAPI service (Postgres-backed, `Bearer`-auth, +ingress `auth = "none"` so programmatic clients work) — the **MCP is just one +frontend over it**. `homelab memory` is a thin HTTP client over the same API, +using the env the hooks already set (`CLAUDE_MEMORY_API_URL` + +`CLAUDE_MEMORY_API_KEY`; defaults to the ingress). Because it talks to the HTTP +API directly, it **works even when the MCP frontend is down** — the recurring +MCP-disconnect problem that motivated claude-memory HA (and that took the MCP +offline for the entire session this was built in). + +Verbs: `recall` (server-side semantic ranking), `list`, `categories`, `tags`, +`stats`, `secret` (read); `store`, `update`, `delete` (write). Validated against +the live API including a store→recall→delete round-trip — full data-plane parity +with the MCP. + +## Deprecation path (deliberate follow-up — NOT done in v0.3) + +The MCP is more than tools: the **per-prompt auto-recall hook** and the +**auto-learn hook** run on every prompt for every agent. Deprecating it safely is +a separate, sequenced change: + +1. Rewire the auto-recall hook to `homelab memory recall` and the auto-learn hook + to `homelab memory store`. +2. Update the CLAUDE.md memory policy to point at the CLI. +3. Uninstall the MCP. + +Done CLI-first (verbs proven before touching the every-prompt path) so a +regression can't silently break auto-recall/auto-learn fleet-wide.