claude-memory-mcp/docs/research/survey.md

Landscape survey: semantic + concept-graph memory for hybrid recall

Status: research input for the ADR-0001 hybrid upgrade. Scope: how the agent-memory and graph-RAG literature builds and retrieves over a personal-memory store, which embedding model to use, how to fuse lexical + dense + graph signals, and how to evaluate the result.

Read this with the decisions already fixed in ADR-0001 –0003: we pursue hybrid (gated on a benchmark beating FTS), embeddings live in pgvector on the existing CNPG Postgres, the concept graph is node/edge tables in Postgres, sensitive memories never egress, and adoption is decided quality-first (recall@k / nDCG@10 / MRR; latency & storage are reported, not gating).

The recurring conclusion below: borrow the ideas, not the engines. None of the four systems surveyed is a drop-in for our stack, but each contributes a mechanism we re-implement natively on Postgres + pgvector.

---

1. Our workload is the opposite of GraphRAG's design target

Before comparing systems it helps to state what we are not. The graph-RAG family was built for global sensemaking ("what are the themes across this corpus") over a static document collection. Our workload is the reverse:

Dimension	GraphRAG target	claude-memory-mcp
Unit	Long documents, chunked	Atomic, already-curated memories (avg ~500 chars)
Corpus dynamics	Static, indexed once	Append-heavy: a few hundred memories/day arriving
Query type	Corpus-wide summarization	Point / multi-hop recall ("what did I decide about X")
Hot path	Offline batch	Every user prompt (auto-recall hook fires before each turn)
Scale	10k–1M+ chunks	~5k memories today → tens of thousands

This mismatch is the lens for everything that follows. The expensive part of GraphRAG — community detection + hierarchical LLM summaries — is the wrong retrieval unit for atomic point lookups, and re-summarizing communities on a sustained append stream is its dominant, unbounded cost. We want a design whose index-time work is proportional to new content, and whose retrieval path has no LLM call (so it fits the per-prompt budget).

---

2. The GraphRAG family — Microsoft GraphRAG, LightRAG, nano-graphrag, LazyGraphRAG

All four turn text into an entity–relation knowledge graph via LLM extraction; they differ on the expensive part (community detection + hierarchical summarization), which is exactly where incremental cost lives.

Microsoft GraphRAG (Edge et al. 2024, arXiv 2404.16130)

Pipeline: chunk → LLM extracts entities + relationships per chunk (with multi-round "gleanings" to catch misses) → summarize duplicate element instances into node/edge descriptions → build graph → Leiden community detection producing a hierarchy (levels C0..C3) → an LLM writes a community report for every community at every level. Two query modes: global (map-reduce over all community reports — corpus-wide sensemaking) and local (start from query-relevant entities, fan out). Indexing is LLM-heavy: ~4,000 LLM calls / ~35 min for one textbook; ~$20–40 per 1M tokens with gpt-4o.

Incremental: the graphrag update command (GraphRAG 1.0) computes deltas and places new entities into existing communities "rather than re-running Leiden," re-summarizing only changed communities — but maintainers warn that once drift crosses a threshold "the worst case degrades to the same performance as a normal indexing." A periodic, unpredictable full-reindex cliff on a sustained append stream. Parquet/file-pipeline oriented, not Postgres-native.

LightRAG (HKUDS, arXiv 2410.05779, EMNLP 2025)

Pipeline: chunk → LLM extracts entities + relations → "profiling" generates a key-value text summary per node/edge → deduplicate merges identical entities/relations across chunks. No community detection. Retrieval is dual-level: the LLM splits the query into low-level keywords (specific entities) and high-level keywords (broad themes via relationship chains); each set is matched by vector similarity against an entity-vector index and a relation-vector index, then one-hop neighbours are pulled from the graph. Modes: naive / local / global / hybrid / mix (mix = default).

Incremental (the crux): a new document goes through the same local indexing to produce a small local graph, then is integrated by set union of node-sets and edge-sets into the existing graph — "eliminating the need to rebuild the entire index graph." No communities ⇒ no global re-clustering or re-summarization, ever ⇒ O(new content) per insert, the only genuinely-incremental member. Ships a PostgreSQL all-in-one backend (PGVectorStorage on pgvector + PGGraphStorage on Apache AGE + KV + doc-status in one DB, PG ≥16.6).

nano-graphrag (~1100 LOC)

A faithful minimal reimplementation of Microsoft GraphRAG and an excellent compact reference for the exact extraction/community/report prompts. Hard NO for incremental: README states plainly "each time you insert, the communities of graph will be re-computed and the community reports will be re-generated" — O(whole graph) LLM cost per append.

LazyGraphRAG (Microsoft Research, 2024)

Defers all LLM work to query time: index time uses only NLP noun-phrase extraction + graph statistics — "indexing costs are identical to vector RAG and 0.1% of the costs of full GraphRAG." Sidesteps the incremental-re-summarization problem entirely by never pre-summarizing communities. The defer-LLM-cost principle is the one to borrow.

Verdict for us

Adopt none wholesale; steal LightRAG's architecture + LazyGraphRAG's defer-LLM principle. LightRAG is the only one whose incremental model (pure set-union, no re-clustering) structurally fits an append-heavy stream, and whose retrieval path (vector + one-hop graph, no query-time map-reduce) is hot-path-viable. But adopting LightRAG-the-product is not recommended: its Postgres graph path needs the Apache AGE extension (not on our CNPG), and that path has documented concurrency/entity-merge instability under append-heavy load (asyncpg pool timeouts at the merge stage; slow upgrades). Our multi-hop is shallow (1–2 hops), expressible in plain recursive SQL CTEs over node/edge tables — no AGE needed.

---

3. Zep / Graphiti — temporal knowledge graph for agent memory

Zep (arXiv 2501.13956) is an agent-memory service; Graphiti is its open engine (Neo4j / FalkorDB / Kuzu backend, ~20k stars, MIT). It is the closest conceptual analog to our hybrid goal — it fuses exactly the three signals ADR-0001 wants.

Three-tier graph: episode subgraph (raw ingested data, the provenance layer ≈ our Memory rows) → semantic entity subgraph (entity nodes + typed relationship edges, each linking back to its source episodes) → community subgraph (clusters with LLM summaries — the GraphRAG "global" layer).

Bi-temporal model: every semantic edge carries four timestamps on two timelines — valid time (t_valid/t_invalid: when the fact held true in the world) and transaction time (t'_created/t'_expired: when Zep learned/retracted it). Facts are never deleted; superseded facts get their validity window closed. This is a principled, queryable version of our "supersede, don't accumulate" memory discipline.

Incremental ingestion (per episode): a sequence of LLM calls — entity extraction → entity resolution/dedup (embed + cosine + BM25 search against existing nodes, then an LLM judges merge vs create) → fact (edge) extraction → fact dedup → temporal extraction (resolve "two weeks ago" against a reference time) → edge invalidation (LLM compares each new edge against related existing edges; on a temporally-overlapping contradiction it closes the old edge). Cost is heavy on write, paid back on reads.

Retrieval (sub-second, NO LLM at query time): three parallel searches fused, then reranked.

• φ_cos: cosine over embeddings of fact text / entity names / community summaries (BGE-m3, 1024-d).
• φ_bm25: BM25 full-text over the same fields.
• φ_bfs: breadth-first n-hop graph traversal from seed nodes — the genuinely graph-native signal.
• Rerank: pluggable — RRF, MMR (diversity), episode-mentions (frequency), node-distance, or a cross-encoder (most accurate, slowest).

Published quality (the strongest evidence in this family): on LongMemEval, Zep reports +18.5% accuracy over a baseline (71.2% vs 60.2% with gpt-4o) and ~90% query-latency reduction; on MemGPT DMR, 94.8% vs 93.4%. These are conversational long-context QA, not personal-fact recall@k — so the headline numbers won't transfer directly, but the fusion recipe is exactly what we benchmark.

Verdict for us

Primary design blueprint for the concept-graph half — but not an adopted dependency. Graphiti has no pgvector backend; adopting the engine forces a new Neo4j/FalkorDB graph DB into the cluster, conflicting with ADR-0002 and reuse-before-building. We borrow four mechanisms, re-implemented on Postgres: (1) the episodic(=Memory rows)/semantic(=new node+edge tables) split; (2) the parallel-search + RRF fusion read path; (3) resolution-via-search to dedupe the graph using our existing FTS+vector; (4) bi-temporal edge invalidation as the queryable form of our supersede discipline. We de-scope the community/summarization tier and the default cross-encoder. Two hard caveats: keep the multi-LLM-call extraction off the hot path (background, like our sync engine), and route extraction through in-cluster llama-cpp / filter is_sensitive per ADR-0003.

---

4. Mem0 / Mem0g — extraction-based, LLM-curated memory

Mem0 (arXiv 2504.19413) is a write-side memory curator, not a retrieval algorithm — it solves a different axis than our gated problem, and is complementary.

Two-phase pipeline. Extraction: on each new message pair, an LLM (fed an async conversation summary + the last ~10 messages) emits a set of concise "candidate facts." Update (the curation step): for each candidate fact, retrieve the top ~10 semantically-similar existing memories, then a function-calling LLM picks one of four ops — ADD (new), UPDATE (merge richer detail into an existing id, gated on information content), DELETE (a contradicted memory), NOOP. Net effect: the store self-deduplicates, self-merges, and self-supersedes instead of accumulating. Two LLM calls per write (extract + decide) + a vector search; async by default (off the user hot path); the read/search path is pure vector similarity with no LLM.

Mem0g (graph variant): a directed labeled entity graph (Alice –lives_in→ SF) on Neo4j; a conflict-detection + LLM update-resolver marks superseded relationships invalid rather than deleting them.

Published quality: on LOCOMO, Mem0 J=66.88 / Mem0g 68.44 beats OpenAI memory (52.90), A-Mem (48.38), LangMem (58.10), ties Zep (65.99), at ~1/15th the tokens of full-context; Mem0g specifically wins temporal reasoning. Reference latencies (gpt-4o-mini): search p95 ≈ 0.20s, total p95 ≈ 1.44s, vs full-context ≈ 17s.

Verdict for us

Adopt the curation loop as a separate, flagged subsystem — it does NOT move the ADR-0001 retrieval metric by itself (its search is vector-only, no lexical+graph fusion). The ADD/UPDATE/DELETE/NOOP loop is the highest-leverage idea Mem0 offers: it automates a discipline our own rules already mandate (every correction stored, supersede-don't-accumulate, tombstones) but currently leave to manual human effort. It is cheap to build against our existing Memory model + update_memory endpoint, runs async off the recall hot path, and respects the is_sensitive boundary. Hard guardrails required: never physically DELETE — supersede to a [SUPERSEDED] tombstone (importance ~0.3, per our convention); log every op; gate behind the non-sensitive filter. Keep extraction optional (our memories are already atomic, so usually only the single UPDATE-decision call is needed). Mem0g's "mark invalid, not delete" and triplet schema (source, relation, dest) are reusable ideas, but implemented on pgvector/Postgres, not Neo4j. Critically: isolate curation behind a flag so the benchmark measures retrieval quality independently of any curation behaviour change.

---

5. HippoRAG / HippoRAG 2 — Personalized PageRank over a concept graph

HippoRAG (NeurIPS 2024, arXiv 2405.14831) and HippoRAG 2 (ICML 2025, arXiv 2502.14802) are the strongest published evidence that a concept graph wins on multi-hop — precisely the query class ADR-0001 says the graph must beat lexical on.

Mechanism (hippocampal indexing analogy): LLM = neocortex; retrieval encoder = parahippocampal region (detects synonyms); open KG = hippocampal index. Offline: an LLM runs OpenIE on each passage → a schema-free KG of noun-phrase nodes joined by relation edges; the encoder adds synonym edges between phrase nodes with cosine > τ=0.8. Online, per query: (1) ONE LLM call does NER on the query; (2) the encoder links query entities to nearest KG nodes = seed nodes; (3) each seed weight is scaled by node specificity (|P_i|⁻¹, an IDF-like rare-phrase boost) and written into the Personalized PageRank reset vector; (4) PPR runs to convergence (damping 0.5); (5) the phrase-node probability vector scores passages. Multi-hop emerges because the random walk reaches passages sharing no query tokens — in one retrieval step instead of iterative retrieve-reason loops.

HippoRAG 2 makes passages first-class nodes (linked to their phrases by "contains" context edges), shifts linking to query→triple + LLM triple-filtering ("recognition memory"), and seeds all passage nodes by embedding similarity (small weight ~0.05) so dense and graph blend in one PPR. Net effect: a single PPR fuses lexical-ish phrase matching, dense passage similarity, and multi-hop traversal into one ranked list.

Published quality (passage recall@5): HippoRAG 2 beats the strongest 7B embedding baseline (NV-Embed-v2) on every multi-hop set — 2Wiki 90.4 vs 76.5 (+13.9), MuSiQue 74.7 vs 69.7 (+5.0), HotpotQA 96.3 vs 94.5 — and is the only structure-augmented method that doesn't regress simple QA (NQ 78.0).

Verdict for us

Adopt the idea (PPR spreading activation over our concept graph), not the framework. Two hard adaptations, both fitting our stack:

1. Drop the per-query LLM (v1 NER / v2 triple-filtering) — the only thing that would blow the hot-path budget — and seed PPR from our existing FTS top-k ∪ pgvector top-k, weighted by fused score × importance × node-specificity. This turns PPR into the fusion layer ADR-0001 wants, with zero added LLM latency.
2. Prefer a memory-node graph (memories as nodes, our typed Relationships as edges) over HippoRAG's phrase explosion (it turns 11.6k passages into ~92k nodes; at our scale that'd be ~43k phrase nodes). Leaner and native to ADR-0002's node/edge tables.

A reproducible PPR latency micro-benchmark on a 5,400-memory graph measured ~2 ms (memory-node graph, transition matrix cached) to ~21 ms (full phrase graph), ~105 ms even at 3× growth — PPR is not the bottleneck; the stock recipe's online LLM is (which we remove). Postgres can store the graph but has no native PPR (pgrouting = shortest-path only), so PPR is computed in Python over a cached scipy.sparse transition matrix loaded from the node/edge tables, rebuilt only on graph mutation. Caveat for the gate: our LLM-free seeding variant is not validated by the papers, and our 5.4k personal corpus is far smaller and less multi-hop-dense than their 90k-node Wikipedia graphs — so the benchmark must confirm the multi-hop win transfers.

---

6. Embedding model survey

Our content and expanded_keywords are short prose (capped ~500 chars), so a model's max-token limit is effectively a non-constraint — quality, dimensionality, and deploy feasibility decide.

Self-hostable (sentence-transformers on the GPU node, or GGUF via llama-cpp; pgvector stores the vector)

Model	Dim	Params / VRAM (fp16)	MTEB(en) avg	License
nomic-embed-text-v1.5	768 (Matryoshka 64–768)	0.1B / <1 GB	62.28	Apache-2.0
bge-base-en-v1.5	768	109M / ~0.5 GB	~63.5	MIT
bge-large-en-v1.5	1024	335M / ~1.3 GB	64.23	MIT
e5-large-v2	1024	0.3B / ~1.3 GB	~62.25	MIT
bge-m3	1024 dense (+sparse +ColBERT)	568M / ~1–2.4 GB	en ~59–60 (strong multiling/BEIR)	MIT
gte-Qwen2-1.5B-instruct	1536	1.5B / ~3.4 GB	67.16 (top of set)	Apache-2.0

Hosted (API call, NON-SENSITIVE memories only per ADR-0003)

Model	Dim	MTEB(en) avg	Price /1M tok	License
OpenAI text-embedding-3-small	1536 (Matryoshka→256)	62.3	$0.02	proprietary
OpenAI text-embedding-3-large	3072 (Matryoshka)	64.6	$0.13	proprietary
Voyage-3.5	1024 (+256/512/2048, int8/binary)	beats OpenAI-3-large ~7.5% on Voyage's eval	$0.06 (first 200M free)	proprietary
Voyage-3.5-lite	1024	beats OpenAI-3-large ~2–3.8%	$0.02–0.03	proprietary
Cohere embed-english-v3.0	1024 (native int8/binary)	~64.5	~$0.10 (sales-quoted)	proprietary

Implementation notes that matter. Use asymmetric prompting (query vs document): sentence-transformers encode_query/encode_document, always normalize_embeddings=True so pgvector cosine == dot product. e5-large-v2 requires manual "query: "/"passage: " prefixes or quality collapses; bge prepends a query instruction; gte-Qwen2 prepends a task instruction to queries only. Pick the dimension once — changing it later forces a full re-embed + HNSW rebuild.

Recommendation (one of each, quality-first)

• Local: BAAI/bge-large-en-v1.5 (1024-d, MIT) — best quality-per-complexity in the self-hostable set for short English memories: strong retrieval, ~1.3 GB VRAM (runs on CPU at ~100 ms), no trust_remote_code, mature ST support. The 512-token cap is irrelevant for our content. (gte-Qwen2-1.5B-instruct is the explicit upgrade candidate if the benchmark says bge-large leaves quality on the table; nomic is the fallback if a long context or sub-768 Matryoshka dims are ever wanted.)
• Hosted: Voyage-3.5 (1024-d) — highest measured retrieval quality of the hosted options, same 1024-d as the local pick so the pgvector column and fusion code are identical whether local or hosted (clean A/B), and our whole corpus embeds inside the free tier. Non-sensitive only; sensitive rows go to bge-large locally. (OpenAI text-embedding-3-small is the pragmatic fallback if no Voyage key.)

Prototype note: the prototype as built used bge-large-en-v1.5 (1024-d, local default, no API key in env). Production should adopt Voyage-3.5 (also 1024-d) for non-sensitive memories per ADR-0003, keeping bge-large as the sensitive-only / no-key fallback. Both 1024-d means the pgvector schema and fusion code are unchanged across the choice.

---

7. Fusion of lexical + dense + graph signals

Three retrieval families produce candidate lists per recall; one fusion function merges them.

Reciprocal Rank Fusion (RRF) — rank-based, scale-agnostic

Cormack/Clarke/Büttcher (SIGIR'09): score(d) = Σ_s 1/(k + rank_s(d)), summed over every signal the doc appears in. k is a smoothing constant — their sweep found k=60 near-optimal but uncritical (≤0.3% MAP swing across [10,100]; k=0 or k=500 costs 3–4%). A doc present in one list but absent from another contributes 1/(k+rank) where it fires and 0 elsewhere — so multi-signal agreement is rewarded and single-signal hits are penalized (the hybrid behaviour we want). Extends to N lists trivially (just sum), which makes a 3-way fuse a one-liner. Weighted RRF: Σ_s w_s/(k + rank_s(d)) — bias a stronger signal, no pre-normalization.

Weighted score fusion / convex combination (CC) — score-based, needs normalization

Bruch et al. (arXiv 2210.11934, TOIS 2023): f = α·φ(semantic) + (1-α)·φ(lexical) with theoretical min-max normalization (TM2C2): cosine min = -1, BM25 min = 0, per-query max — stable across queries. Findings: CC/TM2C2 beats RRF on nDCG and recall in- and out-of-domain (RRF ~3.86% lower nDCG@10 in one replication); Weaviate switched its default from rankedFusion to relativeScoreFusion (min-max CC) for ~6% recall on FIQA. CC is sample-efficient (α tunes from a small labeled set) but requires calibratable scores.

Folding in graph hits

Build a graph candidate list, then feed it to the fuser as just another ranked list: seed (match the query to concept nodes via the same FTS + dense over node labels) → traverse (1–2 hops to reachable memories) → score each reachable memory. Three documented scorings: hop-decay Σ_paths β^hops (β≈0.5–0.7); Personalized PageRank seeded on matched nodes (HippoRAG); or node-degree priority (GraphRAG local search). The Calibrated-Fusion-for-Graph-Vector paper (arXiv 2603.28886) is explicit: naive graph+vector fusion fails on scale incompatibility, so convert graph traversal into a probability-like normalized score before fusing. Crucial consequence: the graph list is sparse (often a handful of memories, sometimes zero). Under RRF that's handled automatically; under CC you must explicitly treat "absent" as the theoretical min or the missing-modality term silently biases the sum.

Cross-encoder re-rank — a separate stage-2, not a fusion function

Retrieve top-N each → fuse → take fused top ~20–30 → score each (query, memory) pair jointly with a cross-encoder (e.g. bge-reranker-v2-m3) → re-sort. Reported lift +5 to +15 nDCG@10 on BEIR/MTEB; cost scales with pair count so it is only ever a small-candidate-set stage.

Recommendation

Weighted RRF over three lists (FTS, dense, graph), k=60, equal weights to start, with importance applied as a deterministic post-fusion prior and a cross-encoder as an optional, benchmark-gated stage-2. RRF is the right default because we fuse three incompatible scales, one of them sparse/often-empty; it is near-parameter-free; and it collapses to exactly today's lexical ordering when dense/graph are empty (the SQLite graceful-degrade path). But because adoption is quality-gated, the benchmark must also run CC/TM2C2 as a challenger — the literature is consistent that CC edges RRF on quality when scores are calibratable. (See the benchmark report for which won on our eval set.)

---

8. Concept-graph construction from memories

Turning flat Memory rows into nodes (concepts/entities) + typed directed edges + memory→concept "mentions" edges. Three extraction families:

• (A) Open LLM triple extraction (schema-free) — prompt an LLM to emit [subject, relation, object] triples. High recall, but relation labels proliferate ("prefers"/"likes"/"favors"), so it requires downstream canonicalization. GraphRAG is the canonical implementation (extract + gleaning + cross-chunk entity summarization).
• (B) Schema-guided — constrain to a fixed ontology. Cleaner, but a fixed schema misses surprises in a heterogeneous personal corpus. EDC (Zhang & Soh, EMNLP 2024) bridges the two: extract open triples → define (LLM writes a one-sentence definition per distinct relation) → canonicalize (embed definitions, retrieve nearest existing relations, LLM verifies map-vs-add). Two modes: target-alignment (fixed schema) and self-canonicalization (grow schema dynamically).
• (C) Entity resolution / canonicalization (the dedup problem — "Svelte"/"SvelteKit"/"svelte framework" are one node): cluster-then-refine on the aggregated graph — embed every surface string, cluster by cosine (HDBSCAN / connected-components over a threshold), optional LLM-as-judge per cluster. KGGEN (arXiv 2502.09956) does iterative LLM-guided clustering; Graphiti uses MinHash+LSH fast-path with LLM fallback. Cost scales with distinct entities (low thousands), not with memory count.
• (D) Lightweight non-LLM — spaCy NER + noun-chunks + co-occurrence edges, or ReLiK (Sapienza, ACL 2024) for typed relations on CPU at up to 40× LLM speed, zero per-doc LLM cost. The natural ablation baseline and sqlite-only fallback.

The tractable recipe for our corpus. Measured: 5,452 non-sensitive memories ≈ 683K content tokens total — tiny. At ~125 content-tokens/memory, ~570 memories pack into one 100K-token request, so the entire corpus extracts in ~10–25 batched LLM calls, not 5,452 sequential calls. Pipeline: (1) batch-extract open triples (each memory tagged with its memory_id so triples map back), parallelized — LangExtract / KGGEN style; (2) aggregate + canonicalize globally once (embed distinct entities, cluster, LLM-judge only ambiguous clusters — tens of calls); (3) optionally one batched LLM "define relations" pass for EDC-style relation canonicalization. Total budget: low tens of calls, minutes of wall-clock, a few dollars hosted or one GPU-node llama-cpp session. Canonicalization quality (the similarity threshold / cluster granularity) is where this lives or dies and must be tuned against held-out data, not eyeballed. Write-time / Graphiti-style per-memory extraction is for incremental updates only — the wrong tool for the one-shot backfill.

---

9. Vector storage in Postgres (production substrate)

pgvector is a proven capability on our exact CNPG cluster (Immich already does vector search there, and claude-memory-mcp is already a tenant of the shared pg-cluster-rw.dbaas behind PgBouncer) — zero new infrastructure, reuse-before-building satisfied.

• HNSW (recommended default): USING hnsw (embedding halfvec_cosine_ops) WITH (m=16, ef_construction=64); query knob SET hnsw.ef_search (default 40). Best speed-recall tradeoff; buildable on an empty table; graph in RAM. IVFFlat is rejected (must be built after data exists — an empty-table footgun — and has a lower recall ceiling).
• halfvec (fp16, 2 bytes/dim) halves index size at ~no recall loss; indexable ≤4000 dims. 768-d halfvec = 1536 bytes/row; at our scale total embedding storage is single-digit MB.
• Filtered ANN: we always filter deleted_at IS NULL (often category). Post-filtering can under-fill top-k; enable hnsw.iterative_scan='relaxed_order', and always add a tie-breaker (, id) since approximate indexes give non-deterministic order.
• Hybrid in one query: each retriever is a CTE producing a per-ranker rank; fuse with RRF via FULL OUTER JOIN on memory id — no score calibration needed across the incomparable ts_rank and cosine scales.
• pgvectorscale / StreamingDiskANN (bounded-RAM disk graph, SBQ compression) is deferred — Rust/pgrx must be compiled into the operand image, and it only earns its keep above ~1–5M vectors. Our corpus is orders of magnitude below that.
• PgBouncer gotcha: per-query GUCs (hnsw.ef_search) must be SET LOCAL inside the recall transaction, not session-level, under transaction pooling.

Not for the prototype (the prototype uses an in-process numpy index); this is the production adoption path if the benchmark clears the gate — an additive Alembic migration (one nullable halfvec(1024) column + HNSW index) plus a Terraform change to the CNPG stack.

---

10. Evaluation methodology

A retrieval test collection = corpus + query set + qrels (relevance judgments). For each query, call recall, take the ordered list of returned memory ids, score against qrels — measuring the retriever in isolation, exactly what ADR-0001's gate needs.

Metrics (compute all; pick one primary):

• Recall@k — "did we surface the right memory at all?" The hot-path metric (auto-recall injects top-N; if the memory isn't in top-k it can't help). Report @5/@10/@20/@30.
• nDCG@k — graded + position-aware; the best single summary (BEIR standard is nDCG@10). Headline quality number for the gate.
• MRR — only the first hit matters; relevant for the exact-lookup stratum.
• MAP — broad binary recall+precision blend; secondary, stable for significance tests.

Stratification (the ADR-0001 hypothesis-targeted design): exact/lexical (FTS already wins — the regression guard); paraphrase/semantic (disjoint vocabulary — the value-of-embeddings test); multi-hop (≥2 memories or a concept link — the graph test).

Qrels generation (the LongMemEval pipeline, inverted for memories): sample seed memories stratified by category + importance → an LLM generates exact / paraphrase / multi-hop queries → label relevant ids, with pooling (union the top-k of every arm, TREC-style) and an LLM-judge on the UMBRELA 0–3 scale. Separate the generator model from the judge model to avoid self-preference leakage. Hand-verify a ≥15–20% sample (oversample multi-hop) and require Cohen's κ(LLM, human) ≥ ~0.6 before trusting auto-labels; always hand-author multi-hop relevant-id sets.

Pitfalls with standard mitigations (all baked into the protocol): LLM judges are systematically lenient (κ gate); "holes" (new arms retrieve unjudged docs — must pool all arms before judging, else the gate is rigged against semantic/hybrid); generator-as-judge leakage (model separation); too-easy self-generated queries (check paraphrase shares no content tokens); adversarial/unanswerable queries have no relevant id and must be kept out of the ranked metrics (mixing them corrupted the disputed Zep-vs-mem0 LOCOMO comparison — 84%→58%).

Sizing: Voorhees & Buckley (2002) — ≥25 topics is the floor, 50 yield reliable rankings, and a ~5–6% absolute gap at n=50 is needed for 95% confidence the ordering holds on a different query set. Since the gate is per stratum, each stratum wants its own ~50 queries.

Honest note on what we actually built: our eval set is 119 queries (40 exact / 40 paraphrase / 39 multihop) — just below the ~50/stratum ideal, and qrels were LLM-generated with lighter hand-verification than the full protocol prescribes. This is a real limitation, tracked in the benchmark report.

---

11. Synthesis — what we borrow, from whom

Source	Borrowed mechanism	Re-implemented as	Adopted?
LightRAG	Incremental set-union graph merge; dual-level retrieve	Native node/edge tables, no AGE; FTS+dense+graph fuse	Idea only
LazyGraphRAG	Defer LLM cost; index-time work ∝ new content	Store-time extraction off hot path	Principle
Zep / Graphiti	Episodic/semantic split; 3-signal RRF read path; bi-temporal invalidation	Memory rows + Postgres node/edge tables; pgvector+FTS+CTE	Blueprint
Mem0	ADD/UPDATE/DELETE/NOOP write-side curation	Flagged async curator over existing update_memory	Complementary, flagged
HippoRAG 2	PPR spreading activation for multi-hop	LLM-free FTS+vector-seeded PPR over memory-node graph (phase 2, gated)	Idea only
Bruch et al. / Cormack	RRF default + CC/TM2C2 challenger	Weighted RRF k=60, post-fusion importance prior	Direct
EDC / KGGEN	Open-extract → define → canonicalize globally	Batched extraction + embedding-cluster canonicalization	Direct
pgvector / Supabase	HNSW + halfvec + RRF hybrid in one SQL query	Additive migration to CNPG (production only)	Production design
LongMemEval / UMBRELA / Voorhees	Stratified LLM-qrels + pooling + κ gate	Our exact/paraphrase/multi-hop eval	Direct

The through-line: a memory-node concept graph, dense pgvector embeddings, and the existing lexical FTS, fused with weighted RRF, with all LLM work pushed to store time — sized for an append-heavy personal store and gated on a benchmark that beats FTS.

---

Sources

GraphRAG family

• Edge et al., "From Local to Global: A Graph RAG Approach…" (Microsoft, 2024) — arXiv 2404.16130
• Microsoft GraphRAG incremental-indexing design — github.com/microsoft/graphrag/issues/741; GraphRAG 1.0 blog (microsoft.com/en-us/research/blog/moving-to-graphrag-1-0…)
• Guo et al., "LightRAG: Simple and Fast RAG" (HKUDS, EMNLP 2025) — arXiv 2410.05779; github.com/HKUDS/LightRAG (incl. issue #2122, PG+AGE concurrency)
• nano-graphrag — github.com/gusye1234/nano-graphrag
• LazyGraphRAG — microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/

Temporal KG memory

• Rasmussen et al., "Zep: A Temporal Knowledge Graph Architecture for Agent Memory" (2025) — arXiv 2501.13956
• Graphiti — github.com/getzep/graphiti; Neo4j writeup (neo4j.com/blog/developer/graphiti-knowledge-graph-memory/)

Extraction-based memory

• "Mem0: Building Production-Ready AI Agents…" (2025) — arXiv 2504.19413; github.com/mem0ai/mem0 (configs/prompts.py)

Graph-PPR retrieval

• Gutiérrez et al., "HippoRAG" (NeurIPS 2024) — arXiv 2405.14831
• "From RAG to Memory" = HippoRAG 2 (ICML 2025) — arXiv 2502.14802; github.com/OSU-NLP-Group/HippoRAG

Embeddings

• bge-large-en-v1.5 — huggingface.co/BAAI/bge-large-en-v1.5; gte-Qwen2-1.5B — huggingface.co/Alibaba-NLP/gte-Qwen2-1.5B-instruct; nomic — huggingface.co/nomic-ai/nomic-embed-text-v1.5; bge-m3 — huggingface.co/BAAI/bge-m3; e5-large-v2 — huggingface.co/intfloat/e5-large-v2
• Voyage-3/3.5 — blog.voyageai.com/2024/09/18/voyage-3/; docs.voyageai.com/docs/pricing
• OpenAI text-embedding-3 — developers.openai.com/api/docs/guides/embeddings; Cohere embed v3 — docs.cohere.com/docs/cohere-embed

Fusion

• Cormack, Clarke, Büttcher (SIGIR'09) — cormack.uwaterloo.ca/cormacksigir09-rrf.pdf
• Bruch et al., "An Analysis of Fusion Functions for Hybrid Retrieval" (TOIS 2023) — arXiv 2210.11934
• Elastic weighted RRF; Weaviate hybrid-search fusion algorithms; "Calibrated Fusion for Heterogeneous Graph-Vector Retrieval" — arXiv 2603.28886
• bge-reranker — huggingface.co/BAAI/bge-reranker-base

Concept-graph construction

• Zhang & Soh, "Extract-Define-Canonicalize" (EDC, EMNLP 2024) — arXiv 2404.03868; github.com/clear-nus/edc
• KGGEN — arXiv 2502.09956; ReLiK (ACL 2024) — arXiv 2408.00103; LightKGG — arXiv 2510.23341; Google LangExtract — github.com/google/langextract

Postgres vector storage

• pgvector — github.com/pgvector/pgvector; pgvectorscale — github.com/timescale/pgvectorscale; CNPG image-volume extensions — cloudnative-pg.io/docs/devel/imagevolume_extensions/; Supabase hybrid search — supabase.com/docs/guides/ai/hybrid-search
• This cluster: infra/docs/architecture/databases.md (claude-memory-mcp is a CNPG tenant); this repo: migrations/versions/001_initial_schema.py, src/claude_memory/api/app.py

Evaluation

• LoCoMo — arXiv 2402.17753; LongMemEval — arXiv 2410.10813; UMBRELA — arXiv 2406.06519; "Judging the Judges" / LLMJudge — arXiv 2502.13908; Voorhees & Buckley "Topic Set Size" (2002); Buckley & Voorhees "Bias and the Limits of Pooling"; BEIR — arXiv 2104.08663