Press Delete to remove.
PrecisionMemBench checks whether a memory returns the current fact and nothing stale. Proofable passed all 77 cases, including all 43 that need the memory to pick the right facts, with precision 0.98 and recall 1.00. Hindsight passed 66 of 77 on the same toolkit. Three full runs on this code scored 77, 77 and 74; every run is published with this record.
Cases passed
Active cases
Full runs
tenurehq/precisionMemBench, single-turn split, as ported by vectorize-io/agent-memory-benchmark
harness 28b644bd4dd653c0207c7507280439390c829155 (2026-11-02); fixtures pinned to upstream b95d6ab
outputs/precisionmembench/proofable/retrieval/single-turn.json (the harness result file, per-case records included)
Proofable typed memory behind the harness interface: each belief stored as one private record with the dataset's scope tags plus state and name tags; retrieval applies the dataset's filter as exact tag matching, then gemini-2.5-flash selects among the filtered candidates and may select none
label extraction at ingest (35 calls) and candidate selection at retrieval (77 calls); the harness scores retrieval mode with no model
src/memory_bench/memory/proofable.py in the Agent Memory Benchmark harness (branch proofable-provider)
No rank is claimed. On this toolkit the other published row is Hindsight at 66 of 77. The benchmark author’s own table (its own code) lists its product Tenure at 77 of 77.
vectorize-io/agent-memory-benchmark outputs/precisionmembench/hindsight/retrieval/single-turn.json.gz (hindsight-cloud, same harness)
uv run amb run --dataset precisionmembench --split single-turn --memory proofable --mode retrieval
PROOFABLE_ACCESS_KEY (the bench profile), PROOFABLE_MEMORY_MODEL=gemini-2.5-flash, a Gemini client configuration (Vertex AI or an API key)
https://docs.proofable.me/evidence/precisionmembench/2026-11-10-77/ (per-case result files for all three runs, the adapter, SHA256SUMS)
Each full run re-ingests every belief, and the label model (gemini-2.5-flash) tags names a little differently each time. The run that scored 74 missed two aliases and kept one extra belief.