Memory benchmark
Agent memory that returns only what is true now.
Old facts, other users’ facts and facts from the wrong project stay out. Your agent reads less and gets it right.
77/77
PrecisionMemBench cases. Mem0 and Zep: 9.
43/43
Cases where memory must pick the right facts
11.1×
Fewer tokens to find the current rule
Every system tested.
- Proofable77 of 77
- Tenure77 of 77
- Hindsight66 of 77
- Supermemory15 of 77
- Mem0, Zep, Cognee9 of 77
| System | Result | Run by | Source |
|---|---|---|---|
| ProofableProofable, on Vectorize’s open toolkit | 77 of 77 | Proofable, on Vectorize’s open toolkit | Public proof |
| TenureTenureHQ, author’s own product | 77 of 77 | TenureHQ, author’s own product | Author table |
| HindsightVectorize, same toolkit | 66 of 77 | Vectorize, same toolkit | Published run |
| SupermemoryTenureHQ, the benchmark’s author | 15 of 77 | TenureHQ, the benchmark’s author | Author table |
| Mem0, Zep, CogneeTenureHQ, the benchmark’s author | 9 of 77 | TenureHQ, the benchmark’s author | Author table |
Proofable and Hindsight ran on the same open toolkit. The other rows are the benchmark author’s own runs, and Tenure is the author’s own product.
Against the alternatives.
PrecisionMemBench cases passed
Proofable77 of 77Agent on its own–Mem0 or Zep9 of 77Cases that need the right facts picked
Proofable43 of 43Agent on its own–Mem0 or Zep0 of 43Precision (share of returned facts that were right)
Proofable0.98Agent on its own–Mem0 or Zep0.06Recall (share of right facts returned)
Proofable1.00Agent on its own–Mem0 or Zep0.66 to 0.99
Mem0 and Zep figures are the PrecisionMemBench author’s own runs. High recall with low precision means the right fact came back buried among wrong ones.
Finding the current rule costs 11.1× fewer tokens.
Tokens, on a 3,621-record vault
Proofable43,169Searching every record479,623Difference11.1× fewerTime for the same questions
Proofable66 sSearching every record178 sDifference2.7× fasterModel calls
Proofable6Searching every record10Difference4 fewer
Same agent and model on both, on the questions both answered correctly. Proofable checks whether a fact is still current before reading it, so it skips everything outdated.
Check the results yourself.
Questions
What does PrecisionMemBench test?
It stores 35 facts for one user, some replaced by newer ones and one belonging to someone else, then asks 77 questions and checks exactly which facts come back. A stale fact, a fact from the wrong area or another user’s fact fails the case. 34 of the questions should return nothing.
How is this different from Mem0 or Zep?
Proofable stores each fact with who it belongs to, what it is about and whether it is still current, so wrong facts are filtered out before anything is ranked. On the benchmark author’s own table Mem0 and Zep pass 9 of 77, and none of the 43 cases that need the right facts picked.
Is 77 of 77 the top score?
It ties the benchmark author’s own system, Tenure. On the same open toolkit the next published memory, Hindsight, passes 66.
Verify this run yourself
Put the same checks on your agents.
Connect an agent. Every action it takes is checked against your rules and leaves a receipt you can show anyone.