Agent Memory Became Infrastructure. The Benchmarks Did Not Catch Up.

Seven memory products shipped in sixteen days. That is not a trend line, it is a category forming in public. Databricks and Autopoetic opened on 20 October. Alibaba Cloud followed on 2 November, Hindsight on 3 November, then Redis, MongoDB with Google and PhoenixAI all on 4 November.

Each one is commercially sensible on its own. Taken together they describe something more useful: the industry has quietly conceded that memory is infrastructure, not a feature you bolt onto a prompt. The harder question, and the one almost nobody is answering honestly, is how you tell whether a given memory system actually works.

What shipped, briefly

The range of approaches is the interesting part, because it shows there is no settled architecture.

Databricks launched Agent Memory on Agent Bricks in beta, built on an architecture it calls MemCurrent. It combines three access patterns: plugins that log and load context automatically, tools the agent calls on demand, and an offline pipeline that consolidates memory in the background. It runs on Lakebase, Databricks' serverless Postgres. Internally the company reports the architecture tripled agent pass rate on enterprise tasks while using 31 percent fewer tokens.^[1]

Redis expanded Iris into six capabilities, four of them generally available today: Intent Router, Agent Playbooks, Agent Memory and Context Gateway. The two in beta are Lens, for tracing an outcome back to the skill or memory that produced it, and Agent Mesh for agent-to-agent messaging. Agent Memory builds records from user actions as well as documents, organises them hierarchically, and offers inspection, audit and provenance. It can also deliberately forget what it should not keep.^[2]

MongoDB and Google shipped an ADK integration with two services. MongoDBSessionService persists sessions, events and scoped state with optimistic concurrency control. MongoDBMemoryService ingests session events and recalls them through keyword matching, with ingestion made idempotent so a re-added session does not duplicate memories. The pitch is architectural rather than algorithmic: keep agent state in the database you already run, rather than standing up a second one.^[3]

PhoenixAI launched Contextbase in private preview, holding semantics, knowledge, memory, traces and evaluations in one governed store on the same cluster as business data. Notably it ships with an application, ObserveStack, for tracing and scoring agent runs against that same store.^[4]

Alibaba Cloud's Memory on Model Studio targets personal agents, assembling a structured user profile from preferences, plans and prior commitments. Developers define memory rules and profile fields, and the console tracks how a memory was formed or changed. It connects through API, MCP and a CLI to environments including Claude Code and Codex.^[5]

Two smaller entries round it out. Autopoetic shipped a standalone long-term memory service built as a hash-chained graph of entities, relationships and events, with no external dependencies and zero inference cost for retrieval itself.^[6] Hindsight took a different angle entirely, publishing small decision models for the judgments inside a memory system: is this fact new, does it contradict what we believe, which memories matter here. It reports a median of 167 milliseconds per decision request against one to seven seconds for a typical LLM call.^[7]

The architecture argument is not settled

Read the seven together and three genuinely different theories of memory appear.

The first is memory as database. Databricks, MongoDB and PhoenixAI all place memory in the storage layer and treat retrieval as a query problem. This has the advantage of mature operational properties: concurrency control, backups, permissions, cost attribution, all inherited from systems that already solve them.

The second is memory as pipeline. Redis and Databricks both emphasise background consolidation, where sessions are distilled into durable memory over time. This acknowledges something important: the raw transcript is not memory. It is evidence, and distilling it is a separate job that should not happen inline with a user waiting.

The third is memory as judgment. Hindsight's contribution is the least glamorous and possibly the most useful. A memory system makes hundreds of small binary calls per session, and answering each with a frontier model is absurd economics. Splitting the judgment into a cheap specialised model and leaving generation to the large one is the kind of engineering decision that shows a team has run this at volume.

None of these is wrong. The fact that credible companies chose three incompatible shapes in the same fortnight tells you the field is pre-standard, and that anyone claiming a settled answer is selling rather than reporting.

The measurement problem is the real story

Here is the part worth reading twice.

Alibaba Cloud's launch included benchmark claims: first place on four factual-memory metrics on HaluMem-medium with a question-answering score of 85.15 percent against 81.71 percent for Zep, and an overall 56.82 on PersonaMem-v2, first among eight systems.^[5] Those are company numbers. The comparison figures for other systems, Alibaba says, come from the OmniMemEval evaluation. None of it has been independently reproduced.

That is normal marketing, and it is not the interesting part. The interesting part is what Alibaba's own engineering team published separately, about its own open-source Qwen Memory Agent.

On a six-persona, 24-query synthetic offline fixture, that system achieved context recall of 1.000 and staleness of 0.000 across token budgets from 8 to 64. Perfect on both. Then a separate ten-scenario live evaluation, which required the decision outcome, the memory-store state and whether the agent consulted memory before acting all to pass, achieved 0.60.

The builder published both numbers and stated the conclusion directly: retrieving the right fact does not guarantee an agent will use it correctly.^[8]

That single sentence is the most valuable thing published in this category all month. It separates two questions that vendor benchmarks routinely blur. Did the system retrieve the correct memory. Did the agent then behave correctly because of it. The first is a search problem with good benchmarks. The second is the only one a customer experiences, and it is barely measured at all.

A system can score near-perfectly on retrieval and still fail the task six times in ten. If you buy on the benchmark, you are buying the half that was easier to measure.

What the numbers do not tell you

Three gaps are worth naming before anyone signs a contract.

Retrieval metrics assume a correct memory exists and is findable. They do not test the common failure where the right fact was never stored, because the moment it was mentioned did not look important at the time. The Databricks pipeline design addresses this by consolidating sessions offline, which helps, but no public benchmark I have seen measures whether a system captured the thing it should have captured.

Staleness is measured less often than accuracy. Human memory handles superseded facts poorly too, which is why good systems are explicit about retirement. Alibaba's Qwen Memory Agent retires superseded facts rather than letting them accumulate, using exact subject-type matching alongside semantic similarity.^[8] That design detail matters more to a working agent than a few points of recall, and it rarely appears in a comparison table.

Provenance is the third gap, and the one that will decide enterprise adoption. Redis Lens, PhoenixAI's tracing and Alibaba's change tracking all point at the same requirement: when an agent acts on a remembered fact, can you reconstruct where that fact came from, when it changed and who or what changed it. An agent that cannot answer this is an audit liability regardless of its accuracy.

The cost question has moved

There is a quieter commercial shift underneath the launches. Every one of these systems is positioned as a way to spend fewer tokens, not more.

Databricks claims 31 percent fewer tokens alongside a tripled pass rate. Redis caches which tools solve a given problem to cut token use, and its Playbooks reuse stable answers rather than regenerating them. Hindsight's entire premise is that a cheap decision model should make the small judgments so a small LLM can do the writing. Its report claims a cheap model with the decision layer scoring within two points of a frontier model at 19 times less cost.^[7]

The logic is now widely accepted: paying a large model to re-read the same context in every session is the expensive way to be forgetful. But be careful with the arithmetic. Cost claims are usually measured against a naive baseline of stuffing everything into the window every time. Against a well-built retrieval system, the saving shrinks. And a memory system adds its own costs: storage, consolidation compute, an extra decision model, and the engineering to run all of it.

The honest framing is that memory moves cost from inference to infrastructure and maintenance. Whether that is cheaper depends on your traffic shape, not on the vendor's benchmark.

What to actually test

A short list, because this is a decision most teams will make this quarter.

Build a fixture from your own domain rather than a public benchmark. Fifty real interactions and twenty questions with known correct answers will tell you more than any leaderboard, because it will include the ambiguities and near-duplicates your business actually produces.

Test refusal. Ask something the system should not know, and see whether it says so or fabricates a plausible recollection. This is the failure that damages customer relationships, and it is almost never in a vendor demo.

Test supersession deliberately. Tell the system one thing, then contradict it a week later, and check which version an agent acts on. Then test the harder variant, where the contradiction is embedded in a long session rather than stated plainly.

Test the path from memory to action, not just the recall. Alibaba's 0.60 is the number to keep in mind here. Ask whether the agent changed its behaviour, not whether the fact was found.

Require a provenance trace. Pick any decision the agent made and reconstruct which memories informed it. If that takes an engineer an afternoon, the system is not ready for regulated work.

What memory becoming infrastructure means

The strongest signal in these sixteen days is not any individual product. It is that memory moved from something every agent team built for itself to something you buy, configure and govern.

That is good news for anyone who has spent months rebuilding the same retrieval loop. It also raises the bar. Once memory is a managed service with a benchmark and a price, the question stops being whether you have memory and starts being whether your memory is correct, current and attributable.

Which brings back the 1.000 and the 0.60. A retrieval score of 1.000 is a statement about a search index. A task success rate of 0.60 is a statement about the product. The gap between them is the work, and no vendor sells that part.