The attack that taught me to take agent memory seriously arrived as product data. On an ecommerce agent solution I worked on for a Microsoft customer, someone tried to poison the system’s knowledge of its own catalog: bad product data, seeded through channels the pipeline ingested on schedule, some of which made it into the RAG corpus feeding the agents. Nothing at ingestion flagged it. Retrieval served it as obediently as any honest document. What finally surfaced the attack was production, where people watching a live system noticed answers that smelled wrong and traced them backward to the planted data.
I’ve been thinking about that incident again because I’ve started moving my own working memory into Hermes, the local-first persistent-memory agent Nous Research shipped in February. Digital brains are becoming standard equipment: Hermes, the memory layers wired into coding agents, the llm-wiki pattern Andrej Karpathy sketched for agent-maintained knowledge bases. The stack all of them run on is missing a subsystem, and the missing subsystem is decontamination. Memory products handle staleness, compaction, and thematic discovery; security tools screen what gets written; neither reaches context that has already been internalized. A personal brain also lacks the thing that saved us on that ecommerce system: a production floor with a support queue and analysts positioned to notice when the answers drift. Downstream of a poisoned memory sits exactly one detector, and it’s the model that got fooled.
None of this threat is hypothetical. In September 2024, Johann Rehberger demonstrated SpAIware, a prompt injection that wrote itself into ChatGPT’s long-term memory and quietly exfiltrated every later conversation until the memory was manually wiped. The research side has kept pace. MINJA corrupts an agent’s long-term memory through ordinary queries, no privileged access required; in its authors’ tests the poison landed in memory more than 95 percent of the time, and later victim queries produced the corrupted behavior about three-quarters of the time. AgentPoison induces backdoor behavior in retrieval-backed agents while poisoning under a tenth of a percent of the store. These attacks persist across sessions and ride channels you can’t close, because ingesting untrusted content is what a digital brain is for.
The economist’s frame for this is George Akerlof’s market for lemons. Akerlof showed in 1970 that when buyers can’t observe quality, bad goods trade at the same price as good ones; the used-car buyer can’t price the rust they can’t see, so the lemon sells for what the honest car sells for, and the honest cars leave the market. A memory store is a lemons market with a single, perfectly credulous buyer. The moment an entry is written, its provenance is flattened into metadata nothing consults at retrieval time. A poisoned memory and an honest one present identically to the model consuming them, so every downstream process prices them identically: retrieval ranks by relevance, compaction summarizes by theme, and nothing asks where anything came from. Akerlof also cataloged the fixes real markets evolved (certification, warranties, brand names), and it turns out the whole defense landscape maps onto that list almost exactly.
What Your Memory System Believes
The memory-subsystem products are good at what they do, and what they do is freshness. Mem0’s original pipeline had an LLM classify every incoming fact as an add, an update, a delete, or a no-op; its 2026 rewrite keeps every fact it ingests, moves the arbitration to retrieval time, and decays entries whose relevance fades. Zep maintains a bi-temporal knowledge graph in which a contradicted fact gets its edge marked invalid, timestamped, and preserved for history rather than deleted. Letta, descended from the MemGPT work at Berkeley, goes furthest of all: a sleep-time agent runs in the background while the primary agent idles, editing its memory, resolving contradictions between stored facts, and abstracting general patterns out of specific experiences. Hermes sits squarely in this tradition. Its pitch is “the agent that grows with you,” one that treats every task as a chance to learn something reusable, which is the property I chose it for and the property that compounds a poisoned lesson just as faithfully.
Every piece of that machinery assumes memories are honest but stale: wrong because the world changed, correctable by trusting newer information. That’s a reasonable assumption, and an adversary inverts it. The newest, most confident, most semantically connected entry in the store may be the attack, and a hygiene loop built to trust recency and coherence will promote it rather than flag it.
The security world, meanwhile, runs in a parallel lane. OWASP gave memory and context poisoning its own slot in its 2026 Top 10 for Agentic Applications, and this June released an early incubator project, Agent Memory Guard, an open-source layer that sits between agent and memory store and dispositions every read and write: allow, redact, quarantine, or block. Academia has post-hoc auditors like MemAudit, which traces a bad output back through causal attribution to the memory entries that produced it. This is good work. Almost none of it has crossed into the memory products, whose documentation talks about token budgets and recall benchmarks while the security lane talks about adversaries; the two literatures share a subject and barely cite each other.
The Rendering Plant
There’s a precedent for maintenance machinery amplifying the contamination it was never designed to see, and it’s a grim one: BSE. British cattle farming in the 1980s rendered slaughterhouse byproducts into protein feed, an efficiency loop that recycled the herd back into the herd. When misfolded prions entered that loop, the rendering plants didn’t filter them out; they distributed them to every trough in the country. The step that made the system efficient became the amplification vector, and the contamination outlived every animal that originally carried it.
Sleep-time compute is a rendering plant. A background agent whose job is to abstract patterns from specific memories will, handed a poisoned memory, abstract a poisoned pattern, and that derived belief survives even if you later locate and delete the original entry. Compaction behaves the same way, laundering a bad entry into a clean-looking digest with no quarantinable source attached. The maintenance loop is where an infection stops being an entry and becomes a belief.
Three Seams
So where does decontamination actually go? I count three seams, and only two of them exist as shipping software.
The write boundary comes first. Provenance tags on every memory, trust scoring on sources, quarantine for content arriving through untrusted channels. This is Agent Memory Guard’s seam and the top of OWASP’s defense stack, and it’s the cheapest place to fight, since a memory stopped at the door costs nothing downstream. In Akerlof’s terms it’s certification at the point of sale. If you run a persistent-memory agent and screen nothing at write time, fix that before you finish this post.
Forensics is the second seam. Snapshot the store at known-good states, roll back when poisoning is confirmed, use causal auditors to trace a bad answer to the entries behind it. Memory Guard ships snapshots; MemAudit does the attribution. The limitation lives in the trigger: forensics starts after you know something happened, and the ecommerce incident is a story about how expensive that knowledge is. It took live traffic and a floor of humans to raise the alarm. Rollback is also a blunt instrument that discards good memories alongside bad ones; for a personal brain, that’s amnesia as the price of disinfection.
The third seam is the one nobody ships: periodic review of the whole accumulated store by a model meaningfully more capable than the one that runs on it daily. Call it a juice cleanse for the digital brain. I’m aware the wellness version is theater; your liver does its own filtering, and no amount of pressed celery improves it. The digital version earns the name, because here the filters really are undersized for the toxins. A local model that accepted a poisoned memory at write time can’t serve as the auditor of record for its own beliefs, and today’s products run their sanitization at exactly that intelligence grade. Capability asymmetry is what does the work: a frontier model reading the store cold, unattached to any belief in it, hunting for entries and summaries that don’t cohere with the rest, the way a strong engineer new to a codebase spots the comment that contradicts the code.
Sending your entire brain to a frontier lab raises the obvious question of what the lab keeps, and the answer depends on which seam you use. Consumer chat apps are the wrong one: ChatGPT and Gemini train on consumer conversations by default, Gemini keeps them for eighteen months out of the box, and Claude’s consumer tier stretches retention to five years if you accept its training option. The API tier is a different contract: Anthropic deletes API inputs and outputs within thirty days and never trains on them, OpenAI’s API doesn’t train by default, and both offer zero-data-retention agreements to customers who qualify, though Anthropic currently keeps its newest frontier models out of those agreements. The strongest retention terms right now sit on Azure, where the hosted frontier models can run with zero data retention once the abuse-monitoring exemption is approved. Run the audit over the API, never through the app, and get the terms in writing if your data warrants it. None of that touches sovereignty, which is the harder residue. For some brains (medical notes, a company’s institutional memory, anything jurisdictionally sensitive) processing on a foreign hyperscaler is disqualifying whatever the retention clause says, and the fallback is an auditor you host yourself. The open-weight models big enough to stand in for the frontier stopped fitting commodity hardware a while ago, so hosting yourself now means a cloud GPU server in a region you choose rather than a second GPU under your desk. Weaker than the frontier, yes, but the mechanism only needs a gap between the auditor and the daily driver, and that gap is rentable by the hour.
The strongest case against the cleanse comes from the boundary camp: write-time screening is deterministic, cheap, and preventive; a periodic audit is none of those things, and it widens the attack surface besides, since a sufficiently clever memory could be crafted to fool or even target the auditor. Most of that lands. Boundary screening should run on every persistent-memory agent, mine included, and if you deploy only one defense, deploy that one. What the boundary can’t do is reach backward. It judges entries one at a time as they arrive, and months in, the thing that needs catching may no longer exist as any entry at all, only as a pattern the maintenance loop distilled from inputs that each looked innocent at the door. The recursion problem also survives this post: who audits the auditor? My only comfort there is that frontier labs patch their injection defenses on a cadence no local checkpoint matches, and a defense that improves between your audits is a different bet than one frozen on your own hardware.
My Hermes brain is brand new. There’s almost nothing in it yet, and no poison that I know of. The write boundary has no screener on it yet either, so I’m currently failing the cheapest test in this post on my own machine. “That I know of” is the entire problem. The ecommerce system had a production floor full of people positioned to catch what the pipeline missed, and the attack still reached production before anyone traced it. My brain has an audience of one: me, consulting the very system I’d be checking. Clean today costs nothing. I still don’t know what tells me a year and ten thousand memories from now.