← All insights

Research · Agentic AI · Security

Agent Memory Integrity Benchmark: Testing Persistent-Memory Security and Utility

A reproducible synthetic evaluation of five memory-governance policies, examining poisoning risk, cross-task contamination and the cost of restrictive controls.

By Muzammal Hussain · 2 November 2026 · 7 min read

Persistent memory can help an AI agent maintain continuity across tasks. It can also create a route through which poisoned, misleading or task-inappropriate information influences later decisions. The security problem is not simply whether an agent should remember something: it is whether a memory should be accepted, retained and retrieved in the right context. The Agent Memory Integrity Benchmark (AMIB) compares five governance policies in a controlled synthetic environment.

Research question

How do write-time filtering and task-scoped retrieval affect memory-poisoning risk, cross-task contamination and benign-memory utility in a controlled synthetic environment?

The benchmark treats memory governance as a security–utility trade-off. A policy that blocks more suspicious writes may also reject useful information. A policy that restricts retrieval by task can limit cross-task leakage, but does not necessarily detect malicious content that belongs to the active task.

Experimental design

AMIB v0.2.0 generates synthetic task episodes containing an active task, a candidate memory's task label, a poison indicator, benign-memory availability, a detector flag and task difficulty. Five policies are evaluated against the same episode sets using seeds 11, 23, 37, 53 and 71. Each seed contains 6,000 episodes.

The five policies are:

• Unrestricted: memory is accepted and retrieved without the additional governance controls tested here.
• Write filter: a simulated write-time filter can reject candidate memories.
• Task-scoped: retrieval is restricted to memories associated with the active task.
• Verified write + task-scoped: combines simulated write verification with task-scoped retrieval.
• No persistent memory: disables persistent memory and serves as a control for memory-mediated effects.

The five policies across 30,000 generated episodes result in 150,000 policy evaluations. The experiment is reproducible using the supplied Python benchmark, fixed seeds and generated CSV/JSON outputs.

Measures

• Attack success rate: proportion of simulated attack opportunities that succeed.
• Benign-memory retention: proportion of useful benign memory retained by the policy.
• Task completion rate: proportion of simulated tasks completed.
• Cross-task contamination rate: rate at which memory from another task influences the active task.
• False rejection rate: proportion of benign writes incorrectly rejected.

Results

All percentages are calculated from the benchmark's generated summary metrics and rounded to two decimal places.

Interpreting the trade-offs

Write filtering achieved the highest task completion rate in this simulation (70.77%) and reduced attack success relative to unrestricted memory. It did not eliminate cross-task contamination: the measured rate remained 21.05%. Its false rejection rate was 6.17%, indicating that the simulated filter also rejected some benign writes.

Task-scoped retrieval reduced cross-task contamination to zero under the benchmark's task-label assumptions. However, attack success remained 9.74%. This illustrates an important distinction in the simulator: restricting where memory is retrieved from does not, by itself, establish that the content is trustworthy.

The combined verified-write and task-scoped policy recorded the lowest non-zero attack success rate (1.82%) among the memory-retaining policies, with zero simulated cross-task contamination. That reduction came with lower benign-memory retention (40.46%) and a 6.17% false rejection rate. It therefore represents a different operating point, not an unconditional winner.

The no-persistent-memory control recorded zero attack success and zero contamination, but it also retained no benign memory and had a 60.51% task completion rate. Its zero-risk figures must be interpreted alongside the complete removal of persistent-memory utility.

Limitations

AMIB is a synthetic simulator. It does not execute an LLM, implement semantic memory in a deployed agent, measure real-world attack rates, or model adaptive adversaries. Its probabilities and scenario rules are explicit simulator assumptions rather than empirical estimates. Zero contamination under task scoping depends on the benchmark's task-label and retrieval assumptions. Results should therefore be interpreted as properties of this experimental design and its policy implementations, not as evidence of production security.

Conclusion and next steps

AMIB provides a reproducible baseline for examining how memory write controls and retrieval boundaries interact. The results show why memory security should be assessed alongside retention and task utility: aggressive restrictions can suppress simulated attacks while also removing useful information. A subsequent research stage should implement these policies in a real agent memory stack, document the threat model, and evaluate them on independently constructed tasks with empirical measurements.

Reproducibility

The benchmark repository includes the source code, fixed seeds, experimental outputs, dataset card, protocol and tests. It requires Python 3.9+ and the standard library.

Code: Agent Memory Integrity Benchmark on GitHub

Version: 0.2.0 · Evaluation type: Synthetic simulation · Author: Muzammal Hussain