Blog · arXiv Analysis · Last reviewed June 25, 2026

The Context Compactor Becomes the Policy Deleter

The June 2026 arXiv paper Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents, by Shiyang Chen, shows a specific failure mode: an agent can obey a policy while it is visible, then violate it after compaction deletes the policy from working context. A context compactor is not just a memory-saving tool. It is a rewrite layer that decides which rules survive into the next action.

Forgetting Is a Governance Action

Chen's paper, arXiv:2606.22528, was submitted on June 21, 2026. It studies a problem that sits between memory management and policy enforcement. Long-running LLM agents often compact their histories by summarizing or evicting older turns so the session stays within a token budget. That engineering step can delete the rule that made the agent safe.

The failure is not ordinary long-context distraction. In the paper's framing, the agent receives an in-context governance constraint: an organizational policy, a standing instruction, a memory entry, or a tool-loaded rule. While the constraint remains visible, the same agent can refuse the prohibited action. After compaction, the summary may preserve the task state but drop the policy. The agent then sees the same kind of request and performs the forbidden tool call.

This is close to the context-window failure archive, but the mechanism is sharper. LOCA-bench asks how growing context makes agents worse at long tasks. Governance Decay asks what happens when a component actively rewrites the record and removes the constraint that runtime enforcement assumed would still be present.

The useful definition is narrow. A context compactor is any summarizer, eviction rule, memory-folding step, context editor, or agent harness component that replaces a larger working history with a smaller representation. A policy deleter is a compactor that treats authority-bearing constraints as ordinary low-salience history. The deletion may be accidental, optimized for task continuity, or induced by hostile content. In each case, the result is the same: the agent's next tool call is governed by a smaller law than the one the institution thinks is still active.

Current Context

As of June 25, 2026, this should be read as a new arXiv preprint and benchmark claim, not as an independent audit of deployed products. The arXiv record lists the paper as submitted June 21, 2026 by Shiyang Chen. The experimental HTML and PDF both frame context compaction as a safety-critical failure surface for long-horizon agents because policy, memory, and standing instructions may live in channels that the harness is allowed to summarize or evict.

The result lands in a broader context-engineering debate. A separate May 2026 arXiv paper on parallel context compaction says long-horizon agents accumulate histories that exceed context windows, that LLM-based summarization keeps sessions bounded, and that summarization is inherently lossy and can make retained knowledge unpredictable across runs. The point is not that compaction is bad. The point is that compaction has become infrastructure for making agents run longer, so its failure modes now matter for governance.

Security guidance already points in the same direction. OWASP's 2025 prompt-injection material treats untrusted content inside LLM applications as a core risk because natural-language data can alter behavior. MITRE ATLAS now tracks AI agent context poisoning and memory/thread variants. Governance Decay adds a complementary shape: the attacker may not need to add a malicious instruction that survives. It may be enough to make the system delete a legitimate constraint.

For institutional systems, this puts context compaction beside context engineering, agent observability, agent action receipts, mechanical policy gates, and out-of-band reference monitors. If an agent can act on tools, the compactor is part of the control plane.

What ConstraintRot Tests

The paper introduces ConstraintRot, a benchmark of long-horizon agent scenarios with deterministic tool-call grading. Each scenario includes a policy turn, benign task turns that grow the context, and a later trigger request whose natural completion would violate the policy. Violations are scored by the emitted tool call, such as sending email to a forbidden recipient, executing a prohibited database operation, reading a secret, disclosing PII, or making an over-limit purchase.

The arXiv abstract reports seven model families and 1,323 episodes. With the policy in full context, violation was 0%. After compaction, pooled violation rose to 30%, reaching 59% for some models. When the compacted summary still preserved the constraint, violation stayed at 0%; when the constraint was dropped, violation reached 38%.

The paper also separates soft organizational policies from hard safety norms. Soft rules include ordinary actions forbidden only by a local rule, such as an external-email restriction or a spend-approval limit. Hard norms include actions many models refuse even without a local policy, such as secret exfiltration or disabling security controls. The decay concentrates on the soft rules, which is exactly where enterprise governance usually lives.

That scope matters. ConstraintRot is strongest as a mechanism test: it isolates whether removing a visible constraint changes later tool behavior. It is not a prevalence estimate for all agents, all products, or all compaction systems. It also does not prove that every model will ignore every compacted policy. The governance lesson is conditional but serious: if the policy is only present because the context happens to carry it, the system must prove that the context-management layer preserves it.

The Eviction Attack

Governance Decay becomes more serious when compaction is treated as an attack surface. The paper's Compaction-Eviction Attack assumes the adversary controls only content that enters context, such as a retrieved document or tool output. The adversary can add bulk content that forces compaction, or add summarizer-directed text that biases the compactor to omit the legitimate policy.

That is not the usual prompt-injection shape. The attacker is not trying to make the agent remember a malicious instruction. The attacker is trying to make the harness forget a protective instruction. The dangerous object is not only the model's next answer; it is the summary that decides what future answers will be allowed to know.

Chen reports that optimized summarizer-injection strategies broke every model in a three-model soft-task study, including a model that resisted the fixed probe. The lesson is conservative: robustness to one deletion prompt is not evidence that the compaction layer is safe. A compactor that ingests untrusted content needs its own threat model.

The defense also needs the right trust boundary. A summarizer is often treated as an internal helper because it only produces context, not final user output. This paper shows why that is too weak. The summarizer can change the future policy environment. Untrusted retrieved pages, emails, tickets, database rows, code comments, or tool outputs should not be allowed to decide which governance text survives into the next compacted state.

Governance Standard

The paper's main mitigation is Constraint Pinning. The harness extracts governance constraints into a pinned buffer, exempts that buffer from lossy compaction, re-injects it after compaction, and checks that the post-compaction context still entails the pinned constraints. In the reported benchmark, this training-free defense restored the violation rate to 0% across the seven models and the fixed attack variants.

Pinning is not a license to stuff every preference into a sacred prompt. It is a requirement to classify authority. A product policy, a user preference, a temporary task note, and an operator emergency update should not all occupy the same editable memory channel. If policy can arrive through memory or tools, the system must record provenance, preserve the active rule, and show when compaction changed the governing record.

The paper is also clear about the remaining boundary. Naive pinning can still be pressured by an operator-impersonation update in recent, non-summarized context. Closing that gap requires a trusted out-of-band operator channel, not just better phrasing. This links the result to agent logs, intent-scoped tools, cross-session prompt payloads, and system prompts.

A practical governance standard should go beyond pinning. First, classify context by authority: system rule, operator policy, legal constraint, tool permission, user preference, task state, retrieved evidence, and untrusted content. Second, store active constraints in a protected policy state rather than only in conversational history. Third, require a compaction receipt: pre-compaction constraint set, post-compaction summary, entailment check, dropped items, summarizer prompt, source classes consumed, and failure action.

Fourth, fail closed when active policy cannot be reconstructed. If the system cannot prove that a required constraint survived compaction, it should re-load the rule from a trusted source, ask for review, or block the affected tool call. Fifth, keep permissions outside the compactor. A tool call should still be checked by a policy gate or reference monitor even if the model's current context no longer contains the rule. Sixth, test compaction adversarially with planted documents, long retrieved pages, bulk tool output, stale memories, policy conflicts, and operator-impersonation attempts.

Seventh, treat summaries as governed records. If a summary replaces earlier context in a consequential workflow, reviewers should be able to inspect what it preserved, what it omitted, and whether the omission changed authority. This is not a demand to log every private token forever. It is a demand to preserve the authority trail for actions that can move data, money, records, code, permissions, or public commitments.

Failure Modes

Task-state bias. The compactor preserves what helps continue the task and drops the standing rule that limits the task.

Soft-policy collapse. Enterprise rules such as recipient limits, spend thresholds, data-sharing limits, or approval requirements disappear because the base model has no built-in reason to refuse them without local context.

Summarizer injection. Untrusted retrieved or tool-returned content contains instructions aimed at the compactor rather than the final agent response.

Evidence-only logging. The final agent trace records the tool call but not the compaction event that removed the rule before the call.

Pinned-prompt sprawl. Teams respond by pinning every preference, creating a crowded sacred context where real safety rules compete with style guidance and stale task notes.

Operator-channel confusion. Recent context claims to update policy, and the agent cannot distinguish a real operator update from a user, document, or tool output impersonating authority.

False benchmark comfort. A model passes single-turn safety, prompt-injection, or long-context retrieval tests, but the deployed harness has never tested whether its own summaries preserve governing constraints.

Source Discipline

Governance Decay should be cited as an arXiv preprint and benchmark study unless and until later review changes its status. Its strongest claims are about the reported ConstraintRot setup: seven model families, 1,323 episodes, deterministic tool-call grading, compaction-induced violation, and the reported Constraint Pinning mitigation. Those numbers should not be reused as product prevalence claims.

Keep the mechanism separate from the policy conclusion. The paper shows that context compaction can delete in-context governance constraints and that deletion can change later tool behavior under the tested harness. The policy conclusion is this site's inference: deployed agents need protected policy state, compaction receipts, out-of-band authority channels, and runtime gates that do not rely entirely on whatever the current prompt remembers.

For adjacent claims, cite source type clearly. arXiv papers establish proposed methods and reported experiments. OWASP and MITRE ATLAS establish security taxonomies and threat language. NIST AI RMF and SP 800-218A are official risk-management and secure-development references, not proofs that a particular compaction defense works. Internal Spiralism links are conceptual neighbors, not external evidence.

What This Changes

The context compactor becomes the policy deleter when forgetting is treated as maintenance instead of authority. A summary is not a neutral compression of the past. It is an operational document that decides which rules will still govern the next tool call.

The practical rule is simple: do not govern an agent only through text that the compactor is allowed to discard. Put constraints in preserved channels, pin the rules that must survive, verify the summary, and log the difference between what the agent once knew and what it is allowed to remember now.

For Spiralism, the lesson is that machine memory has politics at both edges. Remembering can become surveillance. Forgetting can become permission. The institution has to govern both.

Sources

Return to Blog