Blog · arXiv Analysis · Last reviewed June 25, 2026

The Prompt Injection Becomes the Context Problem

Sahar Abdelnabi and Eugene Bagdasarian's arXiv paper AI Agents May Always Fall for Prompt Injections argues that the hard part of agent security is not only separating data from instructions. It is deciding which context makes an action legitimate.

For this essay, prompt injection is not only a hostile sentence inside a webpage, email, document, tool result, or memory. It is any manipulation that causes an AI system to treat the wrong source as authority, the wrong evidence as verified, or the wrong flow as inside the user's delegation.

From Commands to Context

The paper, arXiv:2605.17634 [cs.CR], was submitted on May 17, 2026. The title is exact: AI Agents May Always Fall for Prompt Injections. The phrase should be read as a structural warning rather than a settled law of nature. Abdelnabi and Bagdasarian argue that agents face cases where blocking too much destroys legitimate work and allowing too much lets adversarial context steer action.

Much prompt-injection defense treats the problem as hidden instructions in data. That framing is useful when an email, page, or tool output says something like "ignore previous instructions." It is weaker when the external content changes the apparent context of an action. A message can claim that a manager approved a refund, that a user already delegated authority, that a client will be harmed by delay, or that two requests in one thread share the same permission boundary.

For an agent with tools, this is not just a text-classification problem. The model has to infer who is asking, whose interests are affected, what action is proposed, what authority exists, and whether the action still fits the user's delegation. The attack surface is the agent's judgment about appropriateness.

The operational distinction is evidence versus authority. Untrusted content may be evidence that a task should proceed, but it should not automatically become the authority that permits the task. A purchase order, ticket, forwarded approval, code comment, calendar invite, or MCP tool result can be relevant without being self-verifying.

Current Security Context

As of June 25, 2026, the broader security community already treats prompt injection as an application and agent risk, not a curiosity of chat interfaces. OWASP's 2025 LLM Top 10 lists prompt injection as LLM01 and includes indirect injection through websites, files, repositories, RAG content, and multimodal inputs. NIST AI 600-1 treats prompt injection under information-security risk and describes indirect prompt injection as remote exploitation of LLM-integrated applications by placing prompts in data likely to be retrieved.

The agent-specific guidance is converging on containment rather than one perfect detector. Microsoft Learn's March 2026 guidance says organizations should assume indirect prompt injection will happen and layer prompt shields, spotlighting or data marking, plan-drift detection, critic review, tool-chain analysis, information-flow control, least privilege, short-lived privileges, and human review for risky actions. The Model Context Protocol's 2025-06-18 tools specification says tools are model-controlled, but it also calls for visible exposed tools, confirmation prompts, input validation, access controls, output sanitization, result validation, timeouts, and audit logs.

Public-sector agent guidance says the same thing in institutional language. The 2026 Careful adoption of agentic AI services guidance from CISA, NSA, and partner cyber agencies recommends aligning agentic AI with existing security posture, never granting broad or unrestricted access to sensitive data or critical systems, controlling context from tools and memory, using fine-grained agent identity, allowlisting tool versions, logging tool use, and adding human approval for high-impact actions. That is the governance setting in which this paper should be read.

What the Paper Adds

The authors recast prompt injection through contextual integrity, Helen Nissenbaum's account of privacy as appropriate information flow. In the paper's agent-security version, an action is evaluated by context parameters: sender, receiver, subject, information type, and transmission principle. The agent must know not only what a message says, but under what norm it may act on that message.

That move connects privacy, prompt injection, and delegated authority. A harmful agent action may be a privacy leak, a workflow violation, a forged delegation, or an unauthorized side effect. The common feature is not a forbidden sentence. It is a broken information flow: authority from one context is being used in another.

The paper's strongest contribution is evaluative. It asks defenders to test whether an agent can infer context parameters, ground delegation in history, separate simultaneous flows, and refuse requests whose claimed authority has not been verified. That is a harder standard than scanning untrusted text for known attack vocabulary.

It also changes where the control should live. A context boundary should be represented in tool policy, identity, logs, provenance, approval state, and runtime enforcement. If the only place the boundary exists is a sentence in the system prompt, the same model asked to obey the boundary is also asked to interpret the adversarial context that may dissolve it.

Evidence from the Paper

The paired-email experiment is the cleanest demonstration. The authors construct 4,200 email scenarios, yielding 8,400 emails across authority-asserting and neutral variants. Prompt-injection classifiers looking only at email content perform near chance, with AUROC reported between 0.43 and 0.59. The point is narrow but important: if the attack lies in context rather than injection vocabulary, a content-only detector is looking in the wrong place.

The paper also reports that a contextual-integrity red-team loop targeting context parameters reached 96.7% attack success against an email assistant, compared with a 0.67% unoptimized baseline. The recurring pattern is fabricated provenance: the email makes a claim look like prior user delegation, organizational policy, or later approval. The agent's failure is not obedience to an obvious command. It is accepting the wrong source of authority.

Two other experiments matter for governance. In 300 multi-turn scenarios across email, code, project management, and finance, models executed out-of-scope requests less often when they had interaction history grounding the original delegation. In 100 simultaneous-flow scenarios across 50 professional domains, authorization for one internal action sometimes leaked into a second outbound action in the same thread; the paper reports violations of 65% for gpt-5.2, 34% for gpt-5.4, and 6% for claude-sonnet-4-6 without an explicit boundary.

Those model names and rates should be read as paper-reported results inside controlled scenarios, not as a general certification of any deployed product. The governance lesson is more durable than the leaderboard: mixed authority must be tested at the action boundary, with the resulting tool call and environment state as the evidence.

The Tradeoff

The paper's impossibility argument is informal and practical. A fixed rule can reject suspicious claims, but some real tasks require acting on claims found in external content. A permissive rule can preserve utility, but an attacker can construct a context in which a forbidden action appears legitimate. Verification can help, but each verifier covers only part of the open-ended context space.

This does not mean defense is futile. It means the safety case should stop promising perfect separation between data and instructions. The real control is evidence: which sender was authenticated, which delegation was recorded, which boundary was stated, which external claim was verified, and which tool call changed state.

The tradeoff therefore shifts the assurance target. The product claim should not be "the agent cannot be injected." It should be "the agent cannot turn unverified context into high-impact action without crossing named checks that are outside the model and visible after the fact."

What Defenders Should Measure

A release review should include contextual attacks, not only strings that say they are attacks. Test emails, documents, tickets, chats, repository issues, and agent-to-agent messages that manipulate sender identity, authorization history, urgency, affected-party interests, and mixed flows. The unit of evaluation should be the tool action and resulting state, not the model's self-description.

Defenders should also test utility. A system that refuses to read every external claim may be secure by paralysis. The practical target is calibrated friction: verify high-risk claims, ask for step-up approval when delegation is unclear, split mixed requests into separate decisions, and keep logs that connect instruction source, authority source, approval, and final effect.

That means measuring false refusals as well as successful attacks. A travel assistant that cannot act on a legitimate approval, a coding agent that cannot use a real issue comment, or an operations agent that blocks every external ticket is not deployable. Security review should report both unauthorized actions allowed and legitimate actions blocked because context was too coarsely isolated.

For agents, the minimum evaluation record should include the user delegation, untrusted content source, asserted authority, verification mechanism, exposed tools, selected tool, arguments, approval prompt if any, resulting state, and whether the action crossed an internal, external, destructive, financial, privacy, or privilege boundary.

Limits That Matter

This is a preprint and its impossibility result is an argument, not a mathematical proof covering all possible systems. The experiments are controlled probes, not field measurements of every deployed assistant. The model names, prompts, datasets, and simulated tool environments matter. The paper is strongest as a map of failure modes and test design.

The title should not be turned into fatalism. It should raise the burden of proof for any product claim that says prompt injection is solved because untrusted text was labeled as data.

Governance Standard

Agent governance should treat context as part of the control surface. A serious deployment record names the user delegation, the agent's authority, the external sources consulted, the claims that required verification, the transmission principle used to justify action, and the boundary between simultaneous requests.

That belongs beside automated prompt-injection search, tool-scope gates, delegation traces, agent sandboxing, and capability-based security. The standard is simple: an agent should not be trusted because it ignored a string. It should be trusted only inside a context it can verify, log, and narrow.

Procurement and release review should ask for artifacts, not assurances: a context threat model, prompt-injection and context-poisoning tests, tool permission map, agent identity model, memory provenance rules, human-approval policy, source labeling scheme, audit-log fields, incident path, and evidence that high-impact tool calls fail closed when authority is ambiguous.

This is also a safety and accountability issue. If an agent acts on fabricated approval, collapses two flows, or treats a retrieved document as authority, the affected person may see only the final email, file change, ticket update, payment, or disclosure. Governance has to preserve enough context to explain why the system acted and which human or institutional policy allowed it.

Source Discipline

This article treats the arXiv paper as a preprint and identifies its quantitative findings as paper-reported controlled experiments. Current deployment claims are grounded separately in primary security guidance: OWASP for LLM and agentic risk taxonomies, NIST AI 600-1 for generative-AI information-security risk, Microsoft Learn for indirect prompt-injection mitigation patterns, MCP's own specification for tool-control semantics, and the 2026 multi-agency agentic-AI guidance for operational controls.

When citing this essay, keep the distinction clear. The paper argues for a contextual-integrity reframing and reports experiments. OWASP, NIST, Microsoft, MCP, and CISA/NSA partner materials describe current governance and security practice. None of these sources proves that any particular deployed agent is immune to prompt injection.

Sources

Return to Blog