How to Prevent Prompt Injection: Contain What the Agent Can Do

You cannot reliably prevent prompt injection, so limit what a successful one can make your AI agent do. An attacker does not need your chat box. They need one field your agent reads, such as a supplier email or the PDF attached to it. We treat every such agent as an insider who can be talked into anything by whoever wrote its inputs. Give the reader of outside text no tools, give each agent only the access its task needs, put a human confirm step on writes and payments, and block calls to unknown addresses. Written for owners, CTOs and compliance leads deciding which agents get which access.

The OWASP Top 10 for LLM Applications 2026, published in August, says "no reliable prevention mechanism exists today." Singapore's PDPC named the risk in its Advisory Guidelines on Use of Personal Data in Generative AI on 20 July 2026. And on 5 August, researchers at PromptArmor showed hidden instructions in uploaded files pulling Jira and Confluence data out through Atlassian's Rovo assistant (The Hacker News, 8 August 2026). Atlassian had published no advisory, and no exploitation in the wild was reported.

How does indirect prompt injection work?

Indirect prompt injection plants instructions in content an AI agent will later read, such as an email or a file, and the model obeys them. OWASP's LLM01:2026 entry lists the carriers, from web pages and documents to tool responses and database rows, and adds the detail that matters most: "The user did not supply or see those instructions."

A large language model (LLM) is the engine behind ChatGPT-style tools. An agent is an LLM wired to tools, so it can search your files, update a record or send an email. The model receives your instructions and the content it is working on as one stream of text. Nothing in that stream marks which words are orders and which are data. For a business, that makes the threat surface every field an outsider can write to.

Here is the whole attack in four steps, and the control that cuts each one.

Two things make this worse than ordinary phishing. Your staff can learn to spot a fake invoice; the agent reads a hidden line of white-on-white text with the same attention as the visible ones. And it acts with your access, at machine speed, with no instinct that something is off.

Where does prompt injection enter an SME's systems?

Prompt injection enters wherever an agent reads text that someone outside your company could have written: the inbox, shared drives, the CRM and anything fetched from the web. The record for CVE-2025-32711 in the US National Vulnerability Database (June 2025) describes "AI command injection in M365 Copilot", which Microsoft scored 9.3, critical, with no user interaction required.

Storing text internally does not make it safe. OWASP's 2026 entry counts "internal documents, and mail" among the trusted surfaces where an attacker can still plant text, perhaps "via an unrelated upstream vector such as a public bug-report form." The inventory below is the one worth doing instead: for each source, what could a planted line talk the agent into, and what stops it?

OWASP's 2026 mitigations cover the last row directly: "audit tool descriptions for hidden instructions." The Model Context Protocol (MCP), the open protocol many agents now use to reach tools, makes a related point in its specification of 28 July 2026: clients "MUST consider tool annotations to be untrusted unless they come from trusted servers." The web-page row is OWASP's own Scenario #2, where hidden instructions make a model insert an image whose link carries the conversation out.

For operators, one change follows. Free-text fields are now an attack surface, so the person who reviews an agent's proposed action needs to see the source text that triggered it, not a tidy summary the agent wrote.

Why can't a filter or a smarter model stop it?

Filters and better-trained models cut how often injections work, but they cannot close the hole, because a language model has no reliable way to tell an instruction from data. The UK's National Cyber Security Centre (NCSC) wrote in December 2025 that prompt injection "may never be totally mitigated in the way that SQL injection attacks can be."

The NCSC's reasoning is short: "Under the hood of an LLM, there's no distinction made between 'data' or 'instructions'; there is only ever 'next token'." SQL injection, the classic database attack, was fixed by keeping commands and data in separate channels. LLMs have no second channel. The NCSC suggests treating the model as an "inherently confusable deputy". We read that as an insider: someone with legitimate access who can be manipulated into misusing it. Unlike a human insider, the agent has no motive, which is also why it cannot be deterred.

The research record backs this up. In The Attacker Moves Second, an October 2025 study, a team including Milad Nasr, Nicholas Carlini and Florian Tramèr reported that adaptive attacks, tuned to the specific defence, bypassed 12 recent published defences "with attack success rate above 90% for most." Most of those defences had originally reported near-zero attack success. OWASP's 2026 entry draws the conclusion for builders: "Defense is therefore architectural rather than interceptive."

So what should a CTO do with a vendor slide claiming injection resistance? Treat it as a frequency claim, not a guarantee. A model that resists 99 of 100 planted instructions still fails on the hundredth, and attackers can try thousands of variations for free. Buy the filter if it is cheap. Design as though it will fail.

Containment has a price, and it can be measured. The CaMeL design keeps untrusted data from ever steering the agent's plan. On the AgentDojo benchmark it completed 77% of tasks with provable security, against 84% for an undefended agent (Debenedetti et al., revised June 2025). Seven points of task completion bought provable resistance within the paper's threat model. For anything touching money or personal data, we would take that trade.

What did IMDA's red-teaming challenge find?

IMDA's 2026 Singapore AI Safety Red Teaming Challenge found simple prompts pulled protected data out of generative AI apps, identical prompts gave different results, and regional-language protection was weaker. More than 80 red teamers from 14 Asia-Pacific countries attacked seven simulated apps in January 2026, according to IMDA's Challenge Report (June 2026).

The red teamers typed into the apps themselves, so this was direct injection, not instructions planted in documents. But at least four of the seven apps were wired to tools; those results preview agent incidents. In the Vending Machine challenge, an assistant on MCP tools was told a fake "system-level promotion" overrode its rules, and it called its purchase tool. IMDA describes the outcome as "a completed purchase transaction executed by a legitimately authorised tool, triggered entirely through manipulated reasoning." A research assistant that fetched web addresses applied "no filtering or allowlisting" to them, so red teamers pointed it at an internal cloud address and pulled out credentials.

The language findings are the ones Southeast Asian businesses should circle. One app leaked its secret on a single attempt in Khmer, where 58 attempts had been needed in English. In the report's figures, attacks that played on how text is split into tokens succeeded 21.59% of the time in English and 52.54% on average across regional languages. IMDA also found "Identical user prompts were observed to produce materially different outcomes", and warns developers "not to assume that a single successful refusal is evidence of a secure AI system." A guardrail you tested once, in English, is a guardrail you have not tested.

IMDA's fix list includes better classifiers. It also says to "Separate LLM reasoning from tool-call execution", with a validation layer that checks the model's structured output against security policies before anything runs. That second fix is containment.

What does a containment architecture look like?

A containment architecture assumes some planted instruction will get through and limits what it can achieve. It rests on four controls: an untrusted-content boundary, least-privilege access, a confirm step on consequential writes, and limits on outbound connections. OWASP's 2026 entry calls blast-radius controls "what survive against attackers who can probe the system."

Untrusted-content boundary. Split the job. One component reads outside content and produces a constrained output, such as a category or an amount in a fixed format. A separate component with tools acts only on that output. Never give the agent that reads outside content a send, write or payment tool. The planted line then has nothing to pull. This is the core of the design patterns published by Beurer-Kellner and colleagues in June 2025.

Least privilege. Give each agent its own credential, scoped to its task, never a staff member's login. OWASP's 2026 wording is blunt: "Hold credentials and state-change capability in application code, not the model." The MCP project's 2026-07-28 security best practices recommend a starting scope set covering "only low-risk discovery/read operations", raised step by step when a privileged action is first attempted.

Confirm step on writes. The MCP specification says "there SHOULD always be a human in the loop with the ability to deny tool invocations." Two design details decide whether that human is a control or a formality. First, show the exact action and the source text behind it; OWASP asks for "the exact rendered action rather than a summary", because the agent's summary may be the attacker's words. Second, keep confirmations rare enough to be read. An approval queue of hundreds a day becomes the rubber stamp we described earlier this month. Our guide to designing the confirm step covers the screen itself.

Egress limits. Block calls and links to domains not on an allowlist. Data leaves an agent through URLs as easily as through email: in the Rovo case as reported, the assistant appended search results to an attacker's web address and fetched it. IMDA's research assistant failed the same way, pointed inward.

How do you decide which agents need the strictest controls?

Decide by checking three properties for each agent: does it read untrusted input, can it reach sensitive data, and can it change state or communicate externally? OWASP's 2026 entry adopts Meta's "Rule of Two": an agent with all three "needs per-action human approval", and one with two of them needs "an explicit residual-risk assessment."

Applied to agents a typical SME runs, our mapping looks like this:

Illustrative example: a 40-person distributor runs an accounts-payable agent over 600 supplier emails a month. One email carries white text telling the agent to update a supplier's bank details. In an uncontained setup the agent can write to the supplier master file, so the next payment run sends S$50,000 to the attacker, and the bank's own approval step sees an ordinary payment to a known supplier. In the contained setup, the reading agent can only output "bank change requested" plus the raw email, and the change goes to a confirm queue. At about 20 such requests a month and 3 minutes each, one hour of a finance officer's time a month buys a hard stop on the most expensive failure. The decision rule: if a write is rare and costly, confirm it; if it is frequent and cheap, scope it and log it.

For an owner or MD, this reframes the purchase question. Do not ask an AI vendor "are you safe from prompt injection?" Ask "what is the worst thing one malicious email could make your agent do with the access we give it?"

For compliance leads, the Singapore picture has two layers. This is general information, not legal advice. Binding law: section 24 of the PDPA requires "reasonable security arrangements" against unauthorised access and disclosure of personal data. Advisory guidance: paragraph 8.2 of the PDPC's July 2026 guidelines names "prompt injection attacks, which can be exacerbated by unclear boundaries between control and user data planes" as a risk for system providers. Our walk-through of those PDPC guidelines covers the rest. In our reading, an organisation relying on a filter alone may struggle to show its arrangements were reasonable after a leak.

Where Origin Pi stands

Prompt injection is an access-control problem wearing a language costume. Models will sometimes follow the wrong instructions, and the protocol layer leaves containment to whoever builds the application. Waiting for a model that cannot be fooled is not a plan.

We design agents on that assumption. Readers of outside content get no tools. Every agent gets bounded permissions on its own credential. Consequential writes pass a confirm step that shows the human the source text, not the agent's paraphrase. And an audit trail ties each tool call to what the agent read just before it, so an incident can be traced to the email that caused it. Our agent-ready layer, Cerebrum, is in build around the same pattern.

The first step needs no new software. For each agent you run, list the sources it reads that an outsider can write to, then list the tools it holds. Where those two lists meet on one agent, remove the write tool or add a confirm step before anything else.