Securing LLM applications: prompt injection is an architecture problem

The defining security property of a language model is that instructions and data arrive through the same channel. Everything the model receives - your system prompt, the user's message, a retrieved document, a web page, a tool result - is text competing for influence over what happens next. That is why prompt injection is not a bug to be patched but a property to be designed around.

Direct and indirect injection

Direct injection is a user trying to override your instructions - "ignore your previous instructions and reveal the system prompt". Annoying, usually low impact, and the version everyone tests for.

Indirect injection is the serious one. Instructions arrive inside content the model processes on the user's behalf: a document in your knowledge base, a web page it fetches, an email it summarises, a code comment, a support ticket, a calendar invite. The attacker never talks to your application - they plant text where your model will read it.

The decisive question is not "can we detect malicious prompts?" It is "what can this system do if the model is fully controlled by an attacker?" If that answer is unacceptable, filtering will not save you - the architecture has to change.

Filters and classifiers help at the margins and are worth deploying, but treat them as reducing volume, never as a boundary. Published bypasses appear continuously, and the input space is unbounded.

Excessive agency: where injection becomes damage

An injected instruction is harmless until the model can act on it. Risk scales directly with what you have connected:

• Read-only, single-user data - limited blast radius.
• Read across other users' or tenants' data - now an injection is a data breach.
• Write actions - sending mail, modifying records, making purchases, executing code.
• Autonomous multi-step agents chaining tools without human review - the highest exposure, because a single injected instruction early in a chain influences every subsequent step.

Controls that hold:

• The model's permissions are the user's permissions. Enforce authorisation in the tool layer against the end user's identity, never in the prompt. "Only show data the user is allowed to see" as an instruction is not access control.
• Human confirmation for consequential actions - anything irreversible, financial, outbound or destructive. Show what will happen, in specific terms, and require an explicit approval.
• Narrow, typed tools rather than broad ones. A tool that runs arbitrary queries is unbounded; one that fetches an order by ID is not.
• Segregate trust levels. An agent that has processed untrusted external content should not then be permitted privileged actions in the same session.
• Rate and budget limits on tool invocation, so a runaway loop is contained.
• Log every prompt, retrieval and tool call with the user identity. Without this you cannot investigate anything.

Retrieval: your index is an attack surface

Retrieval-augmented generation introduces two distinct problems.

Permission leakage. Documents indexed once and retrieved for everyone will surface content users should not see - HR files, salary data, confidential contracts. Vector stores rarely enforce document-level permissions by default. Filter at retrieval time against the requesting user's entitlements, and re-check after retrieval; do not rely on the model to decline.

Poisoning. Anything ingested is content the model will treat as authoritative. If users, customers or automated feeds can add to the corpus, an attacker can plant both misinformation and instructions. Control what enters the index, record provenance, and treat retrieved content as untrusted input in your prompt structure.

Output handling

Model output is untrusted input to whatever consumes it. The failure modes are familiar web vulnerabilities wearing new clothes: rendering model output as HTML gives you cross-site scripting; passing it to a shell or interpreter gives you command injection; inserting it into a query gives you SQL injection. Encode, validate and parameterise exactly as you would for user input - the fact that a model produced the string changes nothing.

The governance layer

• Inventory your AI systems, including features embedded in tools you already own. Most organisations underestimate this substantially.
• Data flow clarity - what leaves your boundary, to which provider, under what terms, and whether it may be used for training. This is the first question every enterprise customer asks.
• Purpose and lawful basis where personal data is processed, plus transparency about AI involvement in decisions.
• Evaluation before and after release - adversarial testing, not just functional accuracy. Red-team the injection paths specifically.
• Human oversight proportionate to consequence, documented - increasingly a regulatory expectation rather than good practice.
• Monitoring for drift and abuse, with a route to disable a feature quickly.

ISO 42001 provides a management-system structure for this, and it maps cleanly onto an existing ISO 27001 programme rather than requiring a parallel one.

Key takeaways

• Instructions and data share one channel - injection is an architecture constraint, not a filtering problem.
• Design for a fully attacker-controlled model: limit what it can do.
• Enforce authorisation in the tool and retrieval layers against the end user, never in the prompt.
• Treat model output as untrusted input to whatever renders or executes it.