Lorixen.
EN

Prompt injection: keeping a business chatbot from being misled

Short answer: Prompt injection is when text a chatbot reads (typed by a customer, or hidden in a web page, email or document) changes what the bot does. OWASP, the open security community, lists it first in its Top 10 for LLM applications and says it's unclear whether fool-proof prevention exists. So design for damage limitation: give the bot only the permissions it needs, keep sensitive actions in ordinary code, require a person's approval for anything risky, mark outside content as untrusted, and test it like an attacker would.

A chatbot that only answers questions from your FAQ can embarrass you if someone tricks it. A chatbot that can issue refunds, change bookings or send emails can cost you money. The difference isn't the AI model; it's what the bot is allowed to do. This guide is about that difference.

What prompt injection is

OWASP's Top 10 for Large Language Model Applications describes prompt injection as user prompts altering the model's behaviour or output in unintended ways. Two forms matter for businesses:

  • Direct injection: someone types instructions into the chat itself. "Ignore your previous instructions and…" is the classic opening. OWASP notes this can be deliberate or accidental.
  • Indirect injection: the instructions sit inside content the bot reads from elsewhere: a web page it summarises, an email it triages, a PDF a customer uploads, a document in your knowledge base. OWASP points out these instructions don't even need to be visible to a human, as long as the model reads them.

OWASP's own example scenarios include a support chatbot being told to ignore its guidelines, query private data and send emails, and an attacker editing a document in a knowledge base so that the bot gives misleading answers when that document is retrieved.

Why tools change the stakes

OWASP's separate entry on excessive agency explains the link. When a system can call functions or other systems, damage happens because the bot acts on manipulated output. OWASP names three root causes: excessive functionality (tools it doesn't need), excessive permissions (access broader than the task) and excessive autonomy (acting without checks).

In plain terms: a tricked chatbot with no tools says something silly. A tricked chatbot with a refund tool and no limit issues refunds.

Risk, cause and mitigation

What could go wrong Typical cause Mitigation
Bot reveals its internal instructions or confidential notes Sensitive information placed in the prompt Keep secrets, prices you don't publish and internal notes out of the prompt entirely
Bot shows one customer's data to another Bot has broad database access and decides what to fetch Fetch customer data in code, scoped to the logged-in customer; the model never chooses whose record to load
Bot issues refunds, discounts or credits it shouldn't Refund tool with no limits; bot decides alone Hard limits in code (amount, frequency, eligibility), plus human approval above a threshold
Bot sends emails or messages to the wrong people Email tool accepts any address and any content Restrict recipients (for example, only the customer's verified address), use templates, and confirm before sending
Bot changes or cancels bookings without the customer meaning to Action taken from a single ambiguous message Show the exact change and ask for an explicit "yes" before making it
Bot follows instructions hidden in a web page, email or uploaded file Outside content treated the same as your instructions Label outside content as untrusted data, summarise rather than obey, and block actions triggered purely by that content
Bot gives false answers after a knowledge-base document is altered Weak control over who can edit source documents Limit edit access, keep version history, and review changes to documents the bot relies on
Bot is talked into off-topic or offensive output Broad role and no output checks Narrow the bot's role, filter outputs, and keep the bot to its stated job
No one notices for weeks No logging or review Log conversations and tool calls; review flagged ones and spot-check regularly

The principles behind the table

OWASP's mitigation list for prompt injection lines up with these steps. In business terms:

1. Constrain the role

Tell the bot exactly what it's for and what it isn't for. "Answer questions about our opening hours, menu and bookings" is safer than "be a helpful assistant". This reduces mistakes; it isn't a lock, because instructions in the prompt can still be argued with.

2. Least privilege, enforced in code

OWASP recommends giving the application its own credentials, handling functions in code rather than handing them to the model, and restricting access to the minimum needed. If the bot only needs to read today's availability, it shouldn't hold credentials that can edit the whole calendar.

3. Human approval for high-risk actions

OWASP calls for human-in-the-loop controls on privileged operations. Decide in advance which actions are high-risk for your business: refunds over a set amount, account changes, anything sent to a third party. Those wait for a person.

4. Separate instructions from data

OWASP advises separating and clearly marking untrusted content. When the bot reads an email or web page, treat that text as material to work on, never as instructions to follow.

5. Validate outputs

Where the bot produces something your systems act on (a booking request, a refund amount), check it with ordinary code: is the date valid, is the amount within limits, does the booking belong to this customer?

6. Test like an attacker

OWASP recommends regular adversarial testing that treats the model as an untrusted user. Before launch, and after every significant change, have someone try to make the bot misbehave: ask it to reveal its instructions, offer it fake "manager approval", paste instructions inside a document, request another customer's booking. Write down what worked and fix the controls, not just the wording.

A short pre-launch checklist

  • [ ] The bot's job is written in one sentence.
  • [ ] Every tool it can use is listed, with a reason.
  • [ ] Each tool has limits enforced in code, not just in the prompt.
  • [ ] High-risk actions require confirmation from the customer, a member of staff, or both.
  • [ ] Customer data is fetched only for the verified customer.
  • [ ] Outside content (web pages, emails, uploads) can't trigger actions on its own.
  • [ ] Conversations and tool calls are logged and reviewed.
  • [ ] Someone has spent an afternoon trying to break it.

FAQ

Can a better prompt stop prompt injection?

It helps, but OWASP is clear that it's unclear whether fool-proof prevention exists. Rely on permissions, limits and approvals, not wording alone.

Is a simple FAQ bot at risk?

Less so. Without tools or private data, the main risks are embarrassing or wrong answers. Narrow its role and review its output.

Should we stop using AI agents?

No. Give them less power by default and add power where you can control it. Most of the benefit comes from answering and drafting; the risky part is acting.

Who should test it?

Ideally someone who didn't build it. A fresh pair of eyes tries things the builder never imagined.

To check what your chatbot can actually do with the access it has, send us a list of its tools and permissions.

Sources

WhatsAppLet’s talk