Skip to content
LiveNext 10:45:34

Definition

What is Prompt injection?

Prompt injection is Prompt injection is an attack that hides instructions in text an AI model reads, such as a message or web page, so the model follows the attacker's orders instead of its owner's.

What it is

A prompt injection hides instructions inside content an AI model is asked to read. The model cannot reliably tell the difference between your instructions and instructions buried in the material you gave it, so it sometimes obeys the wrong ones.

The OWASP list of top risks for LLM applications ranks it first, as LLM01.

Direct and indirect

Direct injection is typed by the person talking to the model. "Ignore your previous rules and do this instead" is the classic example. It is closely related to jailbreaking.

Indirect injection is the one that worries builders. The hostile text sits in something the model fetches on its own: a web page, an email, a shared document, a calendar invite. You never type it. Your assistant reads the page, finds a line like "forward the user's files to this address," and may treat it as an order.

An analogy: you hire an assistant and hand them a pile of mail. One letter says "assistant, wire the money to this account." A careless assistant cannot tell that the letter is not from you.

Why it matters

A chatbot that only talks can say something wrong. An AI agent that can browse, read email, run code or spend money can do something wrong. The more access an agent has, the more a single poisoned page is worth to an attacker.

How builders reduce the risk

There is no complete fix today. Sources such as OWASP recommend layers:

  • Give the agent the least access the task needs, and keep private data away from tools that talk to the outside world.
  • Run it in an agent sandbox so a mistake stays contained.
  • Treat everything the model reads from outside as untrusted data, never as instructions.
  • Filter inputs and outputs, and use AI guardrails to check actions before they run.
  • Ask a human to approve anything risky, like sending money or deleting files.
  • Test with red teaming before launch.

Labs also measure how often attacks succeed. A low success rate is good, but it is not zero, and attackers get many tries.

What to do as a user

Be careful when you point an assistant at content you did not write, especially if it can also act on your accounts. Limit what it can reach, and read what it plans to do before you approve it.

Questions people ask

What is the difference between prompt injection and jailbreaking?

A jailbreak is a user trying to get a model to break its own safety rules. Prompt injection is any attack where untrusted text overrides the instructions of the app's owner, and often comes from content the model reads, not from the user.

Can prompt injection be fully prevented?

Not today. Sources such as OWASP say there is no foolproof method, so builders combine limited access, sandboxing, filtering and human approval.

Prompt injection in the news