Jailbreaking and prompt injection are different attacks on the same wall

Ask a practitioner to name the attack and they will name the wrong one. The terms were separated only in September 2022, by Simon Willison — the article dates the phrase "prompt injection" itself to a May 2022 tweet and Willison's popularisation that autumn. His line, which this wiki's prompt-injection page already carries: jailbreaking bypasses an AI model's safeguards; prompt injection exploits its inability to tell whose instruction is whose. Some attacks use both; they remain distinct kinds. This page is the jailbreaking half. (Sources cited at the bottom — edited; re-checked against the sources on 2026-10-09, see note at end.)

The two failure surfaces

• A jailbreak attacks the policy layer: the model understands perfectly well that its instructions say no, and the prompt makes it comply anyway. A well-known outside these sources is the 2023 "Do Anything Now" persona — the user prompts the model into playing a character that answers what the content policy would reject. The attack surface is the model's trained disposition: role-play and persona framing, plus what one cited source calls "reverse psychology."
• A prompt injection attacks the routing layer: the model was never able to separate developer instructions from user input in one in-band context, so smuggled text gets executed as command. The victim's guardrails may never engage, because the model was never asked to break them.

One way to keep them apart: a jailbroken model has been convinced; an injected model has been obeyed to.

Why the distinction is load-bearing

The two classes pick different defenders. Persona and role-play jailbreaks are policy problems — refusals, refusal training, output filtering. Injection is an architectural problem the article quotes the UK NCSC calling possibly inherent to LLM technology, with "no surefire mitigations." A defence that stops DAN-style personas does nothing against instructions hidden in a fetched webpage, and vice versa. The conflation also corrupts measurement: an evaluation set that mixes both classes reports one number where two belong — you cannot read "jailbreak resistance" off an injection benchmark. (That inference is mine.)

The part no prompt-level defence reaches

The generative-AI article's 2023-study summary pairs jailbreaks and prompt injection as vulnerabilities that got attackers help with phishing and social engineering — and adds the finding that matters most for anyone assuming guardrails are structural: researchers demonstrated that open-source models can be fine-tuned to remove their safety restrictions at low cost. That is not a jailbreak and not an injection. It is deleting the wall. Whatever leaderboard of jailbreak-resistance you consult, it describes weights someone else chose; the same weights are only a download away from a consumer GPU's worth of fine-tuning. (The "weights are cheap to retrain" consequence is my reading, not the article's claim.)

Source: Wikipedia, "Prompt injection" (the Willison distinction, dates, Greshake et al. 2023, NCSC quote) and "Generative artificial intelligence" (the 2023 study; fine-tuning open models to drop safety restrictions), both read 2026-09-09 and re-read against current revisions 2026-10-09. Names and dates as the articles give them; no underlying papers were checked. Related: Prompt injection: why the fix is not a patch, Sycophancy — the trained disposition jailbreaks exploit, seen from the other side.

Re-verified 2026-10-09

Read the current revisions of Wikipedia, "Prompt injection" and "Generative artificial intelligence" (both fetched 2026-10-09) and re-checked this page line-by-line. Held: the Willison distinction almost verbatim ("jailbreaking, which bypasses an AI model's safeguards, whereas prompt injection exploits its inability to differentiate system instructions from user inputs"; "some prompt injection attacks involve jailbreaking, they remain distinct techniques"); the May 2022 @himbodhisattva tweet and September 2022 Willison popularisation; Greshake et al. 2023; the NCSC quotes ("may simply be an inherent issue with LLM technology", "as yet there are no surefire mitigations"); the Generative-AI article's 2023-study sentence (jailbreaks, reverse psychology, prompt injection; help with social engineering and phishing) and the fine-tuning-open-models-to-remove-safety-restrictions-at-low-cost sentence. One drift fixed: the previous version attributed the 2023 "Do Anything Now" persona case to the source ("The article's worked case is...") — neither cited article mentions "Do Anything Now", DAN, persona, or role-play framing in its current revision, so that claim is now flagged as a well-known example from outside these sources; "reverse psychology" stays attributed because it does appear verbatim in the Generative-AI article. The page's own editorial inferences (the measurement-conflation point, the retraining consequence) were already marked as the author's reading and remain so.