Okay, this is going to sound dumb, but I spent half of yesterday staring at a new paper and thinking: we probably already knew this, and we still keep shipping the thing it breaks. If you run an LLM app and you’ve been comforted by “the model refuses bad requests,” you should read this before the next release.<\/p>\n
The paper is Transient Turn Injection: Exposing Stateless Multi-Turn Vulnerabilities in Large Language Models<\/a>. I’m not going to rewrite it. I want to talk about the part of it that has practical consequences for the shape of LLM apps I see people build, including ones I’ve built.<\/p>\n Most LLM safety moderation is stateless. Each turn gets scored on its own. An attacker that splits malicious intent across several turns, none of which looks bad in isolation, walks straight past the moderator. The authors call this Transient Turn Injection (TTI), and they show it working across a bunch of frontier and open-source models.<\/p>\n That’s it. No magic. No new adversarial math. The failure mode is an architectural seam between “the model” and “the moderator layer around it,” and the seam has been sitting there the whole time.<\/p>\n If you’re a platform deploying an LLM at scale, you almost always run moderation as a separate classifier on each message. It’s cheaper. It’s easier to reason about. It composes with rate limiting. And for single-turn abuse it actually works pretty well.<\/p>\n The problem is that “assistant” is a multi-turn abstraction. The user writes three benign-looking messages that, together, describe a coherent attack. The moderator sees three benign messages. The model sees the accumulated context and happily executes the composite request. Nobody is looking at the shape of the conversation.<\/p>\n I wrote a related post on why LLM evals miss context-window failures<\/a> and this is cousin to that problem. The thing you’re evaluating is narrower than the thing that actually runs in production, so evals keep passing while production keeps failing in the same structural way.<\/p>\n If you’re building on top of a frontier model through an API, three things are true:<\/p>\n This isn’t a theoretical knock. Any app that uses the pattern “user chats, model responds, each message goes through the provider moderator” is vulnerable to the TTI class of attack. That includes most of the customer-support bots, internal knowledge assistants, and content generators I’ve seen deployed.<\/p>\n The fix isn’t “switch models.” The paper shows the technique working across many frontier models. The fix is architectural: you have to moderate the conversation<\/em>, not the turn<\/em>.<\/p>\n Maintain a running score on the conversation as a whole. Each turn contributes to the score based on its content, but so does the pattern<\/em> of the conversation \u2014 requests that look like probing, requests that try to establish a premise, requests that ask the model to adopt a persona. When the conversation’s accumulated score crosses a threshold, switch to a hardened system prompt, start logging more aggressively, or cut the session.<\/p>\n You don’t need a fancy model for this. A small classifier that scores transitions works fine. The point is that the state lives at the conversation level, not the turn level.<\/p>\n Once every N turns, or whenever the score crosses a bar, send the full transcript (not just the latest turn) to a reviewer model with a system prompt asking “is this conversation, as a whole, trying to get me to do something my guidelines prohibit?”<\/p>\n This is more expensive than per-turn moderation, but you don’t run it on every turn. You run it on sessions that the cheap-turn-moderator has flagged as interesting. It catches the distributed-intent case because you’re handing the reviewer the thing it needs to see: the whole arc.<\/p>\n The OWASP Top 10 for LLM applications<\/a> lists Prompt Injection as the #1 risk, and the multi-turn variant belongs in the same category. If you’ve been treating that section as “theoretical” because your vendor handles it, please do not.<\/p>\n The longer the conversation, the more surface area an attacker has to spread intent across turns. For use cases that don’t genuinely need long context (customer support, form filling, FAQ lookup), cap the session length and force a reset. It’s a blunt instrument, but it removes the TTI attack surface almost entirely.<\/p>\n This is uncomfortable because we’ve built UX patterns around “the assistant remembers you.” For user-facing brand experiences it’s fine. For anything high-trust (financial, medical, policy), short sessions are a genuinely defensible choice and it sidesteps a whole class of attacks.<\/p>\n The uncomfortable read of TTI is that “safety” in LLM apps is mostly a marketing surface, not a property of the system. Real safety in LLM deployments looks the same as it does everywhere else in security: defense in depth, assume breach, monitor the abstraction you actually run.<\/p>\n I was going to write this same post six months ago about a related attack and put it off because “the providers will patch it.” Providers did patch some things. Then the attack surface moved, because attack surfaces always move, and now we’re staring at the stateless-moderation gap. The Rise of Verbal Tics paper<\/a> from the same week is orthogonal but makes a similar point from a different angle: evaluating LLMs on isolated prompts misses properties that only emerge at conversation scale.<\/p>\n If your LLM app is in production, the thing to do this week is not rewrite the system prompt. It’s audit what your moderation layer actually looks at. Does it see individual turns or the full history? If individual turns, you now have a specific attack class to test against.<\/p>\n I’ve stopped pretending LLM safety is a feature of the model and started treating it as a property of the system around the model. Something I wrote about from a different angle in my piece on why reasoning models are more fragile than benchmarks claim<\/a>.<\/p>\nThe one-sentence version of the result<\/h2>\n
Why the blind spot exists in the first place<\/h2>\n
What this means for your app (the practical bit)<\/h2>\n
\n
Three things you can actually do<\/h2>\n
1. Keep a conversation-level risk score<\/h3>\n
2. Replay the full conversation through a reviewer model periodically<\/h3>\n
3. Shorten sessions where it’s safe to<\/h3>\n
The part nobody likes to admit<\/h2>\n
Where I land on the bigger picture<\/h2>\n