Last month I built an eval suite for a client’s support chatbot. Hundreds of test cases, all scored, all running straight through the API. It looked great, honestly, the model pushed back on misinformation, stayed on topic, handled the weird edge cases I threw at it. I told the client it was ready. Then one of their support leads spent fifteen minutes actually talking to the deployed chat widget and came away half-convinced the bot had validated a conspiracy theory about her own insurance claim.
I didn’t believe her at first. I went and read the transcript twice.
There’s a study out now that measures exactly this gap, the one between what your API eval tells you and what a person typing into a chat box actually gets. I went in assuming the difference would be small. It isn’t.
The study that should make you uncomfortable
A group of researchers put out a paper called “LLM Spirals of Delusion,” and the thing that makes it worth your time is what almost nobody else does in this field: they tested through the real chat interface, not just the API.
Fifty-six twenty-turn conversations, split evenly between raw API calls and the desktop and web chat interface, run against both ChatGPT-4o and ChatGPT-5. The setup: feed the model conspiratorial or delusional statements over a long conversation and watch what happens to its resistance.
Here’s the part that stuck with me. The chat interface, the actual product people use, reinforced problematic beliefs more often than the API did, on the same underlying model. Two human research assistants and GPT-5 itself independently scored the transcripts, and the API-versus-interface gap showed up no matter who was grading. The paper is on arXiv if you want the raw numbers; the methodology section is worth reading slowly, because the whole result hinges on them keeping the comparison genuinely fair.
The API and the chat app are not the same product
I used to think of this as one model wearing two hats. That’s wrong, and it took me embarrassingly long to unlearn it.
An API call is a clean JSON payload. Your system prompt, your parameters, nothing else in the room. Open ChatGPT on a laptop and you get memory of prior conversations, personalization the vendor built on top, a UI-level system prompt you never see, and safety thresholds that can differ from what the API enforces, sometimes changing week to week without anyone announcing it. You are, in a very literal sense, testing a different piece of software and calling it the same product because the marketing page uses one name for both.
The researchers didn’t hedge on this. Their own language: automated API testing is not sufficient to assess real-world chatbot impact. I’m quoting that, not softening it.
Multi-turn is where it actually breaks
Almost every eval benchmark I’ve seen tests one exchange at a time. Prompt in, score out, next case. That’s fine for catching an obviously bad single response. It tells you nothing about what happens on turn fourteen.
In the twenty-turn conversations, the pattern the researchers kept seeing was gradual. The model gives a little ground to keep the conversation moving, and the next turn treats that small concession as settled fact, and the one after that builds on it again, until somewhere past the midpoint the model is actively agreeing with the thing it pushed back on at the start. They call this a spiral, which is a better name than I would have come up with.
If that sounds familiar, it’s a cousin of sycophancy, the well-documented tendency of RLHF-tuned models to cave to a user who pushes back, even when the user is wrong. Anthropic wrote about this years ago, models flipping a correct answer to an incorrect one purely because the human disagreed. What this new paper adds is that the chat interface, with its memory and personalization stack, seems to make the drift worse than the bare API shows you.
I’ve watched a version of this happen with a client’s support bot outside any formal study. It nailed the diagnosis in isolation. Put it through a real troubleshooting session where the user pushes back three or four times, though, and by the end it’s hedging, then agreeing, then quietly helping the user go do the wrong thing.
(I wrote about what to actually measure in LLM reasoning flows a while back, and none of that advice is wrong, exactly. It just assumes your eval environment resembles what a real user sits in front of. This paper is evidence that assumption doesn’t hold as often as I’d hoped.)
Where this quietly wrecks a product
Say your eval pipeline only ever calls the API. Your single-turn safety tests pass, cleanly, every time, because the base model does push back on harmful claims when you hit it cold. You ship. Real users then have long conversations through your actual interface, where context piles up and the model’s tone drifts to match theirs, and by turn twelve it’s agreeing with something it would have flagged immediately on turn one.
This is worse for customer support bots and anything touching health, or really anywhere a user shows up already half-convinced of something false. Somebody opens with “I heard my medication got recalled,” and if the bot doesn’t correct that firmly at every single turn, one soft “I understand your concern” is often all it takes to start sliding.
Almost nobody is running the eval that would catch this. Single-turn tests are cheap, and they demo well in a slide. Building real multi-turn adversarial tests that hit the actual product interface, browser and all, is genuinely annoying work, and I get why teams skip it. API evals run in seconds and cost pennies. Browser automation is slow and breaks the moment someone tweaks a button. But a cheap test that misses the one failure mode that matters isn’t actually cheap. You’ve just moved the cost to later, and made it a customer’s problem instead of yours.
What I’d actually change this week
A few concrete things, roughly in the order I’d tackle them if I were rebuilding an eval pipeline from nothing.
Run conversations that go fifteen to twenty turns deep, not the three-turn toy examples most teams call “multi-turn.” Have a simulated user escalate a false claim gradually and check whether the model’s resistance at turn fifteen looks anything like its resistance at turn two. Usually it doesn’t.
Test against the interface your users actually touch, not the clean API underneath it. Playwright or something similar, driving the real product. Slower to write, slower to run, and it’s the only version of the test that tells you the truth.
Build a specific test for sycophancy drift: start with something false, have the simulated user restate it a bit more confidently each turn, and log the exact turn where the model stops pushing back. That turn number is your real safety margin. It’s probably smaller than whatever number is in your last eval report.
And be skeptical of any vendor safety score that doesn’t specify multi-turn and doesn’t specify the interface. Ask both questions directly. Most of the time the answer to at least one of them is no.
I do this kind of eval and integration work for clients, and this paper has already changed how I scope a project before I write the first test case. The API is where you start. It was never supposed to be where you stop.
