Confession: I shipped a feature last spring that confidently told a user our refund window was 60 days. It’s 14. The model made up the number, wrote it in a clean, polite sentence, and I didn’t catch it until the user did. That’s the thing about LLM hallucination examples that nobody warns you about. They don’t look like errors. They look like answers.
I’ve spent a chunk of this year building retrieval-augmented chatbots for small teams, and I’ve collected a small museum of these failures. Some are funny. A few cost me a weekend. What I want to do here is show you actual examples, explain why they happen even when you “did everything right,” and walk through the cheap checks I now run before anything reaches a user.
The hallucinations I keep seeing in real apps
Let me start with specifics, because abstract talk about “factual reliability” never helped me debug anything.
The made-up policy number. The refund example above. The docs said nothing about 60 days. The model filled a gap with a plausible figure because plausible is what it optimizes for.
The phantom citation. I asked a support bot to answer “from the docs only,” and it produced a tidy answer with a reference to docs/billing/proration.md. That file doesn’t exist. It invented a filename that matched the shape of our repo.
The confident API method. This one stung because I trusted it. A coding assistant told me to call client.embeddings.batch_create(). Looks real. Reads real. The method has never existed. I lost an hour before I checked the actual SDK.
The half-right merge. The worst kind. The model pulls two real facts from context and stitches them into one sentence that’s wrong. “Your plan includes 50GB of storage and unlimited seats” when the plan has 50GB and five seats. Both numbers came from the docs. The combination is fiction.
If you want a longer tour of which models I run locally and how often they do this, I wrote up my setup in my notes on running local LLMs in 2026. The short version: every model I’ve tried does this. Bigger ones just do it more fluently.
Why do LLMs hallucinate even with retrieval?
The intuitive fix is retrieval. Give the model the right documents, tell it to answer only from them, and the made-up answers should stop. I believed this for an embarrassingly long time.
They don’t stop. They drop. Big difference.
A token predictor doesn’t “know” where its knowledge ends. When the retrieved chunks don’t contain the answer, the model doesn’t go quiet. It blends what it retrieved with what it half-remembers from training and gives you a sentence that sounds grounded. The retrieval step lowered the odds. It didn’t install a brake.
There’s research that backs up the uncomfortable part of this. A 2026 paper on Evidence Graph Consistency in retrieval-augmented generation evaluated hallucination detectors across six models on thousands of RAG responses and found something I didn’t expect: the structural signal that flags a hallucination in one model family points the opposite way in another. Their conclusion was blunt. Embedding-based graph consistency “cannot serve as a model-independent hallucination detection signal.” Translation for the rest of us: a check that works on your Llama setup might quietly fail when you swap in a different model. There’s no universal smoke detector you can bolt on once and forget.
So retrieval helps, and it’s still worth doing well. It just isn’t the finish line I wanted it to be.
The cheapest check that actually caught things: citation grounding
After the refund incident I stopped trusting the model’s prose and started trusting its receipts. The rule I settled on: every factual claim has to point at a retrieved chunk, and I verify the pointer in code, not in the prompt.
The trick is to make the model emit structured output with explicit source IDs, then check that those IDs were actually in the context you sent. Here’s the shape of it:
import json
def answer_with_citations(client, question, chunks):
# chunks: list of {"id": "doc_3", "text": "..."}
context = "\n\n".join(f"[{c['id']}] {c['text']}" for c in chunks)
valid_ids = {c["id"] for c in chunks}
prompt = (
"Answer using ONLY the sources below. "
"Return JSON: {\"answer\": str, \"citations\": [source_id, ...]}. "
"If the sources don't contain the answer, set answer to "
"\"I don't have that in the docs.\" and citations to [].\n\n"
f"{context}\n\nQuestion: {question}"
)
raw = client.complete(prompt) # your model call
result = json.loads(raw)
# The part that matters: verify the receipts in code.
cited = set(result["citations"])
if not cited.issubset(valid_ids):
bogus = cited - valid_ids
raise HallucinationError(f"model cited unknown sources: {bogus}")
return result
That last block is doing the real work. If the model cites doc_9 and you only sent doc_1 through doc_5, you know it’s improvising before the user sees a word. This single check would have caught my phantom-citation bug. It does nothing for the made-up refund number, though, because the model could cite a real chunk and still misread it. Which is why I don’t rely on it alone.
Detecting the subtle ones with self-consistency
The half-right merges and misread numbers slip past citation checks. For those I lean on a different idea: ask the model more than once and watch whether it agrees with itself.
A hallucinated fact tends to wobble. Confident, real answers tend to repeat. So I sample a few responses at a non-zero temperature and compare. If the answers diverge on the specific claim, I flag it for a human or fall back to “I’m not sure.”
from collections import Counter
def self_consistency_flag(client, prompt, n=5):
answers = [client.complete(prompt, temperature=0.7) for _ in range(n)]
norm = [a.strip().lower() for a in answers]
top, count = Counter(norm).most_common(1)[0]
agreement = count / n
return {"answer": answers[0], "agreement": agreement,
"trustworthy": agreement >= 0.6}
It’s crude. It costs you n times the tokens, so I only run it on high-stakes paths like billing or anything that quotes a number. But it works often enough to earn its keep. A 2026 paper called BEACON formalizes this instinct far better than my five-sample hack: it pulls a 31-dimensional feature vector from multi-pass generation, including paraphrase stability and chain-of-thought consistency, and gets around 0.81 AUROC at flagging hallucinations from black-box model outputs alone. The takeaway I drew from it is that consistency across samples is a genuine signal, not just a folk remedy. You can build something serious on it.
There’s also promising work on structure. One group cut hallucinations on a hard Wikipedia question-answering benchmark by giving the retriever a lightweight graph of the evidence instead of a flat pile of chunks, which improved factual precision and recall. I haven’t shipped that in production yet, but it’s next on my list to try.
What I actually do now, in order
Here’s the pipeline I run, stripped down. None of it is fancy and that’s the point.
First, retrieve well. Better chunks mean fewer gaps for the model to paper over. Most of my early “hallucinations” were really retrieval misses wearing a costume.
Second, force structured answers with citations, and verify the citation IDs in code. Reject anything that points at sources you never sent.
Third, run self-consistency on the money paths only. Numbers, dates, policy claims. If the samples disagree, I’d rather say “I don’t know” than guess on a user’s behalf.
Fourth, log every “I don’t have that in the docs” response. Those are gold. They tell me exactly which questions my retrieval is failing, which is usually a content gap I can fix by writing one more doc.
And fifth, never let one model grade its own homework. I learned this the hard way and wrote about it separately in why I stopped letting one model fix itself. A second model, or a plain code assertion, catches things the first one is structurally blind to.
Try one thing this week
Pick your highest-stakes endpoint, the one that quotes a number or a policy. Add the citation-verification block above. That’s it. Maybe fifteen lines. You’ll either prove your model is well-behaved on that path, or you’ll catch it inventing a source ID, and both outcomes are worth the afternoon.
I keep more of these build notes and the projects behind them in my work if you want to see how this plays out beyond a toy example. Hallucinations aren’t going away. But the gap between “the model lied to a user” and “the model got caught lying before anyone saw it” is mostly a few lines of verification you control. I’d rather own those lines than trust the prose.
