Why your AI chatbot keeps making things up (and how to stop it)
Hallucination isn't a bug you patch. It's what happens when a model is asked to answer without being given the answer — and the fix is simpler than it sounds.
If you've tried a chatbot plugin on your store, you've probably seen it: a customer asks about your return window and the bot confidently invents one. Thirty days becomes sixty. "Tags attached" becomes "any condition." The bot isn't lying — it just doesn't know your policy, and language models are built to answer anyway.
Why it happens
A language model is a prediction engine, not a database. Ask it a question and it produces the most plausible-sounding answer — plausible based on everything it read on the internet, not on your store's actual policies. Generic plugins bolt a chat window onto that engine and hope. When your policy isn't in the model's context, "plausible" is the best you get, and plausible is often wrong.
The fix: make it read before it speaks
The reliable pattern is called retrieval-augmented generation — RAG. In plain terms: before the model answers, the system fetches the exact passages from your real documents that relate to the question, and instructs the model to answer only from those passages.
Customer asks "Can I return a worn jacket?" → system retrieves your actual returns policy → model answers from that text → the answer cites
returns-policy.md, section 2underneath.
Two things change. The answer is grounded in your policy, and the customer can see the receipt. If retrieval finds nothing relevant, the honest move is built in: the assistant says "I'm not sure" and hands off to a human, instead of improvising.
What to ask any vendor (including me)
Three questions separate grounded systems from confident improvisers:
- Where does the answer come from — my content, or the model's memory?
- Can the customer see the source of each answer?
- What happens when it doesn't know?
If a vendor can't answer all three crisply, the bot will make things up. It's not a maybe — it's the default behavior of the underlying technology.
I build support assistants this way for online stores — grounded, sourced, with graceful escalation. If you want to see it working, the live demo is here.