How to stop the chatbot guessing: grounded AI in customer service

Anyone running customer service at a company with a few hundred tickets a week sooner or later gets the question of whether a chatbot can take the simplest ones. The answer depends entirely on where the bot gets its answers. A bot that answers freely can sound confident and still be wrong, and it is the company, not the vendor, that has to stand behind the answer towards the customer.

What does it mean for a chatbot to be grounded?

A grounded chatbot may only answer with content that exists in a defined source, usually the company's knowledge base, and must be able to show where the answer came from.

In practice that means three things:

1. The bot answers only from published articles. When the customer asks about the return period, the bot fetches your returns article and phrases the answer from it. If the knowledge base says nothing about returns, the bot must not invent a period.
2. The bot cites its source. The customer sees which article the answer is built on and can read the full text.
3. The bot says so when the material is missing. A "I do not have an answer to that, let me connect you" is a correct answer. A guessed answer is not.

The technique behind it is often called retrieval-augmented generation, RAG: the system first retrieves relevant passages and then lets the model write an answer from exactly those. A general language model, the kind found in public AI assistants, lacks that link. It has been trained on huge amounts of text and answers everything, whether or not it has your company's terms in front of it. That is why a knowledge base with reviewed articles is the prerequisite for a chatbot in customer service, not an option.

What is a hallucination, and why do language models guess?

A hallucination is an answer that sounds credible but has no support in any material. The model produces the continuation of the text that is statistically most likely, and when the facts are missing that becomes a plausible guess.

Researchers at OpenAI described in 2025 why this happens: the way models are trained and evaluated rewards guessing over admitting uncertainty, rather like a multiple-choice exam where a blank answer never scores but a guess sometimes does. The model learns to always say something.

Connecting the bot to a source reduces the problem but does not remove it by itself. When Stanford RegLab tested commercial legal AI tools in 2024, tools built with exactly this kind of retrieval, they hallucinated in 17 to 33 percent of 202 test questions. The conclusion for customer service is simple: retrieval is necessary, but the bot must also be forced to stick to what was found and allowed to say that the answer is missing.

What can go wrong in customer service? The Air Canada case

The company is responsible for what the chatbot says, just as for an ordinary page on the website. A Canadian tribunal established that in Moffatt v. Air Canada in 2024.

A traveller asked the airline's chatbot about discounted fares after a death in the family. The bot replied that the discount could be claimed retroactively. Another page on the same website said the opposite, and when the customer requested a refund he was refused. The airline argued that the chatbot was a separate entity the company was not responsible for. The tribunal dismissed that: the chatbot is part of the website, and it makes no difference whether the information comes from a static page or a bot. The customer was awarded 650.88 Canadian dollars.

The amount is small. The principle is not. Translated to Swedish e-commerce, the same mechanism can produce:

• An invented discount code or a promise of free shipping the customer then demands.
• The wrong return period, the wrong warranty term or the wrong conditions for withdrawal.
• A statement that an item is in stock or will be delivered on a certain date, without the bot having looked in any system.

On top of that comes a transparency requirement. According to the European Commission's guidance on Article 50 of the AI Act, anyone talking to an AI system must be informed of it at the latest at first contact, and the obligation applies from 2 August 2026.

How do you check that a vendor's chatbot is grounded?

Do not ask whether the bot "uses AI"; ask what it is allowed to answer and what happens when the answer does not exist:

1. Where does the bot get its answers? You want to hear "only from your knowledge base", not "from your knowledge base and the model's general knowledge".
2. What happens when the answer does not exist? Ask for a demonstration and pose a question you know you have no article for. A grounded bot says it does not know or hands over. A guessing bot answers anyway.
3. Does the bot show a source for every answer? The customer should be able to click through to the article.
4. Can we see the list of questions the bot could not answer? That list is your most important material for finding knowledge gaps.
5. Who can publish content the bot uses, and is there review?
6. Can we block topics? Price changes, legal assessments and promises of compensation the bot should never handle alone.
7. How is the bot tested before launch and how are errors measured over time? You want to hear about test questions with known answers and spot checks after launch.
8. What does the contract say about liability for wrong answers? The liability towards the customer lands with you regardless.

A simple test of your own goes a long way: ask ten questions, five of which you have clear articles for, three you have no answer to, and two where your own sources disagree. A grounded bot handles the five, hands over the three and exposes the contradiction in the two.

How should the handover to a person work?

The handover should happen automatically when the bot lacks material, always when the customer asks for it, and as a rule when the issue concerns money, contracts or complaints.

Customers are clear on this point. In a Gartner survey of 3,566 customers in spring 2026, 87 percent said it is essential to be able to reach a person when the company uses generative AI in customer service. Among those who did not want to use AI at all, the option to switch to a person was the most common thing that would change their mind. Another Gartner survey from 2026 shows what happens otherwise: customers are about three times as likely to turn to an external AI service as to the company's own chatbot.

A good handover has three parts:

• The customer does not have to start over. According to Zendesk's CX Trends 2026, 81 percent of consumers want the agent to continue where they left off, and 74 percent are frustrated at repeating themselves. The whole chat history should follow into the ticket.
• The agent sees what the bot said and which article it used. Then the error can be corrected in the article, not just in the individual ticket.
• The contact route is visible. The Swedish Consumer Agency's rules for businesses on customer service say contact details and how to complain must be clearly stated.

The handover is simplest when bot and person work from the same material. In Supportifier the same reviewed articles are used in the help centre, the chat and the team's inbox, so when the ticket changes channel it does not change knowledge; how the parts fit together is described on the platform page. The same principle applies when AI suggests replies an agent reviews before sending, which we go through in the article on AI drafts with review.

What to do

This can be done this week, whether you have a chatbot today or are only considering one:

1. List your twenty most common questions and check that each has a published article with a named owner.
2. Run the ten-question test above on your current bot or on the vendor's demo. Write down every answer and what it should have been.
3. Decide a block list: topics the bot may never handle alone. Compensation, price deviations, legal matters and personal data are a common start.
4. Write down the handover rule on one page: when, to whom, with what information.
5. Book a recurring hour a week to read the bot's unanswered questions and write the articles that are missing.
6. Measure three things: the share of answers with a source, the share of handovers and the share of customers who come back with the same question within a week. The same measures are part of the business case for AI in customer service.

Common questions

Can a chatbot answer wrongly even if it is grounded?

Yes. Grounding reduces errors sharply but does not remove them. The bot can fetch the wrong article, misread an unclear article or answer correctly from an article that is itself out of date. That is why source citations are needed, so the error can be traced, and a routine for correcting the article when an error is found. The quality of the knowledge base sets the ceiling for the bot's quality.

Do we have to tell the customer they are talking to an AI?

Yes. Article 50 of the EU AI Act requires that anyone interacting with an AI system is informed of it at the latest at first contact, unless it is already obvious, and the requirement applies from 2 August 2026. A clear text at the start of the chat is enough in most cases. It is also good customer service: someone who knows they are talking to a bot asks for a person sooner when needed.

What happens if the chatbot promises something we cannot keep?

Assume you will have to honour it. In the 2024 Air Canada case the tribunal rejected the argument that the bot was a separate entity and ordered the company to compensate the customer for the difference between the promised and the actual price. Swedish consumer law is of course tried in Swedish forums, but the principle that the company is responsible for the information on its website is the same. A block list for sensitive topics is cheaper than untangling promises afterwards.

Is a disclaimer on the bot enough?

No. Air Canada had information on its website that contradicted the bot's answer, and the tribunal did not consider the customer obliged to double-check one page against another. A disclaimer in small print weighs little against a concrete statement the customer received in the chat. What actually protects you is that the bot cannot say anything that is not in a reviewed article.

Sources

• Moffatt v. Air Canada, 2024 BCCRT 149 — Civil Resolution Tribunal of British Columbia via CanLII, 2024
• Why language models hallucinate — OpenAI, 2025
• Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools — Stanford RegLab, 2024
• Transparency obligations under Article 50 of the AI Act — European Commission, 2026
• Gartner Survey Finds 87% of Customers Say Companies Using GenAI for Customer Service Must Provide Access to a Human Agent — Gartner, 2026
• Gartner Survey Finds Customers Are 3x More Likely to Use Third-Party GenAI Than Company-Provided Chatbots for Customer Service — Gartner, 2026
• AI Ushers In Era of Contextual Intelligence, Redefining Customer Experience in 2026 — Zendesk, CX Trends 2026
• Konsumentproblem i kundtjänst: regler för företag — Swedish Consumer Agency, read 2026