Legal and data10 min read

GDPR and AI in customer service: what may you do with customer data?

Short answer

You may let an AI read customer tickets on the same legal basis as the rest of customer service, usually the contract with the customer or a legitimate interest, if the vendor is your data processor under a written agreement. The data should stay within the EU/EEA or be protected in another approved way, and tickets may not be used for model training without a basis of their own. Since 2 August 2026 the customer must also be told they are talking to an AI.

Medgrundare, Supportifier (opens in a new tab)
Published

Anyone running customer service at a Swedish e-commerce or service company sooner or later gets the question from management, IT or a customer: may we really let an AI read our tickets? The question is reasonable. Tickets contain names, order numbers, addresses and sometimes more sensitive things, and two sets of rules apply at once: the General Data Protection Regulation (GDPR) and the EU AI Act. This article goes through what applies in practice. It is not legal advice; check with your data protection officer or a lawyer before you sign.

The most common basis is the contract with the customer, and where the contract is not enough, a balancing of interests is usually used. GDPR requires every processing of personal data to rest on one of six legal bases, according to the Swedish data protection authority IMY. When a customer contacts you about a late delivery you process name, order number and address to fulfil the purchase, and that is the contract basis. When the same customer asks a question before buying, or when you keep ticket history to handle a complaint later, it is often a legitimate interest: you have a legitimate reason, the processing is necessary for it, and the customer's interest in protection does not outweigh it.

Two things follow. Consent is rarely the right basis for customer service, because the customer cannot opt out of being helped and consent must be revocable at any time. And the basis should be documented in your record of processing before you change tools, not after.

What applies when an AI reads tickets: processors and processing agreements

Letting an AI service read tickets is as a rule the same processing you already do, carried out with a new tool, and the vendor becomes your data processor. A processor is someone who processes personal data on behalf of the controller; IMY's own example is a cloud service that stores and analyses data for a company (IMY, What is a data processor). You remain the controller. That means you decide the purposes and means, and you are responsible for the processing being lawful even when a tool does the work.

The relationship must be governed by a data processing agreement. Article 28.3 of GDPR sets the minimum requirements, among them that the processor may only process data on your documented instructions, including any transfers to third countries, and that staff are bound by confidentiality (IMY on data processing agreements). IMY's guidance on GDPR and AI has a section of its own on controllership with AI, and the basic question is the same: who decides what.

Where may customer data be stored, and may the vendor train on it?

Simplest is for the data to stay within the EU/EEA; anything else needs a specific basis for the transfer. Within the EU/EEA personal data can move freely because every country is covered by the same data protection. A transfer to a third country, a country outside the EU/EEA, is allowed only if the European Commission has decided the country has an adequate level of protection or if appropriate safeguards are in place, for example standard contractual clauses, and IMY points out that using cloud services often involves exactly such a transfer (IMY on transfers to third countries). So do not only ask where the database sits, but also where the language model runs and where logs end up.

Model training is a separate question. Using customer tickets to train or improve the vendor's own models is a new purpose. It needs a legal basis of its own and its own assessment against the principles of purpose limitation and data minimisation, which IMY highlights as the two hardest to meet in an AI context (IMY, AI and the basic principles). The European Data Protection Board (EDPB) writes in its Opinion 28/2024 that legitimate interest can be a possible basis for developing and using AI models, but only if the processing is shown to be strictly necessary and the balancing against the data subjects' rights holds. For a customer service team the conclusion is simple: the starting point should be that your tickets are not used for training at all, and that should be in the contract.

What is the difference between help centre knowledge and personal data in tickets?

Help centre articles are public knowledge without personal data, while tickets are personal data, and the two should be handled differently. An article about return terms, delivery times or how to change a password says nothing about any individual. It can be read by anyone, indexed by Google and used by a chatbot without GDPR coming into play. A ticket almost always contains a name, an email address and an order number, and sometimes information about health or finances, and is personal data from the first line.

That difference is architecture, not just law. A chatbot that answers from a reviewed knowledge base does not need to see any tickets at all to answer the common questions, while an AI that suggests replies in the inbox by definition reads personal data. If you want a chatbot that answers without guessing, the knowledge base should be the source and the tickets what is protected. So make sure nobody pastes ticket text into a help article.

What does the AI Act require of a chatbot in customer service?

The customer must be told they are talking to an AI, and that requirement has applied since 2 August 2026. The EU AI Act, Regulation (EU) 2024/1689 (EUR-Lex), is risk-based: some systems are banned, high-risk systems have extensive requirements, and a broader group of systems has transparency requirements. A chatbot in customer service is normally not high-risk, but it is covered by Article 50. According to the European Commission's questions and answers on Article 50, an AI system intended to interact directly with people must be designed so that those people are informed they are interacting with an AI, and for a chatbot the information must be given before or at the start of the conversation. The obligations apply from 2 August 2026.

During 2026 the AI Act was amended through the so-called digital omnibus package, which postponed the obligations for high-risk systems to December 2027 and August 2028 respectively. The Article 50 transparency requirements for chatbots were not moved; the only thing postponed was the labelling of AI-generated content for systems already on the market, to 2 December 2026, according to law firm Gibson Dunn's review of the agreement. If you have a chatbot in customer service today, the requirement already applies.

In practice it means a clear text when the chat opens ("You are chatting with our AI assistant") and a route on to a person. Handover is not an explicit requirement of Article 50, but it is what makes the transparency meaningful.

What should you ask a vendor?

Ask the questions in writing and keep the answers; they become the basis for the record of processing and any impact assessment.

  1. Are you a data processor for us, and is there a processing agreement that meets Article 28.3?
  2. Where are tickets stored, where does the language model run and where do logs end up? Is everything within the EU/EEA, and if not, with which safeguard?
  3. Which sub-processors are used, and how do we find out in advance when one changes?
  4. Are our tickets used to train or improve your models? The answer should be no, and it should be in the contract.
  5. How long are tickets, prompts and replies kept, and how do we delete them?
  6. How are permissions controlled, so the AI only sees what the employee using it may see?
  7. How does the chatbot meet the transparency requirement in Article 50?
  8. How is a personal data breach handled, and within what time are we notified?

The answers should match what the vendor publishes openly. How we answer these questions ourselves is on our page on security and data storage.

What to do

This can be done this week without buying anything.

  1. Look up customer service in your record of processing. Is the legal basis there, and does it match how you actually work?
  2. List every tool that touches a ticket today: ticketing system, email, chat, telephony, any AI add-ons. Is there a processing agreement with each?
  3. Separate knowledge from tickets. Go through the help centre and remove anything containing personal data from real tickets.
  4. Open your chat as a customer. Is it clear before or at the start of the conversation that it is an AI? If not, add the text now.
  5. Send the vendor questions above to your existing vendors, not just new ones, and go through the answers with the data protection officer or a lawyer.

Common questions

Normally not. The same legal basis as for the rest of customer service applies, usually the contract with the customer or a legitimate interest, because the AI is a tool for the same purpose. Consent fits poorly in customer service because it must be revocable and the customer cannot opt out of getting help. The customer should, however, be informed in your privacy policy that tickets are processed with automated support and by which processors.

May the AI vendor store data in the US?

Only if the transfer is supported under GDPR. That can be a European Commission adequacy decision for the recipient or appropriate safeguards such as standard contractual clauses, supplemented by an assessment of the risks. Simplest and most predictable is for tickets, language model and logs to stay within the EU/EEA. Ask the vendor explicitly about all three, because it is common for the database to be in the EU while the model runs somewhere else.

Does our chatbot have to say it is an AI?

Yes. Article 50 of the AI Act requires that anyone interacting with an AI system is informed of it, unless it is obvious, and for a chatbot the information must be given before or at the start of the conversation. The requirement applies from 2 August 2026 and was not postponed in the 2026 amendments. Put a clear text in the chat's first message and always offer a route to a person.

Who is liable if the AI answers wrongly about a customer?

You, as the controller and as the contracting party towards the customer. The vendor is responsible under the processing agreement for its part of the processing, but it is your company that answers for what is said to the customer. That is why AI replies in tickets about money, contracts or sensitive data should be reviewed by a person before they are sent.

Sources

Martin Carlsson

By

Martin Carlsson

Martin har fjorton år på Länsförsäkringar Gävleborg bakom sig. Han började med att leda en kundserviceenhet på drygt 20 medarbetare i telefon, mejl och chatt, satt sedan i ledningsgruppen som chef för verksamhetsutveckling och IT, och har de senaste åren ansvarat för den digitala kundupplevelsen. I dag är han affärsansvarig för Alf, Länsförsäkringars uppkopplade tjänst som bevakar hemmet och varnar innan skador uppstår. Dessförinnan digitaliserade och automatiserade han manuella försäkringsflöden på Gjensidige. Vid sidan av jobbet har han byggt egna webbtjänster, bland annat en jämförelsetjänst för elavtal. Martins motto är att automatisera allt som inte behöver mänskligt handlag, med kunden i centrum. Han har läst systemvetenskap vid Högskolan i Gävle och har på senare år certifierat sig inom AI hos Google.

LinkedIn (opens in a new tab)
  • gdpr
  • ai act
  • personal data
  • data processing agreement

This article is also available in Swedish: GDPR och AI i kundservice: vad får ni göra med kunddata?

Next step

Start from your everyday work.

Tell us which question takes time today. We go through what a first step could look like.

Book a walkthrough

A clear first step beats a big promise.

See how the knowledge work, the review and the first channel fit together.

About the knowledge analysis
All articles
From the same question
to a better answer.Book a walkthrough