Should AI run your support? Ask these 5 questions.

Vendors will tell you AI agents can answer most of your tickets. Before you believe it, look at the best independent evidence we could find, and read what it actually measured.

AI as a coach, not a replacement

The most-cited study is "Generative AI at Work", an NBER working paper by Erik Brynjolfsson, Danielle Li and Lindsey Raymond. It followed 5,179 customer support agents at a software firm serving small businesses. When the agents got an AI assistant, they resolved 14% more issues per hour on average. For novice and low-skilled agents the gain was 34%, while experienced agents saw minimal impact. The authors also report better customer sentiment and lower staff turnover.

Read the setup carefully, though. The AI suggested replies to human agents. It did not replace them, and the authors state that the paper is not designed to say what happens to total employment or wages. They also note that AI assistance may reduce the quality of conversations by the most skilled agents.

When the bot gets it wrong

In 2024 a customer, Mr. Moffatt, asked Air Canada's website chatbot about bereavement fares. The bot suggested he could claim the discount retroactively. The airline then refused. The British Columbia Civil Resolution Tribunal sided with the customer. Air Canada had argued that the chatbot was a separate legal entity responsible for its own actions. The tribunal called that "a remarkable submission". It said a chatbot is part of the company's website, and that the company is responsible for all the information on it, whether it comes from a static page or a chatbot. Air Canada was ordered to pay about 812 Canadian dollars.

The amount is small, and it is a small-claims decision, so it does not bind other courts. But the principle is the one to remember: your bot's words are your words.

Regulators and analysts are cautious

• The US Consumer Financial Protection Bureau, looking at banks, found chatbots handle basic questions but become less effective as problems get more complex. It warned about bots that fail to recognize disputes, give inaccurate information, or trap customers in loops with no way to reach a human.
• Gartner, a private research firm, predicts over 40% of agentic AI projects will be canceled by the end of 2027, citing rising costs, unclear business value or weak risk controls. It also estimated that only about 130 of the thousands of vendors calling themselves "agentic" are real. This is a forecast, based on a poll of webinar attendees.
• In a separate release, Gartner predicts that by 2027 half of the organizations that expected to cut their customer service workforce will abandon those plans. Its poll of 163 service leaders found 95% plan to keep human agents.
• A Gartner survey of 5,728 customers in 2024 found 64% would prefer that companies did not use AI for customer service.

Run this for each type of request you get, not for support as a whole. "Where is my order?" and "I want to dispute a charge" belong in different places. The questions are our own synthesis of the sources above, not an official checklist.

1. Is the work routine?

Questions that repeat and follow fixed steps, such as opening hours, delivery status or a returns policy, suit automation. Gartner's own advice is to use automation for routine workflows, assistants for simple retrieval and agents when decisions are needed. It also says many use cases sold as "agentic" do not need an agent at all. OpenAI's guide agrees: unless the task involves complex judgement, rules that are hard to maintain, or lots of unstructured data, "a deterministic solution may suffice". For fixed steps, plain automation is cheaper and more predictable.

2. What does a wrong answer cost?

If a wrong answer would cost money, break a law or hurt someone, do not let a bot answer alone. The Air Canada case is a small version of this. The CFPB's list of risks is the larger one. Put the money, legal and safety questions in front of a person.

3. Can you check the result, and undo it?

Good candidates have a clear "done" you can verify: an order found, a form sent, a ticket tagged. OpenAI's guide says actions that are sensitive, irreversible or high-stakes, such as cancelling orders, authorizing large refunds or making payments, should trigger human oversight until you trust the agent. So start read-only or draft-only, and let a person press the button.

4. Is there always a way to a human?

This is where customers lose patience. The CFPB calls out "doom loops". Gartner's customer survey names difficulty reaching a person as a top concern. OpenAI recommends setting limits on retries and handing over when the agent repeatedly fails to understand the customer. Intercom's escalation guidance has its assistant hand over straight away when the customer clearly asks for a human, and offers escalation when they show anger or repeat themselves. Build in a visible "talk to a person" option and pass the conversation across with its history.

5. Will you be honest about it?

• EU. Article 50(1) of the AI Act says systems intended to interact directly with people must be designed so that people are told they are dealing with AI, unless that is obvious. It has applied since 2 August 2026, and the 2026 Digital Omnibus did not delay it. The duty falls mainly on the provider of the system. See our guide to the EU AI Act.
• California. SB 1001 makes it unlawful to use a bot to mislead people about its artificial identity in order to encourage a purchase or sale. You are not liable if you disclose that it is a bot, in a clear and conspicuous way.
• Maine. A 2025 law prohibits using an AI chatbot in trade with a consumer in a way that may mislead a reasonable consumer into thinking they are talking to a human, unless the consumer is told.
• Marketing claims. The US FTC said in 2024 that there is no AI exemption from the laws on the books, and took action over a chatbot sold as a "robot lawyer" without testing. Do not overclaim what your assistant can do.

Count how many questions you can answer "yes, we're fine" for a given type of request.

1. Week 1Export a month of tickets and group them by type. Mark each type routine or not.
2. Week 2Run the test on the top five types. Write down what a wrong answer would cost.
3. Weeks 3 to 4Pilot an assistant that drafts replies for your agents. This is the setup with the best evidence behind it.
4. Month 2Measure resolution time, customer ratings and handovers. Only then let it answer the easiest question types by itself, with a clear label and a way to reach a person.

If you want the difference between a chatbot and an agent first, read our guide to agentic AI vs chatbots, and our checklist for running agents safely.