Intercom's community post on AI Summaries, Ada's Zendesk Chat handoff documentation and a 2019 paper on summarization evaluation were read on 26 October 2026. Product behavior changes, so check the current documentation before you rely on it.

Conversation summarization· LLM operations

Conversation summarization is the automatic shortening of a chat or call transcript into a brief note covering what the customer wanted, what has been done and what is still open, most often written for a human agent who takes over from a chatbot.

By Chatbotscape Editorial· Methodology· Published 27 October 2026· Updated 27 October 2026

Conversation Summarization — A Short Note So Nobody Re-Reads the Chat

Quick answer: Conversation summarization compresses a long exchange into a few lines. In customer support it is used mainly at handoff: the bot writes a note, the agent reads it in seconds and does not ask the customer to start over. A language model usually writes the note, which makes it fast and also capable of leaving out a key fact or inventing one, so the note needs a structure and a regular check.

Where summaries are used

At handoff. The most common case: a note for the agent who takes over from the bot.
At the end of a conversation. A wrap-up for a ticket or CRM record.
When an old conversation is reopened. A teammate catches up without scrolling.
Inside the bot. Older turns are shortened so a long chat still fits the model's context window. That is a different job, aimed at the model rather than a person.

Two ways to summarize

Method	How it works	Strength	Risk
Extractive	Picks existing sentences or fields and shows them as written	Cannot invent wording	Can be long, and can miss meaning spread over several messages
Abstractive	A model writes new text that restates the conversation	Short and readable	Can state something the customer never said

Most chatbot products that use a large language model are abstractive. Many teams add an extractive element on purpose, such as showing the customer's last message verbatim next to the generated note.

What a handoff summary usually contains

This list is our own working practice, not an industry standard.

Who and what. The customer's name or ID, and the issue in one sentence.
Already done. Steps the bot took or answers it gave, so the agent does not repeat them.
Still open. The question or action that is left, and why the bot stopped.
Facts to use. Order number, dates, amounts and anything the customer supplied.
Mood and urgency. Only if the conversation shows it, stated as what the customer said.

How two vendors describe theirs

Two public descriptions, read on 26 October 2026.

Intercom. Its community post says AI Summaries are generated at handoff, typically when Fin passes a conversation to a teammate, and posted as an internal note so the teammate sees what was covered without reading the full history. They are switched on with an "Add summary note" action in a triage workflow. The post does not describe settings or limits.
Ada. Its Zendesk Chat handoff page says the human agent sees the full AI Agent conversation up to the point the customer joined the queue. Ada can also attach a summary showing why the customer is escalating. It uses up to three of the customer's text messages, ignores button clicks and the AI Agent's own messages, and attaches nothing if it cannot work out the customer's intent.

The two differ in what they feed the summary and where they show it. Neither page states an accuracy figure, so we state none.

A readable summary can still be wrong

A summary that reads smoothly is not evidence that it is faithful. Kryscinski and colleagues, in a 2019 paper on news summarization, found that the common overlap score ROUGE correlated only weakly with human ratings, including ratings of factual consistency, and that in a manual review of 200 summaries from abstractive models, 60 (30%) had consistency problems. That study covered news articles, not support chats, so it does not give a rate for your bot. It does show why overlap scores alone are a poor check and why a person, or a carefully tested LLM judge, should compare summaries with the transcript.

Typical failures in support are an omitted detail (the order number), a wrong detail (a refund amount) and a made-up action ("the bot already refunded you"). The last two are a form of AI hallucination.

Personal data in summaries

A summary can copy names, addresses or card fragments into a ticket or CRM. Decide what is allowed in a note and whether it is stored, and see the data retention policy entry for how long it is kept. This is general information, not legal advice.

Worked example (invented numbers)

A team checks 50 handoff summaries against their transcripts.

41 are correct and complete, 6 miss a key fact and 3 contain a wrong fact. Check: 41 + 6 + 3 = 50, so 82%, 12% and 6%.
At 400 handoffs a month, 6% with a wrong fact would be 400 × 0.06 = 24 notes. If the same share held, that is 24 customers an agent might mishandle.
If a good note saves an agent about 75 seconds of reading, 400 × 75 = 30,000 seconds = 500 minutes, about 8.3 hours a month.

The numbers are made up. The point is that time saved and wrong notes grow together, so both belong in the review.

FAQ

What is conversation summarization in a chatbot?

It is the automatic shortening of a chat into a brief note, usually written for a human agent at handoff.

What is the difference between extractive and abstractive summarization?

Extractive summaries reuse existing sentences. Abstractive summaries are newly written by a model and can be shorter but can state things that were not said.

What should an AI handoff summary include?

The customer's issue, what has been done, what is still open and the facts the agent needs, such as an order number.

Can an AI chat summary be wrong?

Yes. It can omit a detail, get one wrong or invent an action, so summaries should be sampled and compared with transcripts.

Is a summary the same as the transcript?

No. Keep the full transcript available. The summary is a shortcut, not a replacement.

Is ROUGE a good way to check chat summaries?

On its own, no. A 2019 study found it correlated only weakly with human ratings of factual consistency in news summaries.

Human handoff — the transfer the summary supports.
Warm transfer — a handoff where context travels with the customer.
Agent assist — live help for agents, which can include summaries.
Chatbot context window — why a model may need older turns shortened.
AI hallucination — the main way a summary goes wrong.
LLM as a judge — one way to check summaries at scale.

Sources

Intercom Community, Save time and avoid repetition with AI Summaries — community.intercom.com, read 26 October 2026. Source of the handoff note behavior and the triage workflow action.
Ada, Zendesk Chat handoff usage — docs.ada.cx, read 26 October 2026. Source of the transcript, summary and no-intent behavior.
Kryscinski et al., Neural Text Summarization: A Critical Evaluation, 2019 — arxiv.org/abs/1908.08960, read 26 October 2026. Source of the ROUGE correlation finding and the 60 of 200 figure.
Chatbotscape evaluation methodology. /methodology (continuously updated).