A 2019 research paper on out-of-scope prediction, Amazon's Bedrock Guardrails documentation on denied topics and Intercom's Fin Guidance help article were read on 30 October 2026. Product features change, so check each vendor's current documentation.

Out-of-scope detection· Quality and safety

Out-of-scope detection is a chatbot's ability to recognize that a message asks for something outside the job it was built to do, so it can decline politely, redirect or hand off instead of guessing an answer.

By Chatbotscape Editorial· Methodology· Published 31 October 2026· Updated 31 October 2026

Out-of-Scope Detection — Knowing What the Bot Should Not Answer

Quick answer: Every business chatbot has a job: orders, bookings, product questions. Anything else, from poems to legal advice to a competitor's pricing, is out of scope. Out-of-scope detection is the mechanism that notices this and triggers a planned response. It can run through rules in the system prompt, a confidence threshold on intents, a separate classifier, or a guardrail that blocks named topics. Every method makes two kinds of mistake, and you choose how to balance them.

Out of scope versus not understood

Situation	Example	What the bot should do
In scope, understood	"Where is my order?"	Answer
In scope, not understood	"ordr thing late??"	Ask a clarifying question; see reprompt
Out of scope, understood	"Write me a poem about cats"	Decline politely, state what it can do
Out of scope, sensitive	"Is this rash serious?"	Decline and point to a professional or a human

The second row counts toward your fallback rate. The third and fourth should not: the bot understood and chose not to answer.

Why it is hard

Researchers have studied this for years. Larson and colleagues released a 2019 dataset covering 150 intents across 10 domains, plus queries that fit none of them, and reported that classifiers do well on in-scope intents but struggle to identify the out-of-scope queries. Large language models add a new difficulty: they are willing to answer almost anything, so without a boundary a support bot will happily write code or discuss politics on your account.

Four ways to detect it

Prompt rules. The system prompt states the job and what to do with everything else. It is cheap and a good start, but Intercom's help article on Fin Guidance says its guidance "shapes tone, formatting, and style but does not strictly control behavior" and that hard rules belong elsewhere. Treat prompt text as a strong hint, not a lock.

Confidence threshold. In an intent-based bot, a message that matches no intent above a set confidence score is treated as out of scope. Simple, but the threshold needs tuning against real messages.

Separate classifier. A small model or a retrieval-distance check labels each message in scope or out of scope before the main model sees it. More work, easier to measure.

Guardrail topics. Some platforms let you name topics to block. Amazon Bedrock Guardrails, for example, defines a denied topic with a name, a definition of up to 200 characters and up to five sample phrases, supports up to 30 denied topics per guardrail, and can block matching prompts and responses. Its documentation also warns that topics are themes, not words, and should not be written as exceptions. See AI guardrails.

Two errors, one trade-off

Leak: an off-topic question gets answered. Risks include wrong advice, brand damage and wasted model cost.
Over-refusal: a legitimate question gets refused. This frustrates customers and hides your product.

Tightening the boundary lowers leaks and raises over-refusals. You need a test set with both kinds of messages to see the balance; see golden dataset.

Worked example (invented numbers)

A test set holds 100 messages: 60 in scope and 40 off topic.

With prompt rules only, the bot declines 28 of the 40 off-topic messages (70%), so 12 leak (30%), and wrongly refuses 3 of the 60 in-scope ones (5%).

After adding a classifier, it declines 36 of 40 (90%), so 4 leak (10%), but wrongly refuses 6 of 60 (10%).

Scale to 5,000 monthly messages, 20% off topic: 1,000 off-topic and 4,000 in-scope. Prompt rules leak 1,000 × 30% = 300 and over-refuse 4,000 × 5% = 200. The classifier leaks 1,000 × 10% = 100 and over-refuses 4,000 × 10% = 400. Fewer leaks cost more wrongly refused customers; which is worse depends on your business. The numbers are illustrative, not a benchmark of any product.

FAQ

What is out-of-scope detection in a chatbot?

The ability to recognize that a message asks for something outside the bot's intended job, so it can decline, redirect or hand off instead of answering.

How is it different from a fallback response?

A fallback covers messages the bot could not understand. Out-of-scope handling covers messages it understood but should not answer.

Can a system prompt keep a chatbot on topic by itself?

It helps a lot but is not a guarantee; vendors describe such instructions as guidance rather than strict control. Add a check outside the model for anything that matters.

Should the bot refuse or hand off?

Refuse politely for chit-chat and unrelated requests; offer a person for anything sensitive or likely to be a real customer need. See human handoff.

How do I measure it?

Keep a test set of in-scope and off-topic messages and count both leaks and over-refusals after every change.

Is this the same as prompt injection defense?

No. Prompt injection is an attack to override the bot's instructions. Out-of-scope handling is about ordinary questions. The two overlap in tooling; see prompt injection.

Fallback intent — the catch-all for messages the bot did not understand.
Chatbot fallback rate — how often that happens.
AI guardrails — controls on what a model may say.
System prompt — where the job description lives.
Chatbot intent confidence — the score behind threshold detection.
Human handoff — the exit for sensitive questions.

Sources

Larson et al., An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction, 2019 — arxiv.org/abs/1909.02027, read 30 October 2026. Source of the 150-intent, 10-domain dataset and the finding that classifiers struggle with out-of-scope queries.
AWS, Block denied topics (Amazon Bedrock Guardrails) — docs.aws.amazon.com/bedrock/latest/userguide/guardrails-denied-topics.html, read 30 October 2026. Source of the topic definition limits, the 30-topic cap and the caveats.
Intercom, Fin Guidance help article — intercom.com/help/en/articles/10210126-fin-guidance-beta, read 30 October 2026. Source of the statement that guidance does not strictly control behavior.
Chatbotscape evaluation methodology. /methodology (continuously updated).