Human-in-the-loop (HITL)· LLM operations
Human-in-the-Loop — A Person Approves Before the Bot Acts
Quick answer: In a human-in-the-loop (HITL) setup the AI proposes an action and a person decides whether it goes ahead. The pause happens before the action, which is what separates it from reviewing logs afterwards. It is used for actions that are hard to undo, cost money or touch private data. Done well it limits the damage a wrong answer can do; done badly it becomes a queue of clicks nobody reads.
Human-in-the-loop, on-the-loop and handoff
These terms are used loosely. The table gives the meaning we use on this site.
| Term | Who decides | When | Example |
|---|---|---|---|
| Human-in-the-loop | A person, per action | Before the action runs | Agent proposes a $120 refund; a person approves it |
| Human-on-the-loop | The AI, with a person supervising | While and after it runs | Agent acts alone; a person watches a dashboard and can stop it |
| Human handoff | A person takes over the conversation | When the bot cannot or should not continue | Customer is angry; the chat moves to an agent |
| Fully automated | The AI | No human step | Bot answers a store-hours question |
Handoff changes who is talking. HITL keeps the bot talking but puts a person's decision in front of one step. The two often sit in the same product: a bot can hand off a conversation and also ask for approval on a refund.
Which actions usually get an approval step
A common test, and our own rule of thumb rather than a standard, is to ask three questions of each action the bot can take: can it be undone, does it move money or change a record, and does it expose personal data? An action that fails any of them is a candidate.
- Usually approved by a person: refunds above a set amount, changing an order or address after dispatch, deleting or exporting customer data, sending a message to many contacts, account closure.
- Usually automatic: answering a question from the knowledge base, looking up an order status, booking a slot that can be cancelled, tagging a conversation.
Where the line sits is a business decision. The function calling entry explains why the model only requests a tool and your code decides whether to run it, which is exactly where an approval check can be placed.
How tools implement it
Two public examples show the pattern, from documentation read on 24 October 2026.
- LangChain. A
HumanInTheLoopMiddlewarechecks each proposed tool call against aninterrupt_onsetting. A matching call pauses the agent. The reviewer can approve (run as proposed), edit (change the tool arguments), reject (skip it and send feedback back to the agent) or respond (return a message instead of running the tool). The documentation warns against using respond to deny a tool that has side effects, because the model treats the message as a successful result. A checkpointer must be configured so the paused state survives until someone answers. - n8n. When an AI Agent tries to call a tool that has human review enabled, the workflow pauses and sends an approval request showing the tool name and the parameters. The reviewer can approve or deny. Approval channels listed include n8n's chat, Slack, Discord, Telegram, Microsoft Teams, Gmail and WhatsApp Business Cloud, and the approval channel can differ from the one the customer uses.
The details differ, but the shape is the same: proposed action, pause, decision, resume.
Why approval steps fail
A person in the loop is only a safeguard if they can really say no. Common failure modes, which are our reasoning about design rather than measured rates:
- Approval fatigue. If almost everything is routed for approval, reviewers start clicking through. Gate fewer actions and make each one count.
- Rubber-stamping. A request that shows only "Approve refund?" gives the reviewer nothing to check. Show the customer's message, the proposed action, the amount and the order.
- Delay. A customer waiting for an approved refund may wait hours if nobody owns the queue. Decide what the customer is told while they wait.
- Sidestepping. If the agent can reach the same result through a different tool that is not gated, the step protects nothing. Gate the outcome, not only one tool.
A related risk is prompt injection: text in a message or document that tries to steer the agent. An approval step can catch some of those attempts, but only if the reviewer sees what the agent is actually about to do.
Oversight rules for high-risk systems
Article 14 of the EU AI Act, which concerns high-risk AI systems, requires that such systems be designed so people can oversee them effectively. It lists abilities for the people assigned, including understanding the system's limits, staying aware of automation bias (over-reliance on its output), correctly interpreting outputs, deciding not to use the system or reversing its output, and stopping it safely. As published on artificialintelligenceact.eu, the page gives application dates of 2 December 2027 for Annex III systems and 2 August 2028 for Annex I systems. Whether a customer-service chatbot is high-risk depends on its use, so take legal advice rather than assuming. We are not lawyers.
Worked example (invented numbers)
A shop's agent handles 500 refund requests a month. The rule: refunds under $30 run automatically, refunds of $30 or more need approval.
- 380 requests are under $30 and run alone. 120 are $30 or more and go to a person. Check: 380 + 120 = 500, and 120 ÷ 500 = 24%.
- A reviewer takes about 45 seconds per case when the screen shows the message, order and amount. 120 × 45 = 5,400 seconds = 90 minutes a month.
- If the threshold were $10 instead and 420 of the 500 needed approval, the same 45 seconds each would be 420 × 45 = 18,900 seconds = 315 minutes, and more cases to rush through.
The numbers are made up. The point is that the threshold sets the reviewer's workload, and workload affects how carefully each case is read.
FAQ
What does human-in-the-loop mean for a chatbot?
A person approves, edits or rejects a chosen action before the bot carries it out. The bot proposes; the person decides.
Is human-in-the-loop the same as human handoff?
No. Handoff passes the whole conversation to a person. HITL keeps the bot in the conversation and adds a person's decision before a specific action.
What is human-on-the-loop?
The AI acts on its own while a person supervises and can step in. The person is not asked before each action.
Which chatbot actions need human approval?
Usually those that move money, are hard to undo, change records or expose personal data. Low-risk answers and reversible lookups usually do not.
Does a human approval step make an agent safe?
It reduces risk only if the reviewer has the facts and time to refuse. Too many requests, thin context and unowned queues weaken it.
Does the EU AI Act require human-in-the-loop?
Article 14 requires effective human oversight for high-risk AI systems. It does not say every chatbot needs per-action approval. Check whether your use counts as high-risk.
Related terms
- Human handoff — passing the conversation to a person.
- AI guardrails — automated limits that run without a person.
- Function calling — how a model requests an action your code can gate.
- AI agent — the kind of bot that takes actions and so needs gating.
- Audit log — the record of who approved what.
- Prompt injection — one reason to show reviewers the real action.
Sources
- LangChain, Human-in-the-loop — docs.langchain.com/oss/python/langchain/human-in-the-loop, read 24 October 2026. Source of the decision types, the checkpointer requirement and the warning about the respond decision.
- n8n, Human-in-the-loop for AI tool calls — docs.n8n.io/advanced-ai/human-in-the-loop-tools, read 24 October 2026. Source of the approve/deny behavior and the approval channels.
- EU AI Act, Article 14, Human oversight — artificialintelligenceact.eu/article/14, read 24 October 2026. Source of the oversight abilities and the application dates as published there.
- Chatbotscape evaluation methodology. /methodology (continuously updated).