OpenAI's Structured Outputs guide and launch post were read on 16 October 2026. Provider features and limits change often, so check current documentation before you rely on a specific one.

Structured outputs· LLM operations

Structured outputs is a way of making a language model reply in a format you define in advance, usually a JSON object that follows a JSON Schema, so that other software can read the reply directly. In a chatbot it is used when the model must hand back data, such as an order number or a category, rather than prose for the customer.

By Chatbotscape Editorial· Methodology· Published 17 October 2026· Updated 17 October 2026

Structured Outputs — When the Model's Reply Has to Be Readable by Software

Quick answer: A language model normally answers in free text. That is fine when a customer reads it, and a problem when your own system needs to read it, for example to look up an order or route a ticket. Structured outputs solve this by constraining the reply to a schema you supply: a list of fields, their types and which are required. When the feature is switched on in its strict form, the reply is meant to match the schema exactly, so your code can parse it without defensive guessing. What it does not do is check that the values are true. A perfectly shaped reply can still contain the wrong order number.

Three things that get confused

Three terms are often used as if they meant the same thing. According to OpenAI's documentation, they do not.

Approach	What it guarantees	What it does not
Plain prompt ("reply in JSON")	Nothing; the model usually complies	Valid JSON, correct fields, or no extra text around it
JSON mode	The reply is valid JSON	That it matches your schema: fields can be missing or named differently
Structured outputs (strict schema)	The reply matches your JSON Schema, unless it is a refusal or is cut off	That the values are correct

Function calling sits beside this table rather than in it. OpenAI's guide separates the two by purpose: function calling connects the model to your tools, functions and data, while a structured response format shapes the reply that goes to the user or to your code. Many chatbots use both: a function call to fetch an order, then a structured reply to hand the result on. Which of these your provider offers, and under which model names, differs, so check its documentation.

How strict mode works

In OpenAI's version, you pass a schema and set strict to true. The guide's examples mark every object with additionalProperties set to false and list every property as required; that is the pattern shown rather than a rule stated in so many words, so read the current requirements before you copy it. If a value is optional in your business logic, the usual way to express that under such a rule is to let the field accept an empty value, such as null, instead of leaving it out. That wording of the workaround is our explanation.

Only a subset of JSON Schema is supported, so a schema copied from another system may need trimming. Schemas supplied this way are also not eligible for zero data retention, a point worth raising with whoever owns your privacy commitments.

Edge cases that still reach your code

A guarantee has conditions, and three of them matter to a chatbot.

Refusals. The model can still decline a request it judges unsafe. OpenAI reports a refusal in a separate field on the response, so your code can detect it instead of trying to parse a sentence of apology as data. Your code needs a branch for it.

Cut-off replies. If generation stops early, for example because the output hit its length limit, the reply can fail to follow the schema. OpenAI's launch post says a reply that has no refusal and was not cut short should match. So your code should check why generation ended before it parses.

Slow first request. The first request with a new schema is slower, because the schema is processed before use. OpenAI says typical schemas take under ten seconds and complex ones up to a minute, with the result cached for later requests. In a live chat, it is better to send a warm-up request after a deploy than to let a customer's message pay that cost.

Shape is not truth

This is the limit that matters most in practice, and it is our explanation rather than a vendor claim. A schema controls structure: the reply has an order_id string, a category that is one of five allowed labels, a needs_human true or false. It cannot know whether the order number was copied correctly from the customer's message or whether the category was the right one. A guaranteed shape can make wrong answers look more trustworthy because they arrive neatly. Treat the output as untrusted input: check an order number against your system before acting on it, and keep a golden dataset of real messages to measure how often the values are right.

Where chatbots use it

Filling slots from free text. A customer writes "I'd like to move my Thursday booking to Saturday morning" and the model returns date, time of day and action as separate fields, as in slot filling and entity extraction.
Routing and triage. The model returns a category and an urgency value so the system can pick a queue.
Handing off to a person. A short structured summary of the conversation, with fixed fields, saves the agent from rereading the whole chat.
Guardrail decisions. A check step returns pass or fail plus a reason code that your guardrails can act on.

FAQ

What are structured outputs in an LLM?

A way to make a language model reply in a format you define, usually JSON that follows a schema, so software can read it without guessing.

What is the difference between JSON mode and structured outputs?

JSON mode guarantees the reply is valid JSON. Structured outputs, in the strict form, are meant to match your specific schema as well. OpenAI recommends the second where it is available.

Are structured outputs the same as function calling?

No. Function calling lets the model request that your code run a tool. A structured response format shapes the reply itself. They can be used together.

Do structured outputs stop hallucinations?

No. They control the shape of the reply, not whether the contents are true. A neat reply can still hold a wrong value, so check important values against your own data. See AI hallucination.

What happens if the model refuses or is cut off?

A refusal arrives in its own field, so your code can detect it. A reply cut off by a length limit may not match the schema, so check why generation stopped before parsing.

Do all providers support it?

Support, names and limits differ by provider and model. Check your provider's current documentation rather than assuming the same behavior.

Sources

OpenAI, Structured Outputs — developers.openai.com/api/docs/guides/structured-outputs, read 16 October 2026: JSON mode versus schema adherence, strict mode, function calling versus response format.
OpenAI, Introducing Structured Outputs in the API — openai.com/index/introducing-structured-outputs-in-the-api, read 16 October 2026: separate refusal field, cut-off behavior, first-request schema processing time, subset of JSON Schema, zero data retention.
Chatbotscape evaluation methodology. /methodology (continuously updated).

Function calling — the model asking your code to run a tool.
Entity extraction — pulling names, dates and numbers out of text.
Slot filling — collecting the details a task needs.
AI guardrails — checks that act on a structured verdict.
AI hallucination — why a well-formed answer can still be wrong.
LLM evaluation — measuring how often the values are right.