Context Engineering — What Goes Into the Model's Window, Who Decides It, and What Fifteen Platform Reviews Could Not Find

Quick answer: Context engineering is the work of curating what a model sees on a given turn. Anthropic's applied team defines it as "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference, including all the other information that may land there outside of the prompts," and frames it as "the natural progression of prompt engineering." It became a buying question rather than a developer question for two reasons, and the first is that bigger windows did not remove the problem. NoLiMa, an ICML 2025 benchmark, found that at 32,000 tokens eleven of thirteen tested long-context models dropped below half their short-context baseline, with GPT-4o falling from 99.3 percent to 69.7. The second reason is what our own catalogue shows: across the fifteen platform reviews we have published, fifteen sell a knowledge base or retrieval feature, seven record a place to write a system prompt, one records a max-tokens control, and not one records a vendor-disclosed context window size. Every platform on that list does context engineering for you. Almost none shows you the result.

Where the term came from, and why it stuck

The phrase entered general use in June 2025. Shopify's Tobi Lütke posted that he liked "the term 'context engineering' over prompt engineering" because "it describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM." Andrej Karpathy amplified it days later, calling it "the delicate art and science of filling the context window with just the right information for the next step" and listing what that means in practice: task descriptions, few-shot examples, retrieval, related data, tools, state, history, compaction. Both posts are on X and we read them through Simon Willison's contemporaneous write-up rather than on the platform itself, which is a sourcing limitation we would rather state than hide.

What made the term stick was not the naming. It was that the thing being named had quietly become most of the work. A single-turn assistant receives one prompt and answers. An agent runs for dozens of turns, calls tools, reads documents, and accumulates a transcript, and by turn fifteen the prompt you wrote is a small minority of what the model is reading. Thoughtworks moved context engineering from Assess to Adopt on its Technology Radar in April 2026, describing it as having "evolved from an optimization tactic into a foundational architectural concern." That is a consultancy's framing rather than a measurement, and we cite it as one.

Context engineering versus prompt engineering

The two get treated as synonyms and they are not, though the boundary is a matter of scope rather than of kind.

Prompt engineering is authorship. You write an instruction, you test it, you rewrite it. The artifact is a piece of text and it is stable between runs. Our prompt engineering entry covers that craft, and prompt engineering for chatbots covers the SMB version of it.

Context engineering is assembly. It decides what accompanies your instruction into the window on this specific turn: which three of two hundred knowledge-base chunks got retrieved, whether the last forty messages are included in full or summarized, which twelve tools are described and which are withheld, whether a note the agent wrote nine turns ago is pulled back in. Anthropic's own framing of the difference is the useful one. Writing a prompt is "a discrete task"; context engineering "is iterative and the curation phase happens each time we decide what to pass to the model."

The practical test is simple. If changing the thing means editing a document, it is prompt engineering. If changing it means editing retrieval logic, a summarization threshold, a tool manifest, or a memory policy, it is context engineering. On a hosted chatbot platform, the first is usually a text box you can reach and the second is usually not.

What is actually in the window

Anthropic's context-windows documentation is unambiguous about the accounting, and the accounting is where most intuitions break: "Everything in the request counts toward the context window: the system prompt, every message in messages (including tool results, images, and documents), and your tool definitions. The output Claude generates for the turn, including its extended thinking, counts too."

That last clause surprises people. Reasoning tokens the user never sees still consume the budget.

flowchart TB
 subgraph Window[Context window - one turn]
  direction TB
  A[System prompt<br/>role, constraints, tone]
  B[Tool definitions<br/>every tool described, whether used or not]
  C[Retrieved documents<br/>chunks selected for THIS turn]
  D[Message history<br/>full, summarized, or truncated]
  E[Recalled notes / memory<br/>written on earlier turns]
  F[Model output + reasoning tokens]
 end
 G[Knowledge base<br/>hundreds of documents] -->|selection| C
 H[Full transcript<br/>every turn so far] -->|compaction| D
 I[External store<br/>notes, files, state] -->|just-in-time recall| E

Figure 1. The context window is a budget rather than a container you fill once. Six things occupy it, and three of them are decided by code at runtime rather than written by hand: which documents get selected for this turn, how much history survives compaction, and which stored notes get recalled. LangChain describes those same operations as write, select, compress and isolate.

Those three runtime decisions matter more than the list of occupants. A knowledge base with four hundred documents does not go into the window; a selection from it does, and whoever wrote the selection logic is doing context engineering on your behalf. See retrieval-augmented generation for how that selection is normally implemented and chatbot context window for the arithmetic of the container itself.

Why bigger windows did not end the problem

The intuitive response to a context problem is to buy a larger window, and windows did get very large. As of 25 August 2026, Anthropic documents 1M-token context for Claude Fable 5, Opus 5 and Sonnet 5, and states that for every model with a 1M window "1M is the default: you don't need a beta header, and long-context requests are billed at standard pricing." OpenAI documents a 1.05M-token window for the GPT-5.6 family. Google documents 1,048,576 input tokens for Gemini 3.7 Flash.

The research says filling them is a bad idea.

Anthropic's own documentation now states the effect as fact rather than as a research finding: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot. This makes curating what's in context just as important as how much space is available." The company's engineering post is careful to call it "a performance gradient rather than a hard cliff," which is the right calibration. Nothing breaks at a threshold. Reliability erodes, unevenly, in ways that are hard to notice from a demo and easy to notice from a support queue.

The operating conclusion is the one Anthropic states directly: good context engineering means "finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome." Fewer tokens, chosen better.

A fair objection, and the honest answer to it. Most SMB deployments never see 32,000 tokens. A ten-turn support conversation over a five-document knowledge base runs a few thousand, and every number in the table above comes from a regime a small support bot will not enter. The research bites when three things stack: long sessions, a large tool manifest, and aggressive history retention. One of those three is loaded on every single turn regardless of how short the conversation is, and that is the tool manifest. So tool bloat is the part of this that reaches a small deployment first, and the long-context material matters mainly once you are running agents rather than answering questions.

The four ways long context fails

Drew Breunig's June 2025 taxonomy is the one most people have converged on, and it is useful because each mode has a different fix.

• Context poisoning. A hallucination or error "makes it into the context, where it is repeatedly referenced." Once a wrong fact is in the transcript, later turns treat it as established. Our AI hallucination entry covers the origin; the context problem is the persistence.
• Context distraction. The context grows so long that "the model over-focuses on the context, neglecting what it learned during training." Breunig cites a Gemini 2.5 Pro agent past roughly 100K tokens favouring repetition of its own history over new plans.
• Context confusion. "Superfluous content in the context is used by the model to generate a low-quality response." His clearest example is tool bloat, and it rests on two separate pieces of evidence. The Berkeley Function-Calling Leaderboard shows every model tested performing worse once more than one tool is available, on the live tool-definition columns. Separately, Breunig cites a paper on the GeoEngine benchmark in which a quantized Llama 3.1 8B failed a task with all 46 tools supplied and succeeded with 19, well inside its window.
• Context clash. New information or tools conflict with what is already there, and the model has no principled way to choose. Breunig's supporting evidence here is a study that sharded single prompts across multiple turns and found an average 39 percent performance drop, with one model falling from 98.1 to 64.1. We have that finding as he reports it and have not read the paper.

Tool bloat is the one that hits SMB deployments hardest, because it is the failure that looks like generosity. Connecting every available integration to a bot feels like giving it more capability. Every tool definition is tokens in the window on every turn, used or not, and the leaderboard evidence says the marginal tool makes the bot worse at choosing among the tools it already had. See function calling for the mechanism.

The techniques that work

Anthropic's engineering post names three techniques for long-horizon work, compaction, structured note-taking and sub-agent architectures, and discusses two more, just-in-time retrieval and tool design, in other sections of the same piece. We group all five here because they generalize beyond any one vendor. The grouping is ours; the descriptions are Anthropic's.

Compaction. Take a conversation approaching the window limit, summarize it, and restart with the summary. Anthropic calls clearing old tool results "one of the safest lightest touch forms of compaction," which is a good default: raw tool output is usually the largest and least reusable thing in a transcript.

Structured note-taking. The agent writes notes to a file or store outside the window and pulls them back when relevant. A to-do list or a running NOTES.md survives compaction in a way that a transcript does not. This is the practice behind most of what platforms market as memory; our chatbot memory guide covers the SMB implementations.

Sub-agent architectures. A sub-agent burns "tens of thousands of tokens or more" on a task and returns "a condensed, distilled summary of its work (often 1,000-2,000 tokens)." The searching happens somewhere else; only the answer enters the main window. See multi-agent system.

Just-in-time retrieval. Rather than loading everything up front, keep "lightweight identifiers (file paths, stored queries, web links, etc.)" and load the actual content only when it is needed. Anthropic describes Claude Code as a hybrid, with a project file loaded up front and file search performed on demand.

Tool design. Anthropic names "bloated tool sets that cover too much functionality" as "one of the most common failure modes." Fewer tools, described unambiguously, with no two tools a reasonable agent would confuse.

Anthropic's framing device for all five is worth borrowing wholesale: models have "an 'attention budget' that they draw on when parsing large volumes of context." Budgets get spent. They do not get topped up by buying a bigger number.

What our own reviews record, and what that tells a buyer

Here is the part no model vendor and no platform will write, because it is an audit of the market rather than of a product. We publish fifteen platform reviews. Searched on 25 August 2026, they record the following about context control. The searches are printed in Sources so the counts reproduce.

Two rows need their working shown, because in both cases a plain text search gives the wrong answer.

The system-prompt row is 7, not 8. A search on the phrase returns eight files, and one of them, Chatfuel, matches only inside screenshot caption text describing a "Custom instructions / Rules" tab that builds intent-detection rules rather than a prompt. Reading beat the regex there. In the other direction, the search misses Voiceflow, whose review writes the term hyphenated while discussing "system-prompt optimization for non-English" in the default agent template. That implies an authoring surface exists; our review does not record it directly, so it sits in the ambiguous column rather than being counted.

The zero in the last row is the finding, and it deserves care about what it means. Four reviews touch the context window at all, and in three of them the mention is the absence. Our Blip review lists among its cons that the vendor "does not publicly disclose underlying LLM stack (OpenAI? Anthropic? Proprietary?), context window, BYOLLM availability, or per-language accuracy." Our Chatfuel review records that document-upload mechanics and "context window size" are "not documented on vendor pages." Our Tidio review says the same of Lyro: document-upload mechanics "and context-window size are not documented on the vendor's product page." The fourth, Botpress, discusses context only as a billing question, noting that inference is bundled into conversation cost rather than billed per token.

Across the rows, the layer you can reach is the one that feels like writing, and the layers that decide reliability are the ones held by the vendor. Landbot is a good illustration of the ceiling: its AI Agent instructions area gives you a Role, Constraints, Context and Steps scaffold, which is a genuinely thoughtful authoring surface, and it tells you nothing about how much conversation history accompanies those instructions or what happens to it when the transcript gets long. Typebot sits at the other end because it is BYOLLM by design: you bring the key, so you set the model, the temperature and the max-tokens, and the window is whatever the model you chose provides.

The honest caveat, stated plainly. These are counts of what our reviews recorded during their own sessions, not audits of vendor capability. A platform may expose a control we did not exercise, and none of these reviews set out specifically to test context handling. What the table does establish is that a buyer reading the public record cannot find the number, which is itself the point: if fifteen reviews written by people looking closely did not surface a window size, a buyer skimming a pricing page will not either.

What to do with that. Three questions get further than a feature list. How much conversation history does the bot carry into each turn, and what happens when that exceeds the limit: truncation, summarization, or an error? How many knowledge-base chunks are retrieved per turn? Can I see the assembled prompt for a single turn, or only the reply? Vendors that can answer the third question tend to be able to answer the first two.

The case against the term

The objection arrived with the term. Breunig, who is a proponent, records it accurately: "'Another marketing term,' some people said. One HN commenter called it, 'A month long skill, after which it won't be a thing anymore.'"

There is a stronger version, and it is not that the term is rebranded prompt engineering. It is that the discipline is old work under a new name and the name will not last. Eric Porres argued in June 2026 that context engineering is largely library science, information architecture and knowledge management reassembled, and that the job title will dissolve as models get better at managing their own context: "Context engineer could follow the same arc when the models get better at managing their own context. They will." That is a self-published essay rather than institutional research and we label it as one, but the argument is sound.

It also has support from the people doing the most to define the term. Anthropic's own post notes that "smarter models require less prescriptive engineering, allowing agents to operate with more autonomy," and expects agentic design to trend toward "progressively less human curation." The case that this is a temporary specialty is therefore not a sceptic's sniping from outside. It is partly conceded from inside, and we are not going to pretend otherwise.

Where we part company with Porres is on who absorbs the cost in the meantime. A model that manages its own context still runs inside a product that decides how much history to send it, how many chunks to retrieve per turn and how many tools to describe. Those are product decisions rather than model decisions, and on a hosted chatbot platform they are made by someone the buyer never meets. That stays true whether or not anyone is still called a context engineer in 2029, which is why the useful question for an SMB in 2026 is not what to call the work. It is which of those decisions the vendor has already made for you, and whether you can see them.

Related terms

• Prompt engineering — the authorship layer, and the part of this you can usually reach.
• Chatbot context window — the container, its token arithmetic, and what overflows.
• System prompt — the instruction block that opens every window.
• Retrieval-augmented generation — the selection mechanism behind most retrieved context.
• Large language model — what the window belongs to.
• Session context — the live conversation state a platform carries between turns.

FAQ

What is context engineering in simple terms?

It is deciding what the model gets to see when it answers. A language model has no memory between calls and no access to anything except what is in the request, so every answer is produced from a bundle that somebody assembled: your instructions, the tools it is allowed to use, whatever documents were pulled for this question, and however much of the conversation survived. Context engineering is the craft of assembling that bundle well. Anthropic's definition is "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference."

How is context engineering different from prompt engineering?

Scope, and who does the work. Prompt engineering is writing the instruction, a stable artifact you edit by hand. Context engineering also covers what arrives alongside it at runtime, which is decided by code: retrieval, history management, tool selection, memory recall. Anthropic frames the difference as discrete versus iterative, since with an agent "the curation phase happens each time we decide what to pass to the model." A useful test: if you change it by editing text, it is prompt engineering; if you change it by editing logic, it is context engineering.

Do bigger context windows make this unnecessary?

No, and the research is fairly consistent on the point. Windows reached a million tokens across all three major vendors during 2026, and long-context benchmarks kept finding degradation well below that. NoLiMa found eleven of thirteen long-context models below half their baseline at 32,000 tokens. Chroma found that even one distractor degrades retrieval. Anthropic's own documentation calls the effect context rot and concludes that curating what is in context matters "just as important as how much space is available." Bigger windows raise the ceiling on what is possible; they do not raise the floor on what is reliable.

What is context rot?

The observed decline in accuracy and recall as input length grows, even inside a model's stated window. The name comes from Chroma's July 2025 technical report, which tested 18 models and found that "performance grows increasingly unreliable as input length grows." It is a gradient rather than a cliff: nothing fails at a specific token count, but reliability erodes as the window fills, and it erodes faster when the context contains near-miss material rather than clearly irrelevant material.

Can I do context engineering on a no-code chatbot platform?

Partly, and the boundary is worth knowing before you buy. Across our fifteen reviews, seven record a system-prompt field with an eighth ambiguous, so roughly half the market exposes the authorship layer. One records a max-tokens control and none records a vendor-disclosed window size, so the budget layer is mostly not yours. In practice you control what the bot is told to do, you influence what it retrieves by curating the knowledge base, and you generally cannot control how much history it carries or how retrieval is chunked. Platforms with BYOLLM support move more of this back to you, because the model and its parameters become yours to set.

Which techniques give the biggest improvement for a small team?

Two, in order. Cut the tool list: the Berkeley Function-Calling Leaderboard evidence Breunig cites, on the leaderboard's live tool-definition columns, is that every model does worse with more than one tool available, and a bloated integrations list is the most common self-inflicted version of this. Then fix retrieval so fewer, better chunks come back per turn rather than more. Both reduce tokens and improve accuracy at the same time, which is unusual. Compaction and note-taking matter more once conversations run long; our context window guide covers the operational side.

Is context engineering just a buzzword?

The label is a year old and contested; the work is not. The strongest critique is not that it is rebranded prompt engineering but that it is repackaged information architecture and will be absorbed by the models themselves, which is Eric Porres's argument and a reasonable one. What is not in dispute is that on every deployment someone decides what enters the window, and that on a hosted platform that someone is usually the vendor.

Sources

• Anthropic Applied AI team, "Effective context engineering for AI agents," published 29 September 2025, read 25 August 2026. Source of the definition quoted in the Quick answer and the FAQ, the "natural progression of prompt engineering" framing, the discrete-versus-iterative distinction, the "attention budget" metaphor, the "smallest possible set of high-signal tokens" principle, the sub-agent token figures, the "bloated tool sets" failure mode, the "less prescriptive engineering" and "progressively less human curation" statements quoted in the counter-argument section, and the "performance gradient rather than a hard cliff" calibration. On the techniques: the post enumerates three for long-horizon work (compaction, structured note-taking, multi-agent architectures) and discusses just-in-time retrieval and tool design in separate sections. The five-item grouping in this entry is editorial, and the body says so; the descriptions of each are Anthropic's. anthropic.com/engineering
• Anthropic, context windows documentation, read 25 August 2026 (no page date displayed). Source of the enumeration of what counts toward the window including extended thinking tokens, the context-rot statement quoted in the FAQ, and the pricing and default-availability statement for 1M-token models. platform.claude.com/docs
• Vendor model documentation for the context-window figures, all read 25 August 2026: Anthropic model overview (1M tokens for Claude Fable 5, Opus 5, Sonnet 5); OpenAI models reference (1.05M tokens for the GPT-5.6 family, listed without a beta or preview label); Google Gemini 3.7 Flash model page (1,048,576 input tokens, marked Stable, page footer "Last updated 2026-08-13 UTC"). Figures move with model releases and should be re-checked against the vendor page before they decide anything. platform.claude.com · developers.openai.com · ai.google.dev
• Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni and Percy Liang, "Lost in the Middle: How Language Models Use Long Contexts," Transactions of the Association for Computational Linguistics vol. 12 pp. 157-173, 2024. Quoted from the abstract; the ACL Anthology record gives a year rather than a month. aclanthology.org
• Kelly Hong, Anton Troynikov and Jeff Huber, "Context Rot: How Increasing Input Tokens Impacts LLM Performance," Chroma technical report, 14 July 2025, read 25 August 2026. Source of the 18-model scope, the unreliability-with-length finding, and the single-distractor and four-distractor results. A widely repeated claim that degradation begins around 300,000 to 400,000 tokens is NOT in the report text we read and is not attributed to Chroma here. trychroma.com/research
• Ali Modarressi, Hanieh Deilamsalehy, Franck Dernoncourt, Trung Bui, Ryan A. Rossi, Seunghyun Yoon and Hinrich Schütze, "NoLiMa: Long-Context Evaluation Beyond Literal Matching," ICML 2025, PMLR vol. 267 pp. 44554-44570. Source of the 32K threshold result: 13 models all claiming 128K or greater context, 11 below 50 percent of baseline at 32K, GPT-4o from 99.3 to 69.7 percent. proceedings.mlr.press
• Drew Breunig, "How Long Contexts Fail," 22 June 2025, read 25 August 2026. Source of all four failure-mode definitions quoted verbatim, the Gemini 2.5 Pro repetition observation, and the tool-count evidence. Two separate sources sit behind that evidence and are kept separate here, because an earlier draft merged them: the "every model performs worse with more than one tool" finding is the Berkeley Function-Calling Leaderboard, on its live tool-definition columns, while the quantized Llama 3.1 8B result at 46 versus 19 tools comes from a paper on the GeoEngine benchmark that Breunig cites separately. The sharded-prompt study behind the context-clash bullet (average 39 percent drop, one model from 98.1 to 64.1) is a third source. All three are quoted as Breunig reports them; we did not read any of the three primary papers directly. dbreunig.com
• Simon Willison, "Context engineering," 27 June 2025, read 25 August 2026. Source of the Lütke and Karpathy quotations. Sourcing limitation, stated because it affects how much weight the origin story carries: the original posts are on X and we did not read them there. Secondary write-ups date them to 18 June and 25 June 2025 respectively; we have not verified those dates and do not print them as fact. simonwillison.net
• LangChain, "Context Engineering for Agents," 2 July 2025, read 25 August 2026, for the write / select / compress / isolate taxonomy referenced in the figure caption. langchain.com/blog
• Thoughtworks Technology Radar, "Context engineering," entry last updated 15 April 2026, read 25 August 2026, for the Assess-to-Adopt movement and the "foundational architectural concern" quotation. A consultancy's assessment, cited as one. thoughtworks.com/radar
• Drew Breunig, "Why the Term Context Engineering Matters," 24 July 2025, and Eric Porres, "Context Engineer Is the Title of 2026. The Discipline Is a Century Old," Beyond Reason, 3 June 2026, both read 25 August 2026, for the counter-argument section. Porres is self-published opinion, not peer-reviewed or institutional, and is labeled as such where quoted. dbreunig.com · promptedbyeric.substack.com
• Chatbotscape review corpus, searched 25 August 2026 and published so the counts reproduce. Denominator: ls sample-reviews/*-review.md | wc -l returns 15. All searches were run case-insensitively per our own rule after an earlier entry was caught by a case-sensitive grep, and all patterns here are hyphen-tolerant, which they were not on the first pass. That first pass is worth recording rather than hiding, because it damaged two of the five counts: a space-only context window pattern missed Tidio's "context-window," and a space-only system prompt pattern missed Voiceflow's "system-prompt." Every term in this audit is hyphenatable, so every pattern had the same defect. Knowledge base or retrieval: grep -rliE "knowledge base|retrieval-augmented|\bRAG\b" sample-reviews/*-review.md returns 15. System prompt: grep -rliE "system.?prompt|custom instructions|custom prompt" sample-reviews/*-review.md returns 9, and the published figure is 7 confirmed plus 1 ambiguous because two of the nine were reclassified on reading. Chatfuel was removed: its only match sits inside a screenshot caption describing a "Custom instructions / Rules" tab that builds intent-detection rules, which is not a prompt-authoring surface. Voiceflow was added as ambiguous: its only match is the phrase "system-prompt optimization for non-English" in a discussion of the default agent template, which implies an authoring surface without recording one. Temperature: grep -rliE "temperature" sample-reviews/*-review.md returns 2 (BotPenguin, Typebot). Max tokens: grep -rliE "max.?tokens|max_tokens" sample-reviews/*-review.md returns 1 (Typebot). Context window: grep -rliE "context.?window" sample-reviews/*-review.md returns 4 (Blip, Botpress, Chatfuel, Tidio), and every match in all four was read in context before classification rather than classified from the grep line, which is what established that none records a size: three record the vendor's non-disclosure and the fourth discusses inference billing. The last row of the table is therefore a zero derived from reading four files, not from a search returning nothing, and it counts vendor-disclosed sizes specifically. These are counts of our reviews, not audits of vendor capability, and the body says so. None of the underlying reviews set out to test context handling specifically.
• Ahrefs Keywords Explorer, US overview, queried 25 August 2026 — the demand, difficulty, CPC and global-volume figures in this entry's keyword note, including the checks behind declining 'context window management' and 'ai copilot'.
• Chatbotscape evaluation methodology. /methodology (continuously updated).