AI Red Teaming

What the Standard Assumes You Have, and What to Do Without It (2026)

Quick answer: AI red teaming is an authorized, scoped, adversarial exercise run against your own AI system to find failures nobody enumerated in advance. That last clause is what separates it from the probe testing most teams already do: a probe checks whether a control you configured is holding, while a red team asks what the system can be made to do that you never wrote a control for. MITRE published a formal red-team mitigation on 21 July 2026, and it opens by telling you to establish a team. If you rent a chatbot rather than build a model, you will not be doing that. You can still run the exercise, and nineteen of the thirty-three attack techniques MITRE maps to it are reachable from a normal login.

The distinction that decides whether you need this at all

Our chatbot guardrails guide tells you to write a probe for each rail you switched on, one that would fail if the rail were off, and to re-run the set after every change. That is a good practice and most small teams should do it before they do anything on this page.

It is also confirmatory. Every probe in that set exists because someone already decided a rail should be there. The set answers "is my configuration intact," and it is blind by construction to the failure that nobody modeled — the tool nobody thought to restrict, the retrieval path that reaches a document from a different customer, the sequence of five ordinary turns that ends somewhere none of the five would have gone alone.

Red teaming is the exploratory half. You start from an adversary's goal rather than from your configuration, and you are allowed to find things that are not on the list. MITRE's own description of the practice puts it as simulating "realistic adversary behavior to evaluate how attacks could affect the confidentiality, integrity, availability, safety, privacy, and mission performance of an AI-enabled system."

The practical test for whether you need it: if you can look at your rail list and honestly say it was derived from a threat model rather than from a vendor's settings page, your probes may be enough. Almost nobody can say that.

What MITRE actually published, and the sentence that does not fit an SMB

On 21 July 2026, MITRE ATLAS added AI Red Team as mitigation AML.M0035. It is worth knowing what kind of document this is: ATLAS mitigations are mapped to the techniques they defend against, and this one is mapped to thirty-three of them, which we counted from the technique table on the page rather than taking on trust. It sits across three categories — Policy, Technical AI and Technical Cyber — and six lifecycle phases, which is ATLAS's way of saying this is not purely a testing activity.

The mitigation opens: "Establish an AI red team responsible for conducting recurring, authorized, and threat-informed red-teaming exercises to identify and remediate vulnerabilities in AI-enabled systems before deployment and throughout operation."

Read that sentence as a small business and three words do not survive contact. Establish assumes headcount. Recurring assumes a budget line. AI red team assumes people whose job this is. A five-person company running a support bot on a rented platform has none of the three, and the usual response to a standard that does not fit is to conclude the standard is for someone else.

That conclusion is wrong in a specific way. The structure of the exercise survives the loss of the team; only the staffing does. MITRE organizes it into three phases — plan and scope, execute, then assess, report and improve — and each of those is a half-day of work at small scale rather than a program.

The nineteen of thirty-three

Here is the piece of analysis this guide exists for, and it is our reading rather than MITRE's.

Of the thirty-three techniques mapped to AML.M0035, most describe attacks against assets a small business does not possess. You do not train a model, so training-data poisoning is not your exposure. You do not hold weights, so model extraction, inversion and architecture modification are not either. Those belong to your supplier, and the honest thing to do with them is to ask about them during procurement rather than to test them.

The rule we applied: a technique is in scope for a renting operator if it can be exercised through the product surfaces you were given — the chat interface, the content you upload, the integrations you connect, the settings you can change. Everything else is your vendor's exercise, not yours.

That leaves nineteen, and they cluster into five things worth an afternoon each:

The twentieth is arguable and depends on your setup: AI Agent Tool under supply-chain compromise becomes yours the moment you install a third-party tool or connector, which on a modern agent stack usually means a Model Context Protocol server somebody else wrote. If you have installed one, count it in.

Two observations from building that table. The availability and cost cluster is five of the nineteen, and in our reading of the SMB-facing security material we surveyed it is the cluster least often covered — a metered bot with an open front door is a financial exposure that needs no jailbreak at all. And the agency cluster is another five, which is where severity actually lives; that is the same conclusion our LLM jailbreak entry reaches from the other direction, and the same one OWASP's 2026 rewrite reaches when it says the blast radius of an injection "extends from the chat surface to whatever the agent's tools can reach." Our LLM security entry reads the full 2026 list the same way, sorting all ten risks by whether a renting operator can act on them at all.

Phase one: scope, and the authorization question nobody raises

MITRE's planning phase asks for a system diagram, a threat model, success criteria, and rules of engagement covering "authorized systems, accounts, data, techniques, test windows, resource limits, escalation procedures, evidence handling, and stop conditions."

Most of that scales down cleanly. One item does not, and it is the item that should stop you before your first test.

You are about to attack somebody else's production system. The model is your vendor's, or your vendor's vendor's. The infrastructure is theirs. The rate limits you are about to probe are theirs. An internal red team at a model lab is testing an asset its employer owns; you are testing an asset you rent, and the authorization that makes red teaming lawful rather than merely well-intentioned is not automatic. MITRE uses the word "authorized" twice on the page: once in the opening sentence defining the exercise, and again in the rules of engagement, where the first item is authorized systems.

Before anything else, then, read your platform's terms and its acceptable-use policy for language about security testing, automated access, circumvention of safety measures, or attempts to extract the system prompt. Vendors differ, some publish a testing policy and a security contact, and some prohibit exactly this without written permission. If you cannot find a policy, ask your account contact in writing and keep the reply. Authorization is standard in the red-team literature; what we have not found addressed anywhere is the renting operator's version of it, where the system under test belongs to a supplier rather than to the tester. This is not legal advice, and if the stakes are high the question belongs with a lawyer.

The rest of the planning scales to one page:

• Diagram the thing honestly. Not the marketing architecture: the actual list of what the bot reads, what it can write to, who can change it, and which of those crosses a trust boundary. Most teams discover an integration nobody remembered here, and that discovery is often worth more than the testing that follows.
• Write the threat model as three sentences. Who would bother, what they would want, and what they can reach. A support bot for a plumbing company and a bot that quotes insurance have different adversaries and the exercise should not be the same.
• Set stop conditions before you start. The obvious ones: stop if you reach real customer data, stop if you cause a state change you cannot reverse, stop if you trip anything that looks like an outage. Write them down, because the moment you find something interesting is the moment you will want to keep going.
• Decide where evidence lives. Transcripts, timestamps, and crucially the integration-side logs, since a refused action and a declining model are indistinguishable in the chat window. This is the same reading problem our guardrails guide describes at its four-outcome table, and it is the single most common reason a test result is uninterpretable afterwards.
• Use a test tenant if you can, a test account if you cannot. MITRE says to plan destructive, privacy-invasive or high-cost tests for isolated environments with appropriate safeguards. On a rented platform, "isolated" usually means a separate workspace and a knowledge base full of synthetic documents.

Phase two: execute, at the scale you actually have

The full-strength version of this phase is a mix of manual work and automation, where automation generates input variations and replays sequences at scale while humans develop system-specific attacks and investigate unexpected behavior. At SMB scale, drop the automation and keep the sequence discipline.

Three habits carry most of the value.

Work from goals, not from a list. Pick one adversary objective per session — get the bot to reveal how it was configured, get it to state a policy we never wrote, get it to attempt a refund — and pursue it by whatever route the conversation offers. The moment you are working down a checklist you have gone back to probing.

Test across turns, not within them. MITRE's own red-team note against LLM Jailbreak asks you to exercise "manual and automated multi-turn, multilingual, encoded, transformed, and multimodal jailbreaks." Multi-turn is first for a reason, and it is the axis most small-team testing misses entirely: a single message gets evaluated on its own, while a gradual escalation across six exchanges never presents a single message worth blocking. If your product's controls inspect one message at a time, this is where you will find their edge. The multilingual axis is the second-cheapest finding available, and it matters disproportionately for anyone running in more than one market.

Record what the system did, not what it said. An apology followed by a completed action is a failure with good manners.

There is a fourth habit that only matters if you buy tooling rather than do this yourself, and OWASP's August 2026 rewrite of its prompt-injection entry states it more bluntly than any vendor will. The last of its eleven mitigations is to "test against adaptive attackers who have read the deployed defense, and reject static-only attack-success claims," and it cites research finding static attack success near zero while adaptive attack success exceeded 90 percent for most of twelve recent defenses. Translated into a purchasing question: when a red-teaming vendor or a guardrail product shows you a block rate, ask whether the attacker in that test knew what the defense was. If they did not, the number describes a rehearsal.

Then the step that everybody skips. MITRE's execute phase ends: "After testing, remove test accounts, modified data, installed software, persistent instructions, and other exercise artifacts." On a modern agent stack, persistent instructions is not boilerplate. If your bot has memory or long-lived threads, a successful context-poisoning test leaves your own poison in your own bot, and the finding outlives the session that produced it. Clear memory, delete the test thread, remove the synthetic documents from the knowledge base, and verify from a fresh session that none of it survived.

Phase three: turn the finding into something permanent

Assessment is where an exercise becomes a control, and MITRE is specific about the conversion: use demonstrated attacks to improve preventive controls, detection, incident response and recovery, and "where appropriate, convert confirmed failures into regression tests, evaluation datasets, detection logic, monitoring requirements, or deployment criteria."

For a small team that reduces to one rule. Every finding becomes a probe. The successful attack you just ran goes into the permanent set from our guardrails guide, with its expected outcome written next to it, and it gets re-run after every prompt edit, model change, content import and platform update. That is how the exploratory exercise feeds the confirmatory one, and it is the only mechanism that stops the same failure returning six weeks later when your vendor swaps the underlying model without telling you.

Two things to do with findings you cannot fix. If the failure is in the model's behavior, you cannot remediate it — jailbreak prevention lives with whoever trained the model — so the remediation is to reduce what that behavior can reach, which usually means moving a capability behind a permission or a human handoff. And if the failure is in the product rather than your configuration, report it to the vendor through whatever security contact you found in phase one, and record what they said. A vendor's response to a real finding is more informative about a platform than any feature comparison.

If at some point the exercise outgrows an afternoon, the place to go next is OWASP's own AI Red Teaming and Evaluation initiative, which maintains a GenAI Red Teaming Guide announced in January 2025 — MITRE lists it among the references for the mitigation this page is built on — alongside a vendor evaluation criteria document for organizations assessing red-teaming providers and tooling, and a quarterly solutions landscape for the same market. We have deliberately not summarized those here. This guide is the version for a business that is not yet hiring anyone; those are the documents for the moment you are.

MITRE closes the mitigation by noting that red-teaming is a continuous process that should be repeated as the threat landscape evolves and when the system, its components, its intended use or its deployment environment change. For a small business the workable cadence is event-driven rather than calendar-driven: run it when the bot gains a capability. A new integration, a new knowledge source, a new channel, a model upgrade. Those are the moments the threat model changed, and they are cheaper to catch than a quarterly reminder nobody honors.

What our own reviews say about this, which is nothing

We searched all fifteen published platform reviews on 15 August 2026, case-insensitively, for the string jailbreak. The count is zero in every one. A second search using the pattern red.?team returned four matching lines in a single review, and all four are substring matches inside "shared team inbox" rather than references to red-team activity.

So fifteen hands-on evaluations, and not one of them asked a vendor whether they run adversarial testing, publish a security contact, permit customer security testing in their terms, or log a refused action separately from a model refusal. That is a gap in our coverage and it is a strange one, because elsewhere on this site we ship a free prompt-injection tester that audits a system prompt against six attack classes, one of them a role-play jailbreak. We built the instrument and then did not use it in the reviews. Those four questions are now on the list for the next review-refresh pass. Until they are answered, this guide makes no claim about which platform in our catalog handles adversarial testing better, and you should treat any page that ranks them without having asked as guessing.

Frequently asked questions

Related guides

• LLM jailbreak (glossary) — the companion entry, including why two standards bodies define the term differently
• Prompt injection (glossary) — the largest technique cluster in the nineteen, and why no filter closes it
• AI guardrails (glossary) — the five places a control can sit, and which ones actually hold
• Chatbot guardrails guide — the confirmatory half: rails, probes, and reading a block log
• Chatbot security checklist — the pre-launch hygiene pass this exercise assumes you have done
• Chatbot PII handling — what the disclosure cluster is protecting
• QA testing protocol — the functional regression set your new probes join
• Chatbot prompt-injection tester (tool) — free system-prompt audit against six attack classes, useful before you scope an exercise
• System prompt (glossary) — the instruction layer you control, and its limits
• AI agent (glossary) — the deployment shape that makes the agency cluster matter

Sources

• MITRE ATLAS. AI Red Team, mitigation AML.M0035 (page states Created 21 July 2026, Last Modified 21 July 2026, permalink v2026.07) — the opening definition beginning "Establish an AI red team responsible for conducting recurring, authorized, and threat-informed red-teaming exercises"; the statement that red-teaming "simulates realistic adversary behavior to evaluate how attacks could affect the confidentiality, integrity, availability, safety, privacy, and mission performance of an AI-enabled system"; the three-phase structure of plan and scope, execute, and assess, report and improve; the rules-of-engagement enumeration covering "authorized systems, accounts, data, techniques, test windows, resource limits, escalation procedures, evidence handling, and stop conditions"; the instruction to "plan destructive, privacy-invasive, or high-cost tests for isolated environments with appropriate safeguards"; the post-test instruction to "remove test accounts, modified data, installed software, persistent instructions, and other exercise artifacts"; the conversion instruction to turn confirmed failures "into regression tests, evaluation datasets, detection logic, monitoring requirements, or deployment criteria"; and the closing statement that red-teaming is a continuous process. The page's own header states Number of Techniques 33; we counted the technique table independently and reached the same figure. Read in a live browser session on 15 August 2026, because atlas.mitre.org renders client-side and a plain fetch returns an empty shell. atlas.mitre.org
• MITRE ATLAS. LLM Jailbreak, technique AML.T0054 (Created 25 October 2023, Last Modified 27 May 2026, permalink v2026.07) — the red-team mitigation text asking testers to "exercise manual and automated multi-turn, multilingual, encoded, transformed, and multimodal jailbreaks," and the technique's mapping to both Defense Evasion and Privilege Escalation. Read in a live browser session on 15 August 2026. atlas.mitre.org
• OWASP GenAI Security Project. LLM01:2026 Prompt Injection, canonical Markdown source of the published 2026 release (repository states Status Published, Release date August 4, 2026; the project's resource page dates the release 3 August 2026) — the statement that "application-level safeguards help contain it, but effective prevention requires ongoing updates to the model's training and safety mechanisms," which is the basis for this guide's position that jailbreak prevention is not an operator-side control; the statement that where model output drives tool calls "the blast radius extends from the chat surface to whatever the agent's tools can reach"; and the eleventh and last of its prevention and mitigation strategies, "Test against adaptive attackers who have read the deployed defense, and reject static-only attack-success claims," together with its cited finding (Nasr et al., 2025) that static attack success was near zero while adaptive attack success exceeded 90% for most of twelve recent defenses. Read in a live browser session on 15 August 2026. github.com/GenAI-Security-Project
• OWASP GenAI Security Project. AI Red Teaming and Evaluation initiative (page metadata records last modified 28 June 2026) — the existence and scope of OWASP's red-teaming work, described on the initiative page as establishing "comprehensive AI Red Teaming and evaluation guidelines for Large Language Models (LLMs), addressing security vulnerabilities, bias, and user trust," and the three published artifacts named in this guide: the GenAI Red Teaming Guide announced 22 January 2025, the Vendor Evaluation Criteria for AI Red Teaming Providers and Tooling v1.0, and the AI Security Solutions Landscape for AI and Agentic Red Teaming Q2 2026. We have cited the existence, scope and dates of these documents from the initiative page; we have not read the guides themselves, which are distributed as downloads we did not retrieve, and this page therefore quotes nothing from their contents. MITRE's AML.M0035 lists the GenAI Red Teaming Guide among its references. Retrieved 15 August 2026. genai.owasp.org
• Chatbotscape platform reviews, all fifteen published reviews, searched 15 August 2026 — the catalog claim in this guide. Method: a case-insensitive search for the literal string jailbreak across the fifteen *-review.md files in sample-reviews/, excluding *.poc-notes.md, *.backup.md and files prefixed with an underscore, returning zero matches in all fifteen; and a second case-insensitive search using the pattern red.?team, returning four matching lines in one review, all four of which are substring matches inside "shared team inbox." Both the patterns and the counts are published so the searches can be reproduced. The same jailbreak search widened beyond the reviews returns hits in four glossary entries, in this guide and our chatbot guardrails guide, and in the four source files behind our prompt-injection tester — which is why this guide describes the gap as specific to the reviews rather than site-wide. Our companion entry on the LLM jailbreak carries the same finding.
• Chatbotscape. Chatbot guardrails guide — the probe design, the four-outcome reading table and the permanent probe set that this guide's findings feed into. The confirmatory-versus-exploratory framing of the relationship between the two pages is ours.
• Ahrefs Keywords Explorer, US and volume-by-country, queried 15 August 2026 — the search-demand and parent-topic figures in this page's keyword note, including the cannibalization checks that led to declining 'llm guardrails', 'ai guardrails' and 'llm security'.
• Chatbotscape evaluation methodology. /methodology (continuously updated).

About this guide

Chatbotscape launched in 2026 as an independent review site for chatbot platforms. This guide is part of our SMB chatbot Academy. It covers AI red teaming as a scoped exercise for a business that rents a chatbot rather than builds a model: which of MITRE's mapped techniques are reachable from your side of the login, the authorization question that comes before the first test, and how a finding becomes a permanent control. The confirmatory half — rails, probes, block logs — is in our chatbot guardrails guide, and the definitional companion is our glossary entry on LLM jailbreak.

Methodology

Every MITRE ATLAS statement on this page was read from the live page in a browser session rather than a cached fetch, with the page's own creation and modification stamps recorded in Sources. That method was chosen deliberately: atlas.mitre.org is a client-rendered application and a plain fetch of a technique or mitigation URL returns an HTML shell with no content in it, which is exactly the condition under which a cached or partial read produces a confident quotation of something the page does not say.

The thirty-three technique count was verified by counting the technique table on the mitigation page rather than by trusting the page's own header, and both figures agree.

The editorial judgment on this page is enumerated here rather than flagged individually, and the list is deliberately long because almost every prevalence statement in the body is ours rather than a source's: the confirmatory-versus-exploratory split between this guide and our guardrails guide; the rule used to decide which techniques a renting operator can reach, and therefore the nineteen-of-thirty-three figure and the five clusters it is organized into; the position that the twentieth technique is arguable and depends on installed connectors; the recommendation to check vendor terms before testing, and the observation that the renting operator's version of the authorization question is not addressed in the material we surveyed; the event-driven rather than calendar-driven cadence; the priority given to multi-turn over single-message testing and the claim that the multilingual axis is unusually cheap; the estimate that each MITRE phase is about a half-day at small scale; the claims that most rail lists are not derived from a threat model, that most teams find a forgotten integration while diagramming, that the availability-and-cost cluster is the least covered in SMB security material, that artifact cleanup is the step most often skipped, and that reading the transcript instead of the logs is the most common mistake. Each of those is our reading of the practice, not a measurement, and none of them is a MITRE rule. We have not run a red-team engagement against a vendor product and publish no findings, success rates or platform rankings. Nothing here is legal advice. See our methodology for how platform facts are verified.

Last updated

16 August 2026.