Red teaming

Use only on systems you are authorized to test

These strategies generate adversarial prompts designed to make a model misbehave. Run them only against agents you own or have explicit permission to test, in an environment where the outputs stay contained. The breaching transcripts can contain the very content the attack elicited, so handle reports accordingly.

Red teaming answers “can an attacker make my agent misbehave?” It runs jailbreaks (prompts crafted to get a model to do something its instructions forbid) against your agent and scores whether each one got through. While evaluators measure whether an agent does the right thing on cooperative input, red teaming probes what it does under deliberately hostile input: prompts engineered to override its instructions, leak its system prompt or private data, send data out through its tools, or trigger tool calls beyond its authority. An attack that gets through is a breach.

You assemble adversarial cases and one or more attack strategies against your agent in a RedTeamExperiment, run it, and read the breaches off the report. Each attack is driven by an AttackStrategy (a technique drawn from published jailbreak research) and scored by an LLM judge. With the built-in task, you only interact with the experiment; to drive the target yourself, see Writing a custom task.

Why red teaming?

Evaluators tell you the agent is helpful. Red teaming tells you whether it stays safe when someone tries to break it.

Evaluators alone:

• Measure quality on cooperative, in-distribution input
• Assume the user is trying to get a correct answer
• Don’t tell you how the agent behaves under an adversarial prompt

Evaluators + red teaming:

• Probe the agent with attacks engineered to bypass its guardrails
• Cover the threat types that matter for a tool-using agent (prompt injection, sensitive data disclosure, data exfiltration, privilege escalation, excessive agency)
• Score each attack with an LLM judge and surface the conversations that breached

When to use red teaming

Use red teaming when you need to:

• Pre-deployment safety checks: Probe an agent’s guardrails before it ships
• Safety regression checks: Re-run the same attacks after a prompt or tool change to catch new weaknesses
• Coverage across threat types: Probe all seven risk categories, from prompt injection to business-rule violations, in one run
• Comparing defenses: Measure attack success rate across model or system-prompt variants

Risk categories

Cases are organized by risk category: the kind of violation an attack targets. The built-in categories are:

Risk Category	What an attack targets
prompt_injection	Getting the agent to ignore its instructions or adopt a forbidden persona through jailbreaking, role-play coercion, or instruction override
excessive_agency	Getting the agent to make a tool call beyond its authorization scope, such as a refund over the allowed limit
sensitive_data	Getting the agent to reveal information it holds but should not share, such as its system prompt, credentials, or other users’ records
data_exfiltration	Getting the agent to send protected data to an external destination through its tools, such as an email, API call, or webhook
social_engineering	Getting the agent to write messages that manipulate people into unsafe actions, such as phishing emails or impersonating an authority
privilege_escalation	Getting the agent to access resources or operations reserved for higher privilege levels, or to act on behalf of another user
business_rule_violation	Getting the agent to take an action its tools allow but its policies forbid, such as splitting a request to get around a limit

A case’s risk category tells the judge what counts as a breach for that case. Revealing protected data in a reply is sensitive_data; sending it out through a tool is data_exfiltration. These categories align with the threat types in standard taxonomies like the OWASP LLM Top 10 (for example, prompt_injection maps to LLM01, sensitive_data to LLM02 and LLM07, and excessive_agency to LLM06), so findings translate into the language a security review expects.

Renamed categories

guideline_bypass and system_prompt_leak still work but emit a DeprecationWarning, and resolve to prompt_injection and sensitive_data. harmful_content has no replacement: AdversarialCaseGenerator raises a ValueError for it.

Quick example

Define a zero-arg factory that builds a fresh target, generate adversarial cases from it, pass the factory to the run call to attack the case x strategy cross-product in parallel, and read the breaches:

import asyncio

from strands import Agent
from strands_evals.redteam import (
    AdversarialCaseGenerator,
    CrescendoStrategy,
    RedTeamExperiment,
)


def agent_factory() -> Agent:
    return Agent(system_prompt="You are a helpful customer-support assistant.")


cases = AdversarialCaseGenerator().generate_cases(agent=agent_factory(), num_cases=3)
experiment = RedTeamExperiment(cases=cases, attack_strategies=[CrescendoStrategy()])
report = asyncio.run(
    experiment.run_evaluations_async(agent_factory=agent_factory, max_workers=5)
)
report.display()

Red teaming vs evaluators

Aspect	Evaluators	Red Teaming
Question	“How well did the agent do?”	“Can an attacker make it misbehave?”
Input	Cooperative test cases	Adversarial attacks (multi-turn or scripted)
Output	Score + pass/fail	Attack success score + breaching conversations
Driver	Your task function, called once per case	An AttackStrategy that drives the conversation turn by turn
Use Case	Quality evaluation	Safety probing, guardrail regression

Use Together: Evaluate the agent for quality, then red team it for safety. A high quality score and an undefended jailbreak are both true at once.

How it works

A strategy runs against the target one case at a time when you call run_evaluations(), and against the case x strategy cross-product in parallel (default max_workers=5) when you call run_evaluations_async(). Each strategy carries its own cheap in-loop “should I stop?” gate, but the authoritative breach verdict always comes from the AttackSuccessEvaluator over the full conversation and tool trace.

Best practices

Informed by general LLM red-teaming guidance like the OWASP Top 10 for LLM Applications and the NIST AI Risk Management Framework, scoped to what this module does.

• Cover several threat types, not one. Spread cases across the risk categories (AdversarialCaseGenerator does this automatically; report.by_risk_category() breaks results down by type). A pass on one category says nothing about the others.
• Run multiple strategies. Which technique breaks a given target varies, so run several and compare report.by_strategy(): coverage from a portfolio beats betting on one. This is the module’s core capability.
• Test the agent in context, with its tools. Point the strategies at your real Agent (system prompt, tools, guardrails) rather than a bare model: application-layer risks like excessive_agency and data_exfiltration only surface when the tools are present.
• Fix and re-run. A breach is the start of a loop: read the conversation, mitigate, then re-run the same cases to confirm the fix held (see Acting on a breach). Keep your breaching cases as a regression suite.
• Don’t read a clean run as proof of safety. Scores come from an LLM judge over a finite set of cases and strategies, and models are stochastic. A PASS is evidence, not a guarantee.

Next steps

• Quickstart: Run your first red-team experiment end to end
• Attack Strategies: The built-in strategies and how to choose
• Writing Custom Cases: Hand-author cases instead of generating them
• Writing a Custom Task: Build and drive the target yourself, for example to add a guardrail between turns
• Scoring Attacks: How AttackSuccessEvaluator decides a breach
• Reading the Report: Read the breach matrix and act on findings

Related documentation

• Getting Started: Set up your first evaluation experiment
• Evaluators Overview: Score agent performance on cooperative input
• Harmfulness Evaluator: Score a single response for harmful content

Tags

safety19