## Overview

The `InstructionFollowingEvaluator` assesses whether an agent’s response follows all explicit instructions provided in the user’s prompt. It focuses strictly on instruction compliance (whether specific constraints, requirements, and directives were satisfied), regardless of response quality or factual accuracy.

## Key features

-   **Trace-Level Evaluation**: Evaluates the most recent turn in the conversation
-   **Binary Scoring**: Clear Yes (instructions followed) / No (instructions not followed) classification
-   **Async Support**: Supports both synchronous and asynchronous evaluation
-   **Constraint-Focused**: Evaluates compliance with explicit directives, not overall quality

## When to use

Use the `InstructionFollowingEvaluator` when you need to:

-   Verify that agents respect format, length, or style constraints
-   Check compliance with specific answer options or output types
-   Assess whether agents follow multi-part instructions completely
-   Evaluate instruction adherence independently from correctness

## Evaluation level

This evaluator operates at the **TRACE\_LEVEL**, evaluating the most recent turn in the conversation.

## Parameters

### `model` (optional)

-   **Type**: `Model | str | None`
-   **Default**: `None` (uses default Bedrock model)
-   **Description**: The model to use as the judge.

### `system_prompt` (optional)

-   **Type**: `str | None`
-   **Default**: `None` (uses built-in template)
-   **Description**: Custom system prompt for the judge model.

### `version` (optional)

-   **Type**: `str`
-   **Default**: `"v0"`
-   **Description**: Prompt template version.

## Scoring system

| Rating | Score | Description |
| --- | --- | --- |
| Yes | 1.0 | All explicit instructions in the prompt are satisfied |
| No | 0.0 | One or more explicit instructions are not satisfied |

A response passes the evaluation if all explicit instructions are followed (score = 1.0).

## What counts as explicit instructions

The evaluator checks for compliance with specific directives such as:

-   **Information constraints**: “Based on this text passage, give an overview about \[…\]”
-   **Length requirements**: “Summarize this text in one sentence”
-   **Answer options**: “Which of the following is the tallest mountain…”
-   **Target audience**: “Write an explanation for middle schoolers”
-   **Genre**: “Write an ad for a laundry service”
-   **Style**: “Write an ad for a sports car like it’s an obituary”
-   **Content type**: “Write a body for this email” vs “Write a subject line”

## Evaluation rules

-   If a response includes **more** information than requested, it still passes as long as all requested elements are present
-   If a response is purely evasive without any partial or related answer, it defaults to **Yes**
-   If a response is partially evasive but provides a partial answer, the partial answer is judged
-   If there are **no explicit instructions** in the input (casual or open-ended requests), defaults to **Yes**
-   The evaluator does **not** assess factual accuracy, writing quality, or response effectiveness

## Basic usage

Required: Session ID Trace Attributes

When using `StrandsInMemorySessionMapper`, you **must** include session ID trace attributes in your agent configuration. This prevents spans from different test cases from being mixed together in the memory exporter.

```python
import asyncio

from strands import Agent
from strands_evals import Case, Experiment
from strands_evals.evaluators import InstructionFollowingEvaluator
from strands_evals.mappers import StrandsInMemorySessionMapper
from strands_evals.telemetry import StrandsEvalsTelemetry

telemetry = StrandsEvalsTelemetry().setup_in_memory_exporter()

def task_function(case: Case) -> dict:
    agent = Agent(
        trace_attributes={"session.id": case.session_id},
        callback_handler=None
    )
    response = agent(case.input)
    spans = telemetry.in_memory_exporter.get_finished_spans()
    mapper = StrandsInMemorySessionMapper()
    session = mapper.map_to_session(spans, session_id=case.session_id)
    return {"output": str(response), "trajectory": session}

cases = [
    Case(
        name="format-constraint",
        input="List the top 3 programming languages in bullet points."
    ),
    Case(
        name="length-constraint",
        input="Explain quantum computing in exactly one sentence."
    ),
]

experiment = Experiment(cases=cases, evaluators=[InstructionFollowingEvaluator()])
async def main():
    report = await experiment.run_evaluations_async(task_function)
    report.run_display()

asyncio.run(main())
```

## Combining with other evaluators

Pair with quality evaluators to assess both compliance and correctness:

```python
evaluators = [
    InstructionFollowingEvaluator(),  # Did it follow the format/constraints?
    CorrectnessEvaluator(),           # Is the content factually correct?
    ConcisenessEvaluator(),           # Is it appropriately concise?
]
```

## Related evaluators

-   [**CorrectnessEvaluator**](/pr-cms-4519/docs/user-guide/evals-sdk/evaluators/correctness_evaluator/index.md): Evaluates factual accuracy (complementary to instruction following)
-   [**OutputEvaluator**](/pr-cms-4519/docs/user-guide/evals-sdk/evaluators/output_evaluator/index.md): Flexible custom rubric evaluation
-   [**RefusalEvaluator**](/pr-cms-4519/docs/user-guide/evals-sdk/evaluators/refusal_evaluator/index.md): Detect when agents refuse to address prompts
-   [**CoherenceEvaluator**](/pr-cms-4519/docs/user-guide/evals-sdk/evaluators/coherence_evaluator/index.md): Assess logical consistency and reasoning

## Related pages

- [Prompt Engineering](/pr-cms-4519/docs/user-guide/sdk/safety-security/prompt-engineering/index.md) (2 shared tags)
- [Prompts](/pr-cms-4519/docs/user-guide/sdk/agents/prompts/index.md) (2 shared tags)
- [Attack strategies](/pr-cms-4519/docs/user-guide/evals-sdk/red-teaming/strategies/index.md) (1 shared tag)
- [Harmfulness evaluator](/pr-cms-4519/docs/user-guide/evals-sdk/evaluators/harmfulness_evaluator/index.md) (1 shared tag)
- [Reading the report](/pr-cms-4519/docs/user-guide/evals-sdk/red-teaming/reading_the_report/index.md) (1 shared tag)
- [Red teaming](/pr-cms-4519/docs/user-guide/evals-sdk/red-teaming/index.md) (1 shared tag)
- [Refusal evaluator](/pr-cms-4519/docs/user-guide/evals-sdk/evaluators/refusal_evaluator/index.md) (1 shared tag)
- [Responsible AI](/pr-cms-4519/docs/user-guide/sdk/safety-security/responsible-ai/index.md) (1 shared tag)
- [Scoring attacks](/pr-cms-4519/docs/user-guide/evals-sdk/red-teaming/evaluators/index.md) (1 shared tag)
- [Secure for production](/pr-cms-4519/docs/user-guide/sdk/safety-security/index.md) (1 shared tag)