Strands Agents SDK provides an extensible interface for implementing custom model providers, allowing organizations to integrate their own LLM services while keeping implementation details private to their codebase.

## Model Provider Functionality

Custom model providers in Strands Agents support two primary interaction modes:

### Conversational Interaction

The standard conversational mode where agents exchange messages with the model. This is the default interaction pattern that is used when you call an agent directly:

(( tab "Python" ))
```python
agent = Agent(model=your_custom_model)
response = agent("Hello, how can you help me today?")
```
(( /tab "Python" ))

(( tab "TypeScript" ))
```typescript
const yourCustomModel = new YourCustomModel()

const agent = new Agent({ model: yourCustomModel })
const response = await agent.invoke('Hello, how can you help me today?')
```
(( /tab "TypeScript" ))

This invokes the underlying model provided to the agent.

## Model Provider Architecture

Strands Agents uses an abstract `Model` class that defines the standard interface all model providers must implement:

```mermaid
flowchart TD
    Base["Model (Base)"] --> Bedrock["Bedrock Model Provider"]
    Base --> Anthropic["Anthropic Model Provider"]
    Base --> LiteLLM["LiteLLM Model Provider"]
    Base --> Ollama["Ollama Model Provider"]
    Base --> Custom["Custom Model Provider"]
```

## Implementation Overview

The process for implementing a custom model provider is similar across both languages:

(( tab "Python" ))
In Python, you extend the `Model` class from `strands.models` and implement the required abstract methods:

-   `stream()`: Core method that handles model invocation and returns streaming events
-   `update_config()`: Updates the model configuration
-   `get_config()`: Returns the current model configuration

The base class also provides these optional methods you can override:

-   `count_tokens()`: Estimate input token count (defaults to a character-based heuristic)
-   `estimate_utilization()`: Compute the ratio of input tokens to `context_window_limit` (defaults to 200,000 when not configured). See [Utilization Estimation](/pr-cms-3708/docs/user-guide/concepts/agents/conversation-management/index.md#utilization-estimation).

The Python implementation uses async generators to yield `StreamEvent` objects.
(( /tab "Python" ))

(( tab "TypeScript" ))
In TypeScript, you extend the `Model` class from `@strands-agents/sdk` and implement the required abstract methods:

-   `stream()`: Core method that handles model invocation and returns streaming events
-   `updateConfig()`: Updates the model configuration
-   `getConfig()`: Returns the current model configuration

The base class also provides these optional methods you can override:

-   `countTokens()`: Estimate input token count (defaults to a character-based heuristic)
-   `estimateUtilization()`: Compute the ratio of input tokens to `contextWindowLimit` (defaults to 200,000 when not configured). See [Utilization Estimation](/pr-cms-3708/docs/user-guide/concepts/agents/conversation-management/index.md#utilization-estimation).

The TypeScript implementation uses async iterables to yield `ModelStreamEvent` objects.

**TypeScript Model Reference**: The `Model` abstract class is available in the TypeScript SDK at `src/models/model.ts`. You can extend this class to create custom model providers that integrate with your own LLM services.
(( /tab "TypeScript" ))

## Implementing a Custom Model Provider

### 1\. Create Your Model Class

Create a new module in your codebase that extends the Strands Agents `Model` class.

(( tab "Python" ))
Create a new Python module that extends the `Model` class. Set up a `ModelConfig` to hold the configurations for invoking the model.

your\_org/models/custom\_model.py

```python
import logging
import os
from typing import Any, Iterable, Optional, TypedDict
from typing_extensions import Unpack

from custom.model import CustomModelClient

from strands.models import Model
from strands.types.content import Messages
from strands.types.streaming import StreamEvent
from strands.types.tools import ToolSpec

logger = logging.getLogger(__name__)


class CustomModel(Model):
    """Your custom model provider implementation."""

    class ModelConfig(TypedDict):
        """
        Configuration your model.

        Attributes:
            model_id: ID of Custom model.
            params: Model parameters (e.g., max_tokens).
        """
        model_id: str
        params: Optional[dict[str, Any]]
        # Add any additional configuration parameters specific to your model

    def __init__(
        self,
        api_key: str,
        *,
        **model_config: Unpack[ModelConfig]
    ) -> None:
        """Initialize provider instance.

        Args:
            api_key: The API key for connecting to your Custom model.
            **model_config: Configuration options for Custom model.
        """
        self.config = CustomModel.ModelConfig(**model_config)
        logger.debug("config=<%s> | initializing", self.config)

        self.client = CustomModelClient(api_key)

    @override
    def update_config(self, **model_config: Unpack[ModelConfig]) -> None:
        """Update the Custom model configuration with the provided arguments.

        Can be invoked by tools to dynamically alter the model state for subsequent invocations by the agent.

        Args:
            **model_config: Configuration overrides.
        """
        self.config.update(model_config)


    @override
    def get_config(self) -> ModelConfig:
        """Get the Custom model configuration.

        Returns:
            The Custom model configuration.
        """
        return self.config
```
(( /tab "Python" ))

(( tab "TypeScript" ))
Create a TypeScript module that extends the `Model` class. Define an interface for your model configuration to ensure type safety.

src/models/custom-model.ts

```typescript
// Mock client for documentation purposes
interface CustomModelClient {
  streamCompletion: (request: any) => AsyncIterable<any>
}

/**
 * Configuration interface for the custom model.
 */
export interface CustomModelConfig extends BaseModelConfig {
  apiKey?: string
  modelId?: string
  maxTokens?: number
  temperature?: number
  topP?: number
  // Add any additional configuration parameters specific to your model
}

/**
 * Custom model provider implementation.
 *
 * Note: In practice, you would extend the Model abstract class from the SDK.
 * This example shows the interface implementation for documentation purposes.
 */
export class CustomModel {
  private client: CustomModelClient
  private config: CustomModelConfig

  constructor(config: CustomModelConfig) {
    this.config = { ...config }
    // Initialize your custom model client
    this.client = {
      streamCompletion: async function* () {
        yield { type: 'message_start', role: 'assistant' }
      },
    }
  }

  updateConfig(config: Partial<CustomModelConfig>): void {
    this.config = { ...this.config, ...config }
  }

  getConfig(): CustomModelConfig {
    return { ...this.config }
  }

  async *stream(
    messages: Message[],
    options?: {
      systemPrompt?: string | string[]
      toolSpecs?: ToolSpec[]
      toolChoice?: any
    }
  ): AsyncIterable<ModelStreamEvent> {
    // Implementation in next section
    // This is a placeholder that yields nothing
    if (false) yield {} as ModelStreamEvent
  }
}
```
(( /tab "TypeScript" ))

### 2\. Implement the `stream` Method

The core of the model interface is the `stream` method that serves as the single entry point for all model interactions. This method handles request formatting, model invocation, and response streaming.

(( tab "Python" ))
The `stream` method accepts three parameters:

-   [`Messages`](/pr-cms-3708/docs/api/python/strands.types.content#Messages): A list of Strands Agents messages, containing a [Role](/pr-cms-3708/docs/api/python/strands.types.content#Role) and a list of [ContentBlocks](/pr-cms-3708/docs/api/python/strands.types.content#ContentBlock).
-   [`list[ToolSpec]`](/pr-cms-3708/docs/api/python/strands.types.tools#ToolSpec): List of tool specifications that the model can decide to use.
-   `SystemPrompt`: A system prompt string given to the Model to prompt it how to answer the user.

```python
    @override
    async def stream(
        self,
        messages: Messages,
        tool_specs: Optional[list[ToolSpec]] = None,
        system_prompt: Optional[str] = None,
        **kwargs: Any
    ) -> AsyncIterable[StreamEvent]:
        """Stream responses from the Custom model.

        Args:
            messages: List of conversation messages
            tool_specs: Optional list of available tools
            system_prompt: Optional system prompt
            **kwargs: Additional keyword arguments for future extensibility

        Returns:
            Iterator of StreamEvent objects
        """
        logger.debug("messages=<%s> tool_specs=<%s> system_prompt=<%s> | formatting request",
                    messages, tool_specs, system_prompt)

        # Format the request for your model API
        request = {
            "messages": messages,
            "tools": tool_specs,
            "system_prompt": system_prompt,
            **self.config,  # Include model configuration
        }

        logger.debug("request=<%s> | invoking model", request)

        # Invoke your model
        try:
            response = await self.client(**request)
        except OverflowException as e:
            raise ContextWindowOverflowException() from e

        logger.debug("response received | processing stream")

        # Process and yield streaming events
        # If your model doesn't return a MessageStart event, create one
        yield {
            "messageStart": {
                "role": "assistant"
            }
        }

        # Process each chunk from your model's response
        async for chunk in response["stream"]:
            # Convert your model's event format to Strands Agents StreamEvent
            if chunk.get("type") == "text_delta":
                yield {
                    "contentBlockDelta": {
                        "delta": {
                            "text": chunk.get("text", "")
                        }
                    }
                }
            elif chunk.get("type") == "message_stop":
                yield {
                    "messageStop": {
                        "stopReason": "end_turn"
                    }
                }

        logger.debug("stream processing complete")
```

For more complex implementations, you may want to create helper methods to organize your code:

```python
    def _format_request(
        self,
        messages: Messages,
        tool_specs: Optional[list[ToolSpec]] = None,
        system_prompt: Optional[str] = None
    ) -> dict[str, Any]:
        """Optional helper method to format requests for your model API."""
        return {
            "messages": messages,
            "tools": tool_specs,
            "system_prompt": system_prompt,
            **self.config,
        }

    def _format_chunk(self, event: Any) -> Optional[StreamEvent]:
        """Optional helper method to format your model's response events."""
        if event.get("type") == "text_delta":
            return {
                "contentBlockDelta": {
                    "delta": {
                        "text": event.get("text", "")
                    }
                }
            }
        elif event.get("type") == "message_stop":
            return {
                "messageStop": {
                    "stopReason": "end_turn"
                }
            }
        return None
```

> Note: `stream` must be implemented async. If your client does not support async invocation, you may consider wrapping the relevant calls in a thread so as not to block the async event loop. For an example on how to achieve this, you can check out the [BedrockModel](https://github.com/strands-agents/harness-sdk/blob/main/strands-py/src/strands/models/bedrock.py) provider implementation.
(( /tab "Python" ))

(( tab "TypeScript" ))
The `stream` method is the core interface that handles model invocation and returns streaming events. This method must be implemented as an async generator.

```typescript
// Implementation of the stream method and helper methods

export class CustomModelStreamExample {
  private config: CustomModelConfig
  private client: CustomModelClient

  constructor(config: CustomModelConfig) {
    this.config = config
    this.client = {
      streamCompletion: async function* () {
        yield { type: 'message_start', role: 'assistant' }
      },
    }
  }

  updateConfig(config: Partial<CustomModelConfig>): void {
    this.config = { ...this.config, ...config }
  }

  getConfig(): CustomModelConfig {
    return { ...this.config }
  }

  async *stream(
    messages: Message[],
    options?: {
      systemPrompt?: string | string[]
      toolSpecs?: ToolSpec[]
      toolChoice?: any
    }
  ): AsyncIterable<ModelStreamEvent> {
    // 1. Format messages for your model's API
    const formattedMessages = this.formatMessages(messages)
    const formattedTools = options?.toolSpecs
      ? this.formatTools(options.toolSpecs)
      : undefined

    // 2. Prepare the API request
    const request = {
      model: this.config.modelId,
      messages: formattedMessages,
      systemPrompt: options?.systemPrompt,
      tools: formattedTools,
      maxTokens: this.config.maxTokens,
      temperature: this.config.temperature,
      topP: this.config.topP,
      stream: true,
    }

    // 3. Call your model's API and stream responses
    const response = await this.client.streamCompletion(request)

    // 4. Convert API events to Strands ModelStreamEvent format
    for await (const chunk of response) {
      yield this.convertToModelStreamEvent(chunk)
    }
  }

  private formatMessages(messages: Message[]): any[] {
    return messages.map((message) => ({
      role: message.role,
      content: this.formatContent(message.content),
    }))
  }

  private formatContent(content: ContentBlock[]): any {
    // Convert Strands content blocks to your model's format
    return content.map((block) => {
      if (block.type === 'textBlock') {
        return { type: 'text', text: block.text }
      }
      // Handle other content types...
      return block
    })
  }

  private formatTools(toolSpecs: ToolSpec[]): any[] {
    return toolSpecs.map((tool) => ({
      name: tool.name,
      description: tool.description,
      parameters: tool.inputSchema,
    }))
  }

  private convertToModelStreamEvent(chunk: any): ModelStreamEvent {
    // Convert your model's streaming response to ModelStreamEvent

    if (chunk.type === 'message_start') {
      const event: ModelMessageStartEventData = {
        type: 'modelMessageStartEvent',
        role: chunk.role,
      }
      return event
    }

    if (chunk.type === 'content_block_delta') {
      if (chunk.delta.type === 'text_delta') {
        const event: ModelContentBlockDeltaEventData = {
          type: 'modelContentBlockDeltaEvent',
          delta: {
            type: 'textDelta',
            text: chunk.delta.text,
          },
        }
        return event
      }
    }

    if (chunk.type === 'message_stop') {
      const event: ModelMessageStopEventData = {
        type: 'modelMessageStopEvent',
        stopReason: this.mapStopReason(chunk.stopReason),
      }
      return event
    }

    throw new Error(`Unsupported chunk type: ${chunk.type}`)
  }

  private mapStopReason(
    reason: string
  ): 'endTurn' | 'maxTokens' | 'toolUse' | 'stopSequence' {
    const stopReasonMap: Record<
      string,
      'endTurn' | 'maxTokens' | 'toolUse' | 'stopSequence'
    > = {
      end_turn: 'endTurn',
      max_tokens: 'maxTokens',
      tool_use: 'toolUse',
      stop_sequence: 'stopSequence',
    }
    return stopReasonMap[reason] || 'endTurn'
  }
}
```
(( /tab "TypeScript" ))

### 3\. Understanding StreamEvent Types

Your custom model provider needs to convert your model’s response events to Strands Agents streaming event format.

(( tab "Python" ))
The Python SDK uses dictionary-based [StreamEvent](/pr-cms-3708/docs/api/python/strands.types.streaming#StreamEvent) format:

-   [`messageStart`](/pr-cms-3708/docs/api/python/strands.types.streaming#MessageStartEvent): Event signaling the start of a message in a streaming response. This should have the `role`: `assistant`

```python
{
    "messageStart": {
        "role": "assistant"
    }
}
```

-   [`contentBlockStart`](/pr-cms-3708/docs/api/python/strands.types.streaming#ContentBlockStartEvent): Event signaling the start of a content block. If this is the first event of a tool use request, then set the `toolUse` key to have the value [ContentBlockStartToolUse](/pr-cms-3708/docs/api/python/strands.types.content#ContentBlockStartToolUse)

```python
{
    "contentBlockStart": {
        "start": {
            "name": "someToolName", # Only include name and toolUseId if this is the start of a ToolUseContentBlock
            "toolUseId": "uniqueToolUseId"
        }
    }
}
```

-   [`contentBlockDelta`](/pr-cms-3708/docs/api/python/strands.types.streaming#ContentBlockDeltaEvent): Event continuing a content block. This event can be sent several times, and each piece of content will be appended to the previously sent content.

```python
{
    "contentBlockDelta": {
        "delta": { # Only include one of the following keys in each event
            "text": "Some text", # String response from a model
            "reasoningContent": { # Dictionary representing the reasoning of a model.
                "redactedContent": b"Some encrypted bytes",
                "signature": "verification token",
                "text": "Some reasoning text"
            },
            "toolUse": { # Dictionary representing a toolUse request. This is a partial json string.
                "input": "Partial json serialized response"
            }
        }
    }
}
```

-   [`contentBlockStop`](/pr-cms-3708/docs/api/python/strands.types.streaming#ContentBlockStopEvent): Event marking the end of a content block. Once this event is sent, all previous events between the previous [ContentBlockStartEvent](/pr-cms-3708/docs/api/python/strands.types.streaming#ContentBlockStartEvent) and this one can be combined to create a [ContentBlock](/pr-cms-3708/docs/api/python/strands.types.content#ContentBlock)

```python
{
    "contentBlockStop": {}
}
```

-   [`messageStop`](/pr-cms-3708/docs/api/python/strands.types.streaming#MessageStopEvent): Event marking the end of a streamed response, and the [StopReason](/pr-cms-3708/docs/api/python/strands.types.event_loop#StopReason). No more content block events are expected after this event is returned.

```python
{
    "messageStop": {
        "stopReason": "end_turn"
    }
}
```

-   [`metadata`](/pr-cms-3708/docs/api/python/strands.types.streaming#MetadataEvent): Event representing the metadata of the response. This contains the input, output, and total token count, along with the latency of the request.

```python
{
    "metrics": {
        "latencyMs": 123 # Latency of the model request in milliseconds.
    },
    "usage": {
        "inputTokens": 234, # Number of tokens sent in the request to the model.
        "outputTokens": 234, # Number of tokens that the model generated for the request.
        "totalTokens": 468 # Total number of tokens (input + output).
    }
}
```

-   [`redactContent`](/pr-cms-3708/docs/api/python/strands.types.streaming#RedactContentEvent): Event that is used to redact the users input message, or the generated response of a model. This is useful for redacting content if a guardrail gets triggered.

```python
{
    "redactContent": {
        "redactUserContentMessage": "User input Redacted",
        "redactAssistantContentMessage": "Assistant output Redacted"
    }
}
```
(( /tab "Python" ))

(( tab "TypeScript" ))
The TypeScript SDK uses data interface types for `ModelStreamEvent`. Create events as plain objects matching these interfaces:

-   `ModelMessageStartEvent`: Signals the start of a message response

```typescript
const messageStart: ModelMessageStartEventData = {
  type: 'modelMessageStartEvent',
  role: 'assistant',
}
```

-   `ModelContentBlockStartEvent`: Signals the start of a content block

```typescript
// For text blocks
const textBlockStart: ModelContentBlockStartEventData = {
  type: 'modelContentBlockStartEvent',
}

// For tool use blocks
const toolUseStart: ModelContentBlockStartEventData = {
  type: 'modelContentBlockStartEvent',
  start: {
    type: 'toolUseStart',
    toolUseId: 'tool_123',
    name: 'calculator',
  },
}
```

-   `ModelContentBlockDeltaEvent`: Provides incremental content

```typescript
// For text
const textDelta: ModelContentBlockDeltaEventData = {
  type: 'modelContentBlockDeltaEvent',
  delta: { type: 'textDelta', text: 'Hello' },
}

// For tool input
const toolInputDelta: ModelContentBlockDeltaEventData = {
  type: 'modelContentBlockDeltaEvent',
  delta: { type: 'toolUseInputDelta', input: '{"x": 1' },
}

// For reasoning content
const reasoningDelta: ModelContentBlockDeltaEventData = {
  type: 'modelContentBlockDeltaEvent',
  delta: {
    type: 'reasoningContentDelta',
    text: 'thinking...',
    signature: 'sig',
    redactedContent: new Uint8Array([]),
  },
}
```

-   `ModelContentBlockStopEvent`: Signals the end of a content block

```typescript
const blockStop: ModelStreamEvent = {
  type: 'modelContentBlockStopEvent',
}
```

-   `ModelMessageStopEvent`: Signals the end of the message with stop reason

```typescript
const messageStop: ModelMessageStopEventData = {
  type: 'modelMessageStopEvent',
  stopReason: 'endTurn', // Or 'maxTokens', 'toolUse', 'stopSequence'
}
```

-   `ModelMetadataEvent`: Provides usage and metrics information

```typescript
const metadata: ModelMetadataEventData = {
  type: 'modelMetadataEvent',
  usage: {
    inputTokens: 234,
    outputTokens: 234,
    totalTokens: 468,
  },
  metrics: {
    latencyMs: 123,
  },
}
```
(( /tab "TypeScript" ))

### 4\. Use Your Custom Model Provider

Once implemented, you can use your custom model provider in your applications for regular agent invocation:

(( tab "Python" ))
```python
from strands import Agent
from your_org.models.custom_model import CustomModel

# Initialize your custom model provider
custom_model = CustomModel(
    api_key="your-api-key",
    model_id="your-model-id",
    params={
        "max_tokens": 2000,
        "temperature": 0.7,
    },
)

# Create a Strands agent using your model
agent = Agent(model=custom_model)

# Use the agent as usual
response = agent("Hello, how are you today?")
```
(( /tab "Python" ))

(( tab "TypeScript" ))
```typescript
async function usageExample() {
  // Initialize your custom model provider
  const customModel = new YourCustomModel({
    maxTokens: 2000,
    temperature: 0.7,
  })

  // Create a Strands agent using your model
  const agent = new Agent({ model: customModel })

  // Use the agent as usual
  const response = await agent.invoke('Hello, how are you today?')
}
```
(( /tab "TypeScript" ))

## Key Implementation Considerations

### 1\. Stream Interface

The model interface centers around a single `stream` method that:

-   Accepts `messages`, `tool_specs`, and `system_prompt` directly as parameters
-   Handles request formatting, model invocation, and response processing internally
-   Provides debug logging for better observability

### 2\. Message Formatting

Strands Agents’ internal `Message`, `ToolSpec`, and `SystemPrompt` types must be converted to your model API’s expected format:

-   Strands Agents uses a structured message format with role and content fields
-   Your model API might expect a different structure
-   Handle the message content conversion in your `stream()` method

### 3\. Streaming Response Handling

Strands Agents expects streaming responses to be formatted according to its `StreamEvent` protocol:

-   `messageStart`: Indicates the start of a response message
-   `contentBlockStart`: Indicates the start of a content block
-   `contentBlockDelta`: Contains incremental content updates
-   `contentBlockStop`: Indicates the end of a content block
-   `messageStop`: Indicates the end of the response message with a stop reason
-   `metadata`: Indicates information about the response like input\_token count, output\_token count, and latency
-   `redactContent`: Used to redact either the user’s input, or the model’s response

Convert your API’s streaming format to match these expectations in your `stream()` method.

### 4\. Tool Support

If your model API supports tools or function calling:

-   Format tool specifications appropriately in `stream()`
-   Handle tool-related events in response processing
-   Ensure proper message formatting for tool calls and results

### 5\. Error Handling

Implement robust error handling for API communication:

-   Context window overflows
-   Connection errors
-   Authentication failures
-   Rate limits and quotas
-   Malformed responses

### 6\. Configuration Management

The built-in `get_config` and `update_config` methods allow for the model’s configuration to be changed at runtime:

-   `get_config` exposes the current model config
-   `update_config` allows for at-runtime updates to the model config
    -   For example, changing model\_id with a tool call

## Related pages

- [Tool Executors](/pr-cms-3708/docs/user-guide/concepts/tools/executors/index.md) (1 shared tag)
- [Available Sandboxes](/pr-cms-3708/docs/user-guide/concepts/sandbox/available-sandboxes/index.md) (1 shared tag)
- [Building a Custom Sandbox](/pr-cms-3708/docs/user-guide/concepts/sandbox/custom-sandbox/index.md) (1 shared tag)
- [Human in the Loop](/pr-cms-3708/docs/user-guide/concepts/agents/interventions/human-in-the-loop/index.md) (1 shared tag)
- [Sandbox](/pr-cms-3708/docs/user-guide/concepts/sandbox/index.md) (1 shared tag)
- [Cedar Authorization](/pr-cms-3708/docs/user-guide/concepts/agents/interventions/cedar-authorization/index.md) (1 shared tag)
- [Agent Loop](/pr-cms-3708/docs/user-guide/concepts/agents/agent-loop/index.md) (1 shared tag)
- [Hooks](/pr-cms-3708/docs/user-guide/concepts/agents/hooks/index.md) (1 shared tag)
- [Steering (Interventions)](/pr-cms-3708/docs/user-guide/concepts/agents/interventions/steering/index.md) (1 shared tag)
- [Agents as Tools with Strands Agents SDK](/pr-cms-3708/docs/user-guide/concepts/multi-agent/agents-as-tools/index.md) (1 shared tag)


## Implementation

### Python

- [harness-sdk/strands-py/src/strands/models/model.py](https://github.com/strands-agents/harness-sdk/blob/main/strands-py/src/strands/models/model.py)

### TypeScript

- [harness-sdk/strands-ts/src/models/model.ts](https://github.com/strands-agents/harness-sdk/blob/main/strands-ts/src/models/model.ts)