Model
Defined in: src/models/model.ts:379
Base abstract class for model providers. Defines the contract that all model provider implementations must follow.
Model providers handle communication with LLM APIs and implement streaming responses using async iterables.
Extended by
Section titled “Extended by”Type Parameters
Section titled “Type Parameters”| Type Parameter | Default type | Description |
|---|---|---|
T extends BaseModelConfig | BaseModelConfig | Model configuration type extending BaseModelConfig |
Constructors
Section titled “Constructors”Constructor
Section titled “Constructor”new Model<T>(): Model<T>;Returns
Section titled “Returns”Model<T>
Accessors
Section titled “Accessors”modelId
Section titled “modelId”Get Signature
Section titled “Get Signature”get modelId(): string;Defined in: src/models/model.ts:398
The model ID from the current configuration, if configured.
Returns
Section titled “Returns”string
stateful
Section titled “stateful”Get Signature
Section titled “Get Signature”get stateful(): boolean;Defined in: src/models/model.ts:414
Whether this model manages conversation state server-side.
When true, the server tracks conversation context across turns, so the SDK
sends only the latest message instead of the full history. After each invocation,
the agent’s local message history is cleared automatically.
Reads modelInfo.capabilities.stateful, so a provider that keeps state server-side sets it there
(see modelInfo) and a modelInfo config override can change it.
Returns
Section titled “Returns”boolean
false when unknown
modelInfo
Section titled “modelInfo”Get Signature
Section titled “Get Signature”get modelInfo(): ModelInfo;Defined in: src/models/model.ts:428
Everything the SDK knows about the configured model, resolved on each read. Layers, later
ones winning: the shared model info for modelId, temporaryMessages off (a provider that supports
them turns them back on), the contextWindowLimit config value, and the modelInfo config override. Each read returns a new record, so do not compare reads by reference.
A provider that knows better than the shared model info overrides this getter: put its facts over
super.modelInfo, then the user’s getConfig().modelInfo over those so the override still wins:
{ ...info, capabilities: { ...info.capabilities, promptCaching: 'auto', ...config.modelInfo?.capabilities } }.
Do not read stateful inside the override; it reads this getter.
Returns
Section titled “Returns”Methods
Section titled “Methods”updateConfig()
Section titled “updateConfig()”abstract updateConfig(modelConfig): void;Defined in: src/models/model.ts:386
Updates the model configuration. Merges the provided configuration with existing settings.
Parameters
Section titled “Parameters”| Parameter | Type | Description |
|---|---|---|
modelConfig | T | Configuration object with model-specific settings to update |
Returns
Section titled “Returns”void
getConfig()
Section titled “getConfig()”abstract getConfig(): T;Defined in: src/models/model.ts:393
Retrieves the current model configuration.
Returns
Section titled “Returns”T
The current configuration object
stream()
Section titled “stream()”abstract stream(messages, options?): AsyncIterable<ModelStreamEvent>;Defined in: src/models/model.ts:448
Streams a conversation with the model. Returns an async iterable that yields streaming events as they occur.
Parameters
Section titled “Parameters”| Parameter | Type | Description |
|---|---|---|
messages | Message[] | Array of conversation messages |
options? | StreamOptions | Optional streaming configuration |
Returns
Section titled “Returns”AsyncIterable<ModelStreamEvent>
Async iterable of streaming events
countTokens()
Section titled “countTokens()”countTokens(messages, options?): Promise<number>;Defined in: src/models/model.ts:465
Count tokens for the given input before sending to the model.
Used for proactive context management (e.g., triggering compression at a threshold). The base implementation uses a character-based heuristic (chars/4 for text, chars/2 for JSON). It counts only the latest user message’s injected blocks, the ones a model reads.
Subclasses should override this method to use native token counting APIs
(e.g., Bedrock CountTokens, Anthropic countTokens, Gemini countTokens)
for improved accuracy, falling back to super.countTokens() on API failure.
Parameters
Section titled “Parameters”| Parameter | Type | Description |
|---|---|---|
messages | Message[] | Array of conversation messages to count tokens for |
options? | CountTokensOptions | Optional options containing system prompt and tool specs |
Returns
Section titled “Returns”Promise<number>
Total input token count
summarize()
Section titled “summarize()”summarize(messages, options?): Promise<SummarizeResult>;Defined in: src/models/model.ts:490
Summarizes messages.
messages is a request as the caller would send it to stream(), along with its tool specs and system
prompt. The base implementation sends that request closed by an instruction not to act on it or call tools,
followed by instructions (or default ones), and otherwise as given, so a caller that replays its own
conversation can share its prompt cache. The summary is the reply’s text, including a reply cut off at the
output limit or a stop sequence. A reply that ends for another reason, such as a tool call, a cancellation or a
guardrail, or that holds no text, gives an empty result.
Every summary is a summaryBlock. The base implementation leaves its provider unset. A provider that reports
modelInfo.capabilities.serverSideSummary as true overrides this method to summarize server-side and sets
provider to its own name, using the base implementation when the capability is off or the server-side
summary fails, as AnthropicModel does. A summary with a provider can stand in for the messages it
summarized only when they start at the conversation’s first message; a caller that summarizes anything else
uses only its text.
Parameters
Section titled “Parameters”| Parameter | Type | Description |
|---|---|---|
messages | Message[] | The request to summarize, left unmodified |
options? | SummarizeOptions | The request’s system prompt and tools, summarization instructions, cancel signal and agent metadata |
Returns
Section titled “Returns”Promise<SummarizeResult>
The summary and the token usage of the calls made for it
estimateUtilization()
Section titled “estimateUtilization()”estimateUtilization(inputTokens): number;Defined in: src/models/model.ts:503
Estimate the fraction of the model’s context window consumed by the given input token count.
Resolves the model’s context window limit (falling back to DEFAULT_CONTEXT_WINDOW_LIMIT
with a warning when not configured) and returns inputTokens / contextWindowLimit.
Parameters
Section titled “Parameters”| Parameter | Type | Description |
|---|---|---|
inputTokens | number | Total input token count (e.g. from a model event’s projectedInputTokens) |
Returns
Section titled “Returns”number
Token usage ratio (0–1+; above 1.0 means overflow)
streamAggregated()
Section titled “streamAggregated()”streamAggregated(messages, options?): AsyncGenerator< | ContentBlock| ModelStreamEvent, StreamAggregatedResult, undefined>;Defined in: src/models/model.ts:570
Streams a conversation with aggregated content blocks and messages. Returns an async generator that yields streaming events and content blocks, and returns the final message with stop reason and optional metadata.
This method enhances the basic stream() by collecting streaming events into complete ContentBlock and Message objects, which are needed by the agentic loop for tool execution and conversation management.
The method yields:
- ModelStreamEvent - Original streaming events (passed through)
- ContentBlock - Complete content block (emitted when block completes)
The method returns:
- StreamAggregatedResult containing the complete message, stop reason, and optional metadata
All exceptions thrown from this method are wrapped in ModelError to provide a consistent error type for model-related errors. Specific error subtypes like ContextWindowOverflowError, ModelThrottledError, and MaxTokensError are preserved.
Parameters
Section titled “Parameters”| Parameter | Type | Description |
|---|---|---|
messages | Message[] | Array of conversation messages |
options? | StreamOptions | Optional streaming configuration |
Returns
Section titled “Returns”AsyncGenerator<
| ContentBlock
| ModelStreamEvent, StreamAggregatedResult, undefined>
Async generator yielding ModelStreamEvent | ContentBlock and returning a StreamAggregatedResult
Throws
Section titled “Throws”ModelError - Base class for all model-related errors
Throws
Section titled “Throws”ContextWindowOverflowError - When input exceeds the model’s context window
Throws
Section titled “Throws”ModelThrottledError - When the model provider throttles requests
Throws
Section titled “Throws”MaxTokensError - When the model reaches its maximum token limit