Usage
Defined in: src/models/streaming.ts:503
Token usage statistics for a model invocation. Tracks input, output, and total tokens, plus cache-related metrics.
Providers differ on whether inputTokens includes cached tokens. Some count cache reads and
writes inside inputTokens; others count them only in cacheReadInputTokens and
cacheWriteInputTokens. totalTokens always includes cached tokens, so for a single model call:
if inputTokens + outputTokens === totalTokens, inputTokens already includes the cache fields;
otherwise, total input is inputTokens + cacheReadInputTokens + cacheWriteInputTokens. Adding the
cache fields to an inputTokens that already includes them counts cached tokens twice.
Custom model providers should include cached tokens in inputTokens (matching the OpenTelemetry
GenAI conventions), so that totalTokens === inputTokens + outputTokens.
Extended by
Section titled “Extended by”Properties
Section titled “Properties”inputTokens
Section titled “inputTokens”inputTokens: number;Defined in: src/models/streaming.ts:507
Number of input tokens, as the model provider counts them (may or may not include cached tokens).
outputTokens
Section titled “outputTokens”outputTokens: number;Defined in: src/models/streaming.ts:512
Number of tokens in the output (completion).
totalTokens
Section titled “totalTokens”totalTokens: number;Defined in: src/models/streaming.ts:517
Total number of tokens processed, including any cached input tokens.
cacheReadInputTokens?
Section titled “cacheReadInputTokens?”cacheReadInputTokens?: number;Defined in: src/models/streaming.ts:523
Number of input tokens read from cache. This can reduce latency and cost.
cacheWriteInputTokens?
Section titled “cacheWriteInputTokens?”cacheWriteInputTokens?: number;Defined in: src/models/streaming.ts:529
Number of input tokens written to cache. These tokens can be reused in future requests.