NVIDIA NIM

Name: Strands Agents SDK
Author: Strands Agents

strands-nvidia-nim is a custom model provider that enables Strands Agents to work with Nvidia NIM APIs. It bridges the message format compatibility gap between Strands Agents SDK and Nvidia NIM API endpoints.

Features:

Message Format Conversion: Automatically converts Strands’ structured content to simple string format required by Nvidia NIM
Tool Support: Full support for Strands tools with proper error handling
Clean Streaming: Proper streaming output without artifacts
Error Handling: Context window overflow detection and Strands-specific errors

Installation

Install strands-nvidia-nim from PyPI:

pip install strands-nvidia-nim strands-agents-tools

Usage

Basic Agent

import ast
import operator

from strands import Agent, tool
from strands_nvidia_nim import NvidiaNIM

@tool
def calculator(expression: str) -> str:
    """Evaluate an arithmetic expression such as "144 ** 0.5" or "450 / 120".

    Args:
        expression: The arithmetic expression to evaluate.
    """
    ops = {ast.Add: operator.add, ast.Sub: operator.sub, ast.Mult: operator.mul,
           ast.Div: operator.truediv, ast.USub: operator.neg}

    def ev(n):
        if isinstance(n, ast.Constant) and isinstance(n.value, (int, float)):
            return n.value
        if isinstance(n, ast.BinOp) and isinstance(n.op, ast.Pow):
            base, exp = ev(n.left), ev(n.right)
            if abs(exp) > 64:
                raise ValueError(f"exponent too large in {expression!r}: {exp}")
            return base**exp
        if isinstance(n, ast.BinOp) and type(n.op) in ops:
            return ops[type(n.op)](ev(n.left), ev(n.right))
        if isinstance(n, ast.UnaryOp) and type(n.op) in ops:
            return ops[type(n.op)](ev(n.operand))
        raise ValueError(
            f"{expression!r} is not arithmetic; supported: + - * / ** and parentheses over numbers"
        )

    return str(ev(ast.parse(expression, mode="eval").body))

model = NvidiaNIM(
    api_key="your-nvidia-nim-api-key",
    model_id="meta/llama-3.1-70b-instruct",
    params={
        "max_tokens": 1000,
        "temperature": 0.7,
    }
)

agent = Agent(model=model, tools=[calculator])
agent("What is 123.456 * 789.012?")

Using Environment Variables

export NVIDIA_NIM_API_KEY=your-nvidia-nim-api-key

import ast
import operator
import os
from strands import Agent, tool
from strands_nvidia_nim import NvidiaNIM

@tool
def calculator(expression: str) -> str:
    """Evaluate an arithmetic expression such as "144 ** 0.5" or "450 / 120".

    Args:
        expression: The arithmetic expression to evaluate.
    """
    ops = {ast.Add: operator.add, ast.Sub: operator.sub, ast.Mult: operator.mul,
           ast.Div: operator.truediv, ast.USub: operator.neg}

    def ev(n):
        if isinstance(n, ast.Constant) and isinstance(n.value, (int, float)):
            return n.value
        if isinstance(n, ast.BinOp) and isinstance(n.op, ast.Pow):
            base, exp = ev(n.left), ev(n.right)
            if abs(exp) > 64:
                raise ValueError(f"exponent too large in {expression!r}: {exp}")
            return base**exp
        if isinstance(n, ast.BinOp) and type(n.op) in ops:
            return ops[type(n.op)](ev(n.left), ev(n.right))
        if isinstance(n, ast.UnaryOp) and type(n.op) in ops:
            return ops[type(n.op)](ev(n.operand))
        raise ValueError(
            f"{expression!r} is not arithmetic; supported: + - * / ** and parentheses over numbers"
        )

    return str(ev(ast.parse(expression, mode="eval").body))

model = NvidiaNIM(
    api_key=os.getenv("NVIDIA_NIM_API_KEY"),
    model_id="meta/llama-3.1-70b-instruct",
    params={"max_tokens": 1000, "temperature": 0.7}
)

agent = Agent(model=model, tools=[calculator])
agent("What is 123.456 * 789.012?")

Configuration

Model Configuration

The NvidiaNIM provider accepts the following parameters:

Parameter	Description	Example
`api_key`	Your Nvidia NIM API key	`"nvapi-..."`
`model_id`	Model identifier	`"meta/llama-3.1-70b-instruct"`
`params`	Generation parameters	`{"max_tokens": 1000}`

Available Models

Popular Nvidia NIM models:

meta/llama-3.1-70b-instruct - High quality, larger model
meta/llama-3.1-8b-instruct - Faster, smaller model
meta/llama-3.3-70b-instruct - Latest Llama model
mistralai/mistral-large - Mistral’s flagship model
nvidia/llama-3.1-nemotron-70b-instruct - Nvidia-optimized variant

Generation Parameters

model = NvidiaNIM(
    api_key="your-api-key",
    model_id="meta/llama-3.1-70b-instruct",
    params={
        "max_tokens": 1500,
        "temperature": 0.7,
        "top_p": 0.9,
        "frequency_penalty": 0.0,
        "presence_penalty": 0.0
    }
)

Troubleshooting

`BadRequestError` with message formatting

This provider exists specifically to solve message formatting issues between Strands and Nvidia NIM. If you encounter this error using standard LiteLLM integration, switch to strands-nvidia-nim.

Context window overflow

The provider includes detection for context window overflow errors. If you encounter this, try reducing max_tokens or the size of your prompts.