⚙️ How to Configure Non-OpenAI LLM Providers at Different Levels in Agents SDK 🚀

The OpenAI Agents SDK is built to use OpenAI models by default — but what if you’re vibing with other LLMs like Gemini, Claude, or Mistral? 😍✨ Well guess what? You can plug them in effortlessly 🔌💡 using a few smart configuration tweaks.

In this blog, we’ll deep dive into how to configure non-OpenAI providers 🔄 at three levels:

💥 Agent Level — model per agent
⚡ Run Level — model for a single execution
🌐 Global Level — model for the entire app

Plus, we’ll explore using LiteLLM, explain theory 🔍, decode code snippets 💻, and fix common issues 🛠️ — all in your fav aesthetic & confident tone 💅🔥

✅ Prerequisites 🎒

Before we jump into code, make sure you’ve installed the required tools:

uv add openai-agents

Want to use other providers like Claude or Gemini via LiteLLM?

uv add "openai-agents[litellm]"

1️⃣ Agent-Level Configuration (Customized Agents)

This means: assigning a different model just to one agent.

Perfect for:

• Specialized tasks

• Language-specific agents

• Testing providers without affecting other agents

📌 Example: Using Gemini for Just One Agent

import asyncio
from openai import AsyncOpenAI
from agents import Agent, OpenAIChatCompletionsModel, Runner, set_tracing_disabled

gemini_api_key = ""

#Reference: https://ai.google.dev/gemini-api/docs/openai
client = AsyncOpenAI(
    api_key=gemini_api_key,
    base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
)

set_tracing_disabled(disabled=True)

async def main():
    # This agent will use the custom LLM provider
    agent = Agent(
        name="Assistant",
        instructions="You only respond in haikus.",
        model=OpenAIChatCompletionsModel(model="gemini-2.0-flash", openai_client=client),
    )

    result = await Runner.run(
        agent,
        "Tell me about recursion in programming.",
    )
    print(result.final_output)


if __name__ == "__main__":
    asyncio.run(main())

📖 This works great if you only want to override the model for one bot without affecting others.

2️⃣ Run-Level Configuration 🏃‍♀️ (One-Time Switch)

This allows you to configure a specific model for one task execution only.

Ideal when:

• You want to override models just once

• A temporary provider swap is needed

🧪 Example: Use Gemini in a Single Run

from agents import Agent, Runner, AsyncOpenAI, OpenAIChatCompletionsModel
from agents.run import RunConfig

gemini_api_key = ""

#Reference: https://ai.google.dev/gemini-api/docs/openai
external_client = AsyncOpenAI(
    api_key=gemini_api_key,
    base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
)

model = OpenAIChatCompletionsModel(
    model="gemini-2.0-flash",
    openai_client=external_client
)

config = RunConfig(
    model=model,
    model_provider=external_client,
    tracing_disabled=True
)

agent: Agent = Agent(name="Assistant", instructions="You are a helpful assistant")

result = Runner.run_sync(agent, "Hello, how are you.", run_config=config)

print(result.final_output)

💬 Handy when you want to test or isolate logic during a specific job execution.

3️⃣ Global Configuration 🌍 (One Model to Rule Them All)

Want to replace OpenAI for your entire app? Set it up globally and forget about repeating configs. 🎯

Use when:

• You’re building on Gemini/Claude entirely 🔧

• Avoiding OpenAI key usage or costs 💰

🛠️ Example: Gemini as Default Everywhere

from agents import Agent, Runner, AsyncOpenAI, set_default_openai_client, set_tracing_disabled, set_default_openai_api

gemini_api_key = ""
set_tracing_disabled(True)
set_default_openai_api("chat_completions")

external_client = AsyncOpenAI(
    api_key=gemini_api_key,
    base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
)
set_default_openai_client(external_client)

agent: Agent = Agent(name="Assistant", instructions="You are a helpful assistant", model="gemini-2.0-flash")

result = Runner.run_sync(agent, "Hello")

print(result.final_output)

💡 Global setup is clean and ideal when you’re building a mono-provider project.

🔀 Mixing Models Across Agents 🧬

What if you want to use multiple providers or models in one workflow? Mix ‘n match, baby! 🧃💅

Multi-Agent Routing Example:

from agents import Agent, Runner, OpenAIChatCompletionsModel
import asyncio
spanish_agent = Agent(
    name="Spanish Agent",
    instructions="Respond only in Spanish.",
    model="o3-mini"
)
english_agent = Agent(
    name="English Agent",
    instructions="Respond only in English.",
    model=OpenAIChatCompletionsModel(
        model="gpt-4o",
        openai_client=AsyncOpenAI()
    )
)
triage_agent = Agent(
    name="Triage Agent",
    instructions="Detect input language & route.",
    handoffs=[spanish_agent, english_agent],
    model="gpt-3.5-turbo"
)
async def main():
    result = await Runner.run(triage_agent, "Hola, ¿cómo estás?")
    print(result.final_output)
asyncio.run(main())

This makes your app smart enough to assign the right model for the right language or task! 😎

🪄 Bonus: LiteLLM Integration ✨

Install once, unlock multiple providers easily via litellm/ prefix.

📦 Example:

Agent(model="litellm/anthropic/claude-3-5-sonnet-20240620")
Agent(model="litellm/gemini/gemini-2.5-flash-preview-04-17")

LiteLLM handles the routing, tokens, and formatting for you

🛑 Common Errors & Fixes 🧯

❌ Error 💡 Solution 401 Tracing Error Use set_tracing_disabled(True) if you don’t have OpenAI key 404 on Responses API Use OpenAIChatCompletionsModel, not OpenAIResponsesModel Invalid JSON Output Use models that support structured outputs Image/Tool support fails Check if the provider supports tools/multimodal

🧠 TL;DR — You’re in Control Now 💪

✨ Want to use Claude for Agent A, Gemini for Agent B, and OpenAI for Agent C? DONE.
✨ Want one-time override with zero commitment? DONE.
✨ Want global provider setup across your app? DONE.

With Agent SDK, you’re in charge of every decision:

✅ Per-Agent
✅ Per-Run
✅ Global Setup
✅ LiteLLM for simplified access

🛠️

🧠 Final Thoughts from MughalSyntax

In a multi-model world, flexibility isn’t just a luxury — it’s a necessity. Whether you’re testing Gemini’s speed, Claude’s reasoning, or just dancing between models for fun — knowing how to integrate them cleanly is a skill worth mastering. ✨

So, go ahead — experiment, iterate, and fine-tune your agents. The SDK’s got your back, and now… so does your blog. 💻💬

Until the next exploration, stay confident in your code and curious at heart.

~ Ayesha Mughal