Large Language Model Integrations
The LLM is the brain of your voice agent. It understands what callers say and decides how to respond. Bolna lets you switch between models like GPT-5.4 mini, Claude and DeepSeek by changing the provider and model in your agent LLM settings, so you can tune for speed, cost or reasoning depth.
Anthropic
Anthropic Claude writes your Bolna voice agent's replies. Use claude-sonnet-5 for most agents, or claude-haiku-4-5 for faster, high-volume calls.
Azure OpenAI
Azure OpenAI runs GPT models for your Bolna voice agent inside your own Azure account, for teams that need regional data residency and private networking.
DeepSeek
DeepSeek V4 runs the conversation in a Bolna voice agent at low cost per token. deepseek-v4-flash suits high-volume calls, with a 1M token context window.
Google Gemini
Google Gemini runs the conversation in a Bolna voice agent with a 1M token context and strong multilingual skills. gemini-2.5-flash is the recommended default.
OpenAI
OpenAI GPT models write your Bolna voice agent's replies. gpt-5.4-mini is the recommended default, with fast first tokens, function calling and low cost.
OpenRouter
OpenRouter gives a Bolna voice agent one API key for models from OpenAI, Anthropic, Google, Mistral, Meta and more. Pick a model and switch any time.
Perplexity
Perplexity can be the LLM behind a Bolna voice agent. Set the provider to perplexity in the agent config and Bolna routes calls to your chosen Perplexity model.
Custom LLM (OpenAI-compatible)
Run a Bolna voice agent on any OpenAI-compatible LLM endpoint, from Groq or Together AI to your own vLLM, SGLang, TGI, Ollama or LiteLLM server.
Baseten
Baseten runs open models for a Bolna voice agent through shared Model APIs or your own Truss deployment on dedicated GPUs, including fine-tuned weights.
Cerebras
Cerebras serves gpt-oss-120b at around 3,000 tokens per second as the LLM for a Bolna voice agent, so long replies and tool calls finish almost instantly.
Fireworks AI
Fireworks AI serves open models to a Bolna voice agent on serverless per-token access or on-demand GPU deployments, connected as an OpenAI-compatible custom LLM.
Groq
Groq runs Llama and GPT-OSS on fast LPU hardware as the LLM for a Bolna voice agent, shortening the pause between the caller finishing and the agent speaking.
Modal
Modal lets you self-host an open-weight LLM with vLLM on serverless GPUs, in a region near your callers, and use it as the model behind a Bolna voice agent.
Nebius
Nebius Token Factory serves open models such as DeepSeek, Qwen and Llama to a Bolna voice agent, with dedicated endpoints in a region you choose for data residency.
Simplismart
Simplismart serves speed-optimised open models to a Bolna voice agent on a shared endpoint or a dedicated deployment, including your own fine-tuned weights.
Together AI
Together AI gives a Bolna voice agent a large serverless catalog of open models such as Llama, Qwen, DeepSeek and MiniMax, billed per token with no provisioning.
Gemini Live
Gemini Live runs a Bolna voice agent on Google's native-audio speech-to-speech model, with 30 prebuilt voices and session resumption for calls past 15 minutes.
OpenAI Realtime
OpenAI Realtime runs a Bolna voice agent on one speech-to-speech model that hears the caller and speaks back, replacing the separate transcriber, LLM and voice.
LLM options on Bolna compared
| Model family | Best for | Trade-off |
|---|---|---|
| OpenAI gpt-5.4-mini | Most voice agents: lead qualification, reminders, scheduling, FAQs | Less reasoning depth than the full GPT-5 models |
| OpenAI gpt-5.4 and gpt-5.5 | Complex reasoning and sensitive domains such as finance and healthcare | Higher cost; raising reasoning effort adds delay before the first spoken word |
| Anthropic Claude (Sonnet 5, Haiku 4.5, Opus 4.8) | Precise instruction following; claude-sonnet-5 for most production agents, Haiku 4.5 for high volume | Opus is slower than Sonnet; Claude is not in the flat-rate bundle, so it is billed at usage-based rates |
| Azure OpenAI | Data residency, private networking and enterprise compliance (SOC 2, HIPAA, ISO 27001) | Needs an Azure deployment per model; new models arrive after a short lag |
| DeepSeek V4 (Flash, Pro) | Cost-sensitive, high-volume campaigns; v4-flash supports up to 2,500 concurrent requests | For complex English instruction following, gpt-5.4-mini may have an edge |
| OpenRouter | One API key for OpenAI, Anthropic, Google, Mistral, Meta and other models, with provider fallback | Short lag on the newest models; use a direct provider if compliance requires knowing who handles your data |
| Perplexity | Teams that already use Perplexity models | No dedicated Bolna reference page; set the provider to perplexity and use Perplexity's own model IDs |
| Google Gemini (2.5 Flash, 3.x) | Multilingual agents, including Hindi, Tamil and Bengali, and large-context work (1M tokens) | gpt-5.4-mini follows English instructions slightly better; gemini-3.7-flash needs max_tokens of 400 or more |
| Fast inference (Groq, Cerebras) | The shortest silence before the agent speaks, on Llama, GPT-OSS and Qwen | Small model catalogs; shared tiers have rate limits that batch campaigns can hit |
| Open-model hosts (Together AI, Fireworks AI, Baseten, Nebius, Simplismart) | Open or fine-tuned models billed per token, with dedicated endpoints when needed | Serverless capacity is best-effort and rate-limited; model IDs and endpoint paths differ by provider |
| Self-hosted (Modal, vLLM, SGLang, TGI, Ollama) | Full control over the model, GPU and region, for data residency or private weights | You run the server; scale-to-zero causes silence or 503s, so keep a replica warm |
| Speech-to-speech (OpenAI Realtime, Gemini Live) | Lowest turn latency, with one model that hears and speaks directly | No knowledge base or multilingual config; voices limited to the provider realtime voices |
For the full guide, see Choosing an LLM in the Bolna docs.
Frequently asked questions
Yes. The LLM is a single setting: change it on the agent's Intelligence tab, or update provider and model in the llm_config block. GPT-5 models require temperature set to 1, so update that field when you switch to one.
gpt-5.4-mini is the recommended low-latency default, and claude-haiku-4-5 is the fastest Claude model. Keep reasoning_effort at none or low and max_tokens around 150, since extra reasoning and long replies add delay before the first spoken word.
Yes. Connect your OpenAI, Anthropic, DeepSeek, Azure OpenAI or OpenRouter account in Settings → Integrations in the Bolna dashboard. LLM costs are then charged to your provider account.
The flat rate ($0.06 per minute at standard wallet tiers) includes OpenAI and Azure OpenAI models such as gpt-5.4-mini, gpt-5.4-nano, gpt-4.1-mini and gpt-4o-mini, plus OpenAI gpt-5.6-luna. Other models are billed at usage-based rates; the Add Funds panel in your dashboard shows the current list.
Use Azure OpenAI. It runs the same OpenAI models on Microsoft infrastructure with data residency controls and private networking. For on-premise or proprietary models, connect any OpenAI-compatible endpoint as a custom LLM.
Yes. Add a custom model with any endpoint that speaks the OpenAI chat completions API. The dashboard has presets for Baseten, Cerebras, Fireworks AI, Groq, Modal, Nebius, Simplismart and Together AI, and Other (OpenAI-compatible) covers your own vLLM, SGLang, TGI, Ollama or LiteLLM server.
Last updated