Baseten LLM inference for Bolna AI voice agents

What is Baseten used for in a Bolna voice agent?

Baseten runs the LLM for a Bolna voice agent in one of two ways: Model APIs, where Baseten hosts popular open models at one shared URL, or a dedicated deployment of any model, including fine-tunes, on GPUs reserved for you. Choose Model APIs to get running in minutes. Choose a dedicated deployment when you need your own weights, a model Model APIs doesn't offer, or capacity nobody else shares.

How do I set up Baseten with Bolna?

1. Create a key on the API keys page of your Baseten workspace settings.
2. For Model APIs, copy a model slug from Baseten's Model APIs list, for example zai-org/GLM-5.3. For a dedicated deployment, install Truss, write a config.yaml, run truss push and wait until the deployment shows Active.
3. In Bolna, open Settings → Integrations → LLM → Custom LLM → Add model, or choose + Add custom LLM from an agent's LLM Provider dropdown. Select Baseten.
4. For Model APIs, enter the slug, set Endpoint URL to https://inference.baseten.co/v1 and Family to glm, for example. For a dedicated deployment, use https://model-<model_id>.api.baseten.co/environments/production/sync/v1 and the model name from the deployment's example request.
5. On the agent's Intelligence tab choose custom, pick the Baseten model, save and place a test call.

The Baseten setup guide in the Bolna docs walks through both options.

What is the difference between Baseten Model APIs and dedicated deployments?

	Model APIs	Dedicated deployment
Models	A supported set of popular open models	Any model, including fine-tunes
What you deploy	Nothing	A config.yaml pushed with the Truss CLI
Endpoint URL	https://inference.baseten.co/v1	https://model-<model_id>.api.baseten.co/environments/production/sync/v1
Model name in Bolna	The slug, such as zai-org/GLM-5.3	From the deployment's example request
Capacity	Shared, with optional Fast variants	GPUs reserved for you

Baseten's context and output limits for a Model API can be lower than the model's advertised maximum, so check them before writing a very long system prompt.

What can you build with Baseten and Bolna?

Agents on a fine-tuned model. Deploy your own checkpoint with Truss and use it as the agent's LLM without writing serving code. With the TensorRT-LLM engine, no Python code is needed.

Zero-downtime model upgrades. Because the dedicated URL points at the production environment, promote a new deployment in Baseten and your Bolna agents switch to it with no change.

Fast pilots on open models. Start on Model APIs in minutes, then move to a Fast variant or a dedicated deployment as call volume grows.

Teams that want managed GPUs. If you would rather write the serving code yourself, see Modal. For on-demand GPUs with a serverless tier, see Fireworks AI.