Last updated
What is Fireworks AI used for in a Bolna voice agent?
Fireworks AI serves open models to a Bolna voice agent through an OpenAI-compatible API, either on shared serverless capacity paid per token or on dedicated on-demand GPUs. Start on serverless to try a model quickly, then move to an on-demand deployment for no rate limits or a model that isn't on serverless. The one Fireworks-specific detail is the model ID, which is a full resource path.
How do I set up Fireworks AI with Bolna?
- In the Fireworks dashboard, open API keys and click Create API key.
- Find a chat model in the Fireworks model library and copy its full ID, such as
accounts/fireworks/models/llama-v3p1-8b-instruct. Versions usepfor the decimal point, so Llama 3.1 isllama-v3p1. - In Bolna, go to Settings → Integrations → LLM → Custom LLM → Add model, or choose + Add custom LLM in an agent's LLM Provider dropdown. Select Fireworks AI.
- Enter the full model ID, set Endpoint URL to
https://api.fireworks.ai/inference/v1, paste your key and set Family tollama, for example. - Choose custom on the agent's Intelligence tab, pick the model and save.
The Fireworks AI setup guide in the Bolna docs shows how to create an on-demand deployment with firectl.
What is the difference between Fireworks serverless and on-demand in Bolna?
| Serverless | On-demand deployment | |
|---|---|---|
| Capacity | Shared, per-token | Dedicated GPUs for one model |
| Rate limits | Yes | None |
| How you create it | Nothing to create | firectl create deployment |
| Model name in Bolna | accounts/fireworks/models/<model> | accounts/fireworks/models/<model>#accounts/<account-id>/deployments/<deployment-id> |
| Endpoint URL | https://api.fireworks.ai/inference/v1 | Same |
An on-demand deployment created with --min-replica-count 0 can scale to zero, and the next call waits for a GPU to start. Use --min-replica-count 1 for live voice traffic.
What can you build with Fireworks AI and Bolna?
Quick model trials. Try several open models on serverless with no deployment, then keep the one that sounds best on calls. Together AI is another serverless catalog to compare.
Campaigns that outgrow serverless. When a batch starts hitting rate limits, move the agent to an on-demand deployment, which has none.
Agents on models not offered on serverless. On-demand deployments give a wider choice of models. For your own fine-tuned weights, also see Baseten.
Open-model agents with their own billing. Tokens are billed by Fireworks, not Bolna. The custom LLM page covers how custom models work.