What Are Tokens?
Think of tokens as pieces of a puzzle. They are the smallest units of text (like parts of words) that the AI uses to understand your questions and generate a response. Managing your Token Budget allows you to control how your bot “thinks” and where it focuses its effort.
How to Adjust Your Token Budget
Your bot's Token Budget is the maximum amount of tokens it can use for a single request. This single slider replaces the previous system of balancing Context, History, and Response. A larger budget allows the bot to handle longer inputs and provide more detailed replies.
From your bot’s settings page, locate the Model section.
Below the model selection, you will find the Token Budget slider.
Click and drag the slider to choose the maximum amount of tokens (from 8K to 200K) the bot can spend for one request.
Click Save.
Billing and Credit Usage
It's important to understand how this setting affects your credit usage.
Bigger budgets allow for longer inputs and replies.
Billing is for actual usage, but is rounded up to the next 8K tokens.
The estimated credit usage for your chosen budget is displayed directly above the slider.
Common Scenarios (Best Practices)
Not sure where to set the slider? Here are a few common examples.
For most standard Q&A bots: The default setting is a balanced starting point for bots that answer questions based on a knowledge base.
For complex tasks or long documents: If your bot needs to process very long user inputs or search through large, dense documents to find an answer, increase the token budget.
For creative or conversational bots: If you expect long, detailed conversations where the bot needs to generate lengthy and creative replies, increase the token budget.
To conserve credits: If your bot is designed for very short, simple interactions, you can decrease the token budget to minimize credit usage.