The Hidden Complexity of AI Costs
When businesses evaluate AI chatbot solutions, pricing is inevitably one of the first questions. But understanding the true cost of running an AI chatbot requires looking beyond the monthly subscription fee. The underlying large language model - the engine that powers every conversation - has its own pricing structure that directly impacts your bottom line.
This guide breaks down how LLM pricing works, what drives costs up or down, and how to make informed decisions that balance quality with budget.
How LLM Pricing Works
Large language models charge based on tokens - small units of text that the model processes. A token is roughly three-quarters of a word in English, though this varies by language and content type. Code, technical text, and non-English languages tend to use more tokens per word.
Every API call to an LLM involves two types of tokens:
- Input tokens - everything you send to the model: the system prompt, conversation history, knowledge base context, and the user's message
- Output tokens - the model's response text
Providers price these separately, and input tokens are typically cheaper than output tokens. For example, a model might charge $0.50 per million input tokens but $1.50 per million output tokens. This asymmetry exists because generating output requires more computational work than processing input.
The Major Providers and Their Tiers
The LLM market has settled into a clear tiering structure, and understanding these tiers helps you match your needs to your budget.
Budget Tier: Fast and Affordable
Models like Google's Gemini Flash and OpenAI's GPT-4o Mini are designed for high-volume, cost-sensitive applications. They offer impressive capability for their price point - often 10-20x cheaper than premium models. For most customer service chatbot use cases, these models are more than sufficient.
Typical pricing ranges from $0.10 to $0.50 per million input tokens and $0.30 to $1.50 per million output tokens. A typical chatbot conversation might cost between $0.001 and $0.005 - a fraction of a cent.
Mid Tier: The Sweet Spot
Models like GPT-4o and Gemini Pro sit in the middle ground. They offer noticeably better reasoning, more nuanced responses, and better handling of complex queries. The cost is typically 3-5x higher than budget models.
For businesses where response quality directly impacts revenue - such as high-value sales conversations or technical support - the premium is often justified.
Premium Tier: Maximum Capability
The latest frontier models offer the best available performance but at significantly higher cost. These are typically reserved for applications where accuracy is critical and the cost per conversation is justified by the value at stake.
What Drives Your Actual Costs
Understanding token pricing is only half the picture. Several factors determine how many tokens each conversation actually consumes.
System Prompts
Every chatbot conversation starts with a system prompt - instructions that tell the model how to behave, what tone to use, and what information it has access to. A well-crafted system prompt might be 200-500 tokens. This is sent with every single message, so it represents a fixed cost per interaction.
Keep your system prompts concise. Every unnecessary word multiplies across thousands of conversations.
Conversation History
To maintain context in a multi-turn conversation, the chatbot sends previous messages along with each new request. A conversation that has gone back and forth ten times will include all twenty messages (ten from the user, ten from the assistant) in the next API call.
Most platforms limit conversation history to a fixed number of recent messages - typically the last 5-10 exchanges. This caps the cost growth while maintaining enough context for coherent conversation.
Knowledge Base Context
When a chatbot searches its knowledge base to answer a question, the retrieved content is included in the API call as additional context. If the search returns five chunks of website content at 300 words each, that adds roughly 2,000 tokens to the input.
This is one of the most significant cost drivers for knowledge-enabled chatbots. Tuning how much context is retrieved - and how relevant it is - directly impacts both cost and response quality.
Tool Calling Overhead
When a chatbot uses integrations (checking product availability, looking up orders, etc.), each tool call adds tokens. The tool definitions themselves consume tokens, the model's tool call request consumes tokens, and the tool results that get fed back to the model consume tokens.
A conversation that involves two tool calls might use 3-4x more tokens than a simple question-and-answer exchange. This is the price of capability - but for many businesses, the value of real-time data access far outweighs the additional cost.
Strategies for Controlling Costs
Smart cost management does not mean choosing the cheapest model. It means optimising how you use whichever model you choose.
Start with Budget Models
Deploy with a fast, affordable model first. Monitor the quality of responses and only upgrade to a more expensive model if you identify specific areas where the cheaper model falls short. Many businesses are surprised by how capable budget models have become.
Optimise Your System Prompt
Review your system prompt regularly. Remove redundant instructions, consolidate similar rules, and test whether shorter prompts produce equivalent results. A system prompt that is 200 tokens instead of 500 tokens saves 300 tokens on every single message.
Tune Knowledge Retrieval
Experiment with how many knowledge chunks are returned per query and how long each chunk is. Returning three highly relevant chunks is better (and cheaper) than returning ten loosely related ones.
Monitor and Set Limits
Use spending limits to prevent unexpected cost spikes. Set daily and monthly caps that align with your budget, and monitor usage patterns to identify opportunities for optimisation.
The Credit System Approach
Many chatbot platforms, including Morv, abstract LLM costs into a credit system. This simplifies budgeting by providing a predictable monthly allocation of credits with your subscription, plus the option to purchase additional credits as needed.
The advantage of a credit system is predictability. Instead of tracking token prices across different models, you know what your subscription costs each month and can see your usage in credits. The platform handles the complexity of model pricing behind the scenes - on Morv, we also choose and run the model for you, so there is nothing to configure.
Making the Right Choice
The best LLM for your chatbot depends on your specific use case, conversation volume, and quality requirements. There is no universally correct answer. But armed with an understanding of how pricing works and what drives costs, you can make an informed decision that maximises value for your business.
Start affordable, measure results, and scale up only where the data justifies it. That is the most reliable path to a cost-effective AI chatbot deployment.