Why a chatbot isn't cost-effective even with a good LLM

A chatbot can be expensive not because the LLM performs poorly, but because it was assigned the wrong task. If a company lacks clear metrics, up-to-date data, and a well-defined support process, artificial intelligence will simply scale up the existing chaos—along with the cost of tokens.

First the problem, then the bot

The phrase “everyone already has an AI assistant” is not a business objective. Before development begins, you need to establish key metrics: how many inquiries come in, how long it takes to process them, where delays occur, and which outcomes need to improve.

The article provides an illustrative example: an online store spent over a million rubles on a chatbot that was used about 100 times a month, but only 2% of requests were successfully resolved. On a per-resolution basis, hiring an additional customer service representative would have been cheaper.

Therefore, it makes more sense to start with two or three common and routine scenarios—for example, order status or answering frequently asked questions. If the bot doesn’t impact processing costs, response times, sales, or the team’s workload, it’s not automation—it’s just an expensive showcase.

A knowledge base won’t update itself

An LLM doesn’t know which prices, rules, and inventory levels are current today. If you simply upload an old PDF to it, it may confidently offer a discount that doesn’t exist. If it doesn’t have access to the CRM, inventory system, or delivery tracking, it will substitute a polite stock response for the actual answer.

This means that the quality of the assistant isn’t determined by the model alone. You need regular knowledge base updates, verified sources, and integrations with the company’s systems. For critical responses, it’s safer to use approved templates rather than relying on free-form generation.

This also explains why launch isn’t the end of the story. You need to analyze dialogs, troubleshoot errors, update the context, and gradually expand the scenarios. A bot that’s launched and then forgotten will eventually start responding to yesterday’s reality.

Having a Human in the Loop Isn’t a Sign of Failure

Users should always have a quick way to reach a human agent. If the bot is unsure of an answer or gets stuck in a loop, the conversation should be handed off to a human along with the chat history, rather than forcing the customer to repeat everything from the beginning.

How employees perceive the project is also important. When AI is implemented under the slogan “let’s cut back on agents,” the team starts double-checking responses and duplicating work. Formally, tickets are closed, but the workload doesn’t decrease. The bot should take over routine tasks, not turn employees into monitors of every phrase it utters.

What You Must Calculate Before Launch

The main constraints here are quite practical: the total cost of ownership includes not only development but also LLM queries, integrations, support, and regular updates. The economics may look good within free trial limits, but they can deteriorate sharply after scaling.

Security is also a key consideration. Transferring phone numbers, order data, payment information, or documents to an external cloud-based LLM requires a separate risk assessment. Where possible, data should be anonymized, and access to sensitive information should be restricted.

The practical takeaway is simple: first, identify a process where there’s a measurable pain point; then, review the data, integrations, the cost per successfully resolved interaction, and the escalation scenario. Only after that should you select a model and write prompts.

If you had to launch a chatbot tomorrow, which single process would you assign to it—and what metric would you use in a month to determine whether the experiment was a success? Source: All Articles / Artificial Intelligence / Habr