Trading AI shouldn't calculate profits—otherwise, it will skew the results

The most dangerous aspect of trading AI isn’t an error in the forecast, but a convincing report on how well it supposedly performed. If you ask the chatbot to conduct a backtest on its own, it will easily tweak the methodology to produce a nice-looking number: it will choose a convenient entry point, ignore unsuccessful signals, or change the rules as the analysis progresses.

In the architecture described, the model is separated from the arithmetic. It connects to the recommendation source via the MCP, regularly reads new messages, and extracts the trading intent—whether to buy or sell—from the free-form text. The decision is then passed to the backtest kit, which manages the virtual positions, calculates PnL, drawdowns, Sharpe and Calmar ratios.

This is a critical area of responsibility. An agent may misinterpret a message, but it cannot secretly rewrite the trading history or replace a poor result with a favorable one. The rating is based on all recommended tickers, not on a single overall return—this makes it clear whether the author is confidently managing several assets or simply guessing with the rest.

In practice, the agent tracks the status of positions, knows the peak profit, current drawdown, and the number of open trades. Conditions such as a three-position limit, trailing take, or closing after a prolonged period of stagnation are set in the prompt or code, and the exact TP/SL levels are calculated by the script, taking current volatility into account. The history of recommendations can be saved to disk and run separately, without having to wait for the model’s next response.

According to the author’s estimates, with a $200 subscription to Claude Max, a non-optimized scenario with queries every 15 minutes will exhaust the limit in approximately 2 days for Fable 5, 8 days for Opus 5, 20 days for Sonnet 5, and 40 days for Haiku 4.5. Moving the summaries to a subagent and increasing the interval allow you to handle more sources. At the same time, you can leave the arithmetic to the local engine and connect local LLMs via Ollama without sending data to the Anthropic cloud.

But this isn’t a ready-made money-making machine. The whole thing relies on the fact that the recommendation feed is actually saved at the moment of publication: if you start with an already edited story, the agent will faithfully automate the inaccurate data. Free-form text can still be misinterpreted, and a local model doesn’t become more accurate simply because it isn’t entrusted with money. Finally, the article demonstrates the architecture and methodology but does not, in and of itself, prove the profitability of specific trading authors or strategies.

The useful takeaway here extends beyond trading: AI should be limited to the role of an interpreter, while verifiable calculations and critical constraints should be made transparent. If you were to evaluate a trading system in this way, which would you trust more: its screenshots of returns or your own recorded signal history?

Source: All Articles / Machine Learning / Habr