Why a successful AI agent can behave inconsistently

A high success rate does not yet mean an AI agent is dependable. It may solve tasks successfully while taking a completely different logical route to the result each time — and break where the conditions have shifted a little.

The work's authors propose looking not only at the outcome but at the process of execution. Their Behavioral Consistency Metric (BCM) analyses behavioural features in an agent's traces and assesses how similar the behavioural profiles explaining success are across different runs.

An unexpected gap matters here. An agent can be reproducible within a single task: repeated attempts look alike. But when it moves to other tasks no common strategy is found — the behaviour becomes fragmented. The reverse also happens: a system keeps a stable style across tasks while behaving less consistently within an individual one.

That is exactly why a repeatability test alone is not enough. The study analysed around 9,000 traces from six language agents on software development tasks. Systems with similar levels of success differed noticeably in consistency, and the gap between frontier models and open-source systems persisted even when task difficulty was controlled for.

There is an important caveat too. BCM does not measure "reliability" directly: the metric trains a separate model to predict success from behavioural features and then compares the resulting attribution vectors. So the result depends on how meaningfully that predictor links behaviour to success. The authors themselves stress that BCM makes sense only under certain conditions, and the abstract does not list those conditions in detail.

The practical conclusion is simple: when choosing an agent it is worth asking not only "how often does it succeed?" but "how stably does it act in new scenarios?". Would you prefer an agent with a slightly lower success rate but a stable strategy, or a more successful system whose behaviour is unpredictable every time? Source: cs.AI updates on arXiv.org