Where LLMs "Fall Short" in Reasoning: Layer Energy Predicts Errors

When an LLM reasons step-by-step, not all steps carry the same computational load. Until now, interpretability has either focused on the final conclusion or compressed the entire trajectory into a single number—losing sight of exactly where the model “thinks more intensively.” The authors of SARE (Step-Aware Reasoning Energy) propose a geometric framework that measures the effort at the level of each individual step in the chain-of-thought and links it to the semantic progress of the reasoning.

Technically, the method is based on Centered Kernel Alignment (CKA) between the Gram matrices of the hidden states of tokens in adjacent layers of the transformer. This allows us to capture inter-token relational structures without aligning eigenvectors or matching clusters—that is, without heavy preprocessing assumptions. On top of this, SARE models the CoT trajectory as transitions between hidden semantic states, contextualizing the energy within the reasoning logic.

Evaluation on six reasoning benchmarks and three open-weight LLMs yields three observations worth keeping in mind:

• Reasoning energy is extremely uneven across step types, with phase transitions invisible to trajectory-level metrics.
• Incorrect trajectories systematically exhibit lower energy at critical forks—the model “stalls” precisely where it needs to think the most.
• SARE features are comparable to or outperform output-based confidence baselines in most settings.

A practical takeaway for those building agents and pipelines on CoT: the internal geometry of the layers contains a predictive signal that is not present on the surface of the text. If you detect errors based solely on output confidence or self-consistency, you’re missing an early warning sign. SARE is still a research framework rather than a ready-to-use library, but the direction is clear: the next step is to integrate step-aware energy features into runtime reasoning monitoring to catch faulty branches before they trigger an action.

Source: cs.AI updates on arXiv.org