Why There Is Sometimes No “One Right Answer” for LLMs

The main problem with LLMs may not be that the model reasons poorly, but rather that it is expected to provide a single answer in situations where the structure of the task does not guarantee one. The paper on arXiv formalizes this conflict and links it to what the authors call “unsupported canonization”—the reduction of a set of valid conclusions to a single one without sufficient justification.

The authors distinguish three levels of “unambiguity.” First, conclusions may stabilize for a specific initial dataset. Next, they may coincide regardless of the starting point. And only at the third level does a single valid interpretation emerge.

This is an important distinction for LLMs. A stable response does not necessarily mean it is the correct or only possible response. If the input data is incomplete, epistemic multiplicity arises: there may be a single truth, but the system lacks sufficient information. In another case, the multiplicity is structural: several interpretations are simultaneously consistent with the rules.

Hence the practical conclusion: the “always answer confidently and unambiguously” mode may not be a sign of good reasoning, but rather a source of hallucinations. For some problems, an operator is needed that gradually closes the inference system. For others, an explicit selector is required that chooses a variant based on an additional criterion. But such a choice cannot be presented as a consequence of the initial rules themselves.

The work has significant limitations. For epistemically plural theories, the authors demonstrate stabilization of closure, but complete determinization depends on a global convergence property that remains open. For a special class of structurally plural theories, where there are no general upper bounds, determinization is achieved through canonical selection. This is a formal framework, not an experimental comparison of LLMs, so it does not yet specify exactly how to measure hallucinations in a particular model.

When verifying an LLM’s response, do you first look for factual errors, or do you assume that the task itself may have multiple correct solutions? Source: cs.AI updates on arXiv.org