Cancelling a requirement does not work: an LLM may keep following it

The nastiest error in a multi-turn dialogue is not when the model failed to grasp a new requirement but when it failed to let go of an old one. The user cancels a condition, the model confirms the cancellation, and then goes on satisfying it anyway.

The authors call this behavioral relapse — the inertia of cancellation. And it is an important shift in framing the problem: this is not about a model's random "forgetfulness" but about a property of the dialogue state that can be measured for each individual condition.

For that they propose a contract ledger. Each requirement is linked to an executable check, cancelled conditions are marked as tombstones, and before an answer is produced the system assembles the current set of constraints into a single specification.

The idea is practically useful: instead of hoping the model will work things out for itself from a long message history, it is handed an already cleaned contract state in advance. The check works through the model's API, with no access to its internal weights.

In experiments on HumanEval tasks with checkable conditions, the relapse rate for an 8B-class model rose along with the number of constraints. For stronger models the figure stayed at a minimal level. Compiling the state in advance markedly reduced the number of violations compared with a "try, check, repeat" approach at comparable model, checks and budget.

There is a more modest option too: a short note about the cancelled requirement returned roughly a third of the effect of compilation. And an adaptive ladder of extra corrections on top of compilation gave no detectable gain. That is a good antidote to the idea that one more round of self-checking automatically solves the problem.

The limitations are substantial as well. The method was tested on HumanEval tasks and formalised conditions, so transfer to ordinary conversations, where requirements are vague and have no executable checker, remains an open question. The approach requires extra checks and API compute for every result. Besides, the mere fact that the model wrote "the condition is cancelled" cannot be taken as proof: behaviour after that sentence may say otherwise.

If a model cannot reliably forget cancelled requirements, are we ready to trust it with long dialogues without a separate state ledger? Source: cs.AI updates on arXiv.org