2 min read
In short
When training a multi-step agent, the correct trajectory does not always serve as a valid cue: after an error, the state of the environment has already changed. The new approach filters out such cues and significantly improves performance on two tasks.
The main risk of training an AI agent using successful examples is that an example may no longer reflect reality as early as the next step. If the agent goes the wrong way or performs subtasks in a different order, a hint from the reference path may not help but instead cause confusion.
The SMRC-SD paper addresses this very problem. The approach checks each time whether the agent’s current state matches a state for which there is a suitable instruction in the successful trajectory. If there is no match, training on that segment is skipped.
But filtering alone is not enough. For matching states, the method additionally extracts context from the successful trajectory so that it describes specifically the state that has been reached, rather than the entire ideal path. The result is not a generic “do one, two, three” scenario, but a hint tailored to the current situation.
On ALFWorld with the Qwen3-1.7B model, success rates increased from 0.746 to 0.865, and on WebShop—from 0.574 to 0.693. In both cases, SMRC-SD outperformed unconditional training along the full successful trajectory. This is an important implication: more training prompts do not necessarily mean better training. Sometimes quality improves precisely because some prompts were disabled at the right time.
The limitation here is also obvious: the results are shown on ALFWorld and WebShop, so they cannot be used to automatically draw conclusions about any other environments or agent types. Furthermore, the method requires comparing the agent’s state to the reference trajectory at every moment—this is additional logic, not a cost-free configuration.
The practical takeaway is simple: a successful scenario should be used not as a rigid set of instructions, but as a set of local hints applicable only in compatible states. Do your AI agents fail more often due to incorrect actions or because of hints that no longer match the situation?
Source: cs.AI updates on arXiv.org