AI agents were asked to fix problems rather than rewrite themselves from scratch

The main drawback of automatic refinement of AI agents may not lie in the quality of the code, but rather in poor memory. If the system re-examines the history of candidates in every round, it risks repeating unsuccessful fixes and losing the progress it has already made.

In ADIAS, this logic has been reversed. Instead of treating the next version of the agent as the primary focus, the system maintains a list of persistent problems: what exactly is broken, what evidence exists, what stage the fix is at, and how previous attempts ended.

This is a significant shift: the agent is optimized not by the principle of “generate yet another improved version,” but by the principle of “select the next problem and modify the code based on the results already obtained.” This approach should reduce pointless repetition and help assemble incremental improvements into a cohesive system.

Across five interactive benchmarks, ADIAS outperformed the strongest baseline method by an average of 25.2%. This result held true across four baseline models. In control experiments, removing steady-state problems or reverting to a candidate-oriented strategy reduced performance—by up to 40.7%.

However, this is not yet conclusive proof of practical universality. The abstract does not disclose the specific tasks, the cost of iterations, the details of the comparison with the baseline method, or how well the method performs beyond these five benchmarks. Therefore, the main conclusion is more cautious than the promotional claims suggest: ADIAS shows that for complex agents, it is more useful to build up a map of defects than to simply churn out new versions of the code.

If you were to automate the improvement of your own agent, which would be more valuable: dozens of new code variants or a detailed history of the problems you’ve already tried to fix? Source: cs.AI updates on arXiv.org