Verilog verification hit a snag not with the model, but with a narrow debug window

The problem with automatic Verilog generation may not be that the model writes poor code. It may simply not have enough context when trying to understand why the circuit isn’t working.

The authors of VeriTrace attribute the performance ceiling of modern multi-agent systems—about 95% on standard benchmarks—to the narrow scope of actions available during debugging. Agents are restricted in advance to a limited set of signals, a limited time window for analysis, or both parameters at once. As a result, verification turns into a search for a familiar pattern within a predefined slice of the system’s behavior.

VeriTrace offers a different approach. Its Inspector can independently choose:

• which signals to examine;
• the boundaries of the time window;
• how deeply to continue the iterations.

This is called Agentic Temporal Exploration. The agent formulates a hypothesis about the cause of the failure, requests confirmation via the waveform, refines the picture, and repeats the cycle. In other words, the key change lies not so much in the base model as in the actions it is permitted to take during debugging.

The result reported in the paper is remarkable: 100% Pass@1 on VerilogEval-V2—the first such result for this benchmark. Using the same Claude Sonnet 4.0 base, VeriTrace outperformed the strongest reproduced baseline by 5.1%.

But this does not mean that RTL generation as a whole has been fully solved. The metric applies to a specific benchmark, while the paper itself demonstrates a narrower and practically significant thesis: sometimes the final percentage point of quality is achieved not by introducing yet another, more powerful model, but by removing artificial constraints from its tools.

This is a good guideline for agent-based systems. If an agent consistently makes mistakes in the final steps, it’s worth checking not only the prompt and the model, but also the action space itself: what observations are available, what actions are permitted, and whether the agent can revisit its hypothesis after receiving new data.

Source: cs.AI updates on arXiv.org