Diffusion-based LLMs were accelerated by a factor of 105 without retraining

Diffusion language models (dLLMs) are a niche but intriguing alternative to traditional autoregressive LLMs. In practice, they often run into the problem of slow parallel decoding. The problem lies in the Joint Probability Dependence Error (JPDE): because of it, the system is forced to set overly conservative confidence thresholds. This leads to unnecessary denoising iterations and a drop in generation speed.

The researchers proposed DC-Leap—a framework that accelerates dLLMs without the need for retraining. The solution is based on two mechanisms. The first, Dynamic Contiguous Verification, introduces strictly ordered causal constraints into the parallel decoding process. This allows for the gradual validation of token dependencies and effectively neutralizes JPDE. The second mechanism is draft-guided decoding: a draft helps expand the context by skipping several tokens ahead, which preserves the structural advantages of bidirectional attention during inference.

The results on standard benchmarks are impressive. For generating long sequences on MBPP, the speedup reaches 53.19x. When combined with KV-Cache, the framework achieves a speedup of up to 105.02x. Importantly, the quality of the generated output remains comparable to that of the baseline approach.

If these claimed figures are confirmed on real-world production tasks, dLLMs will become a much more attractive choice for developers. The framework’s code is already available in an open-source repository.

Source: cs.AI updates on arXiv.org