Why Routing Weights Do Not Explain the Outcomes of the MoE Model
Sh0ny
10 августа 20262 min read
In short
In Mixture-of-Experts, it is important to know not only which expert received a query, but also why that expert preferred one answer over another. The CoCo analysis shows how to interpret such decisions at the level of answer pairs, rather than based on indirect signals from the router.
Knowing which expert received a query does not necessarily mean understanding how that expert evaluated it. For sparse Mixture-of-Experts reward models, this is a fundamental problem: the routing weights show which examples an expert accepts, but they do not explain why the expert considers one answer better than another.
The authors propose examining an expert’s behavior through pairs of selected and rejected answers. The Contribution-Contrast (CoCo) method identifies pairs where the difference in an expert’s contribution is particularly large. This interpretation links two things at once: which queries the expert is assigned to and which response features influence their preference.
This is more important than simply collecting examples with the highest routing weight. Such a list may describe the expert’s area of expertise, but it’s easy to overlook their actual evaluation criteria. For example, an expert might receive queries on a single topic but evaluate responses based on an entirely different characteristic—such as completeness, style, or adherence to instructions. The abstract does not disclose exactly which patterns the authors identified, but the problem formulation itself appears convincing.
According to the authors, CoCo produced more coherent, accurate, and specialized interpretations than approaches based on a router, final ratings, and sparse autoencoders. At the same time, the accuracy of the reward model remained competitive. In other words, interpretability is not sacrificed here in exchange for prediction quality.
There are limitations, however. This is a paper on arXiv, and the original description lacks numerical results, experimental details, or examples of the interpretations found. Therefore, it is not yet possible to assess how significant the practical benefit from CoCo is or how well the method generalizes to other MoE reward models. The most useful conclusion is a more cautious one: a single router is not sufficient to understand such models; data on how an expert compares answers is needed.
If you’re evaluating a reward model, is it more important to know which queries the expert receives, or which specific differences between answers influence their evaluation? Source: cs.AI updates on arXiv.org