Interpretable AI has been taught to distrust bad concepts

The main problem with explainable AI here is not that the model cannot display its features. It can show concepts honestly, but if one of them is missing or defined incorrectly, the error starts spreading further — together with a convincing but wrong explanation.

ReCBM proposes fixing this not by giving up interpretability but right inside the concept bottleneck. Semantic relations are defined between concepts: co-occurrence, entailment of one from another, and mutual exclusion. That lets the model refine one feature through others.

The key mechanism is uncertainty. The less confidently the system recognised a concept, the smaller its contribution to the exchange of information with neighbouring concepts. That matters more than simply adding more links: without an estimate of reliability, the structure of relations could merely spread the error faster.

By the authors' description, across different datasets ReCBM recovered concepts and downstream tasks better when features were missing or deliberately inverted. The system also supported uncertainty-aware intervention and identified compact sets of concepts important for the specific task, with no loss of final performance.

But there is reason not to turn the result into a universal solution. The abstract contains no numerical comparisons, no dataset names, no details of how the relations are built and no cost for such a scheme. So for now one can speak confidently of a promising approach to unreliable concepts, but not of a proven advantage for any applied AI.

If you were vetting such a system in a product, which would matter more: an intelligible explanation of every decision, or the ability to correct individual concepts safely while it runs? Source: cs.AI updates on arXiv.org