SHAP explains a prediction but does not show the effect of an intervention

The main risk with AI explanations is that they answer the wrong question. SHAP can show why a model predicted a certain outcome for a person, but on its own it does not explain what will happen if you change a condition in the real world.

That gap is exactly what the work on the Causal Attribution Score, or CAS, addresses. The method starts not from the usual contribution of features to a prediction but from an explicitly stated intervention: what precisely we change and how we compare the outcomes. It then distributes the total causal effect across combinations of factors using causal Shapley contributions.

An important detail: CAS does not invent a new Shapley formula. Its novelty is the reporting layer that turns causal contributions into local and global measures. That is, you can see the effect for a specific case, its sign, and the overall picture across the sample separately.

In simulations with a known true causal structure, the mean error of Local CAS was 0.107 against 0.173 for single-feature normalisation and 0.213 for a globally normalised vector of absolute ATEs. Under strong interactions the advantage over the one-at-a-time approach grew to 0.091, while under an additive structure it all but vanished.

The practical meaning here matters more than the method's name. In two DoubleML datasets the feature rankings for prediction diverged noticeably from the rankings of intervention effect modifiers. In the return-to-work bonus data, for example, the feature dep1 — having exactly one dependant — rose from 13th place in the ordinary global predictive ranking to 2nd in Feature-CAS and became the leading local modifier.

The limitations are substantial too. The results come from eight simulations with known ground truth and from two empirical datasets, and the causal estimates depend on the correctness of the causal setup itself and of the DoubleML estimation. CAS does not turn a predictive model into a causal one: first you genuinely have to define the intervention and have data that let its effect be estimated.

So CAS is worth treating not as a replacement for SHAP but as a check on whether we have confused "what predicts the outcome" with "for whom changing a condition will make a difference". If you use model explanations to make decisions, which of those two questions actually underpins your system right now? Source: cs.AI updates on arXiv.org