AI helped resolve an open question regarding the Benjamini-Hochberg method in statistics
Sh0ny
15 июля 20262 min read
In short
Using GPT-5.6, a researcher refuted the hypothesis that the Benjamini-Hochberg procedure controls the false rejection rate in correlated two-tailed Gaussian tests. The result may have conceptual significance for statistics.
Researcher Edgar Dobriban reported that artificial intelligence has been used to solve a long-standing open problem in the field of multiple hypothesis testing. The method in question is the Benjamini-Hochberg (BH) procedure, proposed in 1995 to control the false discovery rate (FDR). This method is widely used in genomics, astronomy, and economics, and the original paper has been cited more than 130,000 times.
What Is the Problem?
Benjamini and Hochberg proved that the FDR is controlled only for independent data. In practice, data are often dependent—for example, genetic variants due to linkage disequilibrium. Later, the method’s validity was extended to cases of positive dependence, but the question of whether FDR control holds for arbitrary correlated Gaussian data in two-sided tests remained open.
The Role of AI
Dobriban notes that using GPT-5.6 Sol Pro, the task was solved in a single pass. The previous version of the model, GPT-5.5, failed to complete the task even after 20 hours of running with multiple parallel agents. According to the researcher, the increase in capabilities was quite significant.
Result
The hypothesis was rejected in the negative sense: the BH procedure does not guarantee FDR control at a given level for correlated two-sided Gaussian tests. The author provided an example of a Gaussian factor model for which, at a nominal alpha level of 0.01, it has been proven that the FDR exceeds 0.0104.
The deviation from the nominal level is small, so the practical implications remain to be determined. The significance of the result is primarily conceptual. Emmanuel Candes of Stanford University previously called the FDR and the BH procedure “one of the two most important achievements in statistics since 1950.”
A preprint of the paper is available on the Wharton School’s website and will soon be posted on arXiv. The accompanying code has been published on GitHub.