Across 100 sci SEEG publications, most null paired AUC comparisons did not exclude gains of 0.02 AUC

We mined 109 paired AUC differences with a 95% CI from the titles and summaries of 100 sci publications on OpenNeuro SEEG datasets (snapshot 2026-10-13, 1065 publications). Most compare two decoders or two pipelines on the same patients. The median comparison had 15 patients (IQR 10-17). From each CI and n, we calculated the implied between-patient SD of the paired differences and the minimum detectable difference at 80% power (MDE80, one-sided alpha 0.05). The median MDE80 was 0.026 AUC (IQR 0.014-0.049; bootstrap 95% CI of the median 0.020-0.031). Only 18% of comparisons could detect 0.01 AUC, and 43% could detect 0.02 AUC. 72 comparisons were not significant. Only 23 of 72 (32%, 95% CI 21-43%) had a CI inside +/-0.02 AUC, and 6 of 72 had a CI inside +/-0.01 AUC. In 39 of 72, a gain of 0.02 AUC or more is not excluded. At the implied SD of each report, the median comparison needs 86 patients to detect a 0.01 AUC gain and 22 patients for 0.02 AUC. The SD depends on the dataset: the memory-load dataset ds004752 had the largest SD (MDE80 0.049; 2 of 20 nulls conclusive). A manual check of 29 extracted rows found no extraction error. Thus, most 'did not improve' claims in sci SEEG decoder work do not exclude BCI-relevant gains of 0.01-0.02 AUC. Null claims should state an equivalence margin, or pool patients across datasets. Assumptions: t-based CIs (median and bootstrap CIs give an approximate SD); n is the first patient count in the title; rows are treated as independent; full bodies were not mined. The package has the snapshot, code, per-row tables and figure.

SEEG · AUC · statistical power · equivalence testing · brain-computer interfaces · decoder comparisons