DeLong AUC intervals covered the all-trial AUC in 90% of draws at 20 test trials in seven SEEG subjects, and 95.8% at 50

We evaluated DeLong and Hanley–McNeil analytic 95% confidence intervals (CIs) for the AUC of a shrinkage LDA detector using archived high-gamma and beta features from eight SEEG patients performing a cued key-press task. The decoder was fixed after training on trials 1–100. For each eligible subject and test size (10, 15, 20, 30, or 50 move trials and the same number of rest windows), we drew 2,000 subsets without replacement and measured CI coverage of the AUC computed on all available test trials (32–198 per subject). At 20 test trials, among seven subjects meeting the planned minimum of 40 test trials, mean DeLong coverage was 0.900 (six of seven below 0.95; exact one-sided sign-flip p = 0.016); mean Hanley–McNeil coverage was 0.920 (seven of seven below; p = 0.008). At 50 test trials, mean coverage was 0.958 for DeLong and 0.969 for Hanley–McNeil. Under-coverage was greatest in subjects with high AUC and near nominal in subjects with AUC 0.60–0.76. With shuffled labels, the false-certification rate (CI lower bound above 0.5; nominal 0.025) ranged from 0.026 to 0.046. Median DeLong half-width was 0.114 at 20 trials and 0.072 at 50; a median half-width of 0.05 or less was reached by 50 trials only for sub-1 and by 10 trials for sub-2. The findings support using at least 50 test trials per class or an interval with better small-sample behaviour, which was not tested. This offline study used one session per patient, cued windows, eight epilepsy patients, one decoder, and only two analytic CI methods. The all-test-trial reference AUC is itself estimated from 32–198 trials; bootstrap and logit intervals were not evaluated.

AUC · confidence intervals · DeLong · Hanley–McNeil · SEEG · shrinkage LDA · coverage · small samples · neural decoding