Across five cognitive SEEG datasets, time-ordered hold-out AUCs were not significantly lower than random-CV AUCs

We tested whether author-reported headline AUCs for within-patient cognitive SEEG decoders were lower when publications reported a time-ordered hold-out than when they reported random cross-validation (CV), comparing results within the same dataset. We used 213 AUCs from sci 6f5fe01a and publication titles and summaries in a 2026-10-13 sci library snapshot. Fixed regular expressions labeled split type; eligible results were within-patient cognitive or sensory/motor targets. Among eligible publications, 19 were labeled TIME, 74 RANDOM, 16 AMBIGUOUS, and 17 UNSPECIFIED. Five cognitive datasets had both TIME and RANDOM labels. Their median AUC differences (TIME minus RANDOM) were -0.010, -0.037, +0.003, -0.005, and +0.006; the mean difference was -0.009, the median was -0.005, and 3 of 5 were negative (exact sign-flip p = 0.375). Across 85 publications in those datasets, the mean difference was -0.015 (within-dataset permutation p = 0.5186). Overall medians were 0.600 for TIME and 0.6205 for RANDOM. The reported split type did not explain a large part of the AUC values; a dataset-level difference larger than about 0.04 AUC was not seen, but small differences cannot be excluded. Labels came from author text, not code; publications on a dataset are not independent; targets, windows, and features differed; and only cognitive datasets had both split types, so this result does not cover motor or sensory decoders. The first run lacked summaries for 104 records; after adding them with sci show and rerunning, it gave mean d = -0.003 and p = 0.56.

SEEG · AUC · brain-computer interface · cross-validation · temporal validation · neural decoding · meta-analysis