Out-of-fold Platt scaling reduced calibration error in a second SEEG movement dataset (6 of 7 patients; within-session, balanced rest/move windows)

This pre-registered replication tested whether out-of-fold Platt scaling reduces over-confidence in shrinkage linear discriminant analysis (LDA) movement probabilities in a second stereoelectroencephalography (SEEG) dataset. In OpenNeuro ds006253 (7 patients; time-ordered split within one session, first 60% of trials for training and last 40% for testing), Platt scaling reduced mean test expected calibration error (ECE) from 0.140 to 0.094 in 6 of 7 patients (difference 0.047; exact one-sided sign-flip p = 0.023). Log loss decreased in 6 of 7 (p = 0.016, Holm-adjusted p = 0.016), and raw confidence minus accuracy was +0.115 in all 7. Across this dataset and the 8-subject ds004473 check, ECE decreased in 12 of 15 subjects (mean difference 0.056; p = 0.0016, Holm-adjusted p = 0.005); the ds004473 high-gamma result matched the reported prior result (0.155 to 0.090, 6 of 8, p = 0.027). AUC was unchanged. Platt slopes were below 1 in all subjects (medians 0.54 for ds006253 and 0.39 for ds004473); temperature scaling gave the same ECE decrease in ds006253. Calibration remained incomplete: mean ECE after Platt (0.094) exceeded the finite-sample ECE floor (0.063). The results apply to balanced rest/movement windows and within-session splits; cross-day performance and calibration in a continuous, unbalanced stream were not tested. Samples were small (7 and 8 subjects).

probability calibration · Platt scaling · linear discriminant analysis · stereoelectroencephalography · movement decoding · expected calibration error · out-of-fold calibration · intracranial EEG