Shrinkage-LDA probabilities were over-confident on new SEEG trials in 8 subjects; out-of-fold Platt scaling reduced mean ECE

We tested whether shrinkage-LDA probabilities from a cued SEEG key-press-versus-rest decoder were calibrated on new trials, and whether recalibration using training trials helped. We used archived high-gamma (70–150 Hz) and beta (13–30 Hz) log-power features for 8 subjects from OpenNeuro ds004473; trials 1–100 trained the decoder and trials 101 onward formed the test set. The plan was written and hashed before results were computed. Raw LDA probabilities were extreme (p < 0.01 or p > 0.99) for 51% of test windows, and 12.0% of those predictions were wrong. Mean test ECE decreased from 0.183 to 0.093 after Platt scaling fitted to 5-fold out-of-fold training scores (7/8 subjects; exact one-sided sign-flip p = 0.0078). Brier score decreased from 0.197 to 0.160 and log loss from 0.887 to 0.501 in all 8 subjects (p = 0.0039 for each). AUC was unchanged at 0.828; accuracy was 0.759 raw and 0.758 after scaling. Temperature scaling gave similar ECE (0.091). The training-set resubstitution ECE was 0.007–0.051. The remaining per-subject mean ECE after Platt scaling (0.093) was near the estimated finite-sample level of about 0.06; pooled ECE was 0.041. This offline analysis covers one dataset, 8 subjects, one linear decoder, and test classes balanced by design. Platt scaling does not correct class-balance drift or later signal changes. The source dataset release was not recorded.

SEEG · brain-computer interface · shrinkage LDA · probability calibration · Platt scaling · expected calibration error · out-of-fold validation