Training-only Platt scaling reduced later-trial LDA calibration error in an 8-subject SEEG key-press study
In an offline analysis of 8 subjects performing cued key presses, a shrinkage LDA trained on trials 1–100 produced overconfident probabilities on later trials. A Platt map fit without penalty to five-fold out-of-fold scores from the training trials decreased expected calibration error (ECE) in 7 of 8 subjects (mean decrease 0.0896; one-sided exact sign-flip p = 0.0078). Raw mean confidence exceeded accuracy by 0.17 in all 8 subjects; after calibration, the mean difference was 0.001. Negative log loss decreased in all subjects (mean decrease 0.4099 nats per window), as did Brier score (mean decrease 0.0365). Accuracy did not change. These results concern balanced rest and move windows from one session per subject and an offline analysis; calibration may differ with movement base rate or across sessions.
SEEG · brain-computer interface · probability calibration · shrinkage LDA · Platt scaling · expected calibration error · key-press decoding · selective prediction
References
3- usesBeta power increased later-trial SEEG key-press AUC in 6 of 8 subjects sci:sha256:b76113878b22b13b8fce6b4b57cf321bb22ce16b618b55ae02c2ef42b972f816
We used the archived ds004473 band-power features (hashes verified) and the shrinkage LDA code (lda_np.py, copied without change) and the same trials 1-100 / 101+ split.
- supportsIn 18 patients, a Platt map improved calibration of SEEG word-recall probabilities, but Brier-skill gains were small sci:sha256:99a311d7d13dd572db26fdd381103a93ee08d7d3216fbc47746dbd9fb5889a14
That publication found miscalibrated SEEG memory-decoder probabilities and an ECE decrease by a Platt map fit on training folds. We found the same effect in a different task (motor key press), decoder (shrinkage LDA) and dataset (ds004473).
- usesdoi:10.18112/openneuro.ds004473.v1.0.2
The publication identifies OpenNeuro ds004473 v1.0.2 as the source dataset for the archived SEEG key-press features.
Cited by
2 publicationsPublic catalog entries on Sciscape.
- Raw LDA probabilities were overconfident in seven SEEG patients, while Platt calibration did not significantly lower ECE
[Legacy relation: reproduces] Repeats the same calibration test (raw LDA vs training-data Platt map, ECE primary endpoint, sign-flip test) on a different SEEG dataset and task (ds006253 mouse movement).
- supportsOut-of-fold Platt scaling reduced calibration error in a second SEEG movement dataset (6 of 7 patients; within-session, balanced rest/move windows)
Same over-confidence of raw LDA probabilities and the same Platt correction occur in ds006253.