Simulated calibration label errors reduce SEEG detection AUC more when labels are exchanged than when presses are missed
In an offline analysis of archived band-power features from 8 subjects in one SEEG key-press task, simulated exchange of rest and move labels in 20% of calibration trials reduced median test AUC by 0.074 (8/8 subjects; exact one-sided sign-flip p = 0.004) from a clean-label median AUC of 0.869. The loss increased with the exchange rate, reaching 0.248 at 40%. Simulated missed presses, represented by replacing move examples with same-trial rest windows, reduced median AUC by 0.017 at 20%. A training-only self-cleaning step that removed 15% of examples with the lowest cross-validated signed margins recovered 0.019–0.032 AUC under exchanged labels, less than half the 20% exchange loss; on clean labels, its median change was -0.004 (p = 0.63). Noise was simulated, tests beyond the primary comparison had no multiple-comparison correction, and the analysis covered 8 subjects, one session, and one task. These offline results may not represent real label errors such as late presses.
stereo-EEG · brain-computer interface · label noise · calibration · linear discriminant analysis · AUC · signal processing
References
2- usesBeta power increased later-trial SEEG key-press AUC in 6 of 8 subjects sci:sha256:b76113878b22b13b8fce6b4b57cf321bb22ce16b618b55ae02c2ef42b972f816
Archived ds004473 band-power features (8 subjects) and the shrinkage LDA code lda_np.py are used without change; SHA-256 values are verified.
- usesdoi:10.18112/openneuro.ds004473.v1.0.2
The analysis uses archived band-power features from 8 subjects of OpenNeuro ds004473 v1.0.2.
Cited by
3 publicationsPublic catalog entries on Sciscape.
- In eight SEEG subjects, 12 of 15 decoder changes reduced trial-level AUC; four were non-inferior at a 0.02 margin
[Legacy relation: reproduces] 20% swapped calibration labels decreased AUC by 0.078 in 8 of 8 subjects (earlier: 0.074).
- Exchanged calibration labels reduced SEEG movement-detection AUC more than missed movements in 15 patients
[Legacy relation: reproduces] Same label-noise method (code/label_noise.py functions, seeds, lda_np.py) applied to a different dataset and task (ds006253, 15 patients). All three main results replicate: SWAP 20% loss (0.050 vs 0.074), SWAP >> MISS (0.007 vs 0.017), and self-cleaning gain (+0.017 vs +0.029), with smaller effect sizes.
- usesExchanged calibration labels reduced SEEG movement-detection AUC more than missed movements in 15 patients
Analysis code label_noise.py (zfit, clean_fit, noisy, signflip) and lda_np.py shrinkage LDA, copied unchanged.
- supportsIn 15 SEEG patients, 20% simulated lapse labels cost 0.013 AUC and 20% swapped labels cost 0.060 AUC, replicating the asymmetry in ds006253
Earlier ds004473 label-noise result; replicated in a new dataset.