Across-day SEEG decoder confidence identified correct trials but probabilities remained overconfident

We tested whether confidence from an offline working-memory load decoder identifies correct trials on a later day, and whether its probabilities are calibrated. We used archived band-power features from OpenNeuro ds004752 v1.0.1 for 10 SEEG subjects and 35 target sessions. A Ledoit-Wolf shrinkage LDA distinguished set size 8 from 4, trained on all earlier-day sessions, and was compared with repeated 5-fold cross-validation within the target session. The plan was written before results were calculated. Across days, accuracy among the 50% most confident trials was 0.740 versus 0.646 across all trials (mean gain +0.094; 10/10 subjects; exact two-sided sign-flip p = 0.002). Confidence-correctness AUC was 0.638 (10/10; Holm-adjusted p = 0.006). Probabilities were overconfident: mean confidence was 0.942, accuracy was 0.646, and expected calibration error (ECE) was 0.331. Overconfidence was similar within one session (cross-day minus within-session difference -0.004; Holm-adjusted p = 0.89). The selective gain was larger across days than within a session (+0.037; 9/10; p = 0.037), but this difference was not significant after Holm correction (p = 0.074). Exploratory temperature scaling fitted using earlier days only reduced ECE from 0.307 to 0.156 (9/10 subjects; p = 0.004); this analysis was not in the plan. The analysis was offline and included 10 subjects, 20–40 test trials per target session, one time window, one feature set, and one decoder. Day was confounded with time between sessions and clinical state. The selective accuracy used confidence ranks from the full target session, so an online BCI would need to set its threshold before seeing that session.

SEEG · working memory · brain-computer interface · decoder confidence · probability calibration · selective prediction · cross-day generalization · shrinkage LDA · OpenNeuro