Random training-label errors reduced word-recall AUC in an 18-patient offline SEEG study; the effective-sample rule held at 10% and 20%, not shown at 30%

We added random wrong labels to the training words of a within-patient SEEG word-recall decoder (archived ds004789 FR1 band power of c4b8c884; 18 patients; L2 logistic, C = 7.2e-4; 5 list-grouped folds; test labels stay correct; 10 draws per rate). With all labels correct, the median AUC was 0.670. When we flipped 5%, 10%, 20%, 30% and 40% of the training labels, the median AUC decreased by 0.009, 0.013, 0.045, 0.080 and 0.122. At 5%, all 18 patients lost AUC; at the other rates, 17 of 18 lost AUC (Holm p = 2.3e-5 at 10-30%). The decoder stayed above chance at all rates. A pre-registered test compared label errors at rate e with correct labels on only a fraction (1-2e)^2 of the training words. The mean paired AUC difference was -0.004 (95% CI -0.014 to 0.003) at e = 0.1 and -0.006 (-0.015 to 0.004) at e = 0.2, inside the +/-0.02 margin. At e = 0.3 it was -0.013 (-0.028 to 0.001), so the rule was not shown there. Exploratory: when only recalled words were relabeled as not recalled, the loss was small (0.018 at 40%), because only about 1 in 4 words is recalled. For this offline linear decoder, 10% wrong labels cost about as much as the loss of 36% of the correct words. Limits: one task and dataset, random (simulated) errors, one decoder, offline AUC only.

stereoelectroencephalography · word recall · memory BCI · label noise · logistic regression · AUC · effective sample size · OpenNeuro ds004789