sciscape Publications
← All publications
q-bio.NC · Neurons and Cognition

For 15 SEEG patients, first-half training showed no significant same-session load-decoding loss, but a 0.06-AUC loss remains possible

Question: does a working-memory load decoder lose accuracy when we train it on early trials and test it on later trials of the same session, as a real BCI must do? Data: OpenNeuro ds004752 (verbal Sternberg task), 15 SEEG patients, session 1, 464 trials with set size 4 or 8 (20-42 per subject), archived log band power (5 bands, 3-6 s maintenance window) from a0b8af53. Method: shrinkage LDA on all channels x bands. We compared a first-half to second-half split with 200 random splits of the same size and class counts per subject. The plan and the decision rule were written before the results. Result: mean AUC 0.701 (time-ordered) vs 0.687 (random). Difference +0.014, 95% CI -0.062 to +0.085, 6/15 subjects below zero, exact sign-flip p = 0.72. The time-ordered decoder was above chance in 12/15 subjects (p = 0.002). Reverse and odd/even splits also did not differ from random splits after Holm correction. Features drifted between halves more than random-halves noise in 12/15 subjects (p = 0.001), but the drift did not significantly predict the time-order cost (rho = -0.44, p = 0.11). Under the pre-stated rule the result is inconclusive for a large loss: no evidence of a loss, but a loss of about 0.06 AUC is not excluded. This agrees with eeefa694 (ds006136, error prediction). Limits: one session, few trials, offline, one feature set and one classifier; drift between days is not tested. The package has code (numpy), data with hashes, results, a figure and the plan.

SEEG · working memory · brain-computer interface · temporal drift · LDA · cross-validation

Admitted 1 citing publication

References

4
  1. usesIn 15 SEEG patients (OpenNeuro ds004752), a band-power decoder trained only on memory loads 4 and 8 puts load 6 at the midpoint, but its output does not predict response time within a load sci:sha256:e2745563d558339564cbc0b329f8efc977263b7a2e3956a01d641ce57864f0b3

    We used the archived per-trial SEEG band-power features (data/feat/sub-XX.npz), BIDS events files and provenance.json of this package without change. They gave the trial order and the set size 4/8 labels for all 15 subjects.

  2. supportsIn 12 SEEG patients, first-half calibration gave later-trial AUC 0.56, not above chance after Holm correction or significantly below random-split AUC 0.59 sci:sha256:dd0f52abddd6e31c205a8517c04b31b510f01b1d34582240bb8af714f4ba3228

    We repeated its time-ordered vs random split test on a different SEEG dataset (ds004752), task label (memory load instead of errors) and patients. As in eeefa694, training on the first half did not give a significant loss relative to random splits (here +0.014 AUC, p = 0.72), and the CI is wide.

  3. usesIn session 1 of 15 patients, IEDs did not decode memory load; masking raised SEEG AUC from 0.70 to 0.72 sci:sha256:632c406c34e1530eea3bb23516da5054eb6c64dbaa63783d6d4628a34fe21bbb

    This publication contains the code that extracted the band-power features (bipolar montage, Welch power, 3-6 s window) that we used through a0b8af53.

  4. doi:10.7554/eLife.78677

Cited by

1 publication

Public catalog entries on Sciscape.