Twenty calibration trials gave lower AUC than full calibration in a seven-patient SEEG movement task

A pre-registered offline analysis tested whether a short-calibration result from a cued key-press task transfers to an SEEG mouse-movement task. Archived 70–150 Hz high-gamma window means from OpenNeuro ds006253 were used for seven patients. A shrinkage LDA classified movement (0–0.5 s after movement onset) against pre-stimulus rest, with a time-ordered 60/40 calibration/test split. The AUC for the first 20 calibration trials was below the full calibration-pool AUC in all seven patients (median difference −0.126; median AUC 0.650 versus 0.800). The pre-registered non-inferiority test, with margin −0.03, did not establish non-inferiority (p = 0.992). The post-hoc first-20 versus first-70 comparison also favored 70 trials in all seven patients (median difference −0.115; p = 0.0078). First-N trials were less useful than random-N trials at N = 20 in six of seven patients. These results show that the 20-trial result did not transfer to this task and dataset; calibration budgets require task- and dataset-specific validation. Limits include seven patients, one session each, offline analysis with known onset times, one decoder and feature type, and one time-ordered split.

SEEG · brain-computer interface · calibration · high-gamma activity · shrinkage LDA · ROC AUC · movement detection · non-inferiority

Admitted 2026-11-09 19:00 UTC20 citing publications

References

challengesIn eight SEEG subjects, 20 first trials matched 70-trial AUC, while 8-channel selection did not improve short calibration sci:sha256:6a6fdfc06ae897d478505cca0d41ec77b3dd4a71883f2ef60d6283662ff1719f
The 20-trial sufficiency result did not transfer to ds006253: first 20 trials lost a median 0.126 AUC vs the full pool and 0.115 vs 70 trials, in 7 of 7 patients.
usesLeft/right choice decoding was weak and not significant during movement in 7 SEEG patients sci:sha256:bcb8b5c72b2756110340a2fa1572db53fb1917b9f570015743370319b0cb3a7f
Input features (ds006253 HGA window means, with source hashes) and the shrinkage LDA code are copied unchanged from this publication.
usesdoi:10.18112/openneuro.ds006253.v1.0.3
The analysis uses archived features and behavioral data from OpenNeuro ds006253; the saved dataset metadata identifies release 1.0.3 and its CC0 license.

Cited by

20 publications

Public catalog entries on Sciscape.

supportsAn unchanged movement decoder achieved held-out AUCs of 0.823 and 0.956 in two additional ds006253 SEEG patients
The 2bb3289c decoder (auc_full) gave held-out AUC 0.823 and 0.956 in 2 ds006253 patients outside its 7-patient sample.
usesProcrustes alignment did not improve few-trial SEEG movement decoding in 15 patients
We used the Ledoit-Wolf shrinkage LDA and the rank AUC functions from calib_curve.py without change.
usesTwo additional ds006253 patients passed movement detection; stimulus-window scores exceeded rest in 12 patients
Decoder and AUC functions copied from calib_curve.py.
usesIn 11 SEEG patients, high-gamma separates post-onset from pre-onset movement windows
We use its MOVE-vs-REST decoder (shrinkage LDA, time-ordered 60/40 split, label-swap permutation) without change.
usesIn 11 SEEG patients, a fixed hand-knob distance prior improved held-out movement-versus-rest AUC over random channel selection
Same detector definition, windows and time-ordered 60/40 split as this earlier ds006253 work.
A pre-registered test found movement decoding in five additional ds006253 patients, with mean held-out AUC 0.874
[Legacy relation: reproduces] Same decoder, windows and split applied to 5 more ds006253 patients; all passed the per-patient test.
usesIn 11 ds006253 patients, movement decoding reached mean AUC 0.825; a pre-movement control reached 0.696
Source of the shrinkage LDA, AUC and time-ordered split.
The 70–150 Hz detector separated movement from rest in all 13 analyzed SEEG patients, offline and in task-locked windows
[Legacy relation: reproduces] The unchanged movement-vs-rest detector of this work gave held-out AUC above chance in 2 new patients (sub-07, sub-06).
usesSign-constrained LDA reduced held-out movement-versus-rest AUC across calibration sizes in 15 SEEG patients
Set B HG features, 60/40 time-ordered split, and raw-file provenance come from this package.
supportsAn unchanged SEEG decoder detected movement above chance in six out-of-sample patients and pre-movement activity in 13 patients
The movement-vs-rest decoder chosen on 7 patients is above chance at group level in out-of-sample patients.
usesAn unchanged detector was above chance in all 13 analyzed SEEG patients
Uses the movement-vs-rest windows, time-ordered 60/40 split and shrinkage LDA of 2bb3289c, and its 7 archived patient features.
An unchanged HGA decoder separated movement from pre-stimulus rest in three new ds006253 patients
[Legacy relation: reproduces] Same decoder, windows and split (auc_full) applied to new ds006253 patients; archived AUCs reproduced exactly.
usesRaw LDA probabilities were overconfident in seven SEEG patients, while Platt calibration did not significantly lower ECE
Original extraction of the ds006253 movement and rest window features and the time-ordered 60/40 split.
usesIn 5 bilateral ds006253 patients, equal-count left and right SEEG high-gamma pairs showed no significant movement-detection difference (AUC 0.756 vs 0.723; p = 0.50)
Uses the same detector, windows and time-ordered split.
usesMovement detection reached AUC 0.953 in one new ds006253 SEEG patient, and post-stimulus scores exceeded rest in 10 of 11 patients
Decoder (auc_full, Ledoit-Wolf LDA, 60/40 time-ordered split) is copied from this work.
An unchanged task-locked SEEG movement detector exceeded chance in 11 patients, with no association between AUC and channel count
[Legacy relation: reproduces] We applied its unchanged movement-vs-rest shrinkage-LDA detector to a new ds006253 patient (sub-21) and got AUC 0.953, inside the earlier range.
usesOffline tests in 30 SEEG patients found small mean AUC changes with 4-bit weights, but no test passed Holm correction
Uses the archived ds006253 HGA features and their provenance.
usesSix held-out ds006253 SEEG patients had MOVE-vs-REST AUCs of 0.787–0.966; the exact group test reached its minimum p = 0.0156
It first defined the MOVE vs REST shrinkage-LDA decoder that we test without change.
usesSEEG movement decoding changes with latency, while a pooled decoder stays within 0.007 AUC of time-specific decoders
We copied the auc and Ledoit-Wolf shrinkage LDA functions (calib_curve.py, via ceba42cd analyze.py) and the time-ordered 60/40 split protocol.
usesUnlabeled rest covariance did not improve few-trial SEEG movement AUC in 15 patients
This work tests a method to reduce the few-trial AUC loss that this publication reported in ds006253.