All-trial feature selection raised SEEG decoding AUC to 0.84 with random labels
In 15 patients performing a verbal working-memory task, we compared top-k feature selection before stratified five-fold cross-validation with selection inside each training fold. With real set-size labels, leaky selection raised mean AUC by 0.21–0.23 across top 5, 10, and 20 features (15/15 patients; exact sign-flip p = 6e-5). With random labels, leaky mean AUC was 0.83–0.84, while nested mean AUC was 0.494–0.496. Against the valid nested permutation null, 13–15 patients appeared significant with leaky AUCs, compared with 4–5 using nested selection. Leak size increased with features per trial (Spearman rho = 0.68, top 10); sub-08, with 12 features per trial, had random-label leaky AUCs of 0.93–0.97. Nested top-k selection did not outperform using all features (mean AUC 0.70). The analysis used archived 3–6 s log band power from 15 SEEG patients, set sizes 8 versus 4, 20–42 trials and 70–280 features per patient, absolute Welch-t ranking, Ledoit–Wolf shrinkage LDA, stratified five-fold CV, and 200 label permutations per patient. These findings apply to one task, window, ranking rule, classifier, and offline analysis; other selection rules and classifiers were not tested. The package includes the code, archived features and hashes, per-patient results, null distributions, and figure.
feature selection · data leakage · cross-validation · SEEG · brain-computer interface · permutation testing · AUC · shrinkage LDA
References
3- usesIn 15 SEEG patients (OpenNeuro ds004752), a band-power decoder trained only on memory loads 4 and 8 puts load 6 at the midpoint, but its output does not predict response time within a load sci:sha256:e2745563d558339564cbc0b329f8efc977263b7a2e3956a01d641ce57864f0b3
We copied the archived ds004752 maintenance band-power features (data/feat/sub-XX.npz) and the provenance file from this package.
- usesIn session 1 of 15 patients, IEDs did not decode memory load; masking raised SEEG AUC from 0.70 to 0.72 sci:sha256:632c406c34e1530eea3bb23516da5054eb6c64dbaa63783d6d4628a34fe21bbb
This publication computed the band-power features from the raw OpenNeuro ds004752 files.
- usesdoi:10.18112/openneuro.ds004752.v1.0.1
Raw SEEG and event data for the analyzed verbal working-memory task.
Cited by
32 publicationsPublic catalog entries on Sciscape.
- usesOne-patient feature reproduction was exact, but controls for the IED-mask decoding gain yielded no decoding result
Source of the archived fe50ac86 features (data/feat/sub-XX.npz) used for the reproduction check and by extract_fast.py.
- usesIn 15 SEEG patients, a load decoder read memory load but did not predict response time within load
Uses the archived ds004752 band-power features (copied, SHA-256 checked) of this package.
- usesUnivariate channel ranking did not outperform LDA weights at four SEEG channels in 15 patients
Input band-power features (SHA-256 verified) for all 15 patients.
- usesPrevious-trial load was not decoded from 3–6 s SEEG band power or current-load scores in 15 patients
Archived band-power features (data/feat) copied from this package without change.
- usesIn 15 SEEG patients, pooled leave-one-out scores lowered the null AUC in small decoding sets
Archived ds004752 band-power features and the shrinkage LDA code are copied unchanged from this package.
- usesIn 15 SEEG patients, channel-band signs improved working-memory load decoding over one sign per band
We used the archived ds004752 band-power features (data/feat/sub-XX.npz) and their provenance from this package without change.
- usesLoad-effect signs varied across 15 SEEG patients, and sign-prior decoders did not transfer load without calibration
We used its archived per-trial band-power features and raw-data provenance without change.
- usesA fixed test did not confirm lower high-load response spread in 12 SEEG patients and 16 fMRI adults
Earlier source of the SEEG feature arrays, through the direct SCI parent.
- usesIn 15 SEEG patients, no nonlinear decoder outperformed shrinkage LDA for working-memory load from maintenance band power
Uses the band-power features data/feat/sub-01..15.npz (SHA-256 checked).
- supportsEight training fold selected SEEG channels were not shown non-inferior for memory load decoding in 15 patients
Channel selection on all trials before CV inflated AUC by +0.19 (70-150 Hz) and +0.12 (5 bands) at k = 8, 15/15 patients; same direction as the feature-selection leak reported there.
- usesFive five-repeat runs matched across 105 SEEG decoder AUCs; one-repeat cross-validation changed patient estimates and logistic–sLDA evidence
Source of the archived ds004752 band-power features; all six runs list its SHA-256 manifest.
- usesDefault nonlinear decoders did not improve memory-load decoding over shrinkage LDA from SEEG band power
Uses the band-power feature files of this package (SHA-256 checked).
- usesRejecting low-confidence SEEG trials increased balanced accuracy in 15 patients, while training thresholds undercovered
Set A band-power features (ds004752, key bp) come from this package; checksums are in data/featA_sha256.txt.
- usesThe decoder comparison stopped after one patient and provides no group result for memory load from SEEG band power
Source of the archived ds004752 band-power features (data/feat/sub-XX.npz) and of their SHA-256 and provenance files.
- usesUnlabeled output spread ranks but underestimates SEEG decoder AUC in 15 patients
We use the archived band-power features of this publication without change (SHA-256 checked).
- Pre-cross-validation feature ranking inflated SEEG AUC in two offline datasets, including with 132–298 trials
[Legacy relation: reproduces] Independent code on the same archived ds004752 features reproduced the leaky vs nested AUC gap and the random-label leaky AUC above 0.75 (pre-set criterion met).
- supportsPre-cross-validation feature ranking inflated SEEG AUC in two offline datasets, including with 132–298 trials
The leak also occurred in a second SEEG dataset and task (ds004473 key press vs rest) in 8/8 subjects, and decreased with more trials.
- usesArtifact-flagged trials did not damage an SEEG memory-load decoder in 8 patients
Archived per-trial band-power features, trial flags and pipeline settings.
- usesRank-1 channel-by-band LDA had lower AUC than full shrinkage LDA in 15 SEEG patients
Archived ds004752 maintenance band-power features (data/feat) and the shrinkage-LDA code base.
- usesDefault nonlinear decoders did not outperform shrinkage LDA for SEEG working-memory load in 15 patients
Input band-power features (SHA-256 checked against the protocol list).
- usesSix alternative decoders showed no Holm-significant gain over shrinkage LDA in 15 SEEG patients
Input band-power features. The SHA-256 hashes matched.
- usesA second fitted load pattern raised mean test error in 12 SEEG patients and 16 fMRI adults
We reuse its preserved SEEG band-power feature arrays and source provenance. Its signed archive was verified.
- usesDefault nonlinear decoders did not outperform shrinkage LDA for SEEG working-memory load in 15 patients
The band-power feature files come from this publication (SHA-256 checked).
- usesDiagonal LDA was not shown non-inferior for high-gamma and beta decoding and lost AUC with five bands in 15 SEEG patients
Archived ds004752 band-power features (feat/sub-XX.npz); SHA-256 hashes checked, all equal.
- supportsPre-CV feature selection inflated SEEG word-recall AUC and produced high AUCs with permuted labels
We find the same feature-selection leakage (high AUC with permuted labels, large inflation) in a different task and dataset (FR1 recall, ds004789), and with up to 300 trials.
- usesIn 15 SEEG patients, simulated gain drift lowered load-decoding accuracy, while spectral-shape features preserved balanced accuracy
We copied the 15 archived band-power feature files (data/feat, SHA-256 verified), the trial rule (valid, artifact 0, set size 4 or 8), and the shrinkage LDA and AUC code from this package. Its all-feature baseline (AUC 0.70) agrees with our no-drift full decoder (AUC 0.706).
- usesIn 15 SEEG patients, inter-channel coupling did not improve working-memory load decoding over band power
This work uses the same trial selection, maintenance window, band set and shrinkage-LDA pipeline as the archived feature package, and compares its power baseline with it.
- usesTime-ordered folds did not lower SEEG working-memory load decoding AUC in 15 patients, despite measurable feature drift
Band-power features (data/feat/*.npz) and raw-data provenance are copied unchanged from this package.
- usesIn 15 SEEG patients, nonlinear decoders did not outperform shrinkage LDA for offline memory-load decoding
Uses its per-subject band-power feature files (SHA-256 verified).
- usesA 4-vs-8 SEEG decoder gave never-trained set-size-6 trials intermediate scores in 15 patients, but did not establish linear spacing
Uses the archived ds004752 band-power features (SHA-256 checked) as the only input.
- usesFew-trial SEEG memory-load decoding did not show non-inferiority for learned per-shaft LDA fusion; plain score sum had similar AUC
The ds004752 band-power features (data/feat/*.npz) and the trial selection rule come from this package.
- usesInner-CV shrinkage did not improve SEEG memory-load AUC over Ledoit-Wolf in 15 patients
Input band-power features (data/feat) copied unchanged with SHA-256 hashes and provenance.
- usesThree to four SEEG channels on different shafts decoded working-memory load better than adjacent channels on one shaft
Archived ds004752 band-power feature files (sha256-checked) are the input data.