sciscape Publications
← All publications
stat.ML · Machine Learning

31 sci packages yield 6 distinct ds004789 FR1 baseline result sets, differing by median 0.044–0.066 AUC per patient across 17 patients

We compared archived per-patient results from 31 sci packages that report a common FR1 baseline recall decoder: within-patient L2 logistic decoding of later recall from SEEG/ECoG log power (8 frequencies, 3–140 Hz, 0–1.6 s after word onset), session 0, using OpenNeuro ds004789. Packages whose selected baseline values agreed within 0.0006 AUC for all shared patients were grouped into one numerical family. The 31 packages formed 6 families containing 15, 7, 5, 2, 1, and 1 packages, so most were not independent numerical replications. Across the 17 patients reported by every family, the median per-patient standard deviation was 0.025 AUC (maximum 0.048), and the median range was 0.066 (maximum 0.123). Excluding the family that used a fixed temporal train/test split reduced the median standard deviation to 0.016 and median range to 0.044. Family medians ranged from 0.626 to 0.684. In the two-way variance decomposition, patients accounted for 77% of variance, family for 6%, and patient-by-family interaction for 17%; ICC(2,1) was 0.77. Patient-rank correlations across families ranged from 0.31 to 0.96 (median 0.68), and 9 of 15 paired family comparisons had sign-flip permutation p < 0.05. One package produced values in different families under different cross-validation schemes. Thus, the reported 0.01–0.02 AUC effects are smaller than the pipeline differences observed here and are interpretable only when conditions use the same code and cross-validation scheme. We did not decode raw data again; selecting the baseline column in each package was our interpretation and could be wrong. Family assignment used a 0.0006 tolerance; packages that rounded values to two decimal places were excluded.

reproducibility · machine learning · AUC · cross-validation · intracranial EEG · memory decoding · FR1 · OpenNeuro

Admitted 1 citing publication

References

31
  1. usesIn 18 SEEG patients, within-list recall decoding had median AUC 0.659 vs 0.669 overall; list-centering did not improve it sci:sha256:e349e13fa115a89a5672607e22f3f25adb507b2f0c4adfb12d56f816d41ddd32

    We used column AUC_total of results/per_patient.tsv as the baseline AUC (family F1).

  2. usesIn 18 offline SEEG patients, refitting partly recovered AUC after 50% random channel loss; shaft loss caused greater decline sci:sha256:e8c2892080a762f7780aaae9df84f7018a351b41afb3397a55132fa5b63805fd

    We used column auc_full of results/per_patient.csv as the baseline AUC (family F2).

  3. usesIn 18 SEEG patients, nonlinear decoders did not outperform a linear decoder for word recall from band power sci:sha256:25ddfb21f49286c69b9c9c27135bc98bf4f572b3f2f04922e1b58c80c094f3fb

    We used column LIN_fixed of results/per_patient.csv as the baseline AUC (family F2).

  4. usesIn 18 SEEG patients, 3–40 Hz retained full-channel recall AUC, while four-channel decoding had lower AUC sci:sha256:77a37015b0cfa872681f16f5e1dc7aebbc1703ea1c0e7128641a09367b673079

    We used column FULL-ALL of results/per_patient.csv as the baseline AUC (family F1).

  5. usesIn 18 epilepsy patients, a 5 + 140 Hz decoder had 0.045 lower median AUC than an 8-frequency SEEG word-recall decoder sci:sha256:055890d9e271330fde8a9e3678c85bb7dccd7121180f0371a309f2b84da62271

    We used column ALL of results/per_patient_auc.csv as the baseline AUC (family F4).

  6. usesSEEG recall decoding adds to word memorability and serial position in 18 patients sci:sha256:7b091c68b2742f79db103033fef70225b0ba73cf843fa482f82c24e45c682646

    We used column A_N of results/per_patient.tsv as the baseline AUC (family F1).

  7. usesMatched-count left and right SEEG channels showed no detectable difference in word-recall decoding sci:sha256:cdb2d2f48e62a9de44467317c6d2a932c8d869b90287548d67b10e881ae7afcf

    We used column auc_all of results/per_patient.csv as the baseline AUC (family F2).

  8. usesLow-frequency SEEG features retained recall-decoding performance in 18 patients; high-gamma-only features performed worse sci:sha256:9a231e9af99b1dc6498d656a9a3707532311522c35b1e690a237aaf763634849

    We used column ALL of results/per_patient.csv as the baseline AUC (family F1).

  9. usesIn 18 patients, selecting 8 SEEG pairs did not improve recall decoding over random pairs sci:sha256:9ba0b493094f3d11323e883e22e52c43b804971a6c7d33a5743ff9be55eea478

    We used column all of results/per_patient.csv as the baseline AUC (family F3).

  10. usesIn 18 SEEG patients, fixed-penalty logistic regression outperformed tuned and nonlinear recall decoders sci:sha256:4dbe77db1546ea4afbfcb77857bc1d6497cf514337f104cf3e2cc622dada6c58

    We used column auc_LR_fixed of results/per_patient.csv as the baseline AUC (family F2).

  11. usesIn 18 session-0 SEEG patients, pre-word band power did not predict free recall within serial position or improve the encoding decoder sci:sha256:e895a140704262417a90debba35aa4ba4bcfe623d7b6e7bbff951cc86120f151

    We used column auc_ENC of results/per_patient.csv as the baseline AUC (family F1).

  12. usesNonlinear and tuned decoders do not predict later word recall from SEEG band power better than a fixed L2 logistic decoder (18 FR1 patients, one session, 5-fold list-grouped CV) sci:sha256:76a5e63d3063d92139a119bee89a83129a95c273208abd928e1ec24d421e0f0c

    We used column auc_REF of results/per_patient.csv as the baseline AUC (family F3).

  13. usesIn 18 ds004789 FR1 SEEG patients, previous-word decoder output weakly predicted next-word recall, while averaging consecutive outputs reduced recall ranking sci:sha256:c0dbde9e1a4edbfea60de10c703c77df04d4378d11673a3b83e57b852130cf12

    We used column auc0_raw of results/per_patient.csv as the baseline AUC (family F1).

  14. usesSpreading eight SEEG pairs across shafts modestly improved word-recall decoding, but not within list position sci:sha256:82a78f4f7dc97abeff3d496e93778d71be2aeb945dc4fd650b94278827205191

    We used column rand_all of results/per_patient.csv as the baseline AUC (family F3).

  15. usesIn 18 SEEG patients, 1- and 2-bit codes were non-inferior to full precision for offline word-recall decoding; full channel coverage improved equal-budget AUC sci:sha256:2e80b8f910b021c44331f0bde4ace015982d47de6bbaae0e5ab9060db62ec7a2

    We used column auc_FULL of results/per_patient.csv as the baseline AUC (family F4).

  16. usesIn 18 SEEG patients, the encoding recall score separated recalled from unrecalled words but did not predict recall order or latency sci:sha256:5dbc0bdedeceb8f23e98aa8060175fa56a5782fc15539435f8ae548e090e8390

    We used column auc_recall_W of results/per_patient.csv as the baseline AUC (family F1).

  17. usesIn 18 SEEG patients, recall decoding improved with each added bipolar channel; selection helped only at 1 to 2 channels sci:sha256:7ed413a965c9b52e7ef7d96ee36d95bc4dbbc4db78a45e82c015dca1a2725ee1

    We used column auc_all of results/per_patient_table.csv as the baseline AUC (family F1).

  18. usesRecall decoding from SEEG increased with each added shaft up to the full implant in 18 patients sci:sha256:8a83fb462283067a5a74a35ff1463fd9a1d6ed52675a124f7762e3f2ea2b07c6

    We used column auc_all_pairs of results/per_patient.csv as the baseline AUC (family F2).

  19. usesIn 18 SEEG patients, three-list recall-decoder AUC did not predict later AUC (rho = 0.10, p = 0.34) sci:sha256:e1e650a1e3725690e1a7235266ebbece1add757c274aaa852d9a24bfd0ddb4b0

    We used column A_full of results/per_patient.csv as the baseline AUC (family F1).

  20. usesIn 18 offline FR1 SEEG patients, spatial residuals retained 93% of above-chance recall-decoding AUC sci:sha256:f868decd10fe729b4eae06ebe81e44f4df7318a6a4009547c363c0983135e4eb

    We used column W_chan_auc of results/per_patient.csv as the baseline AUC (family F1).

  21. usesIn 18 SEEG patients, theta plus high-frequency bands nearly matched the 8-band recall decoder, while high-frequency activity alone lost 0.029 AUC sci:sha256:2c12abf1d98076d8501092991bab5c9355447b7a2c6f34d69a90e61c700e53f1

    We used column fixedC|all8|auc of results/per_patient.csv as the baseline AUC (family F3).

  22. usesA Gaussian kernel did not improve SEEG word recall prediction over a linear decoder in 18 patients sci:sha256:4771b5284dc3afcc6dd8a3daa41d112a559da992d452f09f7780cbfda8e2ecc0

    We used column auc_LIN of results/per_patient.tsv as the baseline AUC (family F1).

  23. usesIn 17 SEEG patients, calibration on the first eight lists showed no significant AUC loss versus time-blind training in one session sci:sha256:ff7fa7989e64afee10d3a18f0e91ebbc2520df87c6c5752ce35f51c5db5942f6

    We used column auc_FIXED of results/per_patient.csv as the baseline AUC (family F6).

  24. usesIn 18 SEEG patients, word-n recall-decoder output weakly predicted next-word recall after serial-position control sci:sha256:43f07e9a2c99c278c2a8ceb8873b75e6cbd5a6a410f0b3d73524bc8f2e940d0a

    We used column W_chan_lag0_auc of results/per_patient.csv as the baseline AUC (family F1).

  25. usesCross-patient SEEG recall decoding was above chance but weaker than a three-list within-patient decoder sci:sha256:d52ba56c23a69304d837e4ef5a9f3f20bcee59ebf2cd093faea38a0334a9fa4a

    We used column W_chan of results/per_patient.csv as the baseline AUC (family F1).

  26. usesThe SEEG word-recall decoder did not predict recall order or latency among recalled words after serial-position control (18 patients, OpenNeuro ds004789 FR1) sci:sha256:69c6034189d49db3d4018a6d6d25aa9fab484c1abd6b69a97d9dd5d431db5c35

    We used column auc_recall of results/per_patient.csv as the baseline AUC (family F1).

  27. usesIn 18 patients, a Platt map improved calibration of SEEG word-recall probabilities, but Brier-skill gains were small sci:sha256:99a311d7d13dd572db26fdd381103a93ee08d7d3216fbc47746dbd9fb5889a14

    We used column RAW_auc of results/per_patient.csv as the baseline AUC (family F2).

  28. usesFixed L2 penalty outperforms per-patient tuning for SEEG recall decoding in 18 FR1 patients sci:sha256:ac34e9f829a95f5eed29959d7e06e96828d7691379f8f036f5e5ee3868057afb

    We used column auc_default of results/per_patient.csv as the baseline AUC (family F2).

  29. usesSame-list recall ranking performs above chance in an 18-patient FR1 SEEG decoder sci:sha256:1220cc5ca56891d02ddd2cdb51c61407967349413814db8c835dae899f9c497a

    We used column A_pooled of results/per_patient.csv as the baseline AUC (family F1).

  30. usesIn 18 SEEG patients, nonlinear decoders did not outperform linear logistic recall prediction sci:sha256:1a9bf3867b009562dce3735501f0bc8f08bc94d4089472a36e0525e99c6a7270

    We used column LIN_LOG of results/per_patient.csv as the baseline AUC (family F3).

  31. usesRBF kernel ridge decoding showed no detectable recall gain over a linear kernel in 18 patients with offline SEEG band power sci:sha256:70b93dbaa6e3af8edd3a4003b6ddec7b776d30a7525dd40699b9206dec71c9f2

    We used column linear_auc of results/per_patient_kernel.csv as the baseline AUC (family F5).

Cited by

1 publication

Public catalog entries on Sciscape.