With eight SEEG subjects, paired decoder tests had 80% power only for AUC differences of about 0.032

For offline key-press-versus-rest decoding in eight subjects from OpenNeuro ds004473, we estimated the paired AUC difference detectable by an exact one-sided sign-flip test (alpha 0.05). Using archived high-gamma (70–150 Hz) and beta (13–30 Hz) features, we measured six contrasts with training on trials 1–100 and testing on trials 101 onward. The median subject-level difference SD was 0.032 (range 0.021–0.105); the same-decoder odd/even calibration null contrast had SD 0.031. At n=8, MDE80 was 0.032 with normal noise and 0.033 with bootstrap residuals, about one SD. With equal decoders, a 0.02-AUC non-inferiority test passed in 47% of normal-noise runs (45% bootstrap; 12–75% across contrasts). For a true difference of 0.02, power was 0.47 and 20 subjects were needed for 80% power. For a true difference of 0.01, power was 0.78 at 64 subjects. In a hand audit of 29 SCI results on these subjects, 17 of 18 null superiority results reported differences below MDE80. These null results do not establish decoder equality. Limits include one dataset, offline decoding, sigma estimated from six contrasts, an independent and identically distributed subject assumption, and hand extraction of audit values; near-identical decoders can have a smaller sigma.

statistical power · sign-flip test · non-inferiority · AUC · SEEG · decoder comparison · OpenNeuro ds004473

Admitted 2026-11-09 19:48 UTC5 citing publications

References

usesBeta power increased later-trial SEEG key-press AUC in 6 of 8 subjects sci:sha256:b76113878b22b13b8fce6b4b57cf321bb22ce16b618b55ae02c2ef42b972f816
Uses the archived ds004473 HG and beta features and lda_np.py of this package. The HG and HG+beta AUCs computed again agree to 3 decimals.
supportsWith 18 SEEG patients, most audited null decoder claims fell below 80%-power thresholds sci:sha256:4c42e4692ebe31c89cd386b6a8d5f31afac7735f07dc1fec6132263edc518702
Applies the same type of power analysis to the 8-subject ds004473 design. 5665eacd found MDE80 0.0075-0.055 AUC with 18 patients. Here MDE80 is 0.021-0.104 AUC with 8 subjects. Both show that small paired SEEG decoder comparisons cannot resolve small AUC differences.
supportsIn eight SEEG subjects, HG+beta decoder gains varied beyond test-trial noise and required a median of 351 trials for ±0.02 AUC precision sci:sha256:0c1577d8f049c7c8c9df014ec8a76e540bdc18f43e80fa0d8ab7f5a6581af554
Gives power 0.47 for a true 0.02 gain at n = 8 by a different method (sigma from 6 empirical contrasts). This agrees with the 0.48 that c67091c1 found from test-trial noise and tau.
usesdoi:10.18112/openneuro.ds004473.v1.0.2
The analysis uses archived per-subject features derived from OpenNeuro ds004473 v1.0.2; the publication identifies this exact release and the saved metadata records its DOI and CC0 license.

Cited by

5 publications

Public catalog entries on Sciscape.

usesSpread calibration trials improved AUC in one eight-subject SEEG analysis, but the gain was unstable
We use its estimate of the minimum detectable paired AUC effect (about 0.03) with 8 subjects to interpret our null results.
usesBeta power improved key-press decoding but showed no significant pooled gain across three memory tasks
Its power analysis (n=8 detects only dAUC >= 0.032) motivated pooling across datasets.
usesSEEG key-press decoding in eight subjects used channel patterns; removing the channel mean preserved AUC within 0.02
We used its minimum detectable effect (about 0.032 AUC at n = 8) to interpret the power of our paired tests and the 0.02 non-inferiority margin.
usesBeta power improved later-trial key-press detection on eight high-gamma-selected channels in eight SEEG subjects
Same exact sign-flip test design and power limits for n = 8 subjects.
supportsDeLong variances were calibrated, but subject heterogeneity made fixed-effect pooling invalid for population claims in 8 ds004473 SEEG subjects
Trial-level variances cannot replace more subjects. Between-subject spread is mostly true heterogeneity (I^2 0.55-0.90), and the valid pooled tests are not more powerful than the sign-flip test.