A leave-one-patient-out threshold reduced held-out F1 versus pair-specific calibration for a hippocampal ripple trigger

We re-analysed archived held-out threshold curves for a causal hippocampal ripple trigger in 20 patients with NREM sleep SEEG from OpenNeuro ds003498. The curves cover 481 bipolar pairs with at least 10 expert marks and two 60 s folds per patient. We compared per-pair labelled calibration, per-patient labelled calibration, leave-one-patient-out (LOPO) calibration without target-patient ripple labels, fixed k = 3, and an oracle. Mean causal held-out F1 was 0.413 for per-pair calibration and 0.300 for LOPO; the patient-level difference was -0.113 (bootstrap 95% CI -0.155 to -0.076), with worse F1 in 18 of 20 patients (exact sign-flip p = 7.6e-6). The prespecified non-inferiority margin of 0.03 F1 was not met. Offline zero-phase mode showed the same pattern (LOPO minus per-pair = -0.123, 95% CI -0.162 to -0.086). LOPO k was stable at 3.0–3.5, while per-pair labelled k ranged from 1.5 to 7.0 (5th–95th percentiles; median 2.75). LOPO reduced false triggers (27 versus 39 per minute) but also reduced recall (0.39 versus 0.43). The median and MAD still came from 60 s of unlabelled signal for each target pair, so label-free means no ripple marks, not no target data. The result is limited to short segments from one dataset and site and measures agreement with expert marks, not stimulation benefit.

hippocampal ripples · SEEG · threshold calibration · leave-one-patient-out · F1 score · closed-loop stimulation · NREM sleep