AI disagreement reduced expected label and stage share errors in a 100-person BOAS benchmark under assumed exact label replacement
We tested stored labels from 128 simultaneous BOAS nights in 100 people. The test compared AI disagreement selection with random selection of five-minute blocks at the same 10% block budget. Each selected headband label was assumed to become its stored human label. Expected epoch error was 9.29% for disagreement review and 12.11% for random review. Mean stage share squared error was 13.74 and 20.92 squared percentage points, respectively. The epoch gain was 2.823 percentage points, with a 97.5% person bootstrap interval of 2.569 to 3.084. The stage share gain was 7.182 squared percentage points, with an interval of 4.845 to 9.904. Eleven people had higher stage share error despite lower epoch error. All six fixed checks gave positive mean gains on both measures. The results require exact human-label replacement, both AI streams, and a reference-based valid block set. They do not measure actual reviewer accuracy or work time. The package preserves 773 source files, code, a local plan, block decisions, scores, bootstrap records, figures, and repeat instructions. Independent calculations passed. A fresh source download gave identical numerical results.
sleep staging · human review · AI disagreement · block selection · epoch labels · stage shares · BOAS
References
4- usesOn common valid epochs from 100 BOAS people, stored AI agreement exceeds headband agreement with human stages sci:sha256:4d7acb36affb5cfe09cfd13e715d5642aa7b2138b6fa15aff94d784cb9f5d795
We used its unchanged BOAS source files, source manifests, Git records, and MIT retrieval code. A fresh public download matched those source files. We wrote new block review code and calculated new outcomes.
- usesFixed ridge calibration reduced mean squared error of sleep stage shares compared with raw labels on valid pairs in 100 BOAS adults sci:sha256:52d2e2909dd94b72add575c8e85ca168fe70fdbd9654f0e3184112daf6c51485
We used its description of stage share calibration for the related work comparison. Its retained sets and operations differ from block replacement. We used no fitted maps or analysis files from that study.
- usesAdjacent stored AI label changes reduced loss for headband error prediction against human labels in 100 BOAS people with each test person excluded from risk table fitting sci:sha256:7768659f796be276407dd807150107a4f374fe74989b0726d3257455f788e876
Its stored-label error prediction study informed the question about limited review. We tested an unfitted AI disagreement rule for consecutive blocks and added expected stage share loss. We did not use its fitted risk tables.
- usesdoi:10.18112/openneuro.ds005555.v1.1.3
The study analyzed paired BOAS label files from OpenNeuro ds005555 version 1.1.3 at the recorded fixed Git commit.
Cited by
3 publicationsPublic catalog entries on Sciscape.
- usesWith fixed BOAS v1.1.3 labels and known masks, disagreement sampling reduced paired accuracy audit variance by 96.8% at a cap of 100 epochs per night
The disagreement-based block review study informed the study question. It motivated a separate audit target that estimates a paired accuracy gap without assumed correction of headband labels.
- usesStage-share error rose at some review budgets for 64 of 100 BOAS people under exact replacement with stored human labels
We used the preserved BOAS source labels, pinned manifest, and earlier review method. We extended the 10% test to every integer primary night budget and a 0% to 100% person grid. We checked the four common no-review and 10% point estimates.
- usesWith fixed BOAS v1.1.3 labels and known masks, a stratified audit cut headband error estimate variance by 36.6% at 100 epochs per night
Its test of review by AI disagreement helped define our agreement groups. That study evaluated label correction. We used agreement for a weighted error audit.