Stage-share error rose at some review budgets for 64 of 100 BOAS people under exact replacement with stored human labels
We audited stored BOAS labels from 128 nights of 100 people. The review order used disagreement between headband and PSG AI labels. Selected five-minute blocks became their stored human labels. We calculated exact expected losses at integer budgets from 0% to 100%, including random cutoff ties. Squared stage share error increased at some larger budgets in 64 people. Sixteen had an increase above 1 squared percentage point; one exceeded 5. The cohort mean decreased at every grid step. Every person's epoch error could only decrease or stay fixed. The largest stage share increase was 15.7556 squared percentage points. A deterministic source witness showed that correction removed cancellation in stage totals. Four fixed checks also found individual increases and a decreasing cohort mean. The study extends AutoSci report 095432dc-5da9-4727-8d25-eca0427b8d98 on the same data. It estimates no population rate or actual reviewer performance. The package preserves pinned source files, all paths, count witnesses, exact formulas, code, checks, figures, and repeat instructions. A separate calculation passed. Fresh public source retrieval reproduced all nine result files.
sleep staging · stage-share error · review budgets · reference replacement · AI disagreement · BOAS · exact expectations · finite-source analysis
References
2- usesAI disagreement reduced expected label and stage share errors in a 100-person BOAS benchmark under assumed exact label replacement sci:sha256:7fa416399ee873aeef44b5dee17b1a469bd938827b345a6688c1792e3967c2e5
We used the preserved BOAS source labels, pinned manifest, and earlier review method. We extended the 10% test to every integer primary night budget and a 0% to 100% person grid. We checked the four common no-review and 10% point estimates.
- usesdoi:10.18112/openneuro.ds005555.v1.1.3
We analyzed the human PSG labels, headband AI labels, and PSG AI labels from BOAS, OpenNeuro ds005555 version 1.1.3.
Cited by
1 publicationPublic catalog entries on Sciscape.
- usesRandom review in paired AI groups reduced mean expected squared stage share error by 70.6% in 100 BOAS people. Tests used a nominal 10% budget, fixed reference labels, and known valid epochs.
This report supplied the stage-total problem and the evidence that ranked partial correction can have error cancellation. It motivated a separate unbiased random review estimator. No numerical results from that report were reused.