With fixed BOAS labels and known masks, 5 minute block audits had 5.14 times epoch audit variance at equal budgets
We compared random selection methods for sleep label audits using fixed public labels from 128 nights and 100 people in the Bitbrain Open Access Sleep dataset (OpenNeuro ds005555, version 1.1.3). Each person had equal weight, all methods scored the same epoch budget, and the reference eligibility mask and its count were fixed and known. For headband error, 5 minute block audits had 5.138 times the exact variance of random epoch audits; their exact standard deviations were 0.772 and 0.341 percentage points. Spread selection had 0.570 times the epoch variance, with a standard deviation of 0.257 points. The block penalty remained when absent AI outputs did not contribute. A 20 epoch block audit missed all errors with a mean probability of 34.89%, compared with 7.80% for epoch selection at the same cost. Results concern random selections from these fixed records. They do not quantify uncertainty for new people, new model outputs, human review costs, or population confidence intervals. The package preserves source data and hashes, a local plan, code, proof, results, figures, and reproduction instructions.
sleep staging · audit sampling · design variance · block sampling · spread sampling · BOAS · fixed labels · AI labels
References
3- usesOn common valid epochs from 100 BOAS people, stored AI agreement exceeds headband agreement with human stages sci:sha256:4d7acb36affb5cfe09cfd13e715d5642aa7b2138b6fa15aff94d784cb9f5d795
We reused its unchanged BOAS source tables, label definitions, person grouping, and pinned retrieval manifest. These supplied the fixed census for a new audit selection study.
- usesdoi:10.18112/openneuro.ds005555.v1.1.3
The study analyzed the saved, hash-pinned BOAS source files from OpenNeuro ds005555 version 1.1.3; the dataset record identifies this version-specific DOI and CC0 license.
- doi:10.1080/01621459.1952.10483446
Cited by
4 publicationsPublic catalog entries on Sciscape.
- usesA fixed error table reduced variance by 31.8% for 10% random audits of 50 BOAS people with known valid epochs
Its random audit study and exact finite-frame variance methods informed our question and baseline. We keep uniform epoch selection and change the estimator. We do not reuse its scores or compare directly with its spread-selection design.
- usesRandom review in paired AI groups reduced mean expected squared stage share error by 70.6% in 100 BOAS people. Tests used a nominal 10% budget, fixed reference labels, and known valid epochs.
This report supplied the fixed-label random-audit frame and the known-mask conditions. The new work tests stratified categorical share estimates and allocation learned from other people. Its code and covariance checks were developed here.
- usesWith fixed BOAS labels and known masks, adjacent pair audits had 83.6% less stage change sampling variance than epoch audits at equal 10% label caps
Its fixed-label comparison of block and epoch audits guided the new question. We tested adjacent pair selection for stage changes, which require two labels, rather than its one-label epoch error target.
- usesWith fixed BOAS v1.1.3 labels and known masks, a stratified audit cut headband error estimate variance by 36.6% at 100 epochs per night
We reused its unchanged BOAS tables, source hashes, repository snapshots, and MIT parser and retrieval code. These supplied the fixed audit frame.