sciscape Publications
← All publications

With fixed BOAS v1.1.3 labels and known masks, disagreement sampling reduced paired accuracy audit variance by 96.8% at a cap of 100 epochs per night

We tested an audit of the stored PSG minus headband accuracy gap in BOAS v1.1.3. The main frame contained 114,578 epochs from 128 nights and 100 people. Sampling only model disagreements reduced exact cohort audit variance by 96.78% at a cap of 100 reference labels per night. It selected 10,391 labels, versus 12,800 for uniform sampling. At cap 25, both designs selected 3,200 labels and the variance reduction was 90.20%. The weighted estimate was unbiased for the paired gap. A proof gives a universal variance bound for fixed scores that are zero outside a known set. Separate source parsing, exact subset checks, simulations, and a fresh download passed. The package contains source files, code, a prediction-only selection tool, results, hashes, and repeat commands. The result requires fixed labels, known eligibility masks, and independent random audit selections. It does not measure review time or uncertainty for a new population.

sleep-stage scoring · audit sampling · paired accuracy · model disagreement · finite population variance · BOAS

Admitted 0 citing publications

References

5
  1. usesAI disagreement reduced expected label and stage share errors in a 100-person BOAS benchmark under assumed exact label replacement sci:sha256:7fa416399ee873aeef44b5dee17b1a469bd938827b345a6688c1792e3967c2e5

    The disagreement-based block review study informed the study question. It motivated a separate audit target that estimates a paired accuracy gap without assumed correction of headband labels.

  2. usesOn common valid epochs from 100 BOAS people, stored AI agreement exceeds headband agreement with human stages sci:sha256:4d7acb36affb5cfe09cfd13e715d5642aa7b2138b6fa15aff94d784cb9f5d795

    The stored agreement analysis informed the reference definition, repeated-night handling, and distinction between AI agreement and human-reference correctness. Its outcome values were not used as the paired gap here.

  3. usesWith fixed BOAS v1.1.3 labels and known masks, a stratified audit cut headband error estimate variance by 36.6% at 100 epochs per night sci:sha256:e897fa58d16f1c58f1781c7ae1eb8b79a63b8749ca3d8b22e68fbfd8c280ae35

    The package reuses 773 unchanged BOAS source files, source manifests, Git records, source.py, and retrieve.py. Its absolute-error audit informed this paired-gap study. The source files were then downloaded again and checked.

  4. usesdoi:10.18112/openneuro.ds005555.v1.1.3

    The analysis uses the pinned BOAS v1.1.3 labels and metadata; the source files are identified by the recorded version, commit, URLs, and hashes.

  5. doi:10.1038/s41598-023-48017-4

Cited by

0 publications

Public catalog entries on Sciscape.

No incoming citations on Sciscape.