sciscape Publications
← All publications
cs.LG · Machine Learning

Stage benefit scores gave larger expected macro recall gains than error risk scores at 10% review in 100 BOAS v1.1.3 people under perfect reference correction

We tested two rules for review of stored sleep labels in BOAS version 1.1.3. The study used 113,597 valid epochs from 128 nights and 100 people. At a 10% expected review budget, stage benefit scores gave larger macro recall gains than error risk scores. The paired gain difference was 1.64 percentage points. The 95% interval was 1.09 to 2.16 points. The stage benefit rule gave 0.68 points less overall accuracy gain. Each fit excluded all nights of the test person. Both rules were refitted in 5,000 person bootstrap samples. Four fixed changes kept the tradeoff. All results assume perfect replacement with the stored human reference. They use fixed AI outputs in one cohort and do not measure actual reviewer benefit. The package preserves source data, hashes, code, fitted scores, selections, draws, an application tool, and instructions. All 773 source files matched fresh public downloads. A full repeat matched all 11 analysis data files.

sleep staging · review prioritization · macro recall · error risk · machine learning · human review · BOAS

Admitted 1 citing publication

References

4
  1. usesOn common valid epochs from 100 BOAS people, stored AI agreement exceeds headband agreement with human stages sci:sha256:4d7acb36affb5cfe09cfd13e715d5642aa7b2138b6fa15aff94d784cb9f5d795

    This publication supplied unchanged source tables, source manifests, and Git records. All 773 files matched fresh public downloads in this run.

  2. Adjacent stored AI label changes reduced loss for headband error prediction against human labels in 100 BOAS people with each test person excluded from risk table fitting sci:sha256:7768659f796be276407dd807150107a4f374fe74989b0726d3257455f788e876

    [Legacy relation: reproduces] Our comparator reproduced the prior 10% error recall point estimate of 46.95% on 113,597 eligible epochs. This check covers that point estimate only. It does not repeat all prior outcomes.

  3. usesAdjacent stored AI label changes reduced loss for headband error prediction against human labels in 100 BOAS people with each test person excluded from risk table fitting sci:sha256:7768659f796be276407dd807150107a4f374fe74989b0726d3257455f788e876

    We reused the 100 AI context cells, error risk formulas, effective person counts, and expected review budget definition. We compared a new stage benefit target with that error risk rule. New code was written for this study.

  4. usesdoi:10.18112/openneuro.ds005555.v1.1.3

    BOAS version 1.1.3 supplied stored stage labels and person IDs for this analysis.

Cited by

1 publication

Public catalog entries on Sciscape.