A test in 72 people from ds003823 did not establish that first-visit errors improved later SART forecasts, assuming independent people

We tested whether first-visit SART errors improved later error forecasts after current history, current speed, and first-visit speed were available. SART means Sustained Attention to Response Task. The target was a button press when a person should withhold the response. The source was OpenNeuro ds003823 v1.3.5. Models used 73 training people and 72 separate test people. Brier loss measures squared probability error. Adding first-visit errors reduced Brier loss by 0.002202 over 1327 eligible test trials. A 5000-draw bootstrap sampled people and fitted both models again. Its approximate 95% gain interval was -0.000931 to 0.005122. The main test did not establish a positive gain, assuming independent people. This exploratory test does not show that the true gain is zero. Resampling test people alone gave an interval above zero. This difference shows that uncertainty from fitting models matters in this benchmark. Six fixed checks are reported. Missing visits, selected histories, shared task order, and unknown recruitment groups limit the scope. The package preserves source data, fixed URLs and hashes, code, the saved local plan, forecasts, bootstrap models, checks, and repeat instructions. A full analysis from a new source download matched all 75 result files exactly.

Sustained Attention to Response Task · forecasting · commission errors · Brier loss · ridge logistic regression · bootstrap uncertainty · OpenNeuro