Adding confidence reduced later-run log loss in fixed accuracy models for 20 people in ds002158 version 1.0.2
We tested whether confidence improved estimates of choice accuracy in later task runs. We used OpenNeuro ds002158 version 1.0.2 and all 20 public participants. Fixed models used response time, dot evidence, and stimulus side. Separate models were fit for each person and condition. Runs 1–3 trained the models. Runs 4–6 tested them. Adding confidence reduced mean test log loss from 0.5741 to 0.2497 nats per trial. The mean gain was 0.3244 nats, with a 95% person-bootstrap interval of 0.2673 to 0.3822. All 20 people had a positive average gain. Six planned checks and one added source-path check retained positive gains. The active-minus-observation gain interval included zero. The result applies to these fixed models and confidence ratings given after a choice. It does not establish a neural cause or performance in new people. The package preserves 125 source files, fixed source hashes, code, all fits and predictions, audits, figures, and instructions. A new source download matched all files. An independent optimizer checked 636 fits. A second offline run reproduced all nine numerical files.
confidence · choice accuracy · metacognition · log loss · ridge logistic regression · person bootstrap · OpenNeuro · later-run prediction
References
2- usesdoi:10.18112/openneuro.ds002158.v1.0.2
The analysis used event tables and participant metadata from this exact OpenNeuro release; the saved version tag maps to the recorded fixed mirror commit.
- doi:10.1073/pnas.1918335117
Cited by
5 publicationsPublic catalog entries on Sciscape.
- usesLogistic maps trained on other people reduced Brier loss in later runs of 20 people in ds002158 v1.0.2 when confidence was used as probability
This work supplies the 125 original source files, source manifest, version record, repository tree, retrieval code, and primary trial screen. We read its positive within-person confidence result before the new plan. The new test fits confidence-only maps across people and compares them with raw confidence used as probability.
- usesFixed confidence rank rules increased later log loss from ds002158 v1.0.2 to ds002739 v1.0.0 and reduced it in the reverse test against source constants
This earlier result showed useful confidence accuracy prediction within ds002158. We used that evidence to define a different test of accuracy rule transfer between two sources.
- usesAdding previous confidence increased observed mean test log loss in 20 ds002158 v1.0.2 and 24 ds002739 v1.0.0 people under fixed models
Its test of current confidence in later dot task runs helped define the new conditional accuracy question and the earlier-run and later-run split.
- usesFixed models with decision time gave no clear log loss gain beyond confidence for 24 people excluded from their fits in ds002739 v1.0.0
We used its later-run comparison of confidence and time as a reference for the question. We test the added value of decision time after confidence in a different dataset, with each test person excluded from fitting.
- usesThe 95% patient bootstrap interval for current confidence gain included zero on later trials from 17 MetaRDK patients under fixed ridge models
Its comparison of current confidence models on later trials informed this study design. Dataset, patient group, features, split, and penalty differ. We make no controlled group comparison.