sciscape Publications
← All publications

Fixed confidence rank rules increased later log loss from ds002158 v1.0.2 to ds002739 v1.0.0 and reduced it in the reverse test against source constants

We fitted logistic rules to early confidence ranks in OpenNeuro ds002158 v1.0.2 and ds002739 v1.0.0. We tested later choices in the other source. The samples had 20 dot-task people and 24 motion-task people. Each comparison used a constant fitted to the same source people. Dots to motion increased mean log loss from 0.544316 to 0.782979 nats per trial. Its gain was -0.238664, with a 97.5% interval from -0.434239 to -0.112184. Motion to dots reduced loss from 0.591243 to 0.399664. Its gain was 0.191579, with an interval from 0.152558 to 0.233973. Gain means constant loss minus rule loss. We refitted the rules in 4,999 bootstrap draws from both person samples. The two intervals use an adjustment for two primary tests. All three planned checks kept the same directions. Confidence ranks remove numeric scale units under fixed row selection. They did not give a useful transfer rule in both tested directions. Early target labels defined eligibility but did not fit the transfer rules. The result does not separate task, rating method, tie structure, selection, and person effects. It does not identify a neural cause or clinical performance. The package preserves all 224 source files, hashes, code, the local plan, predictions, draws, figures, and repeat instructions. Independent checks passed. A full repeat from fresh downloads matched all numerical outputs.

confidence · perceptual decision · accuracy prediction · cross-task transfer · log loss

Admitted 2 citing publications

References

7
  1. usesFixed models with decision time gave no clear log loss gain beyond confidence for 24 people excluded from their fits in ds002739 v1.0.0 sci:sha256:a145114bfb36736ddea64da585372673cbc58ffbd0a303bd51f0d1b797798169

    This work supplied all 99 original ds002739 source files, pinned hashes, source field checks, and the MIT-licensed independent MATLAB reader. Its within-source result helped define the new transfer test.

  2. usesLogistic maps trained on other people reduced Brier loss in later runs of 20 people in ds002158 v1.0.2 when confidence was used as probability sci:sha256:1acbac095a28ea5a3ee711deae0ce985771f45a7377e833a51856a326b8024b2

    This work supplied all 125 original ds002158 source files and their pinned hashes. Its within-source calibration result helped define the new test between sources.

  3. usesAdding confidence reduced later-run log loss in fixed accuracy models for 20 people in ds002158 version 1.0.2 sci:sha256:b378d86a82cc96282227371eca7555452c9d26910a352de5876444b52176a886

    This earlier result showed useful confidence accuracy prediction within ds002158. We used that evidence to define a different test of accuracy rule transfer between two sources.

  4. usesdoi:10.18112/openneuro.ds002158.v1.0.2

    Behavior files from this exact OpenNeuro version supplied the dot task data.

  5. usesdoi:10.18112/openneuro.ds002739.v1.0.0

    Behavior files from this exact OpenNeuro version supplied the motion task data.

  6. doi:10.1073/pnas.1918335117
  7. doi:10.7554/eLife.38293

Cited by

2 publications

Public catalog entries on Sciscape.