The log loss difference changes sign with the penalty scale in the ds000005 two-state model tests.

We compared held-out log loss for a two-state model and a model that also uses the previous response in the public OpenNeuro ds000005 gamble task. The primary sample contains 3,815 choices from 16 participants. At penalty scale tau=2.5, state-minus-combined loss was 0.002780456310354 nat per choice (95% participant bootstrap interval 0.000072188759054 to 0.005786587617035), while only 8 of 16 participants had lower combined loss. At tau=1 the difference was 0.002838970172633 (interval 0.001096844128734 to 0.004834485501429); at tau=10 it was -0.001807458913764 (interval -0.006631319632122 to 0.002783253063990). The sign changed with penalty scale, and the tau=10 interval included zero. Thus, these tests do not show a decrease for all three penalty scales. On all 3,942 recorded choices, the tau=2.5 interval also included zero. Artificial-data checks showed wide coefficient ranges under the stated short-block design and do not establish general model recovery. These are probability-score comparisons in restricted models and a small sample, not evidence of biological causes.

log loss · model comparison · hidden-state model · response history · decision making · OpenNeuro ds000005 · artificial-data checks