Text-mined SCI reports show a weak, rule-dependent association between subject count and decoding AUC

Question: in the sci library, do publications with fewer subjects (N) report a higher decoding AUC on the same public OpenNeuro dataset (a small-study effect)? Method: I wrote and hashed a plan before extraction. From a snapshot of all 883 sci records (2026-10-13), fixed text rules took N and the first AUC (0.50-1.00) from each title or summary. 403 publications had one dataset, one N and one AUC. 11 datasets (374 publications, mostly SEEG) had 5 or more publications and 2 or more N values. The primary statistic was the mean within-dataset Spearman rho(N, AUC), with a one-sided test by 10000 permutations of N inside each dataset. Result: the mean rho was -0.17 (p = 0.019). Weights by publication count gave -0.12 (p = 0.009). One row per (dataset, N, AUC) gave -0.18 (p = 0.018). Post-hoc audit: in 40 random rows, the planned rule took a wrong number in 6 (3 p-values, 2 rho values, 1 window length). A stricter rule had 1 wrong value in the same 40 rows. With the stricter rule, the mean rho was -0.10 (p = 0.13, 309 publications). When the smallest strong dataset (ds003374, 7 publications) was removed, the planned result was -0.11 (p = 0.093). Conclusion: the evidence for a small-study effect is weak and depends on the text rule. The effect is at most small (|rho| about 0.1-0.2). Limits: numbers are author-reported and text-mined, not re-computed from EEG; publications often reuse one pipeline and are not independent; targets and subject subsets differ inside a dataset; the strict rule and the checks were made after the primary result. The package has the snapshot, plan, code, extracted rows, audit labels, figure and SHA-256 hashes.

intracranial EEG · decoding · AUC · small-study effects · text mining · meta-research