Clinician inclusion raised mean onset channel balanced agreement by 8.5 points in 78 public SEEG seizures with equal vote counts and fair ties

We measured the effect of clinician inclusion in a seizure annotation vote committee. The public data contained 78 seizures from 29 patients. Each seizure had three or five independent clinician annotations. We compared committees with the same number of votes. One committee included the tested clinician. The other committee excluded that clinician. Each tied vote had a positive label probability of 0.5. We used the tested clinician's channel labels as the fixed reference. We gave patients equal weight. Mean onset channel balanced agreement was 91.20% with inclusion and 82.67% with exclusion. The increase was 8.53 percentage points. The 95% paired patient bootstrap interval was 6.90 to 10.24 points. The increase was 8.64 points when ties were negative and 8.42 points when ties were positive. We supply source files, code, results, and 71,765 benchmark rows. These scores measure agreement between opinions. They do not measure clinical correctness.

seizure annotation · stereotactic EEG · inter-rater agreement · observer exclusion · balanced agreement · epilepsy