Interrater Agreement for Movement Disorder Classification in Children with Mixed Movement Disorders

Objective: To determine the interrater agreement for the classification component of the Movement Disorders-Childhood Rating Scale (MD-CRS) among five pediatric neurologists with training in childhood movement disorders. Methods: 112 videos with hyperkinetic movement disorders were included. The videos were reviewed independently by five assessors, including a senior pediatric movement disorders neurologist (Rater 1) and 4 individuals trained by him. The raters answered three questions: 1) Is more than one movement disorder? 2) What is the [prominent] movement disorder? 3) What other movement disorders are present? Results: The most frequent movement disorders in the sample were dystonia/athetosis (~30%) and chorea/ballism (~30%), followed by myoclonus, stereotypy, tremor, and tic. Final agreement on number of movement disorders was 58%, with better agreement for videos with one movement disorder. For prominent movement disorder, the highest interrater agreement between any 2 investigators was 89% and the lowest was 76%, but absolute agreement among all five raters was only 42%. For other movement disorders, agreement ranged from 65% to 82% among any 2 investigators. Interrater agreement ranged from poor to good, with highest agreement for prominent movement disorder and the lowest for other movement disorders. Conclusion: Even with formal training, there is substantial disagreement among experts in pediatric movement disorders. We suggest formal training to establish reliability on the MD-CRS prior to use in research studies. We did not test the general assessment or movement disorder severity sections of the scale. For severity scores to be most meaningful, there must be high reliability in classification.
Keywords: childhood movement disorders, classification, training program