This study examines the impact of interrater reliability on the diagnosis of attention-deficit hyperactivity disorder (ADHD). A screening of 6171 elementary school children identified 1573 children with a high risk for ADHD according to teacher rating. Follow-up parent interviews and information from teachers were collected on 243 children. Before screening, health care professionals had diagnosed ADHD in 40% of the identified children. There was low agreement between the parent and teacher reports of ADHD symptoms according to DSM-IV-based questionnaires: Inattentive (r = .34, κ = 0.27), Hyperactive/Impulsive (r = .27, κ = 0.22), and Performance Impairment (r = .31, κ = 0.07). When the two-setting requirement was strictly enforced, poor interrater agreement decreased diagnostic rates for all three types of ADHD in this clinical sample: Inattentive (15%-5%), Hyperactive/Impulsive (11%-3%), and Combined (23%-7%). Parent and teacher agreement was low concerning ADHD symptoms and performance. The recommendation of multiple informants significantly decreased the prevalence. Allowing for observer disagreement by using more lenient core symptom scores could reduce the effect.
Attention-deficit hyperactivity disorder (ADHD) is the most common neurobehavioral diagnosis affecting children today. 1,2 Given the widespread attention that ADHD is receiving in health care and the media, 3-5 a uniform process to evaluate children who present with the core symptoms of inattention, hyperactivity, impulsivity, or academic impairment is warranted. Such a strategy can help minimize overdiagnosis and underdiagnosis. Correct diagnosis is imperative because early detection can help direct individualized management to improve the psychosocial and educational development of children with these symptoms.
The criteria most often used to diagnose ADHD in the United States are from the DSM-IV. 6 These criteria depend on the observation of the individuals who have the most interaction with the child. In the past, the diagnosis has been made mostly from the history provided by parents. Recently, there has been a greater emphasis on the requirement that the symptoms must be present in more than one setting. This emphasis has been incorporated into the DSM-IV criteria, which requires that some impairment from symptoms be present in 2 or more settings and that there must be clear evidence of clinically significant impairment in social, academic or occupational function. How the core symptoms and impairment should be documented, particularly regarding the source of the information, is vague. The American Academy of Pediatrics tried to make the diagnosis more uniform in its ADHD diagnosis guidelines, 7 recommending that the diagnosis require information obtained from both the parent and classroom teacher regarding the core symptoms of ADHD, duration of symptoms, degree of functional impairment, and coexisting conditions. When there is agreement between informants, the diagnosis is simple. However, research has shown low correlation between parent ratings and teacher behavior ratings on questionnaires. 8,9
The vague DSM-IV requirement for two settings 6 does not spell out operationally how to measure the two-setting criterion, so procedures for gathering the information may vary with clinicians and research. A potential problem with the two-observer requirement is bias resulting from between-observer disagreement.
The present study addresses two questions: (1) Is interrater disagreement on questionnaires between parent and teacher a problem in diagnosing ADHD based on the DSM-IV criteria? (2) To what extent does interrater disagreement on questionnaires cause a reduction in the apparent prevalence of ADHD when the two-setting requirement is strictly applied?