Autism spectrum disorder (ASD), a complex and heterogeneous neurodevelopmental disorder characterized by deficits in social communication and interaction, as well as by restricted and/or repetitive patterns of behavior,1 affects 1% to 2% of US children2 and is increasing in prevalence.3 Signs of ASD are almost always evident before the age of 3 years,4 and many caregivers report developmental concerns during the second year or earlier.5 Nonetheless, many children with ASD do not come to attention before school age and fewer receive early interventions.6 Because clinical judgment alone is insufficient for the detection of ASD in young children,7 the use of standard screeners may contribute to the detection of ASD in young children. Despite growing ASD awareness, however, the use of developmental screeners remains low,8 and universal autism screening is an unrealized goal.9 Interestingly, the topic of autism screening has grown controversial, with the US Preventive Services Taskforce (USPSTF) concluding that the harms and benefits of universal autism screening cannot be determined,10 thereby contradicting the recommendations of the American Academy of Pediatrics (AAP) and the Society for Developmental and Behavioral Pediatrics, with regard to screening all children for ASD at 18 and 24 months.11,12
A brief parent-report questionnaire, the Psychological Development Questionnaire-1 (PDQ-1), was developed to represent the expression of social referencing and communication in young children and, inversely, to detect children at risk for ASD. The PDQ-1 was designed to stand as an efficient, single-phase tool, able to identify at-risk individuals, age 18 to 36 months, without a follow-up interview, as required by the Modified Checklist for Autism in Toddlers-Revised/with Follow-up (M-CHAT-R/F), the most utilized and studied ASD screener.13,14 Beginning with 23 questions derived from parent concern, clinical experience and research on the development of social interest, communication, and imitation,11,12,15 the PDQ-1 was reduced to 11 and, ultimately, 10 questions. The child's primary caregiver answers each of the PDQ-1 questions on a 3-point (0–2) scale, and the weighted answers are summed to yield a total score. The PDQ-1 can be self-administered in paper or digital form or administered verbally in <2 minutes and scored in <2 minutes. The aims of this study were to (1) evaluate the concordance of the PDQ-1 with reliable and valid ASD tests; (2) establish the PDQ-1 test-retest reliability; (3) define a screen positive threshold for the general population (Phase 1) and, subsequently, (4) describe the distribution of PDQ-1 scores by age, sex, and socioeconomic status in a large diverse population; and (5) assess the scale's psychometric properties, including sensitivity, specificity, and positive and negative predictive values, based on prospective administration.
Setting and Participants
In the pilot study (Phase 1, Fig. 1), 180 caregivers of children age 12 to 36 months, residing in the New York metropolitan region, participated through a written informed consent process. The psychological status of 42 autism spectrum disorder (ASD)-diagnosed children, recruited from an early intervention program, was evaluated with the Autism Behavior Checklist (ABC)16—a 57-item ASD questionnaire—and the Psychological Development Questionnaire-1 (PDQ-1) for the comparison of test findings from caregivers of 38 age-matched children with developmental delay (DD), but no indication of social deficit, recruited from a high risk infant follow-up program and 100 age-matched children without indication of developmental or neurological delay or disability; typically developing (TD) controls were recruited from 1 urban (n = 50) and 1 suburban (n = 50) pediatric practice. A randomly selected sample (20%) of ASD (n = 8), DD (n = 8), and TD (n = 20) caregivers completed the ABC and PDQ-1 twice, once at baseline and 7 days later, to determine short-term test/retest reliability. ABC and PDQ-1 scores by parents of ASD-diagnosed children (n = 42) were compared with independently administered, case-blind, Autism Diagnostic Interview, Revised (ADI-R) scores for this group. The ASD risk threshold for the new test was defined as the highest total PDQ-1 score observed in the ASD group.
Subsequently (Phase 2), prospective screening with the PDQ-1 was conducted through 16 cooperating pediatric practices and programs, representing a range of primary care types and sizes, serving urban and suburban communities in the New York metropolitan region. Participants were toddlers (age: 18–36 months), and their primary caregivers enrolled with written informed consent. Caregivers were approached for participation while waiting for any appointment type. Children with neurologic, genetic, and/or developmental disorders were excluded, as were those who received neonatal intensive care unit care or were born at ≤37 weeks gestational age. Children whose caregivers lacked sufficient language proficiency were excluded. We sought to enroll and screen every child meeting criteria who presented at cooperating practices and whose caregivers agreed to follow-up. The socioeconomic status of participants was represented by tertiles derived from a multifactorial, community-level index17—the District Factor Group ranking, representing the community of the cooperating practice.
The PDQ-1 is a 10-item questionnaire (Table 1) primarily reflecting the caregiver's evaluation of their child's level of communication and social orientation. Seven PDQ-1 questions bear on the child's level of social interest, imitation, and reciprocity; 2 reflect the child's oral vocabulary. One question inquires about the toddler's responsiveness to sound. The caregiver's answers are scored 0 to 2 points, according to a (predetermined) schema and summed to yield a total score. The maximum PDQ-1 score is 20. A PDQ-1 (cut-point) score ≤12 served as the (screen-positive) risk threshold, based on Phase 1 findings.
Research and cooperating clinical staff established practice-specific procedures for enrollment. Research staff or a designated member of the host practice provided study information, conducted enrollment, and administered the screener. Caregivers of children (18–36 months old) provided written informed consent during the child's visit and completed the PDQ-1. The questionnaire was self-administered by the caregiver, in the waiting area, or was administered verbally, at the parent's request, in a private office, by a researcher. Completed questionnaires were scored by researchers using a standard schema (Time 1: screening phase). Parents of screen-positive children were offered a comprehensive, expedited, (no-cost) developmental evaluation of the child. A subset (n = 10) of randomly selected (screen-negative) individuals (PDQ-1 > 12) was referred for and received evaluation. Evaluations were conducted by a team consisting of a psychologist and developmental pediatrician supervising graduate students and assistants who were research trained in and maintained high reliability on the evaluation measures. Standard tests included the ADI-R,18 a gold-standard semistructured interview for autism classification, and the Mullen Scales of Early Learning (MSEL),19 a standardized test of intellectual functioning in youngsters from 0 to 68 months. The ADI-R behavioral domain was excluded from the autism classificatory algorithm to improve sensitivity.20 Nonverbal intellectual functioning was based on MSEL fine motor and visual reception subtest age equivalents (nonverbal intelligence quotient = mean age equivalent on and Visual Reception/chronological age in months × 100). ASD diagnostic status was determined by an experienced, case blind, developmental pediatrician or clinical psychologist, integrating all available information, including direct observation, using clinical judgment informed by Diagnostic and Statistical Manual of Mental Disorders-IV-TR ASD criteria.21 Children diagnosed with autistic disorder or pervasive developmental disorder-not otherwise specified were defined as having ASD for analytic purposes. Children identified with ASD and/or scoring more than 1 SD below the mean on any MSEL subscale were referred, with the caregiver's permission, to early intervention and/or other providers. In the follow-up phase, when screened cases were aged between 48 and 60 months, researchers contacted caregivers for a standardized, brief interview (Table 2) reflecting the child's developmental status (Time 2: follow-up phase). Multiple strategies to maximize follow-up were implemented, including proactive contact, address, and phone checking and re-engagement through the cooperating practices. The scale was assessed at 3 levels: ≤12, ≤10, and ≤7. PDQ-1 psychometric properties were assessed at follow-up (Time 2, Fig. 2). All aspects of the pilot and the prospective screening project were conducted with institutional review board approval from the host institution. Data analyses were performed using SAS 9.3 statistical software.
In the pilot, Psychological Development Questionnaire-1 (PDQ-1) and Autism Behavior Checklist (ABC) scores varied significantly across autism spectrum disorder (ASD), developmental delay (DD), and typically developing groups (1-way analysis of variance, F test; p-value < 0.001), among children, age 12 to 36 months. Median PDQ-1 scores were 5.0, 16.0, and 17.2, respectively. PDQ-1 and ABC scores were associated (−0.869; p-value < 0.001). PDQ-1 and ABC total scores were consistent (0.997; p-value < 0.001; 0.998; p-value < 0.001) over a 7-day period. PDQ-1 total scores ranged from 0 to 20. The maximum PDQ-1 score recorded from the ASD group was 12. The concordance of the PDQ-1 (and ABC) with Autism Diagnostic Interview, Revised (ADI-R) total score was 100% at ASD diagnostic levels.
Subsequently, in the screening study, investigators requested participation from 2288 caregivers. A total of 325 individuals (14.4%) did not complete the PDQ-1, declined or met exclusionary criteria, yielding 2007 PDQ-1-screened children (18–36 months old). Postscreening, an additional 7 individuals declined further participation, yielding 2000 screened cases at Time 1 (Fig. 2). At follow-up (Time 2), 48 individuals declined participation or could not be contacted, yielding 1959 subjects who participated at Time 1 and Time 2 (Fig. 2). Boys and girls were equally likely to be screened (51%, 49%). Socioeconomic status (SES) distribution skewed low: 55% were from low-SES communities compared with 11% from middle- and 34% from high-SES communities. The mean age at screening was 27 months. Forty-one percent of toddlers were screened at 18 to 24-months, whereas 30% and 29% were screened at 25 to 30 months and 31 to 36 months, respectively. Slight but significant differences in total PDQ-1 scores were observed by sex and SES, and with increasing age, across the screened population (Table 3). Subsequent to a PDQ-1+ (screen-positive) score and consequent diagnostic evaluation, 1959 of 2007 (98%) screened cases had follow-up when children were 48 months or older. At that time, 26 caregivers reported that their child had been diagnosed with ASD, including the 22 identified at Time 1.
Characteristics of the screened children followed at age 4 to 5 years are provided in Table 4. Children diagnosed with ASD were predominantly male (73% vs 27%), skewed younger (50% younger than 24 months vs 41% in the total sample) and were from low SES communities (58%). Table 3 also profiles the 3 (screened) children with PDQ-1 scores ≤12 (screen-positive) at Time 1 but whose caregivers reported no ASD diagnosis at follow-up. These individuals (1 male and 2 females [screened from low SES communities]) received total scores ≤12 (screen positive), who were evaluated at Time 1 and subsequently found negative for ASD by clinical evaluation, and ADI-R interview results can be considered false positives (Table 3). All 3 (false positives) had Mullen Scales of Early Learning (MSEL) scores indicative of language impairment, and 1 individual had MSEL consistent with cognitive impairment. All 3 were evaluated between 18 and 24 months old, the period of lowest test sensitivity. At follow-up, the (false positive) caregivers reported that their children had received services for DD or concern but had not received an ASD diagnosis.
Psychological Development Questionnaire-1 (total) scores ranged from 5 to 20. Among the 1959 (screened and followed) children, 1639 (84%) had a total score between 17 and 20. Twenty-five individuals were screen positive (PDQ-1 score ≤12). Of these, 22 had ASD by clinical evaluation. Four children who scored between 13 and 17 during screening were reported to have an ASD diagnosis at the follow-up, and 3 children scoring ≤ 12 were ASD negative by clinical evaluation and by follow-up report. PDQ-1 total scores differentiated children with (later-confirmed) ASD (mean = 9.8) from those without apparent ASD (mean = 18.3).
The ASD prevalence estimate derived from the screened population was in the range of 13 per 1000. At follow-up, PDQ-1 positive predictive value (PPV) was 88%. Table 5 also presents the sensitivity, specificity, PPV, and negative predictive value (NPV) of the PDQ-1 at different levels of total score cutoff (≤12, ≤10, and ≤7). As expected, as PPV increased across the different cutoff points, test sensitivity decreased. At the ≤7 cutoff, the PPV was 100%, but sensitivity was reduced to 23%. The specificity and NPV of the PDQ-1 ranged from 99% to 100% at all cutoff points (Table 5).
Since population screeners aim to identify individuals with an undetected problem as early as possible, Table 5 shows the screening results by age and indicates highest sensitivity (100%) at the 31- to 36-month level and roughly comparable sensitivities at 18 to 24 months (85%) and 25 to 30 months old (71%). The overall PPV of the PDQ-1 was 88% with lower PPV (79%) at 18 to 24 months and higher PPV (100%) between 25 and 36 months old. Among those who screened positive by PDQ-1 at evaluation, almost all (96%) showed lower than expected (≥1 SD) MSEL expressive and/or receptive language subscale scores and a significant minority (28%) had an MSEL nonverbal intelligence quotient score ≥2 SDs below the mean, consistent with cognitive impairment (Table 6). At Time 2, follow-up interview indicated that parents of PDQ-1+ cases were more likely to report that their child had significant sleep problems and more frequent hospitalization or emergency department visits, over the preceding 12 to 18 months, than parents of children scoring high (≥12) on the PDQ-1 (Table 6).
The findings provide initial evidence in support of a brief autism spectrum disorder (ASD) screener based on parent report. In a large, diverse, low risk population, the instrument detected ASD in toddler-age children without previously suspected deficits and showed good sensitivity (85%) and high specificity (99%)—representing a positive predictive value (PPV) of 88%. The new instrument balanced the advantages of high sensitivity and PPV across the 18 to 36 months age range. The Psychological Development Questionnaire-1 (PDQ-1) can be administered quickly and scored without special training; universal follow-up is not required, and the screener has a clear-cut point (risk threshold). Prospective administration of the PDQ-1 through multiple primary care practices indicated relatively few false positives at the established risk threshold. Consistent with several US population–based studies of the aera,22,23 the ASD prevalence estimate generated from prospective screening with the PDQ-1 was in the range of 10 to 15 per 1000. Data from the pilot study showed that PDQ-1 scores were highly concordant with Autism Diagnostic Interview, Revised (ADI-R) scores at ASD diagnostic levels, thereby affirming the construct validity of the new instrument. In addition, PDQ-1 scores were consistent over the 7-day test-retest period, supporting the likelihood of short-term test stability. At evaluation, most of the PDQ-1+ cases had indications of depressed language functioning on the Mullen Scales of Early Learning (MSEL) and more than one-quarter had MSEL nonverbal intelligence quotient scores consistent with cognitive impairment.
Early detection of heterogeneous developmental disorders, like ASD, is challenging. No single behavioral or observational approach is likely to be simple and reliable across the range of affected individuals. Screening is only a brief assessment designed to identify individuals who should receive a more thorough evaluation. Consistent use of a reliable screening tool may be regarded as a complement to ongoing developmental surveillance and serve as a vehicle for heightened engagement with the caregiver. By systematically eliciting caregiver concern and information, the health provider increases the likelihood of detecting a disorder early and enhances the potential for positive outcomes.
The study has several strengths. It was designed and conducted as a prospective investigation, allowing for an accurate assessment of test sensitivity and specificity. The screened population was demographically diverse and spanned a significant age range (18–36 months), and test performance characteristics were evaluated at multiple ages. Half of all screened subjects were from low socioeconomic status communities, reflecting a plan to include a group that is underrepresented in research and is most likely to have delayed ASD diagnosis.24 The diagnostic evaluation was comprehensive and included an independently administered interview based on parent information (ADI-R) and a DSM-IV–guided (ASD+) diagnostic evaluation based on observation by and clinical judgment of an experienced clinician.
The study included a follow-up phase which allowed for the identification of cases that were screen-negative but later identified with ASD. The investigators achieved a high level of follow-up (98%), by use of multiple strategies, including proactive contact, phone and address checking, and procedures for systematic contact, including through the cooperating providers. The study detected children with ASD who had not come to attention and assisted them in receiving services. The sensitivity and PPV of the PDQ-1 were shown to be good, whereas specificity was excellent in comparison with the best available ASD screener, the M-CHAT-R/F,14 which has a PPV of 54%. PDQ-1 advantages include brevity, ease of administration and basis in parent-provided information, as well as high initial sensitivity, specificity, and PPV.
The study also has limitations. Screening and follow-up were conducted under informed consent conditions. The PDQ-1 operating characteristics may be different under real-life clinical practice conditions. The findings are preliminary and call for replication in large, unselected, and high risk populations. Additional studies are needed with large, unselected populations and, additionally, to investigate the PDQ-1 as a level 2 screener, that is, as a tool for use with individuals who have already known or suspected neurological or developmental conditions. Those children were purposefully excluded from the current study, which evaluated the PDQ-1 as a screener for the general (low risk) population. To be most useful, the PDQ-1 and future ASD screeners should define the extent to which they can discriminate children with ASD from peers with global delay or other (specific) developmental disorders. This study employed a standardized ASD diagnostic interview (ADI-R) and DSM-IV–guided clinical judgment for confirmation of ASD. Future studies could include additional or substitute diagnostic measures such as the Autism Diagnostic Observation Schedule, Toddler Version and the Childhood Autism Rating, Second Edition or other validated standard test.
Autism spectrum disorder diagnosis can only be accomplished through comprehensive evaluation by a professional. Effective screening is but the first step toward diagnosis. Additional study is needed to assess the usefulness of the PDQ-1 with high risk groups and to evaluate the instrument under everyday conditions, with unselected populations. The availability of valid and efficient screeners, like the PDQ-1, may enhance our ability to detect ASD in young children and to expand the number of youngsters receiving early interventions.
The authors thank all the children and caregivers who participated in the study. This study would not have been possible without the assistance of the cooperating pediatric practices and programs, to whom sincere gratitude is extended. Preliminary support for PDQ-1 development was provided by the Healthcare Foundation of New Jersey. This screening project was conducted with support from the Governor's Council for Medical Research and Treatment of Autism. The assistance of Dr. Susan Adubato, Dr. Michael Brimacombe, Vivian Lynn, Nadia Senmartin, Jennifer and Vivian Vidal, Connie Lai, Elise Aportela, Diane Van Driesen, Fonda Mojka, and Sabrina Durant, Vanessa Rodriguez, and David Holmes is gratefully acknowledged.
1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Arlington, VA: American Psychiatric Publishing; 2013.
2. Christensen DL, Baio J, Braun KV, et al. Prevalence and characteristics of autism spectrum disorder among children aged 8 years—Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2012. MMWR Surveill Summ. 2016;65:1–23.
3. Van Naarden Braun K, Christensen D, Doernberg N, et al. Trends in the prevalence of autism spectrum disorder, cerebral palsy, hearing loss, intellectual disability, and vision impairment, metropolitan atlanta, 1991–2010. PLoS One. 2015;10:e0124120.
4. Zwaigenbaum L, Bryson S, Lord C, et al. Clinical assessment and management of toddlers with suspected autism spectrum disorder: insights from studies of high-risk infants. Pediatrics. 2009;123:1383–1391.
5. Zuckerman KE, Lindly OJ, Sinche BK. Parental concerns, provider response, and timeliness of autism spectrum disorder diagnosis. J Pediatr. 2015;166:1431–1439.e1.
6. MMWR. Prevalence of autism spectrum disorders—Autism and Developmental Disabilities Monitoring Network, 14 sites, United States, 2008. MMWR Surveill Summ. 2012;61:1–19.
7. Glascoe FP, Dworkin PH. Obstacles to effective developmental surveillance: errors in clinical reasoning. J Dev Behav Pediatr. 1993;14:344–349.
8. Rice CE, Naarden Braun KV, Kogan MD, et al. Screening for developmental delays among young children—National Survey of Children's Health, United States, 2007. MMWR Suppl. 2014;63:27–35.
9. Pinto-Martin JA, Dunkle M, Earls M, et al. Developmental stages of developmental screening: steps to implementation of a successful program. Am J Public Health. 2005;95:1928–1932.
10. Siu AL, Force USPST, Bibbins-Domingo K, et al. Screening for autism spectrum disorder in young children: US Preventive Services Task Force Recommendation Statement. JAMA. 2016;315:691–696.
11. Bates E. Comprehension and production in early language development (see comment). Monogr Soc Res Child Dev. 1993;58:222–242; discussion 243–252.
12. Bruner JS. Child's Talk: Learning to Use Language, NY: W.W.Norton, 1985. (ISBN13:9780393953459).
13. Chlebowski C, Robins DL, Barton ML, et al. Large-scale use of the modified checklist for autism in low-risk toddlers. Pediatrics. 2013;131:e1121–e1127.
14. Robins DL, Casagrande K, Barton M, et al. Validation of the modified checklist for autism in toddlers, revised with follow-up (M-CHAT-R/F). Pediatrics. 2014;133:37–45.
15. Tomasello M. Joint attention as social cognition. In: Dunham ICMJ, ed. Joint Attention: Its Origins and Role in Development. Hillsdale, NJ: Lawrence Erlbaum Associates; 1995:103–130.
16. Krug DA, Arick J, Almond P. Behavior checklist for identifying severely handicapped individuals with high levels of autistic behavior. J Child Psychol Psychiatry. 1980;21:221–229.
17. New Jersey Department of Education. District Factor Groups for School Districts. Trenton, NJ: New Jersey Department of Education; 1975.
18. Lord C, Rutter M, Le Couteur A. Autism Diagnostic Interview-Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J Autism Dev Disord. 1994;24:659–685.
19. Mullen EM. Mullen scales of early learning: AGS edition. J Psychoeduc Assess. 1995;18:381–389.
20. Wiggins LD, Robins DL. Brief report: excluding the ADI-R behavioral domain improves diagnostic agreement in toddlers. J Autism Dev Disord. 2008;38:972–976.
21. American Psychiatric Association. Diagnostic Criteria from DSM-IV-TR. Washington, DC: American Psychiatric Association; 2000.
22. Autism, Developmental Disabilities Monitoring Network Surveillance Year 2006 Principal Investigators, Centers for Disease Control and Prevention. Prevalence of autism spectrum disorders—Autism and Developmental Disabilities Monitoring Network, United States, 2006. MMWR Surveill Summ. 2009;58:1–20.
23. Zahorodny W, Shenouda J, Howell S, et al. Increasing autism prevalence in metropolitan New Jersey. Autism. 2014;18:117–126.
24. Fountain C, King MD, Bearman PS. Age of diagnosis for autism: individual and community factors across 10 birth cohorts. J Epidemiol Community Health. 2011;65:503–510.