Secondary Logo

Journal Logo

Review Article

Building Capacity for Community Pediatric Autism Diagnosis: A Systemic Review of Physician Training Programs

Guan, Xiaoning MD; Zwaigenbaum, Lonnie MD; Sonnenberg, Lyn K. MD

Author Information
Journal of Developmental & Behavioral Pediatrics: January 2022 - Volume 43 - Issue 1 - p 44-54
doi: 10.1097/DBP.0000000000001042

Autism spectrum disorder (ASD) is a complex developmental condition, characterized by persistent challenges in social interaction and communication and by restricted and repetitive behaviors.1 The estimated prevalence of ASD is between 1% and 1.5%, with relative consistency across studies internationally, making ASD one of the more common developmental disabilities.2 Although evidence has highlighted that ASD can be reliably diagnosed by age 2 years,3 the average age at diagnosis continues to be greater than 4 years.4,5 This lag in diagnosis is even more pronounced for children from racial and ethnic minority backgrounds. Delays in ASD diagnosis lead to a missed window of opportunity for early interventions. Numerous studies have demonstrated the benefit of early intervention, such as improvements in social and communication skills,6 cognitive abilities,7–9 verbal abilities,6 adaptive behavior,9,10 and potential to reduce societal costs.11 Therefore, to improve the long-term outcomes for patients with ASD, it is imperative to address the impediments to timely diagnosis.

Owing to its complex nature, most ASD diagnostic evaluations are conducted by multidisciplinary teams in tertiary centers. This presents a bottleneck for many patients because wait times commonly exceed a year.12 There have been numerous innovative approaches aimed at enhancing early detection of ASD, such as initiatives to increase ASD screening, telehealth-supported models, and novel assessment models.13,14 Recent disruptions to standard care related to COVID-19 encouraged prioritization of strategies aimed at increasing overall diagnostic capacity and accessibility. One of the most promising approaches to improving access is training primary care providers (PCPs; e.g., family physicians or community pediatricians) to provide comprehensive assessments. Encouraging community-based assessments for less complex presentations of ASD is supported by the most recent guidelines from the American Academy of Pediatrics and Canadian Pediatric Society.12,15 Although recent studies have evaluated whether PCPs, with appropriate training, could perform as reliable and valid an ASD diagnostic assessment as a specialist team, it remains unclear whether adding community assessment pathways will reduce wait times and improve access to early interventions. To address these questions and guide future training and evaluation efforts, we conducted a systematic review of all published studies of ASD diagnostic training for PCPs, with the aim of summarizing and critically assessing (1) the training models and how they were implemented and (2) the reported outcomes.

METHODS

An a priori review protocol was developed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement and was registered with PROSPERO. Ethics approval was not required for this review.16

Search Procedures

A systematic search procedure according to the PRISMA guidelines was used to identify studies to include in this review. First, we searched in PROSPERO, a registry for ongoing systematic reviews, to ensure there was no ongoing review of the same topic. Searches were then conducted in 6 electronic databases including Ovid MEDLINE, Embase, PsycINFO, CINAHL, Education Resources Information Center, and Scopus. A gray literature search was also conducted in HSRProj, OAIster database, and Google Scholar to capture conferences, presentations, or other unpublished data that meet our inclusion criteria. The reference lists of included articles were also reviewed, and additional relevant studies were included. We also reviewed all publications by the first authors and corresponding authors of each included study to identify other potentially relevant articles. Therefore, this review includes all peer-reviewed articles and gray literature published in English from all publication years up to and including March 27, 2020. Searches in each database were conducted using search terms related to autism, diagnosis, primary care, and training. The search terms used for all databases were very similar and presented in Supplemental Digital Content 1, https://links.lww.com/JDBP/A339.

Inclusion Criteria

To be included in this review, studies had to meet 3 criteria. First, the study had to include at least 1 participant who was a community-based primary care provider (PCP) who provided care to the pediatric population, such as family physicians or general pediatricians. Second, the study had to describe a community-based training model for autism spectrum disorder (ASD) diagnostic assessment in the pediatric population. Finally, the study needed to report at least 1 outcome related to an evaluation of the implemented training model, such as accuracy of participants' diagnosis for ASD, improvement in access to diagnosis, increase in confidence and knowledge, or family satisfaction. Review papers and commentaries were excluded from this systematic review. Studies that described a training model but did not evaluate any outcome measures were also excluded.

The first author independently reviewed titles and abstracts to identify potentially relevant studies. Once identified, full-text review was completed independently by the first and second authors to determine eligibility based on the inclusion and exclusion criteria. Disagreements were resolved through discussion with the third author.

Data Extraction

Each included study was summarized with the following data extracted: (1) participant characteristics, (2) training program features, and (3) outcome measures and findings. These data are summarized in Tables 1–3.

Table 1. - Characteristics of Included Studies
Study (First Author, Year) Study Design Country Settings Participants (Training Recipients) Number of Participants Number of Patients Assessed by PCPs
Warren, 2009 NRT United States Community pediatric clinics in underserved geographic areas General pediatricians 5 25
McClure, 2010 NRT Scotland Community clinics in the health region of Lomond and Argyll General pediatricians, SLP, and psychologists 16 39
Swanson, 2013 NRT United States Community clinics in different geographic regions General pediatricians 27 14
Harrison, 2017 Pre-post United States Behavioral/developmental access clinic within a tertiary care children's hospital General pediatrician 1 63
Mazurek, 2018 Pre-post United States Community clinics in different geographic regions General pediatrician, family physician, and nurse practitioner 18 47
Ahlers, 2019 NRT United States University developmental assessment clinics associated with the University of UT General pediatricians with advanced training in ASD Not reported 91
ASD, autism spectrum disorder; NRT, nonrandomized trial study design; PCPs, primary care providers; SLP, speech language pathologist.

Table 2. - Training Programs Summary
Study (First Author, Year) Warren, 2009 McClure, 2010 Swanson, 2014 Harrison, 2017 Mazurek, 2018 Ahlers, 2019
Format Interactive workshops
Diagnostic assessment and feedback
Didactic teaching
Clinical mentoring
Didactic workshops
Case-based evaluations
Ongoing training
Shadowing
Self-study
Case-based discussion as needed
Didactic lectures
Case-based learning
Ongoing training
Workshops
Online training
Ongoing supervision
Initial training 2-d workshop 5-d course 2-d workshop Not reported 1.5-d workshop 2-d workshop
Ongoing mentorship duration 4–6 practice assessments 10 wk of mentoring 3.5 yr of ongoing training 6 mo Bimonthly 90-min meeting for 12 mo NA
Training material presented to PCPs MCHAT
STAT
DSM IV criteria
History and interview
Differential diagnosis
Explaining results
Coding for reimbursement
ADOS
ICD-10 criteria
History and interview
Explaining results
Charting
MCHAT
STAT
History and interview
DSM criteria
Explaining results
Code for reimbursement
Early childhood development
Developmental disorders
ASD overview
STAT
History and interview
Diagnostic criteria
Differential diagnosis
Explaining results
Management and resources
ADOS-2
STAT
AMSE
ADOS, Autism Diagnostic Observation Schedule; AMSE, Autism Mental Status Exam; ASD, autism spectrum disorder; MCHAT, Modified Checklist for Autism in Toddlers; NA, not applicable; STAT, Screening Tool for Autism in Toddlers.

Table 3. - Outcome Evaluation: Increased Access to ASD Diagnosis and Accuracy of Diagnosis
Increased Access to ASD Diagnosis Accuracy of ASD Diagnosis
Method Control Sample Size Outcome Method Control Sample Size Diagnostic Decision Expert Team Diagnosis Accuracy of Diagnosis
Warren, 2009 NR NR NR NR Pediatric providers referred selected cases for further evaluation. Initial result was blinded for assessment Autism diagnostic clinic 21 patients ASD vs not ASD: Rank certainty of diagnosis on a 5-point Likert scale (1 = highly uncertain to 5 = highly certain) 71% ASD (n = 15)
29% non-ASD (n = 6)
71% agreement overall. 74% (n = 19) when diagnosed as ASD and 50% when not ASD (n = 2). Agreement in the 4 pediatricians ranged from 57% to 100%
McClure, 2010 Retrospective chart review AAT 38 patients Wait time for local teams was 13 wk compared with 36 wk for specialists Traditional team assessed video-recorded ASD histories conducted by trained providers while blinded to result AAT 38 patients ASD vs not ASD: Specific diagnosis within the spectrum 41% ASD (n = 16)
59% non-ASD (n = 22)
87% (33/38) full agreement on diagnosis; 92% agreement as to whether the child was on the spectrum
Swanson, 2014 Self-reported surveys NA 22 PCP participants Number of children diagnosed within practice by providers involved in this training increased by 85%. Pediatric providers referred selected cases for further evaluation. Initial result was blinded for assessment Comprehensive psychological assessments 14 patients ASD vs not ASD: Rank certainty on a 5-point Likert scale (1 = highly uncertain to 5 = highly certain) 57% ASD (n = 8)
43% non-ASD (n = 6)
86% diagnostic agreement in forced-choice classifications and 93% agreement when uncertain cases were counted as agreement
Harrison, 2017 Retrospective chart review Developmental pediatricians 63 patients Wait time to see a developmental pediatrician was 327 d vs 159 d to see a GP. Wait time was 11 d for new referrals to the access clinic. NR NR NR NR NR NR
Mazurek, 2018 Self-reported surveys NA 15 PCP participants 80% of PCPs reported increase in the number of children with autism on their caseload. 73% reported accepting referrals for ASD assessments at the end of the program. Average of 173 miles of travel avoided for families NR NR NR NR NR NR
Ahlers, 2019 Retrospective chart review Traditional assessment model 143 patients in traditional and 101 patients in alternative models (91 seen by general pediatricians) Time to diagnosis was shorter for all children evaluated by trained pediatricians compared with the traditional assessment model (85 vs 152 d, p < 0.001) Pediatric providers referred selected cases for further evaluation. Initial result was blinded for assessment Traditional assessment model 18 patients ASD vs not ASD: Rank certainty on a 4-point Likert scale (very likely, somewhat likely, somewhat unlikely, or very unlikely) 78% ASD (n = 14)
22% non-ASD (n = 4)
93% diagnostic agreement among pediatricians and psychologists when ASD was ruled in (n = 14) and 100% when ASD was ruled out (n = 4)
AAT, ASD assessment team; ASD, autism spectrum disorder; GP, general practitioner; NA, not applicable; NR, not reported; PCPs, primary care providers.

Risk of Bias Analysis

The methodological quality of the included studies was assessed using the National Institutes of Health Quality Assessment Tool for Before-After (Pre-Post) Study Checklist (NIH Checklist) and the Cochrane risk assessment tool.17,18 The NIH Checklist was used for the pre-post study designs, whereas the Cochrane risk assessment was used for the nonrandomized trial studies. The NIH Checklist includes 12 questions and assesses quality based on study objective, population, intervention, outcome measures, and statistical analysis. The Cochrane risk assessment tool includes evaluation of 7 different areas of bias. The risk of bias analysis was conducted independently by 2 of the coauthors, and discrepancies were resolved by discussion among all authors.

Intervention fidelity was assessed for each study using the Treatment Fidelity Assessment Grid.19 Treatment fidelity is a measure of the reliability and validity of the interventions relative to how it was created, implemented, and received. The Treatment Fidelity Assessment Grid assesses the fidelity for each study in 5 domains: study design, provider training, intervention implementation, intervention receipt, and intervention enactment. Two of the coauthors completed the assessment independently, and the results were discussed among all authors. For the purpose of this review, the ASD diagnostic training for PCPs was considered an intervention. Thus, ‟provider training” refers to training autism experts on how to train the PCPs in ASD diagnostic assessment to ensure autism experts will provide standardized training for PCPs. ‟Intervention implementation” refers to the delivery of the training. ‟Intervention receipt” refers to how well the PCPs understood the training material, and ‟intervention enactment” refers to how well the trained providers were able to apply the diagnostic assessment methods with their patients.

RESULTS

Search Results

The search process identified 2138 studies; 428 duplicates were removed, leaving 1710 unique studies. The identified articles were imported to Covidence,20 an online screening and data extraction tool used for systematic reviews. Titles and abstracts were screened in Covidence by the first author to identify studies that would potentially meet the inclusion criteria; 29 papers passed the primary screen. A further reference review, based on the name of the first author and the corresponding author of each included study, identified 9 additional papers. The full text of those that passed the primary screen and the studies found from the reference review were independently reviewed by the first 2 authors to identify studies for final inclusion. The first author identified 7 studies for inclusion, whereas the second author identified 6 studies. Discrepancies were resolved by the last author after independent review, with final selection of 6 studies for the review (Fig. 1).

F1
Figure 1.:
Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram of study selection.

Study Characteristics

Study characteristics are summarized in Table 1. Two studies used a pre-post study design, whereas the other 4 studies were nonrandomized trials that included a comparison group. Five of 6 studies were conducted in the United States, and 1 in Scotland. There was considerable heterogeneity among studies regarding the training provided and the program evaluation process.

Quality Assessment

The National Institutes of Health and Cochrane risk assessment protocols are summarized in Supplemental Digital Content 2, https://links.lww.com/JDBP/A339. Overall, the quality of the studies included in this review was rated fair to poor with these assessments. None of the studies included a randomized design or blinded outcome assessment, and participating trainees and patients were small in number. The lack of reliable outcome assessment was also common among the studies.

A summary of the fidelity assessment is presented in Supplemental Digital Content 3, https://links.lww.com/JDBP/A339. All studies described their general study design in detail, and 5 of 6 studies provided specific details regarding how the training was implemented. However, none of the studies took steps to ensure fidelity, such as developing a protocol to assess whether the trainers delivered standardized material consistently across different sessions. In addition, the studies did not describe how the expert teams were taught to conduct the training. Four of 6 studies reported steps to ensure intervention receipt, such as requiring trained primary pediatricians to send in videotaped practice administration to demonstrate reliability.

Intervention Description

The included studies are summarized below with a description of participants trained and the diagnostic training program.

Participating Clinicians

The number of participants in the training programs ranged from 1 to 27 primary care providers (PCPs). Most studies trained the PCPs to conduct diagnostic assessments on their own, whereas 2 studies (Mazurek et al., 2018; McClure et al., 2010) trained the providers to make diagnosis in a team setting.21,22 Only 2 studies (Mazurek et al., 2018; Swanson et al., 2014) reported specific participant demographics.21,23 In these 2 studies, most participating clinicians were female (72%), White (83%), and general pediatricians (74%–78%). Detailed participants' demographics were not reported in the other studies. Four of the 6 studies reported on participants' level of training.21,23–25 Only 1 study (Swanson et al.,23 2014) reported years of clinical experience, which ranged from 2 to 39 years (average 17.6 years). In 2 studies (Mazurek et al., 2018 and Ahlers et al., 2019), most participants had previous formal autism spectrum disorder (ASD) training through their residency programs, webinars, or previous experience working in ASD clinics.21,24 By contrast, the training provided by Harrison et al. (2017) was targeted at general pediatricians with no formal training or experience in ASD.

Participating Patients

The number of patients assessed by trained PCPs through these training models ranged from 14 to 91 children (Table 1). Five of 6 studies only assessed patients younger than 5 years, whereas the sixth study (McClure et al.,22 2010) assessed patients with the median age 8 years 9 months and had only 3 patients younger than 5 years. In the 4 studies that reported patient demographics, most (>70%) of the patients were male. Only 1 study reported the race/ethnicity of the patients, with 75% of them being White.24 Most studies did not report how patients were referred or selected; only 2 studies mentioned that patients were young children with a query of ASD within the PCPs' own practices or had a positive autism screen.21,26 None of the studies provided details on what additional information was available to the PCPs, such as the patient's functional level or previous language assessments.

Training Program

Training Format and Duration

There was considerable variability among the training programs reported in the included studies presented in Table 2. The majority (5 of 6) included didactic lectures or workshops as part of the training. Four of 6 studies also included interactive case-based learning with individualized feedback through videoconferencing or videotaped vignettes. Only 1 training program did not include any formal didactic teaching and was mainly based on ‟shadowing” (i.e., observing) developmental pediatricians with teaching provided as needed (Harrison et al.,25 2017). The didactic training portion ranged from 1.5 days to 5 days (median = 2 days). All training programs offered ongoing mentorship in different forms ranging from 10 weeks to 3.5 years. This included observed independent ASD assessment with personalized feedbacks, bimonthly clinics with expert teams for case discussions, or informal mentorship and consultation for challenging presentations.

Training Content

Four of 6 programs included training in administering a standardized ASD screening tool, either the Modified Checklist for Autism in Toddlers or the Screening Tool for Autism in Toddlers and Young Children (STAT).21,23,24,26 Two of the 6 programs provided training in administering and scoring the Autism Diagnostic Observation Schedule, an interactive assessment of autism signs.22,24 Two of the 4 studies required providers to submit video-recorded sessions of themselves interacting with a child to assess their reliability in administering the STAT.21,23 Four of 6 programs provided training on how to establish a diagnosis and how to communicate the diagnosis to patients and families.21,23,26 The diagnostic criteria differed between different training programs; 3 programs referred to the DSM criteria,21,23,26 and 1 program referred to the International Classification of Diseases criteria.22 Two of the 6 programs specified training for coding and billing for reimbursement.23,26 Only 1 of the 6 training programs provided training on resources for patients and families.21

Assessment Setting

The clinical settings of the training programs also varied. Four training programs aimed to implement autism assessment in community pediatrician clinics across different geographical areas, and 2 training programs focused on training PCPs to work in newly created diagnostic clinics associated with a tertiary hospital (Ahlers et al., 2019; Harrison et al., 2017).24,25

Diagnostic Assessment

Assessments in most studies included interpretation of a screening instrument, diagnostic interview of caregivers, direct observation of patient, and delivery of the diagnostic decision. One of 6 training programs incorporated a structured diagnostic interview, whereas the others provided more general guidance to trainees on how to probe for ASD-related symptoms in their history-taking.22 Three studies specified assessment times, ranging from 6026 to 9025 to 120 minutes,24 whereas the other studies did not estimate times.

Diagnostic Decision Making

Trained PCPs from all studies were asked to make diagnostic decisions on whether a child had ASD. In 4 of 6 training programs, providers were asked to make an ASD diagnosis independently after evaluation. Three of these 4 programs had providers rate the certainty of their diagnosis on a Likert scale.23,24,26 In the other 2 programs, the diagnostic decisions were made through discussion with a team.21,22 The members on the team in 1 study included a pediatrician specializing in development and behavior, a clinical child psychologist, a child and adolescent psychiatrist, a social worker, a dietitian, a parent of a child with autism, and other participating PCPs, in the setting of bimonthly training sessions for 12 months.21 The members on the team in the other study included professionals from a range of disciplines, such as psychology, occupational therapy, speech and language therapy, and social work, who were also being trained to participate in the evaluation process.22

OUTCOMES

Among the 6 studies evaluating models aimed at training primary care providers (PCPs) in autism spectrum disorder (ASD) diagnostic assessment, the most common outcomes evaluated were increased access to diagnosis, based on reduced wait times, and accuracy of diagnosis, comparing diagnostic decisions with blinded ASD expert teams. Five studies reported that training PCPs led to improved access in ASD diagnostic assessment, with reduction in wait time by around 50%. There was also a high degree of diagnostic agreement (71%–100%) between trained PCPs and traditional expert teams, as evaluated by 4 studies.

Increased Access to Autism Spectrum Disorder Diagnosis

Measures of access to diagnosis were reported by 5 studies (Table 3). Two studies evaluated the change in wait time before and after implementing the training program.22,25 These 2 studies compared the wait time to see an ASD specialist versus the wait time to seeing a trained PCP. Another study evaluated the change in time to diagnosis by measuring the number of days from receipt of the parent-completed intake package to the date the final diagnostic International Classification of Diseases code was entered.24 All 3 studies reported the wait time to see a trained PCP was approximately 50% the length of the wait time to see the traditional assessment team. Two additional studies reported survey findings of the subjective increase in the number of ASD assessments by the trained PCPs.21,23 In 1 study, 73% of the trained PCPs reported they were accepting referrals for ASD diagnostic evaluation at the end of the program (Mazurek et al.,21 2018). Another study reported the number of children diagnosed within the practice of these trained providers increased by 85% (Swanson et al.,23 2014). One study did not evaluate improved access to autism diagnosis as an outcome.26

Accuracy of Diagnosis

Accuracy of diagnosis was reported by 4 of 6 studies (Table 2). Agreement of clinical diagnosis by PCPs was assessed relative to the local multidisciplinary expert teams who provided the training. Expert teams independently assessed either the video recordings of the diagnostic interview or conducted independent in-person evaluations, blinded to the decisions made by the trained PCPs.22,24,26

There was heterogeneity among the 4 papers on how the diagnostic decision was recorded. Warren et al. (2009) and Swanson et al. (2014) asked PCPs to make forced-choice decisions of ASD or non-ASD and rank their certainty of diagnoses on a Likert scale. Ahlers et al. (2019) asked participants to rank their certainty of diagnosis on a Likert scale. However, Ahler et al only reported the accuracy of diagnosis for which participating PCPs had a high level of certainty, likely inflating these rates and making them less comparable with those of studies that reported accuracy in the full sample.

A small number of patients were included in studies that reported accuracy (n = 14–38). Diagnostic agreement between the trained providers and the expert teams ranged between 74% and 100%. Notably, each of these studies reported absolute agreement, without adjustment for chance agreement; the rates of ASD diagnosis by the expert teams ranged from 41% to 78%. The study (Warren et al., 2009) that reported the lowest mean absolute diagnostic agreement (74%) limited the diagnostic evaluation to 60 minutes and required the trained providers make a binary choice of whether the child had ASD. In that study, diagnostic disagreements often involved presentations with other clinically significant developmental concerns.

McClure et al. (2010) included the most patients (n = 38), but only 3 patients were younger than 5 years. PCPs reported their impression as to whether the child had ASD and stated the specific DSM IV diagnostic subtype. Full agreement between the trained PCP and the expert teams was 92% for the presence/absence of ASD and 87% for specific subtypes. Ahlers et al. (2019) reported the highest diagnostic agreement of 93% for children who were diagnosed as having ASD and 100% when children were diagnosed as not having ASD. Among the 4 studies that evaluated the accuracy of diagnosis, not all patients assessed by the PCPs were included in the analyses. The criteria and reasoning behind the patient selections were not provided or standardized in 3 of the 4 studies. The nonrandomized design for these studies increased the risk of selection bias. In these 4 studies, the expert teams were all blinded to the diagnosis made by the trained PCPs before the evaluation to ensure fidelity. However, the effectiveness of the blinding process was not assessed. In 3 of the 4 studies, it was unclear whether patients were informed of the PCP diagnosis before receiving the expert team's assessment.

Two of the 6 papers reported the complexity of behavioral presentation: 1 stratified the patients into 3 subgroups and a second excluded patients with significant medical complexity.23,27 No study reported the level of diagnostic agreement by complexity.

Provider Practice and Perception

Two of the 6 studies included self-reported changes in provider practice and perceptions.21,23 The first found that the providers' comfort level for discussing ASD diagnosis increased. They also noted a large, statistically significant shift in reports of practice behavior, with 68% of providers reporting that they were more likely to conduct ASD assessments within their practice at the end of the training program.23 The second study reported an increased use of autism-specific screening tools (from 80% to 100%) pretraining to post-training.21 In this study, all providers reported improvements in relationships and interactions with patients with ASD and their families, and 93% reported that the program had a positive impact on their community.21

Family Perspectives

Only 1 of the 6 studies explored family perspectives as an outcome.24 In that study, satisfaction surveys were completed by 14 parents of children evaluated by a trained PCP and 35 parents evaluated by the traditional expert team. Of these 14 parents, 90% of them were either “extremely satisfied” or “moderately satisfied” during the evaluation process. There were no significant differences in respect to parents' perception of shared decision making or family-centered care between the traditional model and the trained PCP model.

DISCUSSION

This review identified 6 studies evaluating training models aimed at preparing primary care providers (PCPs) to assess and provide autism spectrum disorder (ASD) diagnoses. Five of 6 studies demonstrated increased access to ASD diagnoses, with approximately a 50% reduction in wait times. Four studies assessed accuracy of diagnosis by comparing diagnostic decision with blinded ASD expert teams, and the degree of diagnostic agreement was as high as 71% to 100%. These studies have laid the foundational framework for ongoing research. As ASD assessment quality can be considered as a multidimensional construct, there are other important outcomes that should be further evaluated, such as family satisfaction and earlier access to interventions. For many families, satisfaction with the diagnostic experience was associated with clarity of communication and practical information around resources and services available and negatively associated with more professionals consulted and longer time to diagnosis.11,27,28 Diagnostic assessment training could include providing PCPs with practical guidance around navigating the service system and practical resources for patients and families, which was provided during training by 1 of the 6 studies. It is also important that future studies evaluate whether adding a PCP assessment stream to existing diagnostic programs leads to earlier access to intervention services.

Diagnostic wait times for ASD have reached a critical level.29 The development of innovative approaches to address challenges will be of great value to families and systems of care alike.14 There are several barriers to developing and evaluating PCPs' training programs highlighted in the reviewed studies. First, ASD assessment by traditional multidisciplinary teams is very resource and time intensive. Having patients go through similar diagnostic processes twice, as part of the accuracy evaluation, would require significant resource investments and dedication from patient families. This may limit participation. One study video recorded the PCP's interview and assessments, enabling expert teams to form independent diagnostic impressions based on these recordings.22 This approach could reduce the required time commitment and travel for families for confirmatory assessments. Second, training programs are time intensive for PCPs. After training, community pediatricians may not have the time, resources, or infrastructure to incorporate the assessments into their clinical practices or to participate in evaluative studies. In addition, higher-volume and lower-complexity clinical activities may be better reimbursed for fee-for-service primary care physicians; thus, enhanced reimbursement and/or alternate payment models may be needed for sustainability of a PCP-based diagnostic stream. PCPs in these included studies may have had special interest in ASD and/or protected time for ASD assessments and may not be representative of all PCPs. Targeting “local champions” in the care of patients with ASD may be essential to building community capacity. Incorporating a task-sharing approach in primary care, whereby other nonphysician primary care workers (e.g., case managers, community health workers, or therapists) support PCPs’ diagnostic assessments, is another strategy. There are also recent feasibility data on incorporating ASD diagnostic assessment into pediatric residency training, which may also enhance community capacity.30 Furthermore, eligibility for supports and services may depend on including measures (e.g., adaptive skills or other relevant functional domains) that may be less accessible to PCPs than to specialized teams. Mitigating strategies are possible. First, PCPs could work in partnership with other community professionals to complete such measures. These assessments could provide the developmental context to inform differential diagnoses, consistent with current American Academy of Pediatrics practice recommendations.15 Second, leaders from ASD centers could work with decision/policy makers to ensure that requirements for specific measures are aligned with what is essential to inform care and not to inadvertently create barriers to diagnostic access. Ensuring the sustainability of improved access to ASD diagnosis related to PCP training is important. To date, there are no data to indicate to what degree trained PCPs maintain an active role in ASD assessment beyond the initial study period. Reimbursement models (as noted above) and other potential incentives, such as ongoing continuing education/case discussion opportunities, may be needed. Indeed, maintaining a longitudinal relationship to the specialized assessment team would be beneficial to provide PCPs with ongoing support as the clinical landscape and evidence base changes over time. These points argue for consideration of system-level issues and the need for longer follow-up when assessing strategies to increase access to ASD diagnoses.

This review identified solutions, as well as gaps, that can guide and inform future training model evaluation. Future evaluation efforts should aim to include patient and PCP demographics (e.g., age, sex, race, and ethnicity), how patients are referred, additional training and practice settings of PCPs, details of training material and training delivery, and rigorous and comprehensive assessments of outcomes. These outcomes should include family perspectives and timing of access to diagnostic and intervention services. Accuracy of diagnostic assessments should include a traditional expert consultant or team to cross-check the reliability and accuracy of diagnosis by the trained PCPs, the analysis of which should be corrected for chance agreement. Strict methodological standards should be applied to prevent overestimation or underestimation of the diagnostic accuracy. Patients selected for this process should be determined before assessment by either group to reduce selection bias. Although the reviewed training models were generally associated with high levels of concordance between diagnostic decision making by the trained PCPs and local specialty teams, clinical information available to the PCP about the children they were assessing did not generally include language, intellectual, and/or adaptive skill data. Information about the developmental level of the child is needed to inform clinical interpretation of behavioral symptoms, as recommended by clinical practice parameters internationally.12 Future training efforts should ensure this aspect of differential diagnosis is understood by participating PCPs and ensure that assessment data including developmental level are made available. Finally, conducting future studies in settings that serve diverse patient populations such as race/ethnicity and rural/urban settings would be invaluable.

Comparability across studies would also be enhanced by incorporating standard definitions of access (e.g., wait time from referral to initial assessment, which could be compared across community and specialized teams) and total number of children assessed. A retrospective chart review would provide more objective data than self-reported surveys for these outcome measures. Future studies are needed to evaluate improved access to diagnosis to demonstrate that adding a community assessment pathway is not moving the problem downstream, delaying access to intervention. Family experiences with trained PCPs and the expert teams should also be compared. A comprehensive framework for evaluating effectiveness should include a family-centered approach, continuity of care, and expertise pertaining to local resources. This discussion raises broader questions about how to evaluate the quality of diagnostic assessments, regardless of format and setting. A framework is needed that can be applied across the continuum, from traditional multidisciplinary center-based assessments to solo discipline, ‟virtual team” community assessments, and telehealth-based assessments. We should be working toward overall ASD assessment strategies that might involve multiple pathways within a single community.12

There are several limitations to this systematic review. We did not include studies that only targeted community-based psychologists trained as diagnosticians, which could be an additional strategy to increasing overall assessment capacity. The number of studies included is small, which made it difficult to assess publication bias. Study methodology for all the studies included was of low quality (e.g., before-after designs and nonrandomized trials), and none of the studies took steps to ensure fidelity. The diversity of training strategies and outcome assessments limited the ability of the review to compare programs and combine data across studies. The studies reviewed spanned the transition from DSM-IV to DSM-V, further complicating the comparison. As such, there is insufficient evidence from this review to draw conclusions about the effects of PCP training in the ASD diagnostic assessment or to recommend one approach over another.

In conclusion, training PCPs in ASD diagnostic assessment can lead to high levels of agreement with expert teams and potentially add capacity by increasing access. However, current data are limited and difficult to synthesize because of small sample sizes, variable quality of study methods, and heterogenous study design and outcome evaluation. More rigorous training model evaluations addressing diagnostic accuracy, reduction in wait time, family satisfaction, and longer-term outcomes are needed. The potential to apply these training models on a larger scale holds great promise in improving access to ASD diagnosis, with the advantage of embedding diagnosis within the medical home for some children with ASD. This review also highlights important questions about which outcomes to consider when evaluating the quality of ASD diagnostic assessment, applied across the continuum of approaches, from traditional center-based teams to novel community-based models. Ultimately, evidence-based training processes could promote further uptake and increase capacity for high-quality community-based assessments. In combination with tertiary assessments, for patients with greater complexity, these approaches could move us forward toward a more flexible and integrated system of care.

REFERENCES

1. Penner M, Anagnostou E, Andoni LY, et al. Systematic review of clinical guidance documents for autism spectrum disorder diagnostic assessment in select regions. Autism. 2018;22:517–527.
2. Baxter AJ, Brugha TS, Erskine HE, et al. The epidemiology and global burden of autism spectrum disorders. Psychol Med. 2015;45:601–613.
3. Kleinman JM, Ventola PE, Pandey J, et al. Diagnostic stability in very young children with autism spectrum disorders. J Autism Dev Disord. 2008;38:606–615.
4. Baio J, Wiggins L, Christensen DL, et al. Prevalence of autism spectrum disorder among children aged 8 years–autism and developmental disabilities monitoring network, 11 sites, United States, 2014. MMWR Surveill Summ. 2018;67:1–23.
5. Sheldrick RC, Maye MP, Carter AS. Age at first identification of autism spectrum disorder: an analysis of two US surveys. J Am Acad Child Adolesc Psychiatry. 2017;56:313–320.
6. MacDonald R, Parry-Cruwys D, Dupere S, et al. Assessing progress and outcome of early intensive behavioral intervention for toddlers with autism. Res Dev Disabil. 2014;35:3632–3644.
7. Ben Itzchak E, Zachor DA. Who benefits from early intervention in autism spectrum disorders? Res Autism Spectr Disord. 2011;5:345–350.
8. Flanagan JE, Landa R, Bhat A, et al. Head lag in infants at risk for autism: a preliminary study. Am J Occup Ther. 2012;66:577–585.
9. Smith T, Iadarola S. Evidence base update for autism spectrum disorder. J Clin Child Adolesc Psychol. 2015;44:897–922.
10. Baker-Ericzén MJ, Stahmer AC, Burns A. Child demographics associated with outcomes in a community-based pivotal response training program. J Posit Behav Interv. 2007;9:52–60.
11. Zwaigenbaum L, Penner M. Autism spectrum disorder: advances in diagnosis and evaluation. BMJ. 2018;361:k1674–16.
12. Brian JA, Zwaigenbaum L, Ip A. Standards of diagnostic assessment for autism spectrum disorder. Paediatr Child Health. 2019;24:444–460.
13. Daniels AM, Halladay AK, Shih A, et al. Approaches to enhancing the early detection of autism spectrum disorders: a systematic review of the literature. J Am Acad Child Adolesc Psychiatry. 2014;53:141–152.
14. Zwaigenbaum L, Warren Z. Commentary: embracing innovation is necessary to improve assessment and care for individuals with ASD: a reflection on Kanne and Bishop (2020). J Child Psychol Psychiatry Allied Discip. 2021;62:143–145.
15. Hyman SL, Levy SE, Myers SM. Identification, evaluation, and management of children with autism spectrum disorder. Pediatrics. 2020;145:e20193447.
16. Page MJ, McKenzie JE, Bossuyt PM. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71.
17. National Heart Lung and Blood Institute. Quality Assessment Tool for Before-After (Pre-Post) Studies with No Control Group. NIH. Available at: https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools. Accessed May 18, 2020.
18. Cochrane Training. Assessing Risk of Bias in Included Studies. Available at: https://handbook-5-1.cochrane.org/chapter_8/8_assessing_risk_of_bias_in_included_studies.htm. Accessed May 18, 2020.
19. Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent. 2011;71(suppl 1):S52–S63.
20. Harrison H, Griffin SJ, Kuhn I, et al. Software tools to support title and abstract screening for systematic reviews in healthcare: an evaluation. BMC Med Res Methodol. 2020;20:7.
21. Mazurek MO, Curran A, Burnette C, et al. ECHO autism STAT: accelerating early access to autism diagnosis. J Autism Dev Disord. 2019;49:127–137.
22. McClure I, MacKay T, Mamdani H, et al. A comparison of a specialist autism spectrum disorder assessment team with local assessment teams. Autism. 2010;14:589–603.
23. Swanson AR, Warren ZE, Stone WL, et al. The diagnosis of autism in community pediatric settings: does advanced training facilitate practice change? Autism. 2014;18:555–561.
24. Ahlers K, Gabrielsen TP, Ellzey A, et al. A pilot project using pediatricians as initial diagnosticians in multidisciplinary autism evaluations for young children. J Dev Behav Pediatr. 2019;40:1–11.
25. Harrison M, Jones P, Sharif I, et al. General pediatrician-staffed behavioral/developmental access developmental disorders. J Dev Behav Pediatr. 2017;38:12–14.
26. Warren Z, Stone W, Humberd Q. A training model for the diagnosis of autism in community pediatric practice. J Dev Behav Pediatr. 2009;30:442–446.
27. Makino A, Wong PY, King G, et al. Parent perspectives and perceptions of autism spectrum disorder diagnosis: a scoping review. Paediatr Child Health. 2017;22(suppl 1):e9.
28. Moh TA, Magiati I. Factors associated with parental stress and satisfaction during the process of diagnosis of children with Autism Spectrum Disorders. Res Autism Spectr Disord. 2012;6:293–303.
29. Kanne SM, Bishop SL. Editorial Perspective: the autism waitlist crisis and remembering what families need. J Child Psychol Psychiatry Allied Discip. 2021;62:140–142.
30. Hine JF, Wagner L, Goode R, et al. Enhancing developmental-behavioral pediatric rotations by teaching residents how to evaluate autism in primary care. Autism. 2021;25:1492–1496.
Keywords:

autism spectrum disorder; diagnosis; primary care providers; training models; evaluation

Supplemental Digital Content

Copyright © 2021 The Author(s). Published by Wolters Kluwer Health, Inc.