Enhancing Developmentally Oriented Primary Care (EDOPC) is a formal didactic curriculum based on Healthy Steps materials that is designed to improve practicing pediatricians' knowledge and confidence in developmental screening within the medical home. We modified the EDOPC program to provide a formal curriculum to pediatric residents serving children in distressed neighborhoods. Using a pre/post design, we evaluated whether the modified EDOPC program improved their knowledge and confidence regarding developmental screening and referrals to community supports. In particular, we assessed resident knowledge and confidence pertaining to 4 early child disorders: autism, social-emotional risk factors, postpartum depression, and developmental delays. Overall knowledge of development significantly increased, while self-rated confidence in the ability to refer and manage was not affected. Chart audits 1 year after the intervention demonstrated increased use of screening tools and more referrals to community services. This article will discuss lessons about facilitators and barriers to teaching residents about vulnerable preschool children.
Section of Developmental and Behavioral Pediatrics (Drs Bauer, Smith, and Msall), Pritzker School of Medicine (Drs Smith and Msall), and Kennedy Research Center on Intellectual and Developmental Disabilities (Dr Msall), The University of Chicago Comer Children's Hospital, Chicago, Illinois; Division of General Pediatrics, Children's Hospital Boston, Harvard Medical School, Boston, Massachusetts (Dr Chien); Healthy Steps for Young Children Program and Enhancing Developmentally Oriented Primary Care (EDOPC), Advocate Health Care, Park Ridge, Illinois (Ms Berry); and La Rabida Children's Hospitals, Chicago, Illinois (Drs Smith and Msall).
Corresponding Author: Sarah C. Bauer, MD, Section of Developmental and Behavioral Pediatrics, The University of Chicago Comer Children's Hospital, 950 E 61st St, SSC 207, Chicago, IL 60637 (email@example.com).
We are grateful for the support of Dr Madelyn Kahana and The University of Chicago Pediatric Residency Training Program. The authors also thank Drs Leah Durst, Karen Goldstein, Nanah Park, Kyran Quinlan, Heather Johnston, Poj Lysouvakon, and Icy Cade-Bell, the teaching attendings at the Friend Family Health Center. In addition, Patty Mack and Ram Chalasani of Enhancing Developmentally Oriented Primary Care dedicated much time and support to teaching the pediatric residents about developmental screening and surveillance. The authors also thank Phyllis Glink of the Harris Foundation as well as Linda Gilkerson, PhD, of the Erikson Institute for their ongoing support of creating models of health and developmental collaboration for vulnerable children across systems of care.
It is estimated that less than half of the 8% of children with developmental delays are identified prior to school entry (Maternal Child Health Bureau, 2007). As a result, children with developmental delays are not being identified or referred in a timely manner for early intervention services that could help them avoid negative sequelae such as academic underachievement, adolescent risk-taking behaviors, and limited employment opportunities (Glascoe, 2000; Halfon & Inkelas, 2003). Furthermore, children with developmental delays who live in poverty are particularly at risk for poor outcomes. Longitudinal data from the Carolina Abecedarian Project demonstrated that children from low-income families who receive early intervention services starting in infancy have improved educational outcomes. At 21 years of age, they had higher years of education, increased rates of attending a 4-year college, and older age at the birth of their first child when compared with those who did not receive early intervention services (Campbell, Ramey, Pungello, Sparling, & Miller-Johnson, 2002). These data have also been validated for low-income children who participated in a quality early childhood program through the HighScope Perry Preschool Study (Children's Defense Fund, 2005).
As core figures of the medical home, pediatricians are on the frontlines for identifying children with developmental delays. In particular, they see children at least 12 times during the first 5 years of life for well-child visits (American Academy of Pediatrics, 2002). In the medical home, pediatricians have a vital role and responsibility in the early identification of developmental disorders, yet they are not always doing so (King & Glascoe, 2003). The 12 visits may represent missed opportunities to identify children with developmental delays, for some studies demonstrate that only 20% of pediatricians utilize validated screening tools (Halfon & Inkelas, 2003; Glascoe & VanDervoort, 1985). Pediatricians who do not use these tools detect only clinically apparent disabilities and may miss more subtle impairments (Ramey & Ramey, 1992). In fact, the prevalence of developmental delays is approximately 10%, yet early intervention programs serve 2.3% of children younger than 3 years (Sices, 2007).
Historically, the clinical skills required to identify and manage developmental and behavioral impairments have not been emphasized in pediatric training programs. In addition, practicing pediatricians state that they need more training in developmental screening (Carnegie Task Force on Meeting the Needs of Young Children, 1994). Currently, the program that has demonstrated the most success in training practicing pediatricians to identify children with developmental delays in greater need of developmental services is Healthy Steps. Healthy Steps is a multipart intervention for practicing pediatricians that has been shown to increase the discussion of more than 6 anticipatory guidance topics, improve family satisfaction, and increase adherence to well-child visits and vaccinations (Minkovitz et al., 2003). It is a method to enhance discussion of children's behavior and development with families as well as to increase identification of developmental and social-emotional delays.
Given the level of training and support that practicing pediatricians require to be effective in developmental screening and referral, we suspect that pediatricians-in-training may also need specialized curricula to improve their knowledge and confidence. The American Academy of Pediatrics recommends that pediatricians receive formal training in developmental screening and surveillance as part of their responsibility within the medical home (American Academy of Pediatrics, 2006). We are aware of few efforts to improve the developmental screening skills of pediatricians-in-training. One was conducted more than 20 years ago and involved more than 160 hours of a developmental pediatrics rotation and covered, among other topics, issues such as the screening and referral of children with developmental and social-emotional delays (Bennett, Guralnick, Richardson, & Heiser, 1984). A more recent effort targeted the use of the provider-dependent Denver II Developmental Screening Tool with residents' continuity clinics and demonstrated significant improvements in residents' knowledge after 1 year (Barratt & Moyer, 2000).
Because of the success of the Healthy Steps program, the availability of validated screening tools that rely on parental report over provider observation, and the lack of information about the best way to teach this material to pediatric residents, we decided to develop a formal curriculum regarding proper developmental screening and referral practices at The University of Chicago Pediatric Residency Program. Similarly, we decided to use surveys to evaluate its effectiveness at improving resident knowledge and self-rated confidence in proper practices.
Context of study
The University of Chicago is located in the Hyde Park neighborhood, and the population we serve in the surrounding neighborhoods is disproportionately affected by poverty, single parenthood, Medicaid, and high school underachievement. For example, 41% to 58% of residents are affected by poverty; 35% to 72% of households are headed by single parents; and 14% to 31% of individuals have dropped out of high school. These factors have disproportionately negative impacts on kindergarten readiness (Voices for Illinois Children, 2004). Therefore, it is crucial that pediatricians-in-training at The University of Chicago Pediatric Residency Program are particularly attuned to identifying children with developmental delays and ensuring their participation in early intervention services.
Our study population included the pediatric residents in the first and third years of training at The University of Chicago in academic year 2005–2006. Thirty-three pediatric residents were included in this study. Second-year residents and medicine-pediatric residents were excluded from this sample, because their schedules did not allow for regular attendance at noon conference at the main training site.
Our pilot study design was a pre- and postintervention evaluation. Using written surveys, we cross-sectionally assessed resident knowledge and confidence in screening and referral practices pertaining to 4 early child disorders. We evaluated pediatric residents at 2 independent cross sections in time before and after instituting the Enhancing Developmentally Oriented Primary Care (EDOPC) curriculum. We also completed a 1-year follow-up audit of charts at the Friend Family Health Center (FFHC).
The University of Chicago EDOPC curriculum is a formal, didactic curriculum based on Healthy Steps materials that is designed to improve practicing pediatricians' knowledge and confidence in developmental screening and referral practices. In its full form, the EDOPC training program included training multidisciplinary care teams of physicians, nurses, and nonclinical staff in developmental screening and referral. All of the modules were addressed, to some extent, in the Healthy Steps National Curriculum. The Healthy Steps for Young Children Program was piloted in 22 sites across the country and was evaluated by Johns Hopkins University School of Public Health. In 1997, the curriculum was introduced to the pediatric and family practice residency programs with clinics at Ravenswood Hospital in Chicago, Illinois. In 2001, the Advocate Health Care System expanded the program to 2 additional pediatric residency programs, one at Advocate Hope Children's Hospital and the other at Advocate Lutheran General Children's Hospital. A Healthy Steps specialist was hired to work with the families and the residents at each site.
In 2005, the EDOPC project began in the state of Illinois. Its purpose was to provide resources and training for healthcare providers within the state. This is done via office-based training as well as Internet resources. In particular, EDOPC-trained instructors train entire office practices in developmental screening and referral. To implement this program in an urban academic center was unique, and this was the pilot study for the EDOPC curriculum. Our EDOPC curriculum was based on Healthy Step materials via a collaborative effort between the Illinois Chapter of the American Academy of Pediatrics, the Advocate Health Care's Healthy Steps for Young Children Program, the Illinois Academy of Family Physicians, and the Ounce of Prevention Fund (http://www.edopc.org/). The pre- and posttests utilized in this curriculum were the same ones used in our pilot study.
We distilled this training program into 11 case-based didactic lectures presented over 11 hours, and trained EDOPC instructors led these lectures during regularly scheduled residents' noon conferences (Monday through Friday) for 1 hour. Each EDOPC lecture comprised a clinical case, a video clip, and evidence-based recommendations. The following EDOPC lectures were given at The University of Chicago Pediatric Residency Program: Teachable Moments; Developmental Screening and Referral; Social-Emotional Development and Screening; Perinatal Depression, Screening, and Referral; Autism: Early Detection and Referral; Temperament and Promoting Goodness of Fit; Family Risk Factors; Limit Setting; Sleep Approaches and Disorders; Toilet Training and Feeding; Language Development; and Cognitive, Gross, and Fine Motor Development.
We administered our 15-minute survey 2 weeks prior to the first case-based didactic lecture in January 2006 and for the 2-week period immediately following the last one in June 2006. Our pretest was the same as our posttest. The survey was a 34-item self-administered survey that assessed knowledge and self-rated confidence pertaining to 4 early child disorders: (1) autism; (2) social-emotional risk factors; (3) postpartum depression; and (4) developmental delays in motor, communicative, personal-social, learning, and adaptive skills. Table 1 provides examples of questions within each of these domains.
Modified EDOPC intervention chart audits
In 2008–2009, chart audits were performed at the main primary care site for residents' continuity clinics, the FFHC. Twenty-five charts of children who had well-child visits within 2 months of each respective audit date were randomly selected from each age group, and the percentage of children screened was calculated. Only those children who were seen at the FFHC at least 3 times for well-child visits were eligible for audit. Charts included children seen by attending physicians as well as by pediatric residents. Chart audits were performed to ascertain the percentage of children whose mothers were screened with the Edinburgh Postnatal Depression Scale at the 6-week visit and children who were screened with the Ages and Stages Questionnaire (ASQ) at 12- and 24-month well-child visits (Cox, Holden, & Sagovsky, 1987; Squires, Potter, & Bricker, 1999). Similarly, chart audits also encompassed whether or not the Ages and Stages Questionnaire: Social Emotional (ASQ:SE) was conducted at the 18-month well-child visit (Squires, Bricker, & Twombly, 2002).
We used t tests to analyze continuous variables and χ2 tests to examine categorical ones. We did a content analysis of charts.
Twenty postgraduate year-1 (PGY1) residents attended at least 2 case-based didactic lectures, and 12 postgraduate year-3 (PGY3) residents attended at least 2 lectures. Given our small sample size and nature of this study, a dose-response relationship between number of lectures and knowledge and self-rated confidence was not demonstrated. Of the PGY1s and PGY3s surveyed, 18 out of 21 (86%) PGY1s completed the pretest and 18 out of 20 (90%) PGY3s completed the pretest. Eight out of 21 (38%) PGY1s completed the posttest, and 14 out of 20 (70%) PGY3s completed the posttest. Overall, 88% completed the pretest and 54% completed the posttest.
Knowledge of developmental screening and referral practices is presented in Table 2. Overall knowledge of autism and social-emotional risk factors was not affected by the modified EDOPC intervention. Knowledge of screening appeared to be better than knowledge of referral practices. Even if residents identified a developmental delay, they were not aware of how to properly refer patients and their families. Self-rated confidence in the ability to screen and refer is depicted in Table 3. Overall confidence in the ability to screen and refer for postpartum depression, social-emotional risk factors, and development was not affected by the modified EDOPC intervention (P = .1 to P = .3).
No difference in knowledge of autism and social-emotional risk was observed for PGY1s and PGY3s. Overall knowledge of development significantly increased (P ≤ .02). While PGY1 knowledge of development significantly increased (P ≤ .03), PGY3 knowledge was not affected (P ≤ .3). In all domains, neither PGY1 nor PGY3 self-rated confidence was affected by the modified EDOPC intervention.
While the modified intervention increased knowledge of overall development but did not influence knowledge in other domains or self-rated confidence in the ability to refer, the efficacy of this intervention is demonstrated by the results of the chart audits. Prior to our intervention in January to June 2006, 11% of children were screened with the ASQ at the 12-month visit. In January 2009, 100% of charts audited at the 12-month well-visit had documented use of the ASQ. Use of validated screening tools increased across all domains, especially in the 2007/2008 academic year (Tables 4–9).
Our intervention was unique, for it provided a formalized curriculum in developmental surveillance and screening to pediatricians-in-training practicing in an urban center surrounded by concentrated neighborhood poverty. In addition, this served as a pilot study of the EDOPC intervention in an urban primary care setting. Pediatricians-in-training play a vital role in the medical home, for they provide primary care to the population we serve at The University of Chicago. A significant portion of the children are also affected by prematurity, asthma, failure to thrive, child neglect, and family members with disability. It is particularly important for pediatric residents to be aware of the importance of developmental surveillance and screening as well as the benefits of early intervention and child advocacy (American Academy of Pediatrics, 2007).
In this study, our EDOPC training program was not associated with significant improvement in knowledge or self-rated confidence in screening or referral practices for general development, social-emotional risk factors, and autism. There are several reasons that may explain these negative findings and help improve future training efforts. Given the small sample and nature of this study, a relationship between attendance at lectures and knowledge and self-rated confidence was not demonstrated. Attendance in at least 2 sessions was enough to demonstrate a significant increase in knowledge of overall development in first-year pediatricians-in-training. These findings provide optimism but are not enough to draw conclusions about the relationship between the intervention and knowledge and self-rated confidence in screening and referral practices. Even though our educational intervention was one of the most intensive that has been described in the literature and drew upon an effective intervention, it could have been of insufficient intensity and breadth of learning experiences. Other interventions had an “active” component to them, including home visits with early childhood specialists, and the Healthy Steps program had a quality-improvement component. The evidence suggests that these active components may be critical to improving clinical practice, problem solving, and collaboration (Davis et al., 1999; Dunst & Trivette, 2009; Holm, 1998; O'Brien et al., 2007).
Second, the clinical environment may matter. The 2 practice-based educational interventions were effective, but our conference-based one was not. Because of off-site clinical rotations, it is challenging to deliver a uniform curriculum to residents in a 6-month period. For that reason, the residents' core curriculum is delivered on an 18-month rotation to ensure residents experience core topics at least twice during their clinical training. Similarly, we could detect only large changes in our sample size. Ours was a small pilot study, and small changes are more difficult to detect.
While our pre/post survey results did not reflect that our intervention resulted in a significant change of scores, PGY1s demonstrated more significant improvements in developmental knowledge than did PGY3s, although their baseline knowledge was comparable. It is possible PGY1s are more impressionable and receptive to changes in the curriculum than are PGY3s. The implication of this finding is that the timing of the EDOPC intervention may be important. The incorporation of a developmental curriculum has more impact if introduced at the beginning of training for pediatricians than at the end of their training.
Of most importance to the effect the training had on practice were the results of our chart audits. In particular, the overall goal of this intervention was to educate providers at the FFHC, especially pediatric residents, about the importance as well as the use of validated screening tools in their continuity clinics. The chart audits demonstrate that providers at the FFHC dramatically increased the use of validated developmental screening tools in their practice. The reasons for this are likely multifactorial. During and after the modified EDOPC intervention, the authors spent time at provider meetings as well as resident conferences, emphasizing the need and the use of validated developmental screening tools at the FFHC, especially in the residents' continuity clinics. In addition, the most consistent use was during the 2007/2008 academic year, when one of the authors implemented a formalized morning report series on developmental screening. Therefore, the impact of this intervention is reflected more in the chart audits than in the results of our survey. However, we cannot exclude other activities that could have simultaneously occurred during this period such as attendance by primary care attendings at continuing education activities on developmental screening.
The limitations of this study primarily stem from its small size and single-site design (this was conceived as a demonstration project), which impact its generalizability. Questions of validity and response biases (study participants reported self-behavior, they were not observed in clinic) also limit its conclusions. Although our study showed that knowledge changed significantly, we recognize that knowledge alone does not necessarily confer competency. In addition, ours was a pilot study and it did not have a control group. Therefore, changes may not be due to the intervention. It is important to note that detailed developmental training existed in very few other places in the residency curriculum at the time of this study. Since that time, we implemented a developmental morning report series and also included developmental surveillance and screening in the residents' continuity clinic curriculum. Finally, significantly fewer PGY1s than PGY3s completed the posttest. Given that PGY1s increased their developmental knowledge to a greater extent than did PGY3s, changes in knowledge might have been even more significant if more PGY1s had responded.
It is interesting that the residents' knowledge deficit appeared greater in referral practices than in screening ones. This makes sense if one considers the practicalities of forming relationships with providers of developmental and social-emotional services in the community. Research suggests that physicians will not screen for developmental and social-emotional problems if they are not aware of the resources to which they may refer families (Hochstein et al., 2001). Knowledge of referral resources is a major component of community pediatrics and can be easily incorporated into residents' continuity clinics (Garg et al., 2007).
Implementing a formalized developmental screening program into a pediatric residency training program presented numerous challenges. In particular, the curricular demands of residency training programs are constantly increasing. To add additional content requires institutional support as well as advocates for the educational issue at hand. Similarly, change is difficult. To ask experienced pediatricians to alter their approach to developmental screening requires demonstration of evidence as well as persistence. For these reasons, it is crucial to embed developmental screening into pediatric residency programs at the beginning of training, with continued emphasis throughout residency and reinforcement in residents' continuity clinics.
A major component of the EDOPC intervention involves listening to caregivers' opinions of children's developmental and social-emotional health as well as implementing validated developmental screening tools such as the ASQ system. While the didactic approach of our intervention introduced pediatricians-in-training to these concepts, it did not provide opportunities to practice these skills or how to implement the system in the residents' continuity clinics. Similarly, the scoring of the ASQ was not addressed in the curriculum. Future modifications of this study include implementing the ASQ system into the residents' continuity clinics as well as providing them opportunities to practice scoring the questionnaires. These changes may lead to improved knowledge and confidence of validated developmental screening tools as well as referral practices.
Although several important initiatives for children exist in our state, many seen and unseen barriers are present for vulnerable children. In residents' continuity clinics, they have the opportunity to work with the legal advocates who now consult with families and caregivers in the clinic. Legal advocates are an important resource for vulnerable children, as they ensure these children receive the services to which they are entitled to receive by the state.
Future modifications of this study include incorporating an early childhood specialist, such as a Healthy Steps provider, into the residents' continuity clinics. As Table 10 portrays, early childhood specialists are agents of communication and sources of community referral resources for the medical home. Similarly, recent research suggests that Healthy Steps may be integrated into pediatric training programs with early childhood specialists who serve as liaisons to developmental and mental healthcare providers. This not only increases residents' knowledge and confidence of referral services but also can serve as a quality improvement initiative to understand facilitators and barriers to accessing early intervention, Early Start, Head Start, and developmentally promoting preschool educational experiences. In addition, residencies incorporating Healthy Steps demonstrate increased continuity of care within the residents' clinics, which may enhance long-term developmental and social-emotional follow-up of vulnerable children (Niederman, Schwartz, Conneel, & Silverman, 2007).
Furthermore, children with the following complex medical needs may benefit from more extensive developmental surveillance: prematurity, congenital heart disease, Down syndrome, fetal alcohol syndrome, seizure disorders, and multiple congenital anomalies. For that reason, it may be necessary to perform chart audits of these children to ensure they are receiving appropriate medical and developmental surveillance and recommended developmental interventions and family supports.
American Academy of Pediatrics. (2002). Guidelines for health supervision III
(3rd ed., pp. 17–118).
American Academy of Pediatrics, Council on Children with Disabilities, Section on Developmental and Behavioral Pediatrics, Bright Futures Steering Committee, Medical Home Initiatives for Children with Special Health Care Needs Project Advisory Committee. (2006). Identifying infants and young children with developmental disorders in the medical home: An algorithm for developmental surveillance and screening. Pediatrics
Barratt, M. S., & Moyer, V. A. (2000). Pediatric resident and faculty knowledge of the Denver II. Archives of Pediatric and Adolescent Medicine, 154
Bennett, F. C., Guralnick, M. J., Richardson, H. B., & Heiser, K. E. (1984). Teaching developmental pediatrics to pediatric residents: Effectiveness of a structured curriculum. Pediatrics, 74
Campbell, F. A., Ramey, C. T., Pungello, E. P., Sparling, J., & Miller-Johnson, S. (2002). Early childhood education: Young adult outcomes from the Abecedarian Project. Applied Developmental Science
Carnegie Task Force on Meeting the Needs of Young Children. (1994). Starting points: Meeting the needs of our youngest children
. New York: Carnegie Corporation of New York.
Cox, J. L., Holden, J. M., & Sagovsky, R. (1987). Detection of postnatal depression: Development of the 10-item Edinburgh Postnatal Depression Scale. British Journal of Psychiatry, 150
Davis, D., O'Brien, M. A., Freemantle, N., Wolf, F. M., Mazmanian, P., & Taylor-Vaisey, A. (1999). Impact of formal continuing medical education: Do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? Journal of the American Medical Association, 282
Dunst, C. J., & Trivette, C. M. (2009). Let's be PALS: An evidence-based approach to professional development. Infants and Young Children
Garg, A., Butz, A. M., Dworkin, P. H., Lewis, R. A., Thompson, R. E., & Serwint, J. R. (2007). Improving the management of family psychosocial problems at low-income children's well-child care visits: The WE CARE project. Pediatrics, 120
Glascoe, F. P. (2000). Evidence-based approach to developmental and behavioural surveillance using parents' concerns. Child: Care, Health, & Development, 26
Glascoe, F. P., & VanDervoort, R. L., Jr. (1985). The developmental screening and referral practices of physicians in Tennessee. Journal of the Tennessee Medical Association, 78
Halfon, N., & Inkelas, M. (2003). Optimizing the health and development of children. Journal of the American Medical Association, 290
Hochstein, M., Sareen, H., Olson, L., O'Connor, K., Inkelas, M., & Halfon, N. (2001). A comparison of barriers to the provision of developmental assessments and psychosocial screenings during pediatric health supervision [abstract]. Pediatric Research
Holm, J. A. (1998). Continuing medical education: Quality issues in continuing medical education. British Medical Journal, 316
King, T. M., & Glascoe, F. P. (2003). Developmental surveillance of infants and young children in pediatric primary care. Current Opinions in Pediatrics
Minkovitz, C. S., Hughart, N., Strobino, D., Scharfstein, D., Grason, H., Hou, W., et al. (2003). A practice-based intervention to enhance quality of care in the first 3 years of life: The healthy steps for young children program. Journal of the American Medical Association
Niederman, L. G., Schwartz, A., Conneel, K. J., & Silverman, K. (2007). Healthy Steps for Young Children program in pediatric residency training: Impact on primary care outcomes. Pediatrics, 120
O'Brien, M. A., Freemantle, N., Oxman, A. D., Wolf, F., Davis, D. A., & Herrin, J. (2007). Continuing education meetings and workshops: Effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews, 4
Ramey, C. T., & Ramey, S. L. (1992). Effective early intervention. Mental Retardation, 30
Sices, L. (2007). Developmental screening in primary care: The effectiveness of current practice and recommendations for improvement. The Commonwealth Fund
Squires, J., Bricker D., & Twombly, E. (2002). Ages & Stages Questionnaires®: Social-Emotional (ASQ: SE)
. Baltimore: Paul H Brookes.
Squires, J., Potter, L., & Bricker, D. (1999). The ASQ user's guide for the Ages & Stages Questionnaire: A parent-completed, child-monitoring system
(2nd ed.). Baltimore: Paul H. Brookes.
Keywords:©2009Lippincott Williams & Wilkins, Inc.
developmental screening; medical education; pediatric residents