Exposure to adversity in childhood has been shown to impact the development of children dramatically.1–3 There is growing evidence that childhood adversity can lead to toxic stress, which is an overactivation of the stress response that can lead to physiological, cognitive, and even epigenetic changes in a growing body.2,4,5 In 1998, Felitti et al6 were the first to describe a dose–response relationship between childhood adversity and an increased risk of chronic diseases, mental health disorders, and substance abuse during adulthood. The coping skills and external support systems a child has in place to overcome stressors (resilience) is a protective mechanism against the negative consequences of toxic stress in childhood.4,5,7–10 Exposure to ACEs have also been linked with poor early childhood mental health and chronic medical conditions in young children.11 Although there does not exist one single evidence-based treatment for exposure to ACEs, detecting childhood adversity and implementing supports, such as proven behavioral health intervention, can help children build resilience to combat the effects of these experiences.7–10 Nearly, 50% of our nation’s children have been exposed to at least 1 ACE, and 12% have been exposed to 4 or more ACEs.1 A successful method for identifying children and families exposed to adversity is necessary to support the development of resilience. However, only 4% of pediatricians report screening for all ACEs, with only 11%–26% reporting familiarity with the ACEs study.12,13
In our general pediatrics clinic, we struggled with identifying children at risk of experiencing adversity. Conversations about exposures were often challenging due to time constraint, and providers lacked a standardized way to quickly and accurately identify families with adversity, as seen as barriers in previous studies.13 This project aimed to implement a standardized ACEs screening process during annual well-child visits, children ages 9 months through adolescence, with a goal of increasing screening rate from 0% in June 2017 to 80% by July 2018.
Johns Hopkins All Children’s Hospital houses a robust outpatient general pediatrics clinic, conducting over 10,000 visits annually for patients from birth to 21 years of age. Our clinic is staffed by 5 attending physicians, 1 nurse practitioner, and 36 pediatric residents. Seventy-five percent of patients have public insurance. Our improvement team was composed of clinic providers: 5 pediatric residents, 1 chief resident, and 1 attending physician.
Residents and faculty were surveyed before the initiative to better understand potential barriers to the implementation of ACEs screening (Table 1). The survey was modified from its previously published version to increase the precision of the results by including 5 responses instead of 3 on the Likert scale.14 We distributed both presurveys and postsurveys via email, and we sent reminder emails to complete the survey 2 and 4 weeks after the initial distribution. As the initiative occurred over different academic years, only the trainees who were present from the beginning through the completion of the initiative were surveyed (n = 24). We also discussed the barriers to implementation with medical assistants, nurses, patient care technicians, and administrative personnel. The most significant barriers identified were lack of knowledge and exposure to childhood adversity and insufficient time for the families to complete a screening tool (Fig. 1).
Our pediatric practice routinely asks patients to complete other forms such as developmental and oral health screens. These are given at check-in and are ideally completed as the family waits for their appointment. But often these forms are incomplete at the time of the visit and may not be finished by the time the patient finishes with their appointment.
The Johns Hopkins All Children’s Hospital internal quality improvement committee determined that this initiative was quality improvement research and not human subject research. Thus, review and approval by the institutional review board were not required.
We developed a multipronged approach to improve screening for ACEs. First, we selected a screening tool developed by the Center for Youth Wellness.15 It is a screening tool that assesses exposure to the original 10 ACEs described by Felitti et al,6 but also to other childhood stressors including having life-threatening illnesses, exposure to harassment or bullying, living in foster care, and separation from a caregiver due to deportation or immigration. In children younger than 12 years of age, the screen addresses 7 additional adversities and is completed by the parent or caregiver.15 In children 12 years of age or older, the screen examines 9 additional adversities. The patient completed the form in the presence of their parent or guardian. Families receive this form, available in English and Spanish, in a patient packet at check-in with instructions to complete it before their visit. This form does not identify the individual ACEs experience, but instead asks the caregiver or the patient to calculate a cumulative score (ACE score) and report this in the box provided. As a result, the provider only receives a total score during the visit, not a child’s individual adverse experiences.
Next, we developed an electronic form to record a child’s ACE score in the electronic medical record with the help from our institutional health informatics team. This intervention allowed for tracking of ACE scores for a child over time.
Last, we developed a detailed process map (Fig. 2) to ensure a shared mental model with other staff and providers of how the screening tool was to be administered and utilized as part of the clinical encounter. This information, and background information about ACEs, was shared through email before the initiation of the screening.
Support for children who screened positive for the at-risk level of adversity (score of 4 or more)6 was provided via a handout of information and support services in our surrounding community. Information on this handout included psychological and therapy resources, and public assistance programs. This handout was available within the electronic medical record to be included with the after-visit summary. We also placed hard copies in our resident workroom. Referral to mental health providers was provided for interested families. Our complex care coordinator also visited with families of at-risk patients to determine if they needed additional services and to coordinate follow-up with our clinic. If a patient did not score at-risk, but in reviewing the form with the families additional history revealed a concern for unsafe situations such as abuse, we made appropriate referrals. Furthermore, regardless of a child’s score, providers asked if there were any questions or concerns about the form. They answered any clarifying questions and provided counseling on ACE exposure if needed.
The initiative took place from July 2017 to July 2018. Because children attend multiple well-child visits in the first 3 years of life, we screen children at 9, 24, 36 months, and annually after that. These time points were chosen to minimize the number of other screens and vaccinations administered simultaneously. A summary of interventions during the initiative can be found in Table 2. Notably, residents and faculty were exposed to ACEs educational activities during a resident-led advocacy event in March 2017. One month following the start of the initiative, residents and faculty attended an hour-long educational seminar providing background information on ACEs and the procedures for screening for ACEs in the clinic. Clinic staff also attended an hour-long educational seminar.
Providers continued to express discomfort with discussing ACEs with patients and their families during informal in-person feedback sessions. To address this discomfort, we designed a simulation experience where resident providers discussed the results of an at-risk ACE screen score with a standardized caregiver. Residents were tasked with discussing the importance of ACEs screening, disclosing the results, and further assessing what services the simulated child may require. Once the simulation was complete, residents debriefed their experience using concepts of advocacy-inquiry debriefing.16 This simulation experience is now part of the annual curriculum within our residency program.
Five months into our screening initiative, we distributed an email update to our residents and faculty providing baseline data and a reminder of the importance of the screening process.
The time required to complete the screens was identified as a large barrier postimplementation. Our providers did not always have the completed screens to review before the start of the visit. If forms were incomplete, it was difficult to know if a child had been exposed to adversity. If forms were not completed until after the visit, clinicians could not provide counseling and the supportive services that would benefit the child. To address this barrier, we created a sheet that was placed on top of the screening packet that read “To expedite your appointment; please fill out the attached paperwork and place in the bin outside of your exam room. This will let your provider know you are ready to be seen.”
We provided an additional presentation and update to our residents and faculty 8 months into the initiative. At this time, we reviewed ACEs and their impact on the health and well-being of patients and also answered questions and concerns about the current screening process.
Our primary outcome measure was the percentage of ACE screens completed. Comparing the number of completed screens to the number of well-child visits allowed us to calculate the percent of children screened every month. After discussion among the initiative committee, we decided that due to the importance and potential implications of the ACEs screen, 80% completion rate was an attainable and reasonable goal. Additionally, we monitored the duration of the 3-year well-child visit as a balancing measure. As a secondary outcome measure, we assessed resident and attending physician comfort with and understanding of ACEs utilizing the Likert survey given before the start of the initiative.
Data Collection and Analysis
Upon completion and documentation of the visit, we collected the paper screens in a bin in the resident workroom. The team collected these monthly. Additionally, we extracted information on the total number of well-child visits from the electronic medical record utilizing age and appointment type as identifiers. Information with regards to the average duration of the 3-year well-child visit (time from check-in to check-out) was extracted from the electronic medical record starting 7 months before the initiation of the screening tool to provide baseline data.
The percentage of survey completion out of eligible patients seen for well-child visits was plotted on a run chart (Fig. 3) and monitored monthly throughout the initiative. Nonrandom variation was assessed using previously accepted methodology.17 Descriptive statistics were utilized to describe resident and attending physician understanding and comfort with ACEs. Responses of 4/5 or 5/5 were considered positive responses, whereas responses of 1/5 or 2/5 were considered negative; responses of 3/5 were neutral. Preinitiative and postinitiative resident response scores and visit times for the 3-year well-child visit were compared using a 2-sided nonparametric Wilcoxon rank-sum test. We could not compare faculty responses due to an insufficient response during the postinitiative responses. The level of statistical significance for all comparisons was an alpha of 0.05.
ACE Screen Completion
Over the 12-month initiative, we successfully screened 1,206 of 2,569 eligible patients for ACEs. The month following the implementation of the screening, we screened 40.8% (97/238) of patients. Although there was an initial drop-off the following month, with the addition of in-person education and training, we noticed incremental and steady increases in the percentage of patients screened. There was no significant improvement in the percent of patients screened with the change in process or with the last informational conference and email update. However, near the end of data collection, we screened 60% (60/100) of eligible patients, which is below our goal of 80%. Throughout the year, we screened 50.6% of children under 12 years of age (n = 955) and 36.9% of children 12 years of age or older (n = 251).
Resident and Attending Surveys
The response rates before intervention were 46% (11/24) for residents and 100% (5/5) for faculty; postintervention response rates were 42% (10/24) for residents and 60% (3/5) for faculty. Median scores and interquartile ranges for resident and faculty responses before and after the intervention are listed in Table 1. There was no statistically significant change in survey scores before and after the initiative.
The average length of the 3-year well-child visit was 79 min before the initiation of ACEs screening (total visits = 110). Immediately following the initiation of the screening, the average length of the visit was 83 min (total visits = 29). At the end of the initiative (March 28, 2018–July 11, 2018), the average length of the 3-year well-child visit was 75 min (total visits = 69). A 2-sample t test was performed and was insignificant for variance between these means (P > 0.05).
The average total ACE score of our patients was 0.91 (range 0–15) (Fig. 4). Twelve percent of patients (149/1,206) had a high-risk score of 4 or more ACEs.
Over 1 year, we implemented screening for ACEs and had steady incremental increases in the proportion of children screened. After the data collection, we were screening 60% of children presenting for well-child visits. This result was below our targeted goal of 80%. With the initiation of this screen, we have been able to identify multiple at-risk children and provide resources including mental health referrals, information on community resources, and follow-up with our complex care coordinator. Approximately, 12% of our patients had a high-risk score of 4 or more ACEs, which is consistent with previous studies.3,6,18,19 As children who experience adversity are at risk of long-term health consequences, identifying them early and intervening can potentially have a significant impact on their overall well-being. Additionally, with this screening tool, we can track a child’s exposure to adversity over time and, as their exposure increases, provide needed resources.
Housing this initiative in a resident continuity clinic allowed our learners to have hands-on experiences with screening and treating children with exposure to stressors. Residents and faculty reported that including a patient’s history of physical, emotional, and sexual abuse is “extremely important.” However, the initiative did not result in a significant change in provider familiarity with the scientific findings of the ACEs study or self-reported comfort level with discussing abuse with patients. Future interventions with more frequent educational sessions and a more robust simulation curriculum may result in more significant improvements.
Our approach had several limitations that likely contributed to not meeting our goal of screening 80% of eligible patients during this initial initiative. There were some families that they were uncomfortable filling out the form and did not want to disclose this personal information. One family threatened to leave the practice over the form as they felt it was too invasive. We mitigated this situation by counseling the family that form completion was optional, and we provided information with regards to why we ask families to provide this information. Although we did have multiple educational sessions for our providers and staff, we did not target our families for any educational interventions, which may have increased screening compliance and possibly prevented situations where families feel uncomfortable with filling out the screening form. As we discuss possible further interventions, one thought is to provide literature in our exam rooms that discuss ACEs and their impact on child health or find other avenues, such as our hospital’s social media platform, to provide this information. Perhaps, targeting our teen patient population through this platform or others could also help improve form completion within this age group, as adolescents did have lower completion rates as compared with younger patients.
Furthermore, assessing parental perceptions may also benefit the future of this effort. Additionally, form fatigue can occur when families are asked to fill out multiple forms before the initiation of a visit. Although we did strategically attempt to target well-child visits that had fewer screens, it is difficult to mitigate this limitation completely. The limited time within a clinic visit can also impact the completion of the form by patients and/or family members. Perhaps, finding a way for families to complete the screening before their appointment day, either through an online patient portal or via email, may help with completion. Although there was no significant increase in the duration of a well-child visit before and after the initiation of the screen, the perceived time constraints by both providers and families could have affected screening rates.
Third, the way clinic staff processed the form following a visit varied. In some instances, the medical assistants would input the information into the electronic record, whereas in other cases, providers would type the information into their note without placing it in the electronic form. Although completed screens were to be put in a collection bin, there were times the form was lost or thrown out. Therefore, the number of screens administered may have been higher than what was calculated, as we utilized returned forms to track completion. Our future efforts will aim to standardize the scoring and recording process. Also, many of our interventions were focused on education of staff and providers, which is not sustainable with new residents, faculty, or staff members. We have implemented annual curricula within the residency program to help combat this from a provider standpoint. However, more work needs to be done to ensure that this knowledge is not lost with faculty and staff turnover.
ACEs dramatically impact a child’s overall health and well-being.1–3 However, recognizing children exposed to adversity can be difficult for providers. Our resident-lead team implemented a screening tool within a pediatric resident continuity clinic and has been successful in incrementally increasing the number of patients screened utilizing multiple provider and staff-centered educational sessions. Although there is still growth potential for this initiative, we feel the initiation of screening has positively impacted providers and allowed us to provide at-risk patients with needed resources. With the implementation of ACEs screening, we can utilize this information to understand the effect of ACEs on childhood health better, and provide additional resources to support our at-risk patients.
The authors thank Johns Hopkins All Children’s Hospital General Pediatric and Adolescent Clinic for their participation in this quality improvement effort. We want to acknowledge Dr. Phillip Mote for serving on our quality improvement team and Sharon Crabtree for assisting with data extraction.
The authors have no financial interest to declare in relation to the content of this article.
1. Bucci M, Marques SS, Oh D, et al. Toxic stress in children and adolescents. Adv Pediatr. 2016;63:403–428.
2. Karlén J, Ludvigsson J, Hedmark M, et al. Early psychosocial exposures, hair cortisol levels, and disease risk. Pediatrics. 2015;135:e1450–e1457.
3. Bethell CD, Newacheck P, Hawes E, et al. Adverse childhood experiences: assessing the impact on health and school engagement and the mitigating role of resilience. Health Aff (Millwood). 2014;33:2106–2115.
4. Shonkoff JP, Boyce WT, McEwen BS. Neuroscience, molecular biology, and the childhood roots of health disparities: building a new framework for health promotion and disease prevention. JAMA. 2009;301:2252–2259.
5. Ellis BJ, Boyce WT. Biological sensitivity to context. Curr Dir Psychol Sci. 2008;17:183–187.
6. Felitti VJ, Anda RF, Nordenberg D, et al. Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. The Adverse Childhood Experiences (ACE) Study. Am J Prev Med. 1998;14:245–258.
7. Soleimanpour S, Geierstanger S, Brindis CD. Adverse childhood experiences and resilience: addressing the unique needs of adolescents. Acad Pediatr. 2017;17(7S):S108–S114.
8. Luthar SS, Cicchetti D, Becker B. The construct of resilience: a critical evaluation and guidelines for future work. Child Dev. 2000;71:543–562.
9. Rutter M. Resilience in the face of adversity. Protective factors and resistance to psychiatric disorder. Br J Psychiatry. 1985;147:598–611.
10. Masten AS. Ordinary magic. Resilience processes in development. Am Psychol. 2001;56:227–238.
11. Kerker BD, Zhang J, Nadeem E, et al. Adverse childhood experiences and mental health, chronic medical conditions, and development in young children. Acad Pediatr. 2015;15:510–517.
12. Kerker BD, Storfer-Isser A, Szilagyi M, et al. Do pediatricians ask about adverse childhood experiences in pediatric primary care? Acad Pediatr. 2016;16:154–160.
13. Szilagyi M, Kerker BD, Storfer-Isser A, et al. Factors associated with whether pediatricians inquire about parents’ adverse childhood experiences. Acad Pediatr. 2016;16:668–675.
14. Strait J, Bolman T. Consideration of personal adverse childhood experiences during implementation of trauma-informed care curriculum in graduate health programs. Perm J. 2017;21:16–061.
15. Center for Youth Wellness. Center for youth wellness ACEQ & user guide [online]. 2017. Available at https://centerforyouthwellness.org/cyw-aceq/
. Accessed July 27, 2018.
16. Maestre JM, Rudolph JW. Theories and styles of debriefing: the good judgment method as a tool for formative assessment in healthcare. Rev Esp Cardiol (Engl Ed). 2015;68:282–285.
17. Perla RJ, Provost LP, Murray SK. The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual Saf. 2011;20:46–51.
18. Bright MA, Alford SM, Hinojosa MS, et al. Adverse childhood experiences and dental health in children and adolescents. Community Dent Oral Epidemiol. 2015;43:193–199.
19. Wing R, Gjelsvik A, Nocera M, et al. Association between adverse childhood experiences in the home and pediatric asthma. Ann Allergy Asthma Immunol. 2015;114:379–384.