In 1998, Felitti and colleagues published a study showing associations between adverse childhood experiences (ACEs) and downstream negative health outcomes, including chronic disease and premature mortality.1 These adverse experiences include childhood abuse, neglect, and household dysfunction (eg, substance use and domestic violence). A recent meta-analysis estimated that in North America, the annual cost attributed to ACEs was approximately $748 billion.2 The American Academy of Pediatrics published a call to action in 2011 aimed at increasing pediatricians’ awareness of toxic stress and ACEs, yet pediatricians have been slow to adopt and implement ACEs screening.3–5
Pediatricians surveyed regarding obstacles to ACEs screening report concerns regarding the general feasibility of ACEs screening in a busy practice, caregiver resistance, the time needed to administer screening, lack of confidence in discussing ACEs, and lack of interventions to address positive screens.5–7 A recent qualitative study that examined caregivers’ responses to ACEs screening found that caregivers accept and appreciate the screen and believed it could help connect their family to needed services.8 The clinic in which this study was conducted was awarded a grant to develop Project LAUNCH (Linking Actions for Unmet Needs in Children’s Health), which aims to improve social-emotional health. The integrated mental health professionals on the LAUNCH team offer the following: evidence-based caregiving support during well-child visits, short-term therapy within the clinic, and connection to needed services after ACEs screening.
The purpose of this work was to effectively and efficiently incorporate caregiver and child ACEs screening for 1 age group into a primary care pediatric clinic using a quality improvement model, with a primary goal to achieve ≥ 75% screen completion of eligible patients.
We conducted the current study at an urban, hospital-based outpatient pediatric clinic. Eight physicians and additional rotating residents serve approximately 7600 unique patients per year; 90% of them have Medicaid, and 50% are Spanish-speaking. The LAUNCH team, 2 physicians, and a medical student pursued integrating ACEs screening. A total of 267 caregivers and 199 children received screening for ACEs during this study. Our institutional review board reviewed and approved this study.
The project occurred between November 2018 and February 2020. The clinic adopted the Center for Youth Wellness Adverse Childhood Experiences Questionnaire for the screening tool, cited as a preferred screening method.9,10 We used the de-identified version of the screen so caregivers disclosed only the number of ACEs instead of identifying specific ACEs they had experienced. The team made 3 minor tool changes to improve clarity (see document, Supplemental Digital Content 1, which shows the final screen used.”9, https://links.lww.com/PQ9/A321).
The project was divided into 3 phases: Pilot 1, Pilot 2, and the Implementation Phase (Table 1). Baseline data were not available because ACEs screening was not in place before this project. Iterative process changes were made via Plan-Do-Study-Act (PDSA) cycles.11 A key driver diagram was created before implementation (see document, Supplemental Digital Content 2, which shows the key driver diagram created before the project started, https://links.lww.com/PQ9/A321). The secondary drivers identified as targeted changes to test were 4-fold: developing efficient and clear explanations of the screen, who administers screening, screened age groups, and developing buy-in from clinic staff. The final clinic flow process map and the script for delivering the screen is described in Figure 1.
Table 1. -
Timeline and Details of PDSA Cycles
||No. Physicians Participating
(0- to 5-year-olds)
|1. Percent complete
2. ACEs scores
3. Qualitative parent reception
||Front desk staff
(0- to 1-year-olds)
|1. Percent complete
2. ACEs scores
3. Time to deliver screen
4. Time to discuss screen
6. Qualitative parent reception
(0- to 1-year-olds)
||Parent/child (0- to 1-month-olds)
||1. Percent complete
2. ACEs scores
4. Qualitative parent reception
*Only cycle in which written delivery method was used.
Pilot 1. During the small pilot that began PDSA cycle 1, patients of only 2 physicians underwent ACEs screening on a day the screen administrator was present in the clinic if they met the following criteria: aged 0–5 years, arrived for their well-child check, and spoke Spanish or English. Each person who administered the verbal screen throughout the project spoke both English and Spanish. If there was a score of 4 or higher for the caregiver and/or child, the medical student notified the physician, and a LAUNCH referral was made. In PDSA Cycle 2, all 8 physicians participated, significantly increasing the number of screened patients, although only on days when a screen administrator was present. To address long-term feasibility and efficiency in rooming patients, caregivers received a typed letter describing the purpose and procedure instead of a verbal screen delivery; caregivers completed the written screen in the waiting room. The physician discussed the screen in the patient’s room.
Pilot 2. Data from Pilot 1 informed significant changes for Pilot 2 because the written letter was not a successful change in the clinic. Thus, the clinic returned to verbal delivery of the screen. The 2 original physicians participated with a narrower age range (0–1 year) to decrease patient numbers as adjustments were made. Pilot 2 began with an “ACEs Week” to increase buy-in from all staff. Staff and physicians watched a documentary about ACEs followed by a discussion and educational displays within the clinic. After that, if the caregiver ACEs score was 4 or higher, they received a handout called “6 Tips to Protect and Heal From Toxic Stress.”12 The staff also asked the following question based upon Felitti’s recent publication: “How do you think these experiences have impacted your health today and how do you interact with your child?”13 Then, using shared decision-making, the patient and physician reached an agreement regarding whether a LAUNCH referral was indicated for behavioral health services. In PDSA Cycle 5, with successful processes underway for caregivers, patients were also screened to establish their baseline score. If the child’s ACEs score was 4 or more, a LAUNCH referral was made. The “6 Tips to Protect and Heal From Toxic Stress” was delivered, and the caregiver was asked if the child was safe in his/her current environment.12 In PDSA Cycle 6, meetings were held with staff and physicians to understand the screening barriers and identify opportunities to optimize the process.
Implementation Phase. During the Implementation Phase, all 8 physicians participated, and the eligible age-range narrowed further to 0- to 1-month-olds. PDSA Cycle 8 entailed flagging eligible patients in the electronic medical record (EMR) schedule to identify patients eligible for screening. In PDSA Cycle 9, to further “flag” eligible patients and their caregivers, a bright yellow paper was placed on the front of the 1-month-old WCC packets as a reminder for staff to administer the ACEs screen.
The primary measure, percent completed ACEs screens, was calculated by dividing the number of completed screens by the number of families eligible for screening. A secondary goal was to address perceived barriers to ACEs screening. The average ACEs score throughout each PDSA cycle was a process measure. Balancing measures were average time to administer the screen as a function of screen administrator [medical student versus medical assistant (MA)], the average time of physician-caregiver discussion of the screen during the visit, the number of referrals generated by ACEs screening, and caregivers’ perceptions of the ACEs screen. De-identified ACEs scores were captured from the screening sheet or EMR documentation. The medical student and MA measured the times to deliver (explaining ACEs and screening instructions to caregiver) and discuss the screen (physician time to discuss results). Novel LAUNCH referrals resulting from ACEs screening were collected weekly from chart review. The physicians and staff collected caregiver quotes during conversations about ACEs to provide qualitative data about caregiver response.
QI Macros SPC Software for Excel developed by Jay Arthur was used to create control charts.14 Standard SPC rules were applied to detect special cause variation. Control limits were set at 3 standard deviations (per convention).15 During Pilot 1, single tests of change were employed to identify and develop the necessary infrastructure for sustained implementation of ACEs screening. Once this infrastructure was established, weekly ACEs screening rates were measured beginning with Pilot 2, which served as the baseline. All other analyses were run in SPSS (version 26). An independent samples t test was run to examine differences in mean ACEs scores between PDSA Cycles 2 and 9, taking into account an unequal homogeneity of variance as assessed by Levene’s Test for Equality of Variances. An independent samples t test was performed to look for mean differences in the time to administer the screen as a function of administrator. We conducted a 1-way ANOVA between-subjects to compare the average discussion time for PDSA Cycles 3, 4, and 5. Emerging themes categorized caregivers’ comments about the screen.
Pilot 1, Pilot 2, and the Implementation Phase lasted 10, 6, and 17 weeks, respectively, during which 55, 52, and 125 families received screening, respectively.
Screening Completion Rates
The screening completion rate ranged from 58% to 100% (Table 2). Of the 284 intended screens, 58 (20%) screens were incomplete. Reasons for incomplete screens included staff forgetting to deliver the screen, physicians failing to document, caregivers misunderstanding instructions, misplaced screens, and caregiver refusals. There were 5 total refusals, making up 9% of incomplete screens and 2% of total families screened. The completion rate was > 80% for the final 10 weeks of data collection (Fig. 2).
Table 2. -
Number of Screens and Mean Adverse Childhood Experiences Scores for each PDSA Cycle
||Percent of Goal Screened
The overall average caregiver ACEs score was 2.32 (SD = 3.42). ACEs scores of 4 or greater occurred in 23% of caregivers screened (Fig. 3). Scores in Cycle 9 (M = 2.40, SD = 3.42), which had a verbal delivery method, were significantly higher than scores in Cycle 2 (M = 0.51, SD = 1.10), the only cycle with a written delivery method (P < 0.001). The average caregiver ACEs score for each week demonstrated decreased variation over time (Fig. 4).
Time to Deliver and Discuss Screen
The average time to administer the caregiver-only screen was 68.06 seconds (SD = 29.88), and the average time to administer the caregiver and child screen was 92.44 seconds (SD = 21.57). There was no significant difference in time when the screen was administered by the medical student (M = 78.69, SD = 33.16) versus the MA (M = 58.85, SD = 24.14) (P = 0.079). The average time to discuss all screens was 86.78 seconds (SD = 79.59) with a range of 0–420 seconds. Six of the 42 timed discussions were longer than 2 minutes.
There were 12 (6%, n = 199) novel LAUNCH referrals during Pilot 2 and the Implementation Phase. Of the 50 (25%, n = 199) caregivers reporting an ACEs score of 4 or greater, 8 (16%, n = 50) were deemed clinically appropriate for a new LAUNCH referral. Many caregivers declined because they reported feeling well-supported. Five (3%, n = 149) additional referrals were made for caregivers who reported an ACEs score of < 4.
Qualitative Caregiver Reception of Screen
Five themes emerged from caregiver feedback: gratitude, the notion these events were in their past, an acknowledgment that these events have had an impact on both their health and the health of their child, a desire to seek professional help to process these experiences, and the idea that these experiences have made them stronger. Comments include:
“Thank you for taking the time. It means you care.”
“I have gone to therapy about these things, and now I am just enjoying my life.”
“[The screen] made me realize how important it was to raise my kid better than how I was raised. You know, break the cycle.”
“I hope to teach my son not to bully others but to love everyone. I was bullied as a kid.”
Using Improvement Science methodology, we embedded ACEs screening of 1-month-old children and their caregivers into a busy pediatric clinic.11 There was at least an 80% completion rate during the final 10 weeks of data collection, higher than reported in a similar study.16 While ACEs screening does not replace acute psychosocial needs assessments that ensure the child’s safety, ACEs screening can contribute to a healthier tomorrow for our children and families. With ACEs screening, providers have a conduit to initiate a conversation within the clinic about toxic stress, to empower parents to help their child build resilience, and to offer strategies to decrease the effects of toxic stress both for the child and the parent.8,12,17
Sustainable screening processes can be established. The sustained completion rate demonstrates that ACEs screening in 1 age group is feasible in a busy primary care clinic serving a vulnerable population. Strategies such as choosing the 1-month-old WCC because there are relatively few forms requiring completion probably allowed for meaningful caregiver engagement with the ACEs screen. Additionally, we believe EMR flagging and brightly colored paper as visual cues increased successful screenings. Following recent literature, buy-in was relentlessly pursued from all clinic staff to enable a culture shift in prioritizing ACEs screening, including incomplete screens and lack of documentation experienced, which was observed early in the project.18 Buy-in strategies included 1-on-1 provider coaching during on-boarding, an “ACEs Week,” and providing staff and physicians updates and opportunities to give feedback throughout the implementation.
Face-to-face delivery is important. Literature shows that physicians feel ill-equipped to deliver and discuss ACEs information with families.5,6 Our data suggest that caregivers were significantly more willing to disclose, as measured by increasing average ACEs scores, when a verbal delivery method was used instead of a written delivery. These findings are consistent with previous literature.19 Twenty-three percent of caregivers screened in our study reported an ACEs score of 4 or more, when compared with 16% of the US population.20 This suggests that (1) our population is at higher risk than the general population, (2) the delivery method used makes them feel safer to disclose, and/or (3) this population is more comfortable disclosing than the general population. Other reasons for this finding could be sample size, lack of clarity and brevity in the written letter, or that our clinic’s literacy level rendered a written letter delivery method ineffective.
Time added to visit: Less than 2 minutes per patient. The time to deliver data suggests that delivery is a skill that can be efficiently learned and embedded within an MA’s clinic duties. It is patient-centered for the MAs to provide the screen because they usually have developed an element of rapport with the patients throughout the rooming process. On average, it took 92 seconds to deliver the child and caregiver screens. Also our assumption was that ACEs screen delivery did not add to the patient visit time because it was administered when the patient was awaiting the physician.
However, time added for physician discussion of the screen potentially adds to visit duration, a concern voiced by physicians in this clinic and reported in the literature.5–7 The average physician time to discuss the screen was 87 seconds, although these data were limited by sample size. While a 2-minute addition to each visit is significant, it is also possible that the screen helps streamline the conversation, ultimately decreasing visit times, but more studies are needed to determine how this affects total visit time. One study found that screening for ACEs did not affect average clinic visit duration.21
Physicians infrequently referred with a positive screen. One of the biggest concerns in the clinic was that screening for ACEs would result in an unrealistic number of referrals to the behavioral health team. However, the referral rate was only 6% of those screened for ACEs. Referral rate could be low because physicians either do not see the benefit of referral or find referral burdensome. Possibly, the script provided a level of comfort that reduced the perceived need for additional behavioral health intervention. Learning from postpartum depression screening, it could be that the act of screening alone is a form of intervention.22 Additionally, another plausible reason is that families in this clinic are already connected to behavioral health services, allowing ACEs screening to serve as an educational platform to discuss the negative effects of toxic stress. This clinic already had established behavioral health services before this quality improvement project, which may limit the generalizability of this study.
Caregiver gratitude. Caregivers appreciated the clinic’s willingness to ask questions about their past. One caregiver noted that they felt the clinic better understood his family as a result of the screen. One of the perceived barriers held by pediatricians is that caregivers will negatively perceive the screen.5,6 Findings in this and other studies suggest that caregivers are receptive to screening and deem it important in the pediatric clinic.7,8,23 Of the 5 refusals, only 1 caregiver refused because they seemed distrustful of the screen, and the other 4 were due to clinic logistics not conducive to a successful screen completion. Asking for feedback in person during the visit potentially introduced biases into our responses, resulting in more positively skewed responses, although previous literature also demonstrates generally positive caregiver reception.8
There is controversy about whether caregiver ACEs screening should be conducted in a pediatric clinic due to potential triggering or lack of positive screen interventions.24–26 We did not experience this. Caregivers showed that they learned about ACEs through the screening process. Recent literature suggests that children of caregivers with high ACEs scores are at an increased risk of missing preventive healthcare visits and delayed acquisition of developmental milestones, suggesting that ACEs are intergenerational. Thus, an approach targeting both the caregiver and child is needed to address ACEs adequately.18,22,27 Screening caregivers allowed us to educate about ACEs, discuss resilience factors, and ultimately empower caregivers to mitigate the risks of toxic stress for both themselves and their child. Furthermore, ACEs screening identified patients at risk for developing new ACEs and ACEs sequelae. Early behavioral health referrals may mitigate such sequelae. This anticipatory guidance aimed at developing strategies to mitigate the effects of toxic stress is an intervention both feasible for the primary care pediatrician and supported by the literature when addressing a high ACEs score in clinic.17
There are several limitations to our study. First, it was conducted at a single center and our interventions may not work in other centers, although most of the changes we made are fundamental change concepts that we believe should be effective in other settings. As discussed above, the behavioral health services already established in the clinic allowed for more accessible interventions when a patient had a high ACEs score. Second, our work was conducted in a single age group (1-month-olds). Because the 1-month-old visit often has fewer items that need discussion and the physical examination is relatively brief compared with that in older age groups, care providers may have been more receptive to adding this additional screen to the visit. The sample size of our study was another limitation specifically for the time data. With a larger sample size, more accurate conclusions may be reached regarding the amount of time ACEs screening adds or saves during a clinic visit. Finally, there were limitations in which the quotes were elicited because they occurred during a clinical visit, and they were elicited by any member of the team, both of which likely introduced biases into the responses. However, our results are consistent with other literature demonstrating generally positive caregiver reception of ACEs screening.8
ACEs screening sustainability for 1-month-olds and their caregivers will be ensured through ongoing performance review of data with clinic staff and physicians. Over time, additional age groups will be included in the screening with concurrent PDSA cycles to inform the development of streamlined processes.
This study successfully reached a sustained ACEs screening completion rate of greater than 80%, demonstrating that ACEs screening in 1 infant age group is feasible in a busy primary care clinic serving a vulnerable population. ACEs screening can start the conversation about toxic stress and build trust with caregivers.8 We believe that for every child to reach their full potential, screening tools such as the ACEs screen are key elements that when used effectively, with the correct training for providers, and within the context of a trauma-informed community, we can improve the community health to that which we all aspire for our patients and families.28–30 In the words of ACEs research giant Felitti, “Asking...coupled with listening and implicitly accepting the person who had just shared his or her dark secrets, is a powerful form of doing.” 13(p5)
The authors have no financial interest in relation to the content of this manuscript.
The authors thank the physicians of Raleigh Campus Pediatric Primary Care WakeMed Physician Practices. Additional thanks to The University of North Carolina’s Institute for Healthcare Quality Improvement for facilitating the study.
1. Felitti VJ, Anda RF, Nordenberg D, et al. Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. Am. J. Prev. Med. 1998;14:245–258.
2. Bellis MA, Hughes K, Ford K, et al. Life course health consequences and associated annual costs of adverse childhood experiences across Europe and North America: a systematic review and meta-analysis. Lancet Public Health. 2019;4:e517–e528.
3. Garner AS, Shonkoff JP, et al. Committee on Psychosocial Aspects of Child and Family Health, Committee on Early Childhood, Adoption, and Dependent Care, and Section on Developmental and Behavioral Pediatrics. Early childhood adversity, toxic stress, and the role of the pediatrician: translating developmental science into lifelong health. Pediatr. 2012;129:e224–e231.
4. Kerker BD, Storfer-Isser A, Szilagyi M, et al. Do pediatricians ask about adverse childhood experiences in pediatric primary care? Acad Pediatr. 2016;16:154–160.
5. Szilagyi M, Kerker BD, Storfer-Isser A, et al. Factors associated with whether pediatricians inquire about parents’ adverse childhood experiences. Acad Pediatr. 2016;16:668–675.
6. Finkelhor D. Screening for adverse childhood experiences (ACEs): cautions and suggestions. Child Abuse Negl. 2018;85:174–179.
7. Eismann EA, Theuerling J, Maguire S, et al. Integration of the Safe Environment for Every Kid (SEEK) model across primary care settings. Clin Pediatr (Phila). 2019;58:166–176.
8. Conn AM, Szilagyi MA, Jee SH, et al. Parental perspectives of screening for adverse childhood experiences in pediatric primary care. Fam Syst Health. 2018;36:62–72.
9. Center for Youth Wellness. ACE-Q Materials. ACE-Q Materials. Available at https://centerforyouthwellness.org/aceq-pdf/
. Published 2017. Accessed October 21, 2018.
10. Oh DL, Jerman P, Purewal Boparai SK, et al. Review of tools for measuring exposure to adversity in children and adolescents. J Pediatr Health Care. 2018;32:564–583.
11. Langley G, Moen R, Nolan K, et al. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. Jossey-Bass Publishers; 2009.
12. Center for Youth Wellness. What can I do? stress health. 2019. Available at https://www.stresshealth.org/what-can-i-do/
. Accessed August 2, 2019
13. Felitti VJ. Health appraisal and the adverse childhood experiences Study: national implications for health care, cost, and utilization. Perm J. 2019;23:18–026.
14. Arthur J. QI Macros SPC Software for Excel. (888) 468-1537. 1997. Available at www.qimacros.com
. Accessed September 25, 2019.
15. Carey R, Lloyd R. Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control Applications. 1st ed. ASQ Quality Press; 2001.
16. Marsicek SM, Morrison JM, Manikonda N, et al. Implementing standardized screening for adverse childhood experiences in a pediatric resident continuity clinic. Pediatr Qual Saf. 2019;4:e154.
17. Gilgoff R, Singh L, Koita K, et al. Adverse childhood experiences, outcomes, and interventions. Pediatr Clin North Am. 2020;67:259–273.
18. Stein S, Swerlick A, Chokshi B. Employing an adaptive leadership framework to childhood adversity screening. Pediatrics. 2020;145:e20183820.
19. Eismann EA, Folger AT, Stephenson NB, et al. Parental adverse childhood experiences and pediatric healthcare use by 2 years of age. J Pediatr. 2019;211:146–151.
20. Centers for Disease Control and Prevention. Adverse childhood experiences resources. Available at https://www.cdc.gov/violenceprevention/acestudy/resources.html
. Published 2019. Accessed August 15, 2019.
21. Bryant C, VanGraafeiland B. Screening for adverse childhood experiences in primary care: a quality improvement project. J Pediatr Health Care. 2020;34:122–127.
22. ACOG Committee Opinion No. 757: screening for perinatal depression. Obstet Gynecol. 2018;132:e208–e212.
23. Barnes AJ, Anthony BJ, Karatekin C, et al. Identifying adverse childhood experiences in pediatrics to prevent chronic health conditions. Pediatr Res. 2020;87:362–370.
24. Lê-Scherban F, Wang X, Boyle-Steed KH, et al. Intergenerational associations of parent adverse childhood experiences and child health outcomes. Pediatrics. 2018;141:e20174274.
25. Schickedanz A, Halfon N, Sastry N, et al. Parents’ adverse childhood experiences and their children’s behavioral health problems. Pediatrics. 2018;142:e20180023.
26. Gillespie RJ, Folger AT. Feasibility of assessing parental ACEs in pediatric primary care: implications for practice-based implementation. Journ Child Adol Trauma. 2017;10:249–256.
27. Folger AT, Eismann EA, Stephenson NB, et al. Parental adverse childhood experiences and offspring development at 2 years of age. Pediatrics. 2018;141:e20172826.
28. Kemper AR, Kelleher KJ, Allen S, et al. Improving the health of all children in our community: the nationwide children’s hospital and Franklin County, Ohio, Pediatric vital signs project. J Pediatr. 2020;222:227–230.
29. Shonkoff JP. Capitalizing on advances in science to reduce the health consequences of early childhood adversity. JAMA Pediatr. 2016;170:1003–1007.
30. Selvaraj K. What’s the punchline?: promoting child and teacher resilience through pediatrician-teacher partnerships. Pediatrics. 2018;141:e20173538.