The Power of Connections: AAP COVID-19 ECHO Accelerates Responses During a Public Health Emergency : Journal of Public Health Management and Practice

Secondary Logo

Journal Logo

Research Reports

The Power of Connections: AAP COVID-19 ECHO Accelerates Responses During a Public Health Emergency

Bernstein, Henry H. DO, MHCM, FAAP; Calabrese, Trisha MPH; Corcoran, Peter MPH; Flint, Laurence E. MD, MS, MBA, FAAP; Munoz, Flor M. MD, MSc, FAAP

Author Information
Journal of Public Health Management and Practice: January/February 2022 - Volume 28 - Issue 1 - p E1-E8
doi: 10.1097/PHH.0000000000001466
  • Open

Abstract

Pediatric providers across the United States have sought guidance on how to care for the nation's children during the uncertain historic times of the COVID-19 pandemic.1 The health care community has been challenged by the unprecedented burden of caring for patients when they have evolving guidelines and limited information about the effects of the virus on children.

In response, the American Academy of Pediatrics (AAP) rapidly launched a COVID-19 ECHO (Extension for Community Healthcare Outcomes) program. The goal of this national initiative was to increase child health professionals' knowledge, skills, and self-efficacy during the pandemic. The result was a powerful educational model for disseminating “just-in-time” information and creating peer-to-peer support during a rapidly evolving public health crisis.

Project ECHO is an innovative hub and spoke program, designed to create communities of learners by using technology to connect specialty care providers and subject matter experts with health professionals including rural and underserved areas. ECHO programs utilize an “all-teach, all-learn” approach that includes a combination of didactic learning about clinical best practices and case studies from individual spoke participants. ECHO has demonstrated success in increasing provider capacity to identify, treat, and manage patient care within the medical home.2,3

Since 2015, the AAP has served as an ECHO Superhub, operating as the pediatric arm of the ECHO Institute to train more than 90 organizations in the ECHO model while hosting nearly 100 AAP ECHO programs that have built and supported dynamic learning communities of peers and national experts. Serving as a Project ECHO Superhub, the AAP recruits an interdisciplinary faculty team to spearhead the ECHO programs. This experience positioned the AAP well to accelerate its rapid response to an evolving public health emergency by promptly launching its COVID-19 ECHO program. Past ECHO programs took several months to develop and implement. In contrast, organizers recruited participants, created didactic materials, arranged sessions, and identified and trained interdisciplinary faculty members and facilitators to host sessions, all within 20 days. Because of the short implementation time frame and “just-in-time” nature of the content, continuing medical education (CME) credits were not offered for this program, although the AAP does typically attach CME and maintenance of certification (MOC) credits to its ECHO programs.

Program Description

AAP ECHO sessions are held over Zoom, a virtual teleconferencing system that is optimized for ECHO sessions and is free to ECHO hubs through the ECHO Institute. Participants may access Zoom at no cost via laptops, cell phones, or tablets. Table 1 describes the activities, time frame, and responsible parties during a typical ECHO session. During the COVID-19 ECHO session, participants were encouraged to type in their names, roles, and locations at the beginning of each session, which helped encourage and facilitate discussion. Participants also were offered the opportunity to join a consistent community of learners throughout the program, that is, a cohort that met on a recurring day/time. The COVID-19 topics included scientific and clinical overviews and updates; incorporation of telehealth into practice; rise in child abuse cases; updates on COVID-19 prevention, testing, diagnosis, and treatment; evaluation of children's mental health needs; and return-to-school considerations. The bidirectional conversation between faculty and participants aided in the establishment of relationships within the sessions. This bidirectional learning is paramount to the “all-teach, all-learn” approach. As the understanding of COVID-19 expanded and new needs emerged among practitioners, the ECHO model responded dynamically with new and updated content.

TABLE 1 - Typical AAP ECHO Session
Activity Time, min Responsible Party
Welcome, guidelines/expectations, and introductions 10 AAP moderator
Topic-related presentation 15 Subject matter expert faculty
Question-and-answer segment 5 All
Case-based scenario presentation 10 Self-selected participants
Facilitated group case discussion 15 All
Closing remarks 5 AAP moderator
Abbreviation: AAP, American Academy of Pediatrics.

The virtual platform allowed participants to share resources and links in real time through the chat function. As comfort within groups grew, participants verbally shared more of their own experiences, and many discussions and storytelling continued offline. Faculty presentations and additional resources were e-mailed to audience participants at the end of each session to help reinforce active learning, and many participants reported sharing these materials with their colleagues. Session topics were selected in a “just-in-time” fashion based on participant feedback from the prospective postsession surveys and faculty recommendations; speakers brought data and expert recommendations that were often, literally, “hot off the press.”

Methods

The AAP COVID-19 ECHO program used a mixed-methods approach to evaluate the program's effectiveness in meeting its educational objectives. The 2-phase evaluation collected quantitative and qualitative data through an integrated feedback structure that utilized prospective postsession and retrospective postprogram surveys, along with postprogram focus groups.

After each session, ECHO participants were given a link to an online survey where they could provide feedback to ECHO faculty and AAP staff about the session. This prospective postsession survey had 8 questions rating elements of the session. A 5-point Likert scale was utilized in 4 questions, 3 questions were open-ended, and the final question was a list of topics participants were interested in learning about during upcoming AAP COVID-19 ECHO sessions. The list was based upon the overarching ECHO curriculum and the most current and pressing subject matters being discussed in the literature. To evaluate the effectiveness of the ECHO model, confidence and knowledge gains were assessed on the basis of participation in the ECHO program. The retrospective survey had 7 confidence items, 9 knowledge items, and 9 ECHO satisfaction items. All items used a 5-point Likert scale and to access statistical significance, the P value was set at .05, and a paired-sample t test was conducted.

Two 1-hour focus group discussion sessions with a combined convenience sample of 6 participants were also conducted. Participants were pediatricians and comprised twice as many females as males who attended at least three-fourths of the 8 ECHO sessions. Both focus groups were conducted virtually with a single moderator. All participants gave verbal informed consent, and there was no remuneration for each contributor's time. Their open-ended feedback was evaluated and analyzed on the basis of the context of the educational objectives. Focus group data were audiotaped, transcribed, de-identified for confidentiality, and analyzed and reviewed by 3 AAP ECHO program evaluation staff members. Each member of the research team read through each transcript, coding key statements. The iterative coding process involved clustering of key words, followed by thematic analysis, whereby emerging themes were identified and recorded on the basis of similarity and emphasis. Thematic analysis also involved the identification and refinement of themes through careful reading and rereading of the data.

Results

Each ECHO session brought together child and public health professionals from urban, rural, and suburban areas of the United States. The initial AAP COVID-19 ECHO program hosted more than 900 participants in 127 individual virtual sessions, with approximately 25 participants per session. The evolving nature of the pandemic necessitated dynamic and continuous bidirectional flow of concerns and information relevant to participants.

Table 2 lists the participant demographic characteristics collected in the retrospective postprogram surveys, offering continuous, diverse perspectives on the ECHO sessions. A convenience sample of 33 individuals completed the retrospective postprogram survey, 70% of whom answered the open-ended questions in the survey. Focus group participants were pediatricians who attended at least 6 ECHO sessions and had more than 10 years of practice, and the majority (67%) described their practice to be in a suburban setting while the rest (33%) were in an urban setting. Half of the participants were independent primary care practice providers, one was from a multispecialty group practice, and the remaining 2 were practicing at a hospital-affiliated or academic medical center.

TABLE 2 - Demographics of Retrospective Postprogram Survey Participants (N = 33)
Frequency (%)
Practice setting
Rural (<9999 residents) 1 (3)
Suburban (between 10 000 and 49 999 residents) 14 (42.4)
Urban (≥50 000 residents) 18 (57.6)
Credentials
Physician (MD, DO) 31 (93.8)
Other (public health, hospital administrator) 2 (6.2)
ECHO sessions attended
0-2 2 (6.1)
3-5 14 (42.4)
6-8 17 (51.5)
Number of years in practice/profession
≤5 7 (21)
6-10 6 (18)
11-15 2 (6)
>15 18 (55)
Practice description
Independent primary care practice 13 (39.4)
Multispecialty group practice 5 (15.2)
Federally qualified health center 3 (9.1)
Academic medical center 5 (15.2)
Local public health 2 (6)
Hospital 2 (6)
Hospital-affiliated practice 3 (9.1)
Sex (biological)
Female 24 (72.7)
Male 9 (27.3)

Satisfaction, confidence, and procedural knowledge were examined by assessing self-reported ECHO experience and participant benefits. An overwhelming majority of survey respondents agreed/strongly agreed that the ECHO model was a valuable use of their time (100%; n = 30), makes them better at their job (96.7%; n = 29), and their understanding of the subject matter has improved (100%; n = 30) (Table 3). Participants self-reported statistically significant improvements across all knowledge and confidence items. Although participants reported improvement across all content areas, the highest gains were reported in the following areas: managing potentially exposed patients who present in the office setting (MD = 1.548, SD = 1.150, df = 30), conducting telehealth visits for patients (MD = 1.600, SD = 1.192, df = 29), and integrating telehealth into longitudinal care for patients (MD = 1.567, SD = 1.278, df = 29) (Table 4).

TABLE 3 - ECHO Satisfaction From Retrospective Postprogram Survey (N = 30)
% Rating Agree/Strongly Agree (n)
Was a valuable use of my time 100 (30)
Contributed to my professional network 66.7 (20)
Makes me better at my job 96.7 (29)
This ECHO series provided an appropriate balance between instruction and case-based learning 100 (30)
The content of this ECHO series was relevant to my patient population 93.3 (28)
My understanding of the subject matter has improved as a result of having participated in this ECHO series 100 (30)
My interest in the subject matter has increased as a result of having participated in this ECHO series 86.6 (26)
Participating in this ECHO increased my professional satisfaction 96.7 (29)
Participating in this ECHO made me feel less isolated 100 (30)

TABLE 4 - Quantitative Analysis of Confidence and Knowledge From Retrospective Postprogram Survey
Mean Pre-ECHO (SD) Mean Post-ECHO (SD) Mean Difference Post/Pre-ECHO (SD) t df P
Confidence statementsa
Managing potentially exposed patients who present in the office setting 2.387 (1.145) 3.935 (0.772) 1.548 (1.150) 7.496 30 <.001
Understanding best practices about personal protective equipment (PPE) in the context of your office 2.969 (1.231) 4.219 (0.706) 1.250 (1.078) 6.562 31 <.001
Supporting patients during a public health emergency 2.545 (1.201) 3.848 (0.712) 1.303 (0.984) 7.609 32 <.001
Supporting colleagues during a public health emergency 2.697 (1.132) 3.909 (0.723) 1.212 (0.992) 7.016 32 <.001
Finding resources to keep you informed and support talks with parents 3.031 (1.150) 4.094 (0.818) 1.063 (1.045) 5.750 31 <.001
Integrating telehealth into longitudinal care for patients 2.367 (1.273) 3.933 (1.048) 1.567 (1.278) 6.714 29 <.001
Providing well child visits and other primary care for patients during a pandemic 2.699 (1.228) 3.966 (0.910) 1.276 (1.099) 6.254 28 <.001
Knowledge statementsb
Identifying prevention options for patients related to COVID-19 2.688 (1.203) 4.125 (0.871) 1.438 (1.076) 7.559 31 <.001
Identifying treatment options for patients related to COVID-19 2.226 (1.146) 3.581 (0.992) 1.355 (0.915) 8.247 30 <.001
Identifying strategies to support patients and families in times of stress 2.938 (1.105) 3.750 (0.803) 0.813 (0.780) 5.890 31 <.001
Managing patients exposed to COVID-19 who present in the clinical setting 2.581 (1.119) 4.000 (0.966) 1.419 (1.088) 7.260 30 <.001
Treatment of mental and emotional issues in patients during a pandemic 2.844 (1.139) 3.750 (0.842) 0.906 (0.817) 6.271 31 <.001
Conducting telehealth visits for patients 2.467 (1.408) 4.067 (0.980) 1.600 (1.192) 7.352 29 <.001
Identifying Multisystem Inflammatory Syndrome in Children (MIS-C) 2.188 (1.330) 3.625 (0.793) 1.438 (1.045) 7.779 31 <.001
Management of well child visits during the pandemic 3.036 (1.201) 4.179 (0.772) 1.143 (1.044) 5.793 27 <.001
Infection prevention and control related to COVID-19 2.758 (1.299) 4.182 (0.846) 1.424 (1.062) 7.707 32 <.001
aFive-point Likert scale was used with the anchors to score: very confident = 5; confident = 4; somewhat confident = 3; slightly confident = 2; not confident = 1.
bFive-point Likert scale was used with the anchors to score: very knowledgeable = 5; knowledgeable = 4; somewhat knowledgeable = 3; slightly knowledgeable = 2; not at all knowledgeable = 1.

Table 5 lists the major themes that emerged from the qualitative data, including the following: (1) knowledge acquisition about this novel virus and disease; (2) diffusion of knowledge as a pipeline for credible information in a state of continuous adjustments; (3) communal endeavor and support through the creation of an ECHO community that supported not only the participants' learning experiences but also their emotional and mental well-being as the pandemic raged on; and (4) confidence building as expressed by participants in how each ECHO session gave them opportunities to discuss and act upon their knowledge acquisition to offer care, testing recommendations, guidance on childcare, back-to-school, and more. Self-reported participant feedback documented the ECHO model's unique attributes. These themes are further outlined as follows.

TABLE 5 - Themes Identified Through Qualitative Analysis of Open-Ended Retrospective Survey and Focus Groups
Theme Subtheme Sample Quotes
Knowledge gained Relevant and timely information
Value of ECHO format
Faculty experts
  • We're continuously learning as well, everything from the CDC guidelines changes about testing and quarantine..., there's just a lot to learn and then shift into your practice, so it's great to get that reinforced in these ECHO sessions.

  • Improved knowledge base about how to make decisions about care and counseling positive cases.

  • Improved knowledge of strengths and limitations of testing for COVID-19.

  • I have to say that this one really satisfied my sense of urgency, because we were dealing with a brand-new disease that was rolling out.... Impressed with the speed with which you guys got that thing rolling, considering where we were in our knowledge of pediatric COVID issues ... and so they came along at a very timely point.

  • Things are changing so rapidly and the guidance was evolving quickly, everything from the value of rapid testing to understanding the multi-inflammatory system condition in children. I thought it was important..., because we kind of all knew the facts, it was really the deep dive and the subtleties of it and allowing us to ask the gray area of questions and understand it more in depth that really added the value to it.

  • We learned from each other as well as learning from the subject matter expert, but it was great to be able to really do a much deeper dive after the present or with the presentation, because the facts are what they are.

  • This ECHO has been comprehensive for a reliable tap for accurate information & state of the science as of any given week.

Diffusion of knowledge Packaged format
Dissemination to peers and patients
Rich discussion
  • Having a robust discussion about asymptomatic carriers and how to handle their presence was helpful. Discussing best practices in going back to school helped strengthen my resolve to: (1) be proactive with the school district on how best to reopen schools and (2) feel good that the best decision for my children's education and safety was going back to school.

  • I was able to help that way of transmitting information with various colleagues, and honestly, I found that with Coronavirus, information was therapy, it was so critical to people to be able to have a pipeline of information while they were just in that state of frantic adjustment to what was happening around them.

  • Many things are relevant, and I put together a package into our SharePoint site ... and then we discussed it all at this meeting every couple of weeks, and its been super valuable because the information is so cutting edge and if it's something that any one of us feels needs to be acted on sooner than the next meeting.

Community endeavor Peer support
Safe space
  • I felt much less isolated especially in the beginning of the project. It felt nice to have a community of others who were seeking to get as much knowledge as they could to support patients and themselves.

  • I love the collaborative aspect of it, I felt like it was enriching to be with this big group on Zoom all sort of echoing a lot of the same feelings of challenge and whatever that we all were. And then being able to support and help each other through them. So it was valuable from that standpoint, it was great to have other people who are kind of in the same boat. Waiting through the muck together. That was good. I agree, I love being in a big group and was people that weren't all just from New York.

  • Mostly I benefited from feeling professional support from colleagues, experts and the AAP.

  • I am more optimistic about providing safe care and have been encouraged by the camaraderie experienced in the groups.

Confidence building
  • It's definitely led to a much more confidence feeling around my knowledge base that I can go out there then and proclaim with confidence and quite certainty what we know now.

  • ECHO sessions have really provided me that confidence in having resources that I can turn to and information that I can draw from to gain that confidence. I do feel like the ECHO sessions did help me get scripts and strategies for helping build confidence in my staff around feeling safe and doing testing....

  • As a result of our ECHO sessions, it's definitely led to a much more confidence feeling around my knowledge base that I can go out there and then proclaim with confidence and quite certainty what we know now.

Abbreviation: AAP, American Academy of Pediatrics.

Theme: Knowledge acquisition

Participants reported that they had gained new COVID-19 clinical knowledge and skills through participation in the ECHO program. This unfolded through the combination of a didactic lecture overview on the current evolving guidelines that would form the starting point in the case discussions; knowledge and skills were freely exchanged among the ECHO faculty members and various participants. In most cases, knowledge and skills development pertained to screening, testing, or infection prevention control approaches to care, including management of patients and staff.

Theme: Diffusion of knowledge

The educational opportunities were not limited to just the actual ECHO sessions. Participants received the didactic lecture slides and emphasized personal lessons learned during discussions when reporting back to their local networks. One focus group participant indicated,

I put together a package into our SharePoint site ... and then we discussed it all at this meeting every couple of weeks, and it's been super valuable because the information is so cutting edge and if it's something that any one of us feels needs to be acted on sooner than the next meeting.

Participants applied the acquired knowledge and skills to their patient populations and reported disseminating information to policy makers, staff, as well as colleagues.

Theme: Communal endeavor and support

A distinct relational element also was effectively demonstrated in each ECHO session for the theme of communal endeavor and support. From the outset, faculty and providers struggled not only with professional uncertainties of the pandemic but also with their personal and societal challenges. Participants brought their personal COVID-19 stories into the conversation, sharing the pandemic's impact on patients, staff, colleagues, and their own families. As the ECHO sessions progressed, a growing number of participants, or their loved ones, were directly affected by a COVID-19 illness; it was not uncommon for these individuals to share very personal stories. In times of anxiety, shutdowns, and isolation, the program provided participants with the opportunity to share and gain wisdom from a larger community, all of whom were experiencing the same challenges.

Participants reported that the combination of access to a panel of experts and being able to participate interactively made ECHO a unique learning experience, both personally and professionally. They also stated how they appreciated the:

... rich discussion of [the] evolving COVID landscape. I really appreciate the peer-to-peer discussions about how practices are adapting and I have shared recommendations on infection control, patient care, etc. with administration in hopes of making changes within our practice.

As another participant noted,

The ECHO offers connectedness and support to others as well as resources. As COVID was not as bad in my location, it was very valuable to hear from my peers on how to better handle the pandemic during the case study.

Another learner shared, “I did not feel as alone during the ECHO. I would hear from others about their challenges and concerns. It was very supportive as I too faced similar issues in my practice.”

Theme: Confidence building

The fourth theme of confidence building emerged as a significant benefit of participating in this ECHO initiative. Participants described that the combination of access to a panel of experts and being able to participate interactively helped reassure them that their approaches to the complex and challenging conditions COVID-19 created for patient care were in line with best practice, based on the uncertainty in which we all are operating. Focus group participants highlighted how the ECHO sessions aided in the development of or reaffirmation of their clinical actions and guidance in offering care and disease management to the wider public. This reassured participants and increased their self-confidence and motivation to have conversations with patients, caregivers, staff, and administrators about testing, screening, and care management during the pandemic.

Discussion and Conclusion

The participants of this AAP COVID-19 ECHO program expressed how they had a more in-depth understanding of COVID-19 and a greater confidence in managing and improving patient-centered and team-based care practices during the pandemic. They shared how they brought lessons learned from the ECHO program back to their practices and local networks to support patient and staff education on self-protection in preventing infectious disease transmission, building confidence that their approaches were in line with best practice. These findings are consistent with similar themes documented with another ECHO program utilizing case-based tele-education sessions that extended the reach of child maltreatment experts to statewide community emergency department providers with little access to consultation.4

Our experience during the AAP COVID-19 ECHO program emphasizes how powerful this model can be for disseminating timely information and creating peer-to-peer support during a rapidly evolving public health crisis. It was initially designed to build capacity in the health care workforce. However, the strength of a network of individuals experiencing similar crisis circumstances, along with the establishment of relationships, has helped support and guide pediatricians during this time of unprecedented challenges and unpredictable gaps in knowledge. As they learned together, members of these learning communities built trusting relationships.5

One notable limitation was the relatively small sample size for the retrospective postprogram surveys and focus groups compared with the number of individuals who participated in the ECHO program. However, this was somewhat anticipated, given the continuous shifting personal and professional responsibilities and novelty associated with the pandemic.

Most importantly, the expeditious manner with which this ECHO program was launched was critical since it enabled “just-in-time” disease-specific information and practical solutions to be shared broadly. The ECHO model enabled participants who had already experienced cases to share their experiences with others who had not. The experience also provided early local advocacy for resources and changes to practice patterns for the latter group. Through sharing practice experiences, clinicians engage in the process of improving patient care.6

Implications for Policy & Practice

  • The ECHO model is a tele-mentoring program designed to create communities of learners among health professionals and subject matter expert faculty members using didactic and case‐based presentations that foster an “all-teach, all-learn” approach.
  • This model highlights its power in disseminating credible and timely information and creating peer-to-peer support during a rapidly evolving public health emergency, such as during the pandemic.
  • Health care organizations can utilize the ECHO initiative to increase health professionals' knowledge, skills, and self-efficacy during the ongoing pandemic.

In summary, as the COVID-19 pandemic surges once again and another influenza season is upon us, this national AAP educational initiative continues to build child health professionals' knowledge, skills, and self-efficacy during an unprecedented and ongoing public health emergency. The spirit of the initial COVID-19 ECHO program continues, informing the creation of several new AAP ECHO programs to support health care providers' response to the needs of children during the COVID-19 pandemic and beyond.7

References

1. World Health Organization. Tedros Adhanom Ghebreyesus. World Health Organization Director-General's statement on IHR Emergency Committee on Novel Coronavirus (2019-nCoV). https://www.who.int/dg/speeches/detail/who-director-general-s-statement-on-ihr-emergency-committee-on-novel-coronavirus-(2019-ncov. Published January 30, 2020. Accessed December 5, 2020.
2. Arora S, Thornton K, Jenkusky SM, Parish B, Scaletti JV. Project ECHO: linking university specialists with rural and prison-based clinicians to improve care for people with chronic hepatitis C in New Mexico. Public Health Rep. 2007;122(suppl 2):74–77.
3. Arora S, Thornton K, Murata G, et al. Outcomes of treatment for hepatitis C virus infection by primary care providers. N Engl J Med. 2011;364(23):2199–2207.
4. Tiyyagura G, Asnes AG, Leventhal JM, et al. Impact of Project ECHO on community ED providers' perceptions of child abuse knowledge and access to subspecialists for child abuse and neglect. Acad Pediatr. 2019;19(8):985–987.
5. Hess D, Reed G, Turco M, Parboosingh J, Bernstein H. Enhancing engagement in practice improvement: a conceptual framework. J Contin Educ Health Prof. 2015;35(1):71–79.
6. Aveling EL, Martin G, Armstrong N, Banerjee J, Mary Dixon-Woods M. Quality improvement through clinical communities: eight lessons for practice. J Health Organ Manag. 2012;26(2):158–174.
7. American Academy of Pediatrics. Project ECHO (Extension for Community Healthcare Outcomes). Itasca, IL: American Academy of Pediatrics. www.aap.org/projectecho. Accessed January 28, 2021.
Keywords:

clinical practice; COVID-19 pandemic; pediatrics

© 2022 The Authors. Published by Wolters Kluwer Health, Inc.