Secondary Logo

Share this article on:

Building a Global, Online Community of Practice: The OPENPediatrics World Shared Practices Video Series

Wolbrink, Traci A. MD, MPH; Kissoon, Niranjan MD; Mirza, Nabila MPH; Burns, Jeffrey P. MD, MPH

doi: 10.1097/ACM.0000000000001467
Innovation Reports

Problem Health care professionals are familiar with engaging in local communities of practice (CoPs) within their hospital, region, and/or country, but despite the availability of online technologies that facilitate online global collaboration, the health care sector has yet to fully embrace these tools.

Approach In 2013, OPENPediatrics (an online social learning platform) launched the World Shared Practices video (WSP) series to engage and coalesce the global community of critical care clinicians. Each month, a 30- to 45-minute video featuring a pediatric critical care medicine expert, interspersed with questions for the audience, is released. Viewers contribute to the community discussion by leaving comments that display alongside the video. Clinicians are encouraged to asynchronously host an educational conference so they can watch the videos and participate in the discussion together.

Outcomes From March 2013–November 2015, 28 WSPs were launched on a variety of topics. They were viewed over 18,414 times by 1,864 viewers in 132 countries and 760 hospitals; 1,155 comments were submitted. Attending physicians/consultants were the largest audience (36% [671/1,864]), and 37% (30/81) of responding viewers that commented in WSPs watched in small groups. The WSP series was reported to add value to respondents’ learning or teaching and to have had a positive impact on their knowledge or practice.

Next Steps Future research will focus on further describing the context and structure of the CoP and on more deeply investigating its higher-level outcomes and impact. More work is needed to identify barriers and strategies that improve online community engagement.

Supplemental Digital Content is available in the text.

T.A. Wolbrink is assistant professor, Department of Anaesthesia, Harvard Medical School, and associate, Division of Critical Care Medicine, Department of Anesthesia, Perioperative and Pain Management, Boston Children’s Hospital, Boston, Massachusetts.

N. Kissoon is professor, Department of Pediatrics and Department of Emergency Medicine, University of British Columbia, and vice president of medical affairs, British Columbia Children’s Hospital and Sunny Hill Medical Center, Vancouver, British Columbia, Canada.

N. Mirza is a data analyst, OPENPediatrics Program, Boston Children’s Hospital, Boston, Massachusetts.

J.P. Burns is professor, Department of Anaesthesia, Harvard Medical School, and chief, Division of Critical Care Medicine, Department of Anesthesia, Perioperative and Pain Management, Boston Children’s Hospital, Boston, Massachusetts.

Funding/Support: In-kind support was generously given by the Department of Anesthesia, Perioperative and Pain Management, Boston Children’s Hospital, and the IBM Corporation.

Other disclosures: The authors have no personal financial disclosures to report. OPENPediatrics is a not-for-profit entity in collaboration with the World Federation of Pediatric Intensive and Critical Care Societies.

Ethical approval: This project and its component analyses were approved for exemption by the Boston Children’s Hospital institutional review board.

Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A402 and http://links.lww.com/ACADMED/A403.

Correspondence should be addressed to Traci A. Wolbrink, Boston Children’s Hospital, 300 Longwood Ave., Bader 634, Boston, MA 02115; telephone: (617) 355-7327; e-mail: traci.wolbrink@childrens.harvard.edu.

Back to Top | Article Outline

Problem

Communities of practice (CoPs) have been defined as “groups of individuals formed around common interests and expertise” that, when leveraged, can “solve problems [and] transfer best practices.”1 CoPs include a combination of both active and passive members and facilitate master–apprentice learning, whereby novices gradually increase knowledge and experience with the help of experts, often through passively watching or listening to experts, while experts gain knowledge through more active interaction with the community.2 Health care professionals are familiar with engaging in local CoPs within their hospital walls, region, and/or country, but despite the availability of online technologies that facilitate online global collaboration, the health care sector has yet to fully embrace these tools,3 which may be attributable to cultural preferences, privacy concerns, technical issues, perceived time constraints, difficulty in adapting these tools for health care professionals, and/or difficulty in engaging health care professionals with these tools.4

CoPs are heterogeneous in audience, scope, and goals, and thus, there is no standard validated framework for evaluating CoPs.2,3,5 Reported evaluation frameworks of medical CoPs include assessments of one or more of the following components of the CoP5:

  1. Goals/scope: description of what the CoP is seeking to accomplish;
  2. Stage of development: description of the CoP’s development stage (i.e., building, growth, adaptive, closed);
  3. Context: the external environment in which the CoP exists or the characteristics of the CoP itself (i.e., people, organization, system);
  4. Structure: description of the composition of the CoP, often using description measures (i.e., size, existence of subgroups) or social network analysis;
  5. Process/activities: description of CoP activities and ways the CoP intends to achieve its goals;
  6. Outcomes: positive or negative results of the CoP from an individual, group, or organizational level; and
  7. Level of impact: value or potential value of the CoP to the individual or entire group.

Studies most commonly report these components by describing site activity information and member surveys or interviews.2,5

In this report, we describe our innovative approach to initiate a global, online pediatric critical care CoP through an activity that combines both traditional in-person educational conferences and online interaction, and we evaluate it using the above components (we have listed the components in parentheses where we report on the corresponding information, with the exception of the outcomes, which are reported throughout the Outcomes section of this report).

Back to Top | Article Outline

Approach

OPENPediatrics6 is an online social learning platform developed at Boston Children’s Hospital in collaboration with the World Federation of Pediatric Intensive and Critical Care Societies and the IBM Corporation. Released in September 2012, OPENPediatrics has acquired over 7,443 registered users from over 1,800 hospitals in 132 countries as of November 2015. OPENPediatrics has three sites through which users can access information: a public open-access site, a secure clinician site, and a YouTube channel. On average, each month the open-access site receives 15,000 page views, the clinician site receives 2,000 log-in sessions, and the YouTube channel receives 10,000 page views.

OPENPediatrics launched the World Shared Practices video (WSP) series in March 2013 as a strategy to engage and coalesce the global community of critical care clinicians (CoP stage of development—building and growth). Each month, a well-known, international expert in pediatric critical care medicine is invited to be a speaker for the WSP series by a group of four faculty members in pediatric critical care medicine at Boston Children’s Hospital, all of whom are professors at Harvard Medical School and themselves experts in the field. The WSPs are filmed during the experts’ visits to Boston Children’s Hospital, and subsequently edited and reviewed by the speaker and one or two critical care faculty members before their release on the OPENPediatrics sites. Two to five questions pertaining to context and practice are interspersed into each 30- to 45-minute video, often with the speaker stating their expert opinion after posing the question to the audience.

Once released, a WSP can be viewed whenever it is convenient for the viewer. Viewers contribute to the community discussion by leaving comments that display alongside the video (CoP goals/scope—using the WSP as primary medium, community discussion is encouraged), without formal comment moderation. As CoPs are more likely to share knowledge when a combination of in-person and online activities are used,2 clinicians are encouraged to hold an educational conference at their hospitals so that they can watch the videos together, discuss responses to the questions, and submit a single group comment.

Back to Top | Article Outline

Conceptual framework of the WSP series

In a worldwide needs assessment conducted prior to developing OPENPediatrics, approximately half of respondents requested a question-and-answer forum (53% [228/429]), best practice webinars (48% [208/429]), and a place to network and talk with doctors and nurses worldwide (43% [186/429]).7 To address these needs, we conceptualized the WSP series. We chose an asynchronous format for the WSP series for several reasons. First, videos could be optimized for online viewing by incorporating engaging graphics, animations, and figures. Second, videos allow and encourage equal opportunities for participation worldwide, a goal that is unlikely to be accomplished by webinars presented at a set time which may be convenient for some but inconvenient for others. Although webinars can be recorded and archived for later viewing and discussion, in our experience, few clinicians return to watch archived webinars. Additionally, clinicians may be less likely to actively contribute to an educational opportunity for which a live portion has already been completed. Finally, having a scheduled release date and encouraging the incorporation of WSPs into their local educational conference schedule might enhance participation.

Back to Top | Article Outline

WSP release and global interaction

A WSP is released on the fourth Tuesday of each month, a recurring and predictable event that can easily fit into local educational conference schedules. While watching the video, the audience can respond to the interspersed questions posed to the global CoP, such as “How do you make the diagnosis of low cardiac output?” Viewers can see the responses from other participants and comment on their own practices (CoP context—global conversations about individual practices in pediatric critical care medicine) (see Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/A402).

Back to Top | Article Outline

Measures of community engagement and data analysis

We used embedded site analytics and Web-based surveys (see below) to gather the following data from March 2013 to November 2015 (CoP process/activities—metrics used to assess viewer engagement):

  • The total number of video views and unique viewers, viewer names and roles, hospital names and locations, and comments on the clinician site were reported by IBM Cognos Analytics (IBM Corporation, Armonk, New York).
  • The number of views on the public site and YouTube channel were reported by Vimeo (New York, New York) and YouTube (San Bruno, California) analytics. (As no log-in is required for these public sites, unique user information was not available.)
  • The annual OPENPediatrics user experience survey was sent to all OPENPediatrics users on April 15, 2015, via SurveyMonkey (Palo Alto, California).
  • A survey was sent to the viewers who commented on a WSP on the clinician site asking about WSP-viewing activity on November 19, 2015, via SurveyMonkey (Palo Alto, California).

We analyzed data using Microsoft Excel 2013 (Microsoft Corporation, Redmond, Washington) and SAS 9.4 (SAS Institute Inc., Cary, North Carolina).

Back to Top | Article Outline

Outcomes

Between March 2013 and November 2015, 28 WSPs were launched on a variety of pediatric critical care topics; these were viewed 18,414 times by 1,864 unique viewers in 132 countries and 760 hospitals. On average, each WSP was viewed over 247 times in the first 30 days following its release; 69% (4,748/6,928) of these views came from the clinician site, 24% (1,671/6,928) from the public site, and 7% (509/6,928) from the YouTube channel (see Supplemental Digital Appendix 2 at http://links.lww.com/ACADMED/A403).

On the clinician site, user engagement was most active during the first 30 days following a WSP release, with videos receiving almost half (4,748/11,109) of their lifetime views during this time. Log-ins to the clinician site were highest during the week prior to and the week of a WSP release, when release and save-the-date e-mails were sent to the OPENPediatrics distribution list. Approximately one-third (2,934/10,627) of the total clinician site visits during WSP release weeks included a WSP view. The average lifetime total views for each WSP on the clinician site were approximately 400 views by about 186 viewers (see Supplemental Digital Appendix 2 at http://links.lww.com/ACADMED/A403).

Twenty-five percent (1,864/7,443) of our registered users, a diverse global audience (CoP structure—the size and composition of our CoP) (see Table 1), viewed at least one WSP. Attending physicians/consultants, an audience that, in our experience, is often reluctant to engage with new technology, made up the largest group of WSP viewers at 36% (671/1,864).

Table 1

Table 1

Videos that covered sepsis, cardiovascular, and respiratory topics generated over twice as much participation and, on average, were viewed 130 times by 60 viewers each, as compared with videos on history and quality improvement, which, on average, were viewed 55 times by 30 viewers each (see Supplemental Digital Appendix 2 at http://links.lww.com/ACADMED/A403).

Eight percent (149/1,864) of WSP viewers on the clinician site contributed to community discussions and left 1,155 comments. On average, each video received approximately 40 comments from about 14 viewers on the clinician site (see Supplemental Digital Appendix 2 at http://links.lww.com/ACADMED/A403). After controlling for the number of viewers from each region, there was no statistically significant difference in the number of comments received by location (F = 0.89) (see Table 1). Forty-seven percent (876/1,864) of WSP viewers viewed at least two WSPs, and 15% (280/1,864) viewed five or more. There is a positive correlation between the number of comments submitted by viewers and the number of WSPs they viewed (Spearman correlation coefficient = 0.32, P < .001). Viewers that commented are highly engaged members in this global, online CoP, watching on average seven or eight WSPs and commenting half of the time, suggesting that there are some champions (or very engaged users). Of viewers who left comments who responded to our survey (81/149 from 40 countries), 37% (30/81) reported viewing the WSPs in small groups with an average of 8 clinicians. Although the number of actual commenters was small, reporting the number of comments is not an adequate metric to capture and reflect vigorous discussions that might be occurring locally when individuals in a hospital view such an online educational lecture together, as described in a local pediatric intensive care unit CoP in Canada.8 The small number of comments may also be indicative of the magnitude of health care professionals’ online engagement.9

Ninety-six percent (1,111/1,155) of comments were responses to the interspersed questions, and 4% (44/1,155) were general comments or questions. We thought that interspersing questions in the videos would stimulate discussions among clinicians locally and globally, and indeed, most of the comments were related to these questions. However, our hope that these questions would allow clinicians to become comfortable with commenting in an online platform and stimulate additional discussions over time was not realized. Instead, we observed a decrease in additional commenting over time. Possible explanations include clinicians not feeling comfortable asking additional questions when no other peers were doing so, clinicians feeling intimidated to ask questions of the expert WSP speakers, or perhaps additional questions were being answered in local discussions. To try to encourage ongoing discussion, the WSP speakers responded to questions for the lifetime of the WSP, and 48% (21/44) of general questions were answered by the WSP speaker or the speaker’s designee.

Although our study was not designed to assess impact in practice or clinical outcomes, our 2015 annual OPENPediatrics user experience survey (480/7,443 respondents from 84 countries) offered insight into perceived satisfaction and behavioral changes related to the WSPs (CoP level of impact—satisfaction with and change in knowledge or practice resulting from CoP engagement). Fifteen percent (56/376) of respondents reported that the goal of their last visit to OPENPediatrics was to watch a WSP. Ninety-six percent (310/323) of respondents reported that the WSPs were valuable or very valuable to their learning or teaching, and 15% (21/144) of respondents reported that the WSP series led to a positive change in their knowledge or practice.

Twenty-one percent (1,107/5,226) of viewers on the clinician site watched an entire video in a single sitting (see Supplemental Digital Appendix 2 at http://links.lww.com/ACADMED/A403), which is much higher than expected for the approximately 40-minute videos. A recent study by the online education site edX about student engagement with online math and science videos found that watching an entire video declined rapidly when videos were more than 6 minutes long.10 The difference seen on our clinician site was likely due to our audiences having different motivations for viewing the WSPs. We used well-known experts as speakers, likely increasing interest. Also, scheduling a structured time to watch together in a traditional educational conference may have helped promote longer view times. Finally, clinician viewers likely found the videos relevant and valuable to their practice, as supported by the number of viewers that returned to watch more than one WSP and reported satisfaction with the WSP series (see above).

Our study has some limitations. One limitation is the difficulty in providing an evaluation of our CoP that allows us to benchmark our work against other CoPs. This is related to the heterogeneous nature of CoPs and the lack of agreement on a standardized framework for evaluating CoP activities.2,3,5 Although we described our CoP using basic metrics similar to those reported in the literature, more work is needed to better detail the context and structure of our CoP and to more deeply investigate its higher-level outcomes and impact.

Back to Top | Article Outline

Next Steps

Emerging technologies hold the potential to enhance global collaboration in the field of medicine across clinical, education, and research domains. Through our novel approach with the WSP series, we have demonstrated the potential to leverage online technologies to coalesce a global, online CoP in a format that could be used to stimulate discussion and collaboration within other health professional CoPs. The success of this video series may be due to using well-known subject experts to increase interest, encouraging local group viewings, and providing an asynchronous format for convenience. WSPs are viewed by a large percentage of our OPENPediatrics users, who reported finding the videos to be valuable for clinical practice. It is often difficult to engage busy clinicians in online activities, but we found that there were some champions who participated in the WSPs regularly. Active discussions may occur in local CoPs but cannot be adequately captured by current metrics.

Our future research will focus on further describing the context and structure of our CoP and more deeply investigating its higher-level outcomes and impact through more formal structured qualitative evaluation strategies. We will also seek to identify barriers that limit and strategies that improve clinician engagement with online communities through online surveys and structured interviews. Using enhanced analytic capabilities, we will also aim to identify and report additional metrics from other social interaction tools on the OPENPediatrics site to better explore the social learning behaviors of our CoP.

Acknowledgments: The authors would like to thank the pediatric critical care community of practice, especially the hospital champions who organize the local conferences, for their engagement with the World Shared Practices videos (WSPs), both viewing and commenting. The authors would also like to thank the expert speakers who agreed to be interviewed and filmed for the videos and the entire OPENPediatrics team for their hard work, creativity, and attention to detail required to produce the WSPs, especially Annalise Littman for her medical video editing and Aron Willey, Karen Gerofskey, Brandon Moye, and David Humphreys for their expert production. Finally, the authors wish to thank the IBM Corporation for their in-kind support in developing the platform to host OPENPediatrics.

Back to Top | Article Outline

References

1. Wegner E, McDermott RA, Snyder W. Cultivating Communities of Practice: A Guide to Managing Knowledge. 2002.Boston, MA: Harvard Business School Press Books.
2. Barnett S, Jones SC, Bennett S, Iverson D, Bonney A. General practice training and virtual communities of practice—A review of the literature. BMC Fam Pract. 2012;13:87.
3. Li LC, Grimshaw JM, Nielsen C, Judd M, Coyte PC, Graham ID. Use of communities of practice in business and health care sectors: A systematic review. Implement Sci. 2009;4:27.
4. Cheston CC, Flickinger TE, Chisolm MS. Social media use in medical education: A systematic review. Acad Med. 2013;88:893–901.
5. McKellar KA, Pitzul KB, Yi JY, Cole DC. Evaluating communities of practice and knowledge networks: A systematic scoping review of evaluation frameworks. Ecohealth. 2014;11:383–399.
6. OPENPediatrics. Homepage. http://www.openpediatrics.org. Accessed August 10, 2016.
7. Wolbrink TA, Kissoon N, Burns JP. The development of an Internet-based knowledge exchange platform for pediatric critical care clinicians worldwide. Pediatr Crit Care Med. 2014;15:197–205.
8. Collet JP, Skippen PW, Mosavianpour MK, et al. Engaging pediatric intensive care unit (PICU) clinical staff to lead practice improvement: The PICU participatory action research project (PICU-PAR). Implement Sci. 2014;9:6.
9. Chan T, Thoma B, Lin M. Creating, curating, and sharing online faculty development resources: The medical education in cases series experience. Acad Med. 2015;90:785–789.
10. Guo P. Optimal video length for student engagement. edX blog. November 13, 2013. http://blog.edx.org/optimal-video-length-student-engagement. Accessed August 10, 2016.

Supplemental Digital Content

Back to Top | Article Outline
© 2017 by the Association of American Medical Colleges