Journal Logo

Empirical Investigations

Learners' Perceptions During Simulation-Based Training

An Interview Study Comparing Remote Versus Locally Facilitated Simulation-Based Training

Duch Christensen, Margrethe MD; Oestergaard, Doris PhD, MD, FSSH; Dieckmann, Peter PhD, Dipl-Psych, FSSH; Watterson, Leonie MBBS, FANZCA, MClinEd, GradCertOrgCoach&Lship

Author Information
Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare: October 2018 - Volume 13 - Issue 5 - p 306-315
doi: 10.1097/SIH.0000000000000300

Abstract

Remotely facilitated simulation-based training (RF-SBT) is a synchronous distance education method where instructors facilitate in real time via web or video conferencing and simulators are, in some instances, operated by remote control. Several studies have reported that RF-SBT is effective and feasible.1–5 Notwithstanding, statistically significant differences have been observed in several studies, whereby some learners rate the RF-SBT less positively than conventional formats, in which the instructor provides locally facilitated (LF-SBT) face-to-face instruction. These results revealed that in response to RF-SBT, some participants reported feeling uncomfortable, where communication was a barrier to learning, and the quality of instruction was inferior and/or expressed unwillingness to participate in future training.3,5,6 These findings are consistent with our observations, drawn from unpublished evaluations of a large-scale RF-SBT education program that our group had previously provided for rural health professionals. Furthermore, we observed that participants appeared willing to overlook RF-SBT's deficiencies because these were outweighed by the perceived benefits of having access to SBT within their local setting.

Simulation literature published to date does not explain what aspects of RF-SBT technology negatively impact learners. Consequently, it is not possible to posit which educational theories best explain these effects. This issue may be informed by studies of synchronous distance education, such as videoconference-delivered tutorials and supervision, where negative learner experiences have also been reported, particularly in relation to the challenges faced by instructors to master the technology and to achieve “telepresence” with learners.7,8 These findings are replicated in research on participant and instructor “presence” within asynchronous distance education and virtual environments.9,10 However, they may not wholly translate to RF-SBT.

Meanwhile, several theoretical models or conceptual frameworks published within the simulation literature could be applied to RF-SBT to explain negative learner experiences of RF-SBT. Cognitive load theory (CLT) proposes that learning is most effective when mental workload associated with engaging with a task (intrinsic workload) and effort to control attention away from extraneous factors (extraneous workload) are minimized.11 In RF-SBT, the requirement for learners to interact with an instructor via a monitor may potentially increase intrinsic mental workload by impairing nonverbal communication during debriefing. Meanwhile, extrinsic mental workload may increase if the monitor acts to distract the learners from the task of managing the clinical scenario. Taking a different perspective, Rudolph et al12 recognized that learners' emotional activation also influences their cognitive processing. Their conceptual model of psychological safety may explain learners' disquiet with RF-SBT as it emphasizes the importance of achieving contextual understanding and respectful relationships between learners and instructors as prerequisite conditions for learner engagement. In their theoretical discussion of realism, Dieckmann et al13 also proposed that learners' cognitive and emotional responses to simulation are interdependent. They proposed that learners' capacity to immerse fully in a simulation is dependent on the following three factors: learners' interpretation of their physical environment, their conceptualized meaning of events in scenarios, and their emotional responses to these.13 Other work has demonstrated that low levels of realism reduce learners' immersion.14

Research conducted further afield in the information technology sector identifies the technology acceptance model (TAM) as an another potentially useful theoretical construct to guide use of RF-SBT.15 The TAM (see Figure (a), Supplemental Digital Content 1, https://links.lww.com/SIH/A363; which demonstrates the model diagramatically) predicts that users' behavioral intentions to adopt new technology are determined by their negative or positive attitudes toward it. These are in turn determined by two sets of factors: the technology's “perceived usefulness,” defined as the extent to which users feel the technology will lead to improved job performance, and its “ease of use,” defined as the extent to which users believe that the technology will be easy to use. The TAM also predicts that behavioral intent is most sensitive to perceived usefulness. That is, the impact of negative attitudes on behavioral intent will be outweighed and overridden by highly perceived usefulness. The TAM has been applied in higher education research to guide and/or evaluate innovative approaches to e-learning16,17 and seems to be closely aligned with our previously mentioned observations on users acceptance of RF-SBT, despite reporting negative attitudes toward it.

Aims

We explored newly graduated nurses' and doctors' perceptions of RF-SBT and LF-SBT, aiming to explore their perceptions of the relative value of the two and to identify factors that might influence their attitudes to, and acceptance of, these methodologies. This addressed our broader aims to understand the factors that contributed to negative attitudes toward RF-SBT and to identify theory that explains its potentially negative impact.

METHODS

Overview

Telephone interviews were conducted with course participants shortly after completing a standardized simulation course that included simulation scenarios delivered in both RF-SBT and LF-SBT formats.

Setting

The study was conducted at the Sydney Clinical Skills and Simulation Centre, between April 2013 and January 2014, after approval from the Northern Sydney Local Health District Human Research Ethics Committee (1302-062M). The setting for the study was an established course for newly graduated doctors and nurses addressing management of the deteriorating patient. The study was conducted with the cohort of learners reported in the previously mentioned study by Christensen et al.6 Detailed descriptions of the setting for this are available in that publication.

The 4-hour course included four, 30-minute simulation scenarios, of which one scenario used the RF-SBT format and three scenarios used the LF-SBT formats. The remainder of the course comprised interactive small group presentations. Scenarios followed the “pause, discuss, and coach” line of enquiry and discussion, whereby the instructor paused the scenario one or two times to enable discussion before continuing from the stop point. Participants worked in groups of six to eight of whom four to six were allocated an active role in the scenario and the remaining participants observed. All participants were actively included in the discussion during pauses and after the scenario. A postscenario debriefing followed each scenario. Instructors' briefings were standardized with scripts aligned to the principles of psychological preparation described by Rudolph et al.12,18 We also attempted to achieve consistency in the delivery of the debriefings by equipping instructors with debriefing prompts similar to those reported by other simulation educators.19

Differences Between LF-SBT and RF-SBT

To achieve optimal psychological safety, participants were briefed about learning with SBT at the beginning of the course and just before each scenario. A script developed specifically for the RF-SBT format pre-empted some of the potential challenges of this technology, regarding interaction with the instructor and other participants.18 In the LF-SBT–formatted scenarios, the instructor remained collocated with participants in the simulation room. The RF-SBT scenarios were delivered in a similar manner except that the instructor interacted with the group via a monitor and speakers from another room located on-site using a CISCO conferencing system. A surveillance camera transmitted video images of the participants to the instructor (Fig. 1). In all respects, the technology set-up was similar to that reported in previously published studies.1–3,6

FIGURE 1
FIGURE 1:
Demonstration of simulation training using RF-SBT format. View of instructor conducting training workshop via RF-SBT. Monitor is split to display images of participants and instructors.

Instructors

The instructors had completed the course-specific instructor accreditation program, which assumed a minimal level of competence with constructive student enquiry-focused debriefing20 and the course instructional design, as described previously. They had variable experience delivering RF-SBT. The more experienced instructors regularly used RF-SBT as part of an established simulation education program for rural health professionals. Inexperienced instructors received 4 hours of instructor training. As previously reported, participants were allocated to groups to ensure even exposure to experienced and inexperienced instructors.

Participants

Course participants comprised newly graduated nurses and doctors in their first 2 years of postgraduate employment. Thirty of the 157 course participants consented to interview. An iterative approach to sampling was used, whereby the responses from interviews were analyzed continuously for new content and new participants were interviewed until saturation was reached, providing confidence that key themes were captured.13

Interviews

All telephone interviews were conducted by the first author (MDC) and recorded. The interviews used a standardized semi-structured template (see Table, Supplemental Digital Content 2, https://links.lww.com/SIH/A364; which demonstrates examples of interview questions). An iterative questioning approach was used in that issues and themes that appeared to emerge in early interviews were included as questions in latter interviews. The protocol also allowed the interviewer to ask probing questions at her discretion to explore emerging themes.13 This aimed to prompt the participants to recall how they reacted and responded in the moment to the experience and consequently reveal their acceptance of the methodologies and preferences, if these existed.

Data Analysis

The transcripts were transcribed verbatim by a professional transcription service and crossed-checked for accuracy by MDC. Transcripts were deidentified and analyzed using Malterud method for systematic text condensation.14 This method, derived from Giorgi psychological phenomenological analysis, uses a four-stage approach to thematic cross-case analysis of qualitative data, whereby text analysis passes through stages of gestalt impression, systematic coding, condensation of codes to themes and in the final stage, synthesis, and conceptualization. In accordance with this method, the authors worked independently and collectively in stages with two authors (MDC and LW) participating in all stages and two authors (DO and PD), consulting on the last stage of condensation and conceptualization, as follows. In stage 1, they independently read the transcripts to get an overall impression of the content with an open mind while resisting any temptation to code text elements. Subsequently, they shared their initial impressions and generated a preliminary set of themes. Guided by these, in stage 2, they independently re-read the transcripts coding narrative according to these. In stage 3, the authors discussed and reached consensus regarding the meaning of these themes and condensing participants' citations into definitions, such as those shown in Table 2. In stage 4, the themes were developed into a conceptual model that reflected their interrelationships.

TABLE 2
TABLE 2:
Enabler Factors Identified in Learners' Responses to SBT

Rigor

Several measures were undertaken to ensure the authors' own presence, and involvement in the study did not unduly influence results. The first author, who conducted the interviews, abstained from teaching in the courses or developing any familiarity with participants. A script was followed at the beginning of the interviews to encourage participants to be open with their responses. After text analysis, one author (LW) subsequently returned to the original transcripts to cross-check and validate the coding, themes, and the conceptual model.

RESULTS

Participants

A total of 21 participants were interviewed. Participants comprised twelve nurses and eight doctors. Nurses included eight females and four males and doctors included four of each sex. In general, the participants had little previous exposure to immersive simulation with 13 participants reporting they had participated in SBT on fewer than three previous occasions. The interviews were conducted within 48 hours of the course and their duration ranged from 9 to 25 minutes.

Participants' Attitudes to and Acceptance of SBT

All participants appraised both formats as acceptable and beneficial to their learning. Ten participants expressed a preference for the LF-SBT format, one expressed a preference for the RF-SBT format, seven indicated that they had no preference, and three did not state a preference. As described in the following sections, different aspects of the training evoked positive or negative reactions from the participants whose individual responses varied.

Key Themes and their Relationships

Participants responses revealed that they were highly aware of their reactions and responses to different aspects of the SBT scenarios. From their citations, we generated three broad themes, which were categorized into perceptions, enablers, and precursors, after agreeing that participants used consistent phrasings and applied consistent meaning to these. As a key finding, participants also actively attributed their perceptions to enabler and precursor factors, in a causal relationship. Precursors, representing factors present before the simulation and enabling factors, relating to how the simulation was delivered, seemed to impact on perceptions as if in an “input-process-output” relationship (see Figure, Supplemental Digital Content 3, https://links.lww.com/SIH/A365; which demonstrates the relationships between these themes). The results presented in the following sections are organized according to these themes. Supporting citations are presented in Tables 1 to 3.

TABLE 1
TABLE 1:
Participants' Perceptions of SBT

Perceptions

The term perception was used to denote statements that revealed participants' awareness of the SBT's immediate impact on them.15 Key perceptions are presented in Table 1. Participants generally revealed their perceptions in relation to a positive or negative attitude toward one aspect of either format; however, examples of each perception were reported in each format. For example, the RF-SBT format was associated with most comments about surveillance; however, probing questions revealed that discomfort or dissatisfaction also occurred from being observed in the LF-SBT format. Perceptions seemed to have predictable positive or negative impact on participants' attitudes. For instance, high levels of perceived surveillance were consistently associated with a negative attitude (See Figure, Supplemental Digital Content 3, which shows the direction of the arrow beside each listed perception as a predictor of its positive or negative impact on participants' attitudes to the SBT).

Text analysis also revealed that perceptions were often associated with phrases evoking emotions and/or social relationships with others and/or the participants' understanding of the situation. We coded the impact as psychosocial or cognitive, respectively (Table 1, column D). Some perceptions were expressed with predominantly psychosocial phrasing, for example, the perception of belonging, whereas others, for example, control of attention, conscious mental effort, and task engagement, were predominantly concerned with the participants' cognition. However, other perceptions, including surveillance, responsibility, realism, and contextual understanding, had a psychosocial impact on some participants and an impact on the cognitive processes of other participants.

Participants frequently used the term “presence.” On some occasions, it was used to describe a state of being present with the instructor or relating to the instructor in a social sense that led to a perception of belonging. On other occasions, as explained in the following section, participants referred to the instructor's physical presence or interactive behaviors that enabled the participants' learning or perceptions.

Enablers

Participants generally attributed their perceptions to one or more other factors. These associations, outlined in Table 2, were classified as enablers if they described aspects of the delivery of the simulation.

Instructors' interactive behaviors were consistently mentioned as a key enabler across both formats. Participants frequently mentioned the instructors' face-to-face interactivity in the LF-SBT in positive terms, whereas they reported mixed reactions to interacting with instructors via video conference monitors, in the RF-SBT format. The lack of face-to-face interaction in the latter impacted negatively on several additional factors. These included the following: the interpretation of nonverbal cues, the efficiency of communication, the instructor' capacity to physically demonstrate skills or to use their physical proximity to enhance practical learning, and their ability to achieve a sense of belonging through their interactive behaviors. We coded the latter two factors as physical and social presence, respectively, deriving these terms from existing literature.9 Notably, however, participants also reported that instructors could mitigate this negative impact through effective behaviors such as using participants' names, using direct eye gaze to the camera and animated body language.

Equipment-related enablers included the quality of audiovisual technology and/organization of the learning environment. The audiovisual quality was reported to have a slightly negative impact by some learners mainly due to audio delays that interrupted the flow of communication. This resulted in increased mental effort, reduced contextual understanding, reduced realism, and/or reduced task engagement. However, when probed, participants generally rated audiovisual quality as good. Participants that reported a negative experience of the technology more frequently described factors related to the organization of the learning environment. Participants recalled their unsettled response to entering the scenario room, devoid of the facilitator, their feelings of dissociation from viewing an image of the instructor and themselves on screens, the awkwardness that they felt from shifting their gaze from peers in the simulation laboratory to the instructor on the screen, and their cognizance of the separation of audio and video communication. Participants also recalled the intrusive positioning of the camera, the instructors' nonsustained eye gaze into the camera, and, on some occasions, the fact that some participants were cut out of the camera field. Collectively, these factors impacted all eight perceptions in a negative manner.

Participants who were allocated active roles in the scenarios also tended to mention aspects of instructional design that impacted on their experience, including the inclusion of procedures that may require practical demonstration.

As with perceptions, it was evident that enablers could have a psychosocial impact on some participants and a cognitive impact on others. For instance, in the following citation, a sense of belonging seems to be engendered by the instructor's use of the environment and social presence:

“The way that the (RF-SBT) screen was positioned, it almost felt like the instructor was standing, speaking to us. So, it felt very personal.” [Participant S]

In contrast, the following quote suggests that the same factors had an impact on the mental effort of the participant.

“There was quick interaction (with LF-SBT). Everything was normal… It felt right. There was a quicker reaction time between questions.” [Participant B]

Interdependence between enablers was commonly observed. As an example, the following quote demonstrates a relationship between the instructor's questioning technique, videoconferencing technology and group interaction, and the impact of these on participants' sense of social connectedness (belonging).

“I don't think the discussion (in RF-SBT format) was as open (as that) with the face-to-face instructor. We weren't sort of bouncing off each other, (…) one person at a time was asked a question and then they responded. Whereas, when it was face-to-face, it was more of a group discussion which I felt a lot better about.” [Participant D]

Precursors

Factors that were present before the commencement of the simulation and had a causal association with participants' perceptions were coded as precursors (Table 3). Participants' previous experience and preparation for simulation training seemed to be a major factor. Similarly, participants appeared able to identify whether instructors were experienced in respect to their mastery of the RF-SBT technology, their clinical expertise, and their general interactive behaviors.

TABLE 3
TABLE 3:
Precurser Factors Identified in Learners' Responses to SBT

Implementation Issues

All participants indicated that the RF-SBT format had advantages in some circumstances, for example, by improving access to education for geographically remote learners. All agreed that it had potential disadvantages and attributed these to reduced ability to hear and see the instructor, for example, if it diminished the instructors' opportunities to physically demonstrate skills or if it led the instructor to miss important information about the participants in the scenario. Other examples related to losing closeness with the instructor although when probed, all felt that the effectiveness of RF-SBT was probably subject to conditions. Participants suggested strategies to optimize the RF-SBT format (Table 4).

TABLE 4
TABLE 4:
Strategies to Increase RF-SBT's Ease of Use

DISCUSSION

Key Findings

As a key finding of this study, learners undergoing simulation training reported being actively aware of multiple perceptions, representing their awareness of its immediate impact on them at an emotional, social, or cognitive level. These perceptions positively or negatively influenced the participants' attitudes to the learning and contributed to a preference for the LF versus RF formats.

As mentioned, learners also actively attributed their perceptions to enabler and precursor factors, in a causal relationship, appearing to impact on perceptions as if in an input-process-output relationship. Human interaction emerged as a significant multidimensional enabling theme influencing participants' experience of SBT. This comprised a range of factors, several of which have been previously reported, such as the availability of nonverbal cues and the capacity for instructors to mitigate this with effective use of eye gaze and verbal communication.16

However, in our opinion, the participants' elucidation of what it meant to “be present” with the instructor in the LF-SBT format compared with remote interaction provided most new insights into the potential limitations of RF-SBT. Face-to-face interaction with the instructors increased the participants' sense of having a personal social relationship with them, which we coded as belonging, and contributed positively to other psychosocial and cognitive perceptions such a realism, surveillance, and contextual understanding. Meanwhile, participants distinguished between the enabling effects of instructors' social behaviors versus their capacity to provide physical support. The main implication of these findings is that faculty should endeavor to mitigate the negative impact of the RF-SBT format on learners' emotional and cognitive responses through a range of interactive strategies that act to increase their social presence with the learners and to identify strategies that compensate for their inability to perform physical tasks.

As expected, technology and the way instructors interact with it were also key enablers. These factors are potentially able to be improved with careful attention to room set-up and instructor training. Suggested strategies, such as those presented in Table 4, provide practical guidance to providers wishing to introduce the RF-SBT technology.

Theoretical Basis of the Findings

A broad aim of the study was to identify theories and conceptual frameworks that explained RF-SBT's potentially negative impact on learners. Several appear relevant to this goal.

Cognition: CLT

Cognitive load theory proposes that learning is most effective when learners have spare cognitive capacity to actively reflect on the learning activity in which they are engaged (germane workload). This is achieved when mental workload allocated to completing the task at hand (intrinsic workload) or to resisting extrinsic stimuli that complete with and distract from the task (extrinsic workload) are minimized.11 Cognitive load theory seems to explain some of the findings of this study, in relation to perceptions. Here, task engagement and conscious mental effort are closely aligned to existing concepts of intrinsic cognitive workload, whereas control of attention and realism tend more to reflect extraneous cognitive workload (Table 1, Column D).

Social Practice: Situated Learning Theory and Related Concepts

The dynamic interdependence of the learners' perceptions and their emotional, social, and cognitive responses to the simulation tasks were key findings in this study. The foundations of simulation as a form of social practice are well recognized. In this regard, simulation is conceptually aligned with situated learning theory by emphasizing learning as an unconscious act that is embedded in the context of the situations in which it normally occurs.21 As previously mentioned, Dieckmann et al13 conceptualized the construct of realism in terms of social practice, where it reflects a confluence of the following three elements: the learners' interpretation of their physical environment, the semantical meaning of the task, and their phenomenal response. By involving “emotions, beliefs, and self-aware cognitive states,” the latter closely approximates the perceptions identified in this study. Rudolph et al12 elaborated these concepts in the construct of the fiction contract, one of four conditions underpinning psychological safety and consequently, learner engagement. The other elements of psychological safety—attending to logistics, achieving contextual understanding, and demonstrating respect—also appear well aligned to the perceptions of belonging and contextual understanding.

Instructor Presence: The Social Presence Model

Research conducted in the fields of distance and virtual education contributes to our understanding of the impact of instructor presence in RF-SBT. Three types of presence are recognized in virtual educational settings. Physical presence explains a learner's sense of being situated in place; co-presence indicates a learner's sense of participation in an activity with others and social presence signifies the salience or perceived importance of interacting with another person.10 The social presence model (SPM) proposes that the meaning learners derive from a learning activity reflects the quality of interactions among learners and instructors.10 In virtual learning environments, increasing social presence and emotional involvement correlate positively with learner engagement and perceived relevance and are considered to improve recall through association.22 The SPM extends our understanding of instructor behaviors in novel simulation settings. For instance, strategies employed by instructors to create telepresence through immediacy and intimacy in distance education may also achieve these in RF-SBT. Examples include the following: being friendly and attentive to greetings and closings to engage learners; self-disclosure, using humor and encouragement and otherwise building relationships with learners by addressing them by name; using eye gaze to suggest direct eye contact; and using inclusive terms such as “we.”8,23,24

Behavioral intent: The TAM

The TAM is one of several theories that describes behavioral intent,15 and while initially created to describe users' adoption of information technology, it has been used widely to evaluate adoption of information technology–based education.16 In our view, and more so than any of the previously mentioned theories or concepts, the TAM addresses the incongruence between participants' reported acceptance of RF-SBT despite holding some negative attitudes toward it. To elaborate this, we have adapted schematic representation of the TAM by Davis et al15 (See Figure (b), Supplemental Digital Content 1, which demonstrates the alignment between these study findings and the TAM). Here, perceptions equate with ease of use factors and are determined by enabler and precursor factors, as previously described. The TAM predicts that learners' acceptance of RF-SBT reflects the balance between these perceptions and its perceived value, with the latter outweighing and potentially overriding the former.

This has two key implications. Firstly, RF-SBT is likely to be well received when its perceived value is high. From our experience, this is likely to occur when learners otherwise have limited access to SBT or the opportunities with which it is associated, such as the following: preparation for clinical roles, team-based training, rehearsal of rare events, or training in specific disciplines. Secondly, without careful attention to delivery, learners may reject RF-SBT, either by refusing to participate or disengaging from it. This may occur if the balance of perceptions is strongly negative, reflecting a breach of the learners' psychological safety and/or excessive intrinsic or extrinsic cognitive workload.

Study Strengths and Limitations

A key strength of the study was that the learners were exposed to both RF-SBT and LF-SBT formats, thus allowing them to compare the two. This comparison demonstrated that while several perceptions were commonly associated with negative experiences of RF-SBT, the same perceptions can arise in either format to a greater or lesser degree. This finding should encourage providers to be attuned to learners' psychosocial and cognitive responses to SBT regardless of format.

A limitation of the study was the use of post hoc interview to investigate real-time thought processes. Arguably, a more precise method to achieve this could involve immediate recall methods, such as think aloud verbalizing or stop and probe questioning, which have been used in studies of clinical reasoning and situation awareness, respectively.20 However, these methods may potentially increase cognitive load and were avoided for this reason.

The results of this study should also be interpreted within the context of precursor factors that characterized participants, instructors, and instructional design elements in this study. For example, the study focused on a cohort of new graduates most of whom were not familiar with immersive simulation methods. Consequently, we might predict that perceptions reflecting cognitive workload will increase compared with those of learners who are more familiar with SBT. Moreover, participants were exposed to only one scenario in the RF-SBT format. This relative lack of familiarity with the RF-SBT format may have contributed to a reduced sense of social presence, as predicted in other studies.22 Although instructor experience was not found to influence participants' quantitative evaluation of the RF-SBT format in our cohort,6 participants perceived different levels of experience and skills among the instructors based on their verbal and nonverbal communication. Elsewhere, these skills are predicted to influence participants' appreciation of social presence7,8 and may have had some bearing on the findings of this study.

CONCLUSIONS

This study demonstrates that SBT evokes a range of emotional, social, and cognitive responses in learners that influence their attitudes toward learning with this method. These findings address the study aims by elucidating unexplained factors that negatively impacted on learners' perceptions of the RF-SBT format despite the high perceived value they attributed to it. By comparing learners' perceptions of both RF-SBT and LF-SBT formats, we conclude that the perceptions, enablers, and precursors identified in this study are thematically relevant to SBT whether it is delivered face to face or by remote facilitation. In this sense, the study provides additional means by which to evaluate SBT in any setting and can be applied to faculty education and program quality control.

REFERENCES

1. Treloar D, Hawayek J, Montgomery JR, Russell W. Medical Readiness Trainer Team. On-site and distance education of emergency medicine personnel with a human patient simulator. Mil Med 2001;166(11):1003–1006.
2. Ikeyama T, Shimizu N, Ohta K. Low-cost and ready-to-go remote-facilitated simulation-based learning. Simul Healthc 2012;7(1):35–39.
3. Hayden EM, Navedo DD, Gordon JA. Web-conferenced simulation sessions: a satisfaction survey of clinical simulation encounters via remote supervision. Telemed J E Health 2012;18(7):525–529.
4. Ohta K, Kurosawa H, Shiima Y, et al. The effectiveness of remote facilitation in simulation-based pediatric resuscitation training for medical students. Pediatr Emerg Care 2017;33:564–569.
5. Ahmed R, King Gardner A, Atkinson SS, Gable B. Teledebriefing: connecting learners to faculty members. Clin Teach 2014;11(4):270–273.
6. Christensen MD, Rieger K, Tan S, Dieckmann P, Østergaard D, Watterson LM. Remotely versus locally facilitated simulation-based training in management of the deteriorating patient by newly graduated health professionals: a controlled trial. Simul Healthc 2015;10(6):352–359.
7. Davies R, Yeung E, Mori B, Nixon SA. Virtually present: The perceived impact of remote facilitation on small group learning. Med Teach 2012;34(10):e676–e683.
8. Cameron M, Ray R, Sabesan S. Remote supervision of medical training via videoconference in northern Australia: a qualitative study of the perspectives of supervisors and trainees. BMJ Open 2015;5(3):e006444.
9. Bulu ST. Place presence, social presence, co-presence, and satisfaction in virtual worlds. Comput Educ 2012;58:154–161.
10. Kiliç Çakmak E, Çebib A, Kan K. Developing a “social presence scale” for e-learning environments. Educ Sci: Theory Prac 2014;14(2):764–768.
11. Young JQ, Van Merrienboer J, Durning S, Ten Cate O. Cognitive load theory: implications for medical education: AMEE Guide No. 86. Med Teach 2014;36(5):371–384.
12. Rudolph JW, Raemer DB, Simon R. Establishing a safe container for learning in simulation: the role of the presimulation briefing. Simul Healthc 2014;9(6):339–349.
13. Dieckmann P, Gaba D, Rall M. Deepening the theoretical foundations of patient simulation as social practice. Simul Healthc 2007;2(3):183–193.
14. Hagiwara MA, Backlund P, Söderholm HM, Lundberg L, Lebram M, Engström H. Measuring participants' immersion in healthcare simulation: the development of an instrument. Hagiwara et al Advances in Simulation. 2016;1(17):1–9. Available at: https://advancesinsimulation.biomedcentral.com/articles/10.1186/s41077-016-0018-x. Accessed December 16, 2017.
15. Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: a comparison of two theoretical models. Manag Sci 1989;35(8):982–1003.
16. Persico D, Manca S, Pozzi F. Adapting the technology acceptance model to evaluate the innovative potential of e-learning systems. Comput Hum Behav 2014;30:614–622.
17. Tarhini A, Hassouna M, Sharif Abbasi M, Orozco J. Towards the acceptance of RSS to support learning: an empirical study to validate the technology acceptance model in Lebanon. Electron J e-Learn 2015;13(1):30–41.
18. Christensen MDRK, Tan S, Dieckmann P, Østergaard D, Watterson LM. Appendix 1, Remotely Versus Locally Facilitated Simulation-based Training in Management of the Dete-riorating Patient by Newly Graduated Health Professionals: A Controlled Trial.: Simulation in Healthcare; 2016 Available at: http://journals.lww.com/simulationinhealthcare/_layouts/15/oaks.journals/ImageView.aspx?k=simulationinhealthcare:2015:12000:00004&i=TT3&year=2015&issue=12000&article=00004&type=Fulltext. Accessed December 16, 2017.
19. Arafeh JM, Hansen SS, Nichols A. Debriefing in simulated-based learning: facilitating a reflective discussion. J Perinat Neonatal Nurs 2010;24(4):302–309.
20. Eppich W, Cheng A. Promoting Excellence and Reflective Learning in Simulation (PEARLS): development and rationale for a blended approach to health care simulation debriefing. Simul Healthc 2015;10(2):106–115.
21. Palaganas J, Flanagan B, Simon R. Debriefing – theory and techniques. In: Richard H. Riley, editor. Manual of Simulation in Healthcare. 2nd ed. London: Oxford University Press; 2016;166–184.
22. De Leo G, Diggs LA, Radici E, Mastaglio TW. Measuring sense of presence and user characteristics to predict effective training in an online simulated virtual environment. Simul Healthc 2014;9(1):1–6.
23. Fullwood C, Doherty-Sneddon G. Effect of gazing at the camera during a video link on recall. Appl Ergon 2006;37(2):167–175.
24. Lahaie U. Strategies for creating social presence online. Nurse Educ 2007;32(3):100–101.
Keywords:

Remote facilitation; teledebriefing; distance education; simulation; telemedicine; debriefing; qualitative study; technology acceptance model

Supplemental Digital Content

Copyright © 2018 Society for Simulation in Healthcare