Journal Logo

Feature Articles

Assessing the Impact of Telehealth Objective Structured Clinical Examinations in Graduate Nursing Education

Phillips, Tiffany A. DNP, RN, NP-C; Munn, Allison C. PhD, RN; George, Tracy P. DNP, RN, APRN-BC, CNE

Author Information
doi: 10.1097/NNE.0000000000000729


Telehealth is a way to provide remote health care services to patients utilizing digital technology.1 Telehealth is increasingly being used to provide care in the United States. In 2014, 90% of health care executives reported their organizations were creating or implementing telehealth programs.2 There is growing acceptance of telehealth visits by patients. One study in 2017 found that 65% of patients were very or somewhat interested in telehealth visits with their primary care provider.2 Patients report high satisfaction with telehealth visits and cite convenience, perceived quality of care, and decreased cost as very important.1,3

To meet the demands of a rapidly changing health care system, it is important to include telehealth in nurse practitioner (NP) education.4 Informatics, including telehealth, can be integrated into multiple NP courses so that graduates are better prepared for clinical practice.5 Simulation experiences with telehealth may promote acceptance and skill with telehealth delivery among NP students.6 In 2018, the National Organization of Nurse Practitioner Faculties (NONPF) released a white paper supporting the incorporation of telehealth in NP education in the didactic or practicum settings and/or simulation experiences.7 There is a need to prepare students to use telehealth through didactic education, telehealth simulation experiences, clinical telehealth rotations, and by involvement in telehealth projects.6 Erickson et al8 implemented both didactic and clinical experiences successfully in an NP program, which increased student knowledge of telehealth.

Telehealth is a growing area of health care delivery, especially in rural areas. NPs may work in rural settings and need experience with telehealth, and simulation may improve their competence with it. Therefore, there is a need to incorporate more telehealth simulation experiences among NP students. The purpose of this article is to describe the implementation of a no-cost telehealth simulation and its impact on students in their first NP clinical course.

Literature Review

The effectiveness of simulation has been established among NP students. Simulation has been used to teach NP students how to take health histories and perform health assessments and a wide range of other skills and for evaluation.9,10 In NP curricula, simulation in the form of objective structured clinical examinations (OSCEs) has been used to evaluate a variety of skills including history taking and physical assessment and to integrate cultural humility.11,12 OSCEs are a standardized way for faculty to evaluate clinical competencies using scenarios and grading rubrics.13 Through OSCEs, students can apply knowledge to a clinical scenario, which enables faculty to evaluate student performance, provide strategies to strengthen clinical skills, and better prepare students for the clinical setting.13,14

Telehealth simulation can be an effective teaching strategy. Palmer and colleagues15 found significant improvements in NP students' knowledge and confidence regarding the practice of telehealth when compared with another group of students who did not participate in telehealth simulation. Additionally, an interprofessional telehealth simulation was used to effectively teach team-based communication skills among NP, physical therapy, and occupational therapy students.16 Rutledge et al17 implemented a multimodal approach, including a simulation workshop, telehealth simulation immersions, and written projects, which were well accepted by students.


Design and Sample

This study was conducted at a public university in rural South Carolina. The study was approved by the authors' University Institutional Review Board. NP students in the hybrid master's degree program complete a 2-year plan of study. The participants in this study consisted of 28 first-year NP students in their first clinical course.

Telehealth OSCEs

Didactic telehealth content is provided in the advanced health assessment course, which is taught prior to the first clinical course. Students were introduced to the telehealth OSCE during a live course orientation at the beginning of the semester, wherein the details of and grading of the experience were thoroughly explained. Two faculty members participated in all telehealth simulations, with 1 faculty member acting as the scripted patient and the other acting as the nurse and telepresenter. The faculty members were unchanged throughout all telehealth OSCEs. The NP students were in the role of the telehealth providers for this simulation.

Telehealth OSCEs took place during weeks 5 and 6 of a traditional 15-week semester and included only head-ear-eyes-nose-throat (HEENT) conditions that were taught during week 1 of the course and tested during week 5 of the course. Students were informed during the live course orientation that HEENT complaints and diagnoses would be the focus of the telehealth OSCE experience.

Telehealth OSCEs were conducted using a free video conferencing system, which allowed the students and faculty to communicate, view physical assessment images, and hear audio files of the cardiac and respiratory sounds via screen sharing. No telehealth-specific equipment was required for this OSCE. During the OSCE, each student completed a remote telehealth visit, in which he/she asked history questions of the patient and instructed the nurse to conduct appropriate portions of the physical assessment. Representative images and assessment sounds were used from internet sources and shared with students, including images of the tympanic membrane, throat, and nasal mucosa and audio clips of the cardiovascular and respiratory sounds. Each student then presented his/her diagnosis and plan to the scripted patient. At the conclusion of the visit, the student was debriefed by both faculty members. Each student had 30 minutes to complete the telehealth simulation visit, consult evidence-based resources, and present his/her diagnosis and plan to the scripted patient. Feedback was provided to students immediately after the telehealth OSCE by both faculty members.

Evaluation Methods

Students completed an anonymous online survey about the impact of telehealth OSCE simulation both prior to and after completion of the OSCE simulation. The survey was adapted from an instrument developed by Palmer and colleagues,15 and the authors granted permission for survey utilization and adaptation in this investigation. The original survey established content validity through an expert panel review.15 The adapted survey did not establish further measures of validity or reliability. The survey assessed students' telehealth knowledge and skills and included 15 items to determine student understanding of telehealth, attainment of skills necessary to implement telehealth, and confidence and comfort with the implementation of telehealth. Survey items were scored using a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree). Students completed 5 open-ended survey questions where they had an opportunity to further express their perceptions of the telehealth OSCE experience.

Data Analysis

Preimplementation survey and postimplementation survey responses were compared using Wilcoxon signed ranks testing with an α set at .05. The postimplementation survey also included student demographic and clinical experience items and contained the 5 open-ended questions. χ2 analyses were performed to determine differences between student groups by age, race/ethnicity, and clinical work experience. Mann-Whitney U tests were performed to determine if there were differences in preimplementation and postimplementation survey scores in terms of years of clinical experience or age.

Qualitative analyses of the open-ended survey responses were conducted using Microsoft Excel and directed content analysis.18-20 The directed approach provides the researcher with a clear and concise participant description of the experience with minimal interpretation. Two authors coded the text responses, labeled high-frequency words as nodes, and determined categories based on those nodes. From the categories, themes emerged about the telehealth OSCE experience. A third author reviewed the responses and separately coded the text. Agreement was reached on any differences.


Students' Characteristics

Twenty-three students completed the preimplementation survey, and 22 students completed the postimplementation survey (Supplemental Digital Content, Table 1, Participants included 22 females. Seventeen (73.9%) of students were white/Caucasian, 5 (21.7%) were black/African American, and 1 (4.3%) was Asian. Age range categories included 18 to 25 years (17%), 26 to 35 years (57%), 36 to 45 years (13%), and 46 to 55 years (13%). Students were relatively equal in distribution between groups for years of clinical experience as an RN (1-3 years [30%], 4-6 years [30%], 7-10 years [17%], and >10 years [22%]).

Telehealth Knowledge, Skills, and Comfort

All Likert scale survey items demonstrated an increase in scores from preimplementation to postimplementation reflecting improved student knowledge, skill, and comfort after the telehealth experience. Thirteen of the 15 items yielded statistically significant differences in preimplementation and postimplementation survey responses (Supplemental Digital Content, Table 2, Students' general understanding of the field of telehealth was improved after the telehealth OSCE experience (preimplementation survey median = 3.00, postimplementation survey median = 4.00, z = −3.11, P = .002).

Student Characteristics and Response to Telehealth

Chi square analyses revealed no significant differences between groups by age, race/ethnicity, and clinical work experience (all P > .05). Mann-Whitney U tests determined that there were no differences in the groups' rating of comfort and confidence in telehealth by both age and years of clinical work experience. Thus, the telehealth OSCE experience was beneficial for students of all ages and experience levels.

Open-ended Responses

Students provided rich feedback about the telehealth OSCE experience and offered many positive comments about the experience and learning environment. Student reports aligned with quantitative results that the experience was beneficial and provided new learning and tools for practice. Students reported feeling nervous at first, but then finding comfort in providing services through telehealth by the end of the simulation.

Directed content analysis of the responses revealed 3 themes: usefulness of telehealth, benefit in role preparation, and technology. Students were surprised about the level of care that can be provided through telehealth visits. One student reported, “You can do a number of things remotely that I wasn't aware you could do.” Students perceived telehealth as useful in treating a variety of patient populations: “It's a way to provide care to an underserved population that currently is lacking in appropriate access to providers.” Students believed their history-taking and physical examination abilities improved through the telehealth simulation. Statements such as “I feel more comfortable with my assessment skills” and “this experience may prepare me in the future as an NP to participate in a telehealth job” reflect the benefit of the experience to build student confidence and familiarity with an innovative method of patient care delivery (Supplemental Digital Content, Table 3,

Reliance on technology was a limitation to the experience, and 1 student had difficulty with the telehealth OSCE due to her remote location and the lack of a reliable internet connection. This served as a valuable lesson learned to ensure that Wi-Fi signal, computer capabilities, and examination tools are functioning at a proper capacity before treating patients using telehealth methods.


Both quantitative data and open-ended survey responses revealed positive student perceptions of the telehealth OSCE simulation, and students responded well to the experience regardless of age or years of clinical experience. All students indicated that they learned from the experience and could easily perform a telehealth visit with patients. Students were exposed to the telehealth methods and technology and stated a likelihood that they would use telehealth in their future practice due to increased comfort with the technology after the simulation experience. This is a valuable skill for NPs who will likely practice in underserved areas with rural populations who could greatly benefit from telehealth services.

Telehealth OSCEs are an effective way to incorporate telehealth into NP education as recommended by NONPF.7 Additionally, the use of simulation early in NP clinical courses allows for the identification of students whose clinical skills are subpar so they can receive additional guidance.13 This telehealth simulation experience ensured all students received direct experience with telehealth while in the NP program. Similar to the findings of Palmer and colleagues,15 the students in this study had improvements in their knowledge and understanding of telehealth. Additionally, the telehealth OSCEs were incorporated without the use of specialized equipment, which would have required additional purchases by the university. The use of representative physical assessment images allowed faculty to incorporate abnormal findings into the telehealth OSCE simulation. In an online program, telehealth OSCEs could possibly strengthen faculty-student connections and engagement.

A limitation of this study is that it involved a small number of students at 1 university over a single semester. In addition, it was at a rural university, and internet connection was an issue with 1 student because of the remote location. Also, the survey used did not have evidence of reliability or validity. Future studies could include students in various clinical courses in the NP program, not only in the first clinical course. Additionally, telehealth simulation could be integrated into the prelicensure nursing program. The use of telehealth equipment could be incorporated into the telehealth simulation OSCE.


As the delivery of health care continues to evolve through telehealth, NP programs must stay current in their preparation of graduates through didactic, clinical, and simulation experiences. The use of telehealth simulation is an effective strategy to assess clinical competency, provide individualized feedback, and ensure students are using evidence-based practice.


1. Polinski JM, Barker R, Gagliano N, Sussman A, Brennan TA, Shrank W. Patients' satisfaction with and preference for telehealth visits. J Gen Intern Med. 2016;31(3):269–275.
2. American Well. Telehealth index: 2017 consumer survey. 2017. Available at Accessed March 5, 2019.
3. Powell RE, Henstenburg JM, Cooper G, Hollander JE, Rising KL. Patient perceptions of telehealth primary care video visits. Ann Fam Med. 2017;15(3):25–229.
4. Hawkins SY. Telehealth nurse practitioner student clinical experiences: an essential educational component for today's health care setting. Nurse Educ Today. 2012;32(8):842–845.
5. Swenty CL, Titzer JL. A sense of urgency: integrating technology and informatics in advance practice nursing education. J Nurse Pract. 2014;10(10):e57–e67.
6. Rutledge CM, Kott K, Schweickert PA, Poston R, Fowler C, Haney TS. Telehealth and eHealth in nurse practitioner training: current perspectives. Adv Med Educ Pract. 2017;8:399–409.
7. Rutledge C, Pitts C, Poston R, Schweickert P. NONPF supports telehealth in nurse practitioner education. 2017. Available at Accessed March 5, 2019.
8. Erickson CE, Fauchald S, Ideker M. Integrating telehealth into the graduate nursing curriculum. J Nurse Pract. 2015;11(1):e1–e5.
9. Rutherford-Hemming T. Learning in simulated environments: effect on learning transfer and clinical skill acquisition in nurse practitioner students. J Nurs Educ. 2012;51(7):403–406.
10. Tiffen J, Corbridge S, Shen BC, Robinson P. Patient simulator for teaching heart and lung assessment skills to advanced practice nursing students. Clin Simul Nurs. 2011;7(3):e91–e97.
11. Loomis JA. Expanding the use of simulation in nurse practitioner education: a new model for teaching health assessment. J Nurse Pract. 2016;12(4):e151–e157.
12. Ndiwane AN, Baker NC, Makosky A, Reidy P, Guarino AJ. Use of simulation to integrate cultural humility into advanced health assessment for nurse practitioner students. J Nurs Educ. 2017;6(9):567–571.
13. Beckham ND. Objective structures clinical evaluation effectiveness in clinical evaluation for family nurse practitioner students. Clin Simul Nurs. 2013;9(10):e453–e459.
14. Rutherford-Hemming T, Jennrich JA. Using standardized patients to strengthen nurse practitioner competency in the clinical setting. Nurs Educ Perspect. 2013;34(2):118–121.
15. Palmer RT, Rdesinski RE, Galper A, et al. Assessing the impact of a telemedicine simulation on clinical learners. J Family Med Community Health. 2017;4(5):118–121.
16. Shortridge A, Steinheider B, Ciro C, Randall K, Costner-Lark A, Loving G. Simulating interprofessional geriatric patient care using telehealth: a team-based learning activity. MedEdPORTAL. 2016;12:1–7.
17. Rutledge CM, Haney T, Bordelon M, Renaud M, Fowler C. Telehealth: preparing advanced practice nurses to address healthcare needs in rural and underserved populations. Int J Nurs Educ Scholarsh. 2014;11(1):1–9.
18. Bengtsson M. How to plan and perform a qualitative study using content analysis. NursingPlus Open. 2016;2:8–14.
19. Hesse-Biber SH. The Practice of Qualitative Research: Engaging Students in the Research Process. 3rd ed. Thousand Oaks, CA: Sage; 2017.
20. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–1288.

nurse practitioner students; objective structured clinical examination (OSCE); patient simulation; telehealth

Supplemental Digital Content

Copyright © 2020 Wolters Kluwer Health, Inc. All rights reserved