Nursing education is challenging, exciting, and busy. Students are eager to learn and begin doing the things “that nurses do.” Student engagement is directly linked to student success.1,2 However, maintaining student engagement throughout a rigorous program is a difficult undertaking. Students have long days of class, laboratory, clinical practice, and hours of reading and homework. Understanding of effective educational practices is essential to promote student learning and retention and is critical to successfully prepare nursing students.
It is imperative that students are given opportunities to engage in active learning to improve their satisfaction, be more engaged, enhance learning, and retain knowledge. With current media and technology platforms, students have 24/7 access to learning tools. In addition, today's students who grew up with computers are technologically savvy and interested in more innovative ways of learning.3 Research has found that students were more satisfied and engaged when study content included technology4,5 and were more motivated and engaged when learning was gamified.6
Evidence shows up to 80% of university students do not read the assigned materials.7,8 For years, faculty have reported that students do not read all the assigned pages.9 In fact, some students never even purchase the required textbook.10 Knowing that students may not complete assigned readings in traditional textbooks, engaging didactic and interactive pedagogy driven by educational technology may incentivize students to learn. Such technology can address various learning preferences and create an active learning environment to keep students engaged and focused on learning. Thus, the use of technology may be helpful in promoting active learning and student engagement and satisfaction, especially as technology evolves, resulting in online interactive programs, videos, emails, podcasts, and web-based systems.11 Research has demonstrated students perceive that technology would improve their performance and facilitate their learning.12 In addition, faculty and students alike have become comfortable with the idea that learning can effectively occur through the use of technology.13
According to Kavanagh and Sharpnack,14 the current model of nursing education is disconnected from what is needed, and schools of nursing are losing ground on preparing practice-ready nurses. Therefore, faculty should move beyond outdated pedagogy to improve students' knowledge and thinking skills.15 Innovative technology is being integrated into nursing education. Educators have an opportunity to learn about these innovations and find ways to integrate them into their programs. Educators should embrace technology to transform education to meet the learning needs of students.14
Despite research demonstrating the use of technology in promoting student learning, there are other important factors to consider. Understanding the gap and finding the balance between didactic content and interactive technology could improve approaches to promote student learning. To determine student engagement in a textbook-replacement product that contains multiple modes of technology-based learning, researchers conducted this retrospective study.
Purpose
The purpose of this study was to examine the impact of an online interactive education program (OIEP) that incorporates multiple modes of technology-based learning for students. This may maximize engagement and knowledge acquisition in prelicensure nursing students. There are several advantages of interactive programs, but to our knowledge, this is the first study that assessed the efficacy of a product designed to replace traditional textbooks. Specifically, we assessed the following constructs in students: the perceived satisfaction with the OIEP; the perceived efficacy of the OIEP (eg, its usefulness and ease of use); and student engagement from a cognitive, affective, and behavioral perspective. Finally, students were asked to compare the OIEP with traditional textbooks and were asked whether this tool will help prepare them for the NCLEX and reduce burnout. Faculty were asked to report their satisfaction with the OIEP and efficacy; report on their students' cognitive, affective, and behavioral engagement; compare the OIEP with traditional textbooks; and report on this program's ability to help students prepare for the NCLEX and reduce both student and faculty burnout.
Methods
Sample and Design
The retrospective study followed a mixed-methods design, using both quantitative and qualitative data. Surveys were administered to both students and faculty midway through the semester, after gaining some experience using the OIEP, and again at the end of the semester. The OIEP is an online program that provides evidence-based, need-to-know content, including readings, text-to-speech listening, podcasts, simulations, animations, annotation and highlighting tools, note-taking, documentation practice in an online electronic health record, quizzing, and test banks using National Council State Boards of Nursing (NCSBN) Next Generation NCLEX (NGN) items. Students provided feedback throughout the program.
Data were collected at 2 time points to gauge the usability and efficacy of the OIEP after prolonged exposure to the content. In addition, collecting data at 2 time points allowed us to assess whether prolonged exposure to the OIEP was related to changes in students' perceptions of the benefit of using an online interactive tool versus the use of traditional textbooks. Qualitative data included open-ended questions about the program, student perceptions of its use as a textbook replacement, and how it compared with a traditional textbook.
Email invitations were sent to students and faculty from schools using products of the creators of the OIEP. These individuals were selected from a national sample, including a diverse geographical and educational background. Participants were asked about using a textbook-replacement product in their fundamentals of nursing courses. The surveys were administered via electronic survey software, and links to the surveys were provided to participants. The institutional review board approved the study as an exempt study.
Measures
Descriptive data of the student and faculty participants were collected. Kirkpatrick's Model of Assessment16 was used as the framework to assess the impact of the OIEP in relation to the constructs of interest stated earlier. Specifically, both the student and faculty surveys assessed satisfaction with the OIEP using 2 items (recommend the OIEP to other nursing students/instructors and how they would rate the OIEP, on a scale from poor to excellent). Furthermore, the OIEP's efficacy was assessed from both student and faculty perspectives using one item with multiple subcomponents rated on a scale from (1) not at all to (5) a very great extent. These items were adopted and modified from the Eden et al17 scale that assesses the learner's belief in a tool, which has been shown to correlate with changes in behavior.
In addition, 3 scales were included in both the student and faculty surveys to measure student engagement: (1) the Cognitive Engagement Scale, which measured how students process and retain new information using multiple statements; (2) the Affective Engagement Scale, which measured students' affect toward new learning content using multiple statements; and (3) the Behavioral Engagement Scale, which measured students' behavior toward classroom participation, attention, and engagement with learning material using multiple statements. For all 3 subscales, students responded on a scale ranging from (1) strongly disagree to (5) strongly agree. These scales were modified on the basis of the Hart et al18 validated Student Engagement in Schools Questionnaire (SESQ).
Students were asked questions comparing the OIEP with traditional textbooks. These items were based on Webster and Ho's19 article describing a set of constructs that assess audience engagement with multimedia presentations. The constructs included how challenging the medium is perceived by students, feedback of the learning medium, whether students felt control of their learning, whether a variety of media types that facilitated learning were used, and the level of student engagement when using this learning medium. Students' perceived comparison of the OIEP with traditional textbook items was rated on a 5-point Likert scale ranging from (1) not at all or to a very little extent to (5) to a very great extent.
Predictive measure items asked students to rate the extent to which they agreed that interactive learning materials such as the OIEP better prepared them for the NCLEX and reduce nursing student burnout. Finally, students were asked open-ended questions such as (1) Why they would recommend the OIEP to other students; (2) Whether they liked the OIEP more than a traditional textbook and why; (3) What they liked best about the OIEP; (4) What they liked least about the OIEP; and (5) What they would change to make the program more effective.
Faculty were also asked to compare the OIEP with a traditional nursing fundamentals textbook using the same measures that were described for students but framed for faculty response. Faculty were asked to what extent the OIEP would better prepare their students for the NCLEX, reduce students' burnout, and decrease their own burnout and answered the same open-ended questions as students.
Data Analysis
Quantitative data were analyzed using descriptive statistics for both student and faculty data (frequency counts for Likert scale responses, means, and standard deviations). Furthermore, student data on engagement were compared from time point 1 to time point 2 using the independent-samples t test to assess whether mean change occurred from time point 1 to time point 2.
The primary objective of the open-ended questions for students was to assess student perceptions of the OIEP and to also explore those perceptions in comparison with using a traditional textbook as a supplement to a prelicensure nursing course. Qualitative analysis was conducted via MAXQDA, a software program designed to streamline the process of coding qualitative data into useful themes.20 The coding process was primarily manual; however, the lexical search feature in the MAXQDA software allowed for key word search capabilities that streamlined our coding process. Frequency counts were generated after coding schemas were created.
Results
Student Data: Descriptive Statistics
The student sample at the first time point (N = 81) comprised an average age of 24 years, with 94% female. The student sample at the second time point (N = 78) had an average age of 30 years, with 89% female. At time point 1, more than 95% of the sample stated that they had in-person instruction, were full-time, and were a first-semester student. At time point 2, 88% of students stated that they had in-person instruction, 74% stated that they were full-time, and 78% stated that it was their first semester. In the first time point, 60 students reported their program of study: 72% associate degree in nursing (ADN), 18% licensed practical nurse (LPN)/licensed vocational nurse (LVN), 8% bachelor of science in nursing (BSN), and 2% diploma. In the second time point, 58 students reported their program of study: 59% ADN, 32% LPN/LVN, and 9% BSN.
Student Survey Results
Students reported high levels of satisfaction with the OIEP specifically at time point 2; 31% of students rated the OIEP as good, 35% rated it as very good, and 32% rated it as excellent. Most (91%) would recommend the OIEP to other nursing students. At time point 2, 80% of students rated the OIEP to be a useful tool, 71% well-constructed, 79% easy to use, 77% able to provide relevant client examples, 73% likely to help patients, 78% using realistic media, and 85% based on client scenarios that are relevant to the nursing practice.
Descriptive statistics were used to assess perceived student behavioral, cognitive, and affective engagement at both time points (Table 1). An independent-samples t test was used to compare the means of the items between both time points. Furthermore, independent-samples t tests were conducted to assess change from the semester's midpoint to the end of the semester on whether students changed in their perception of the OIEP relative to a traditional textbook the longer that they used the OIEP (Table 2).
Table 1. -
Student Engagement Survey Results
|
Post 1, M (SD) |
Post 2, M (SD) |
t Test P Value |
| OIEP increased my confidence in my ability to: |
| Understand the material better by relating it to things I already know |
3.83 (0.900) |
4.11 (0.808) |
<.05 |
| Figure out how the information might be useful in the real world |
3.84 (0.834) |
4.11 (0.713) |
<.05 |
| Understand how the things I learn fit together with each other |
3.86 (0.860) |
4.10 (0.783) |
ns |
| Think through topics and decide what I'm supposed to learn from them, rather than studying the topics by just reading them |
3.87 (0.900) |
3.96 (0.977) |
ns |
| Combine different pieces of information from the material in new ways |
3.84 (0.817) |
4.03 (0.857) |
ns |
| OIEP increased my: |
| Interest in what I am learning |
3.85 (0.940) |
4.17 (0.804) |
<.05 |
| Enjoyment for learning new things |
3.75 (0.940) |
4.06 (0.889) |
ns |
| As a result of the OIEP: |
| I work harder |
3.52 (0.968) |
4.01 (0.893) |
<.01 |
| I participate more |
3.62 (0.974) |
4.04 (0.842) |
<.01 |
| I pay more attention |
3.67 (0.944) |
4.09 (0.847) |
<.01 |
| My mind wanders less |
3.36 (0.958) |
3.75 (1.063) |
<.05 |
| I am better able to review difficult material until I understand it |
3.78 (0.982) |
4.16 (0.760) |
<.05 |
Abbreviations: ns, nonsignificant; OIEP, online interactive education program.
Table 2. -
Textbook Comparison
a
| Compared With Traditional Textbooks, the OIEP: |
Post 1, M (SD) |
Post 2, M (SD) |
t Test P Value |
| Sufficiently challenged me to learn more |
3.38 (0.975) |
3.91 (0.972) |
<.01 |
| Encouraged me to think more |
3.49 (0.925) |
4.03 (0.928) |
<.001 |
| Provided more direct feedback |
3.65 (1.039) |
4.09 (0.872) |
<.01 |
| Provided more timely feedback |
3.59 (1.084) |
4.18 (0.802) |
<.001 |
| Provided more clear and understandable feedback |
3.66 (1.024) |
4.00 (0.905) |
<.05 |
| Allowed the instructor to maintain more control over the direction of their lectures |
3.41 (1.084) |
3.92 (1.094) |
<.01 |
| More effectively supported lecture content |
3.61 (0.904) |
3.94 (1.029) |
<.05 |
| Utilized more of a variety of different types of media to facilitate learning |
3.68 (0.938) |
4.00 (0.945) |
<.05 |
| More effectively integrated media to facilitate learning |
3.61 (0.991) |
4.02 (0.893) |
<.05 |
| Held more of my attention |
3.47 (0.982) |
3.75 (1.160) |
ns |
| Excited my curiosity more |
3.56 (1.000) |
3.85 (1.041) |
ns |
| Aroused more of my imagination |
3.41 (0.984) |
3.62 (1.225) |
ns |
| Was intrinsically more interesting |
3.50 (0.969) |
3.76 (1.082) |
ns |
| Overall was more engaging |
3.59 (0.950) |
3.97 (1.073) |
<.05 |
Abbreviations: ns, nonsignificant; OIEP, online interactive education program.
aRatings based on the extent of agreement with statement.
Students were asked to indicate their agreement with the statement, “Courses that adopt interactive learning materials like the OIEP will better prepare me for the NCLEX if they are used throughout my nursing coursework.” There was a significant increase in mean agreement from time point 1 (M = 3.79, SD = 0.90) to time point 2 (M = 4.26, SD = 0.730, P < .001). We also asked students to indicate how much they agree with the statement, “Courses that adopt interactive learning materials like the OIEP will reduce nursing student burnout.” There was a significant increase in agreement from time point 1 (M = 3.39, SD = 1.1) to time point 2 (M = 3.88, SD = 1.1, P < .01).
Faculty Data: Descriptive Statistics
The average age of the faculty sample at the first time point (N = 28) was 38 years, with 85% female. The average age of the faculty sample at the second time point (N = 7) was 56 years, with 100% female. The majority of faculty members were employed full-time and taught synchronous in-person sections. The majority of faculty members who reported their degree level had either their master's (6 faculty members) or doctorate (2 faculty members) at time point 1; at time point 2, 5 faculty members had their master's and 1 had their doctorate.
Quantitative Measures-Faculty
Faculty were highly satisfied with the OIEP. At time point 1, 41% of the faculty rated the OIEP as good and 44% rated it as very good; 81% would recommend this program to other faculty members who teach nursing students. During this same time point, 74% of faculty rated the OIEP to be a useful tool, 67% well-constructed, 56% easy to use, 70% able to provide relevant client examples, 70% likely to help patients, 70% using realistic media, and 78% based on client scenarios that are relevant to the nursing practice. Unfortunately, the sample size at time point 2 was too small (N = 7) to conduct any meaningful comparisons using inferential statistics.
Descriptive statistics were used to assess faculty's perceptions of student behavioral, cognitive, and affective engagement in the OIEP at both time points (see Supplemental Digital Content, Table 1, available at: https://links.lww.com/NE/B302). Descriptive statistics were also used to assess faculty feedback when comparing the OIEP with traditional fundamentals nursing textbooks (see Supplemental Digital Content, Table 2, available at: https://links.lww.com/NE/B303). Faculty agreed that courses that adopt interactive learning materials such as the OIEP will better prepare students for the NCLEX if used throughout their nursing coursework (time point 1: M = 3.61, SD = 0.78; time point 2: M = 4.14, SD = 0.90), will reduce nursing student burnout (time point 1: M = 3.04, SD = 1.2; time point 2: M = 4.14, SD = 0.90), and will reduce nursing instructor burnout (time point 1: M = 3.54, SD = 0.66; time point 2: M = 4.14, SD = 0.69).
Student Data: Qualitative Measures
Students' open-ended questions include descriptions of (1) why nursing students would recommend the OIEP to other students, (2) whether they liked the OIEP more than a traditional textbook and why, (3) what they liked best about the OIEP, (4) what they liked least about the OIEP, and (5) what they would change to make the program more effective. Most (91%) students indicated they would recommend using the OIEP to other nursing students because it is informative and instructive (17%), is easy to understand (17%), easy to navigate (7%), and that they like the rationale and explanations (7%). Other themes we captured in this open-ended question were that students enjoyed the portability of the resource, could potentially pair it with a textbook, and it was applicable to real life.
From a sample of 68 students who answered this question, 52 students (76%) preferred the OIEP over the traditional textbook. Students cited similar reasons as they did for the previous open-ended question. From the responses, 12% of the students indicated that it was easy to navigate, 17% cited the interactivity as a benefit, 5% preferred the use of practice tests, and 5% favored this online learning resource due to its helpful videos.
Faculty Data
Although the faculty sample size was too small for a qualitative analysis, there were themes identified that mirrored student responses. Of the 13 faculty members who answered the survey, 69% indicated that they were using the OIEP as a replacement for a textbook for their classes and 86% would recommend it to other faculty members as they prefer it over the traditional textbook. Reasons for recommendations included that the program was current, concise, and engaging. It was easy to use, comprehensive, and easy to navigate. Students only had to sign into one platform and look for information, which helped decrease confusion. Faculty also mentioned that the OIEP integrated active learning and helped assess students' understanding of the material.
Discussion
Quantitative and qualitative results demonstrate the benefits of the OIEP along with student and faculty preference over using a textbook that students often do not purchase or read. Students were satisfied using the OIEP at both time points. The effect of using the OIEP showed a positive impact on students' perception of behavioral, cognitive, and affective engagement at both time points. The OIEP allowed students to have an increased perception of understanding, interest, and focus after using the product. Some student perceptions on engagement and preference over a traditional textbook increased from time point 1 to time point 2. In addition, students believed that products such as the OIEP can help better prepare them for the NCLEX and reduce burnout.
The faculty, similar to students, were also satisfied with the OIEP. However, because of the sample size limitations, we were unable to make conclusions about the change in faculty perception of behavioral, cognitive, and affective engagement, in addition to changes regarding preference over a traditional textbook.
Qualitative results demonstrate the benefits of the OIEP as an interactive learning tool. Students preferred using the OIEP over the traditional textbook, as it augmented their ability to access information provided an iterative learning experience and more practice opportunities. Integrating interactive components in the OIEP allows learners to review content areas that they may need remediation and helps students identify their own learning needs. The program allowed students to navigate directly to content that was identified as areas requiring additional learning more efficiently than with a traditional textbook. The practice tests exposed students to NCLEX examination questions and allowed them to receive timely feedback.
The OIEP provides formative and summative feedback along with scaffolding throughout the learning process, which is critical for good instruction. The scaffolding of information throughout the program and the activities for reinforcing learning are integrated within each module of the OIEP. In summary, the OIEP was well received by students and faculty. Nursing students would benefit from wider integration of the OIEP and other online instructional tools into nursing curricula.
Limitations
Limitations include inability to conduct inferential statistical analyses between time points among faculty members due to the small sample size. Another limitation was the inability to perform a group comparison through a matched-sample t test for student and faculty responses between time point 1 and time point 2. Because student responses were unable to be matched, a group analysis using an independent-samples t test was performed. However, we do not know to what extent the sample overlapped from time point 1 to time point 2. Although nursing program students were recruited and had previously been exposed to other company resources, some participants may have been biased toward the OIEP.
Conclusions
The preliminary results of this retrospective study indicate that nursing students could benefit from the integration of educational technology such as the OIEP into prelicensure nursing curricula. Integrating interactive learning tools offers nursing students opportunities to be more engaged in their learning, allowing for flexibility in their learning approach, how they engage with new information, and how they review previously taught concepts. Online learning tools allow students and faculty to better monitor progress, offer students immediate feedback, and are easily combined with coursework to support the acquisition and retention of knowledge. This is achieved by providing multiple opportunities for reviewing content identified as knowledge gaps and practice tests. Both student and faculty can use the feedback to enhance student learning.
References
1. Reckmeyer M. Focus on Student Engagement for Better Academic Outcomes. Gallup.com. October 30, 2019. Accessed July 25, 2022.
https://www.gallup.com/education/267521/focus-student-engagement-better-academic-outcomes.aspx
2. Hudson H. Student Engagement Equals Student Success: Engaging Students Inside & Outside the Classroom. Xello. July 28, 2020. Accessed December 20, 2022.
https://xello.world/en/blog/strategies-student-engagement
3. Culha I. Active learning methods used in nursing education. J Pedag Res. 2019;3(2):74–86. doi:10.33902/jpr.2019254174
4. Ortega R, Akhtar-Khavari V, Barash P, Sharar S, Stock MC. An innovative textbook: design and implementation. Clin Teach. 2017;14(6):407–411. doi:10.1111/tct.12587
5. Chan SL, Lin CC, Chau PH, Takemura N, Fung JTC. Evaluating online learning engagement of nursing students. Nurse Educ Today. 2021;104:104985. doi:10.1016/j.nedt.2021.104985
6. Killam LA, Timmermans KE, Shapiro SJ. Motivation and engagement of nursing students in 2 gamified courses. Nurse Educ. 2021;46(6):E173–E187. doi:10.1097/nne.0000000000001065
7. Zeivots S. Up to 80% of Uni students don't read their assigned readings. Here are 6 helpful tips for teachers. The Conversation. Accessed August 27, 2022.
https://theconversation.com/up-to-80-of-uni-students-dont-read-their-assigned-readings-here-are-6-helpful-tips-for-teachers-165952
8. Anning R. Are textbooks becoming obsolete? May 2, 2019. Accessed July 22, 2022.
https://elearningindustry.com/textbooks-becoming-obsolete
9. Smale M. “It's a lot to take in”—undergraduate experiences with assigned reading. IMPACT. 2020;9(1). Accessed July 22, 2022.
https://academicworks.cuny.edu/ny_pubs/316
11. Gee BM, Salazar L, Porter J, Clark C, Peterson TW. Overview of instructional
technology used in the education of occupational therapy students: a survey study. Open J Occup Ther. 2017;5(4). doi:10.15453/2168-6408.1352
12. Harerimana A, Mtshali NG. Nursing students' perceptions and expectations regarding the use of
technology in nursing education. Afr J Nurs Midwifery. 2019;21(2):1–20. doi:10.25159/2520-5293/5103
13. Roth A, Dominique T. Putting the Ed before the tech: evaluation and assessment of elearning. Comput Libr. 2020;40(6). Accessed December 20, 2022.
https://escholarship.org/uc/item/43j3w4xc
14. Kavanagh J, Sharpnack P. Crisis in competency: a defining moment in nursing education. OJIN. 2021;26(1):2. doi:10.3912/OJIN.Vol26no01Man02
15. Forneris S, Fey M, League N. Critical Conversations: The NLN Guide for Teaching Thinking. Wolters Kluwer; 2018.
16. Kirkpatrick DL, Kirkpatrick JD. Evaluating Training Programs: The Four Levels. Readhowyouwant.com; 2010.
17. Eden D, Ganzach Y, Flumin-Granat R, Zigman T. Augmenting means efficacy to boost performance: two field experiments. J Manage. 2008;36(3):687–713. doi:10.1177/0149206308321553
18. Hart SR, Stewart K, Jimerson SR. The Student Engagement in Schools Questionnaire (SESQ) and the Teacher Engagement Report Form-New (TERF-N): examining the preliminary evidence. Contemp Sch Psychol. 2011;15:67–79.
19. Webster J, Ho H. Audience engagement in multimedia presentations. ACM SIGMIS Database. 1997;28(2):63–77. doi:10.1145/264701.264706
20. VERBI Software. MAXQDA. VERBI Software; 2020.
