INTRODUCTION AND REVIEW OF LITERATURE
Physical therapist education programs are challenged to provide students with exposure to an expanding amount of content in order to produce competent entry-level practitioners within a doctoring profession.1-3 The knowledge base now required of Doctor of Physical Therapy (DPT) degree students to successfully become a physical therapist has expanded due to advances in medicine and the growing amount of evidence related to our profession.4 Programs have achieved this increased exposure to content with relatively little change in time to degree completion since the transition to the DPT degree. This increase in exposure has been accomplished with the same or less time for practice in the classroom, while placing more demand on acquisition and integration of material in the clinical settings.1-3
Challenges exist for educators to ensure students are educated at a doctoral degree level and that the breadth and depth of knowledge allows for practice across all settings.3,5 Students are expected to integrate and apply knowledge during patient care activities.1 Increased student expectation places responsibility on faculty to ensure students are knowledgeable in basic foundational coursework and have higher-level critical-thinking skills prior to participation in hands on clinical practice during internships. A burning question often asked by educators today is “How do we fit it all in?” Educators are attempting to use a variety of instructional strategies in the classroom to assist in learning, and in retaining and applying content required for successful clinical practice with a focus on the higher-level cognitive skills of problem solving and clinical-decision making. In recent years, technology has emerged as one of the primary resources used by educators to facilitate student learning.3,6-8
Technology in the form of online learning resources is a growing trend in health professional education.5,9,10 Online learning, also referred to as web-based learning and e-learning, is multi-faceted with a variety of instructional configurations and instructional methods. Online learning is learning that takes place partially or entirely over the Internet.11 Instructional configurations include use of email, learning management systems, multimedia, online resources available through hyperlinks, technology driven presentations, patient simulations, or other online tutorials.9 Through these configurations, educators have applied web-based instructional methods, such as homework, reflection activities, practice exercises, feedback, and patient case discussions.9,12 Online technology has changed the mode of classroom delivery, which now ranges from fully integrated online classrooms, hybrid/blended learning classrooms,7,13,14 or simply integration of some forms of multimedia in the traditional on-campus classroom. Online learning can also be designed to be a synchronous or an asynchronous learning environment. Synchronous environments use technology to interact simultaneously with learners at various locations,11,15 while asynchronous environments use technology to promote self-study at convenient times determined by the learner.11 Both methods can facilitate learning in traditional on-campus education.11 Variation in the use of technology in health care professional education exists and is dependent upon the faculty member, the students, and the material to be presented.8,15-17
Hybrid or blended learning courses combine face-to-face (F2F) classroom time with online instruction to provide learning enhancement.11 Designs for hybrid learning courses vary and are dependent on the amount of time an instructor dedicates to online assignments versus the amount of time dedicated to traditional F2F classroom time in on-campus education. Researchers report various course designs for hybrid learning that range from a percentage (5%-90%) of classroom time dedicated to online learning to courses.11,13,18-21 In some blended courses, educators essentially “flip” the classroom and move selected content to an asynchronous, online format, while using in-class meeting time for other, more interactive purposes as described below.
In a flipped classroom, the overall F2F classroom time does not change, but the time is used differently. For example, a flipped classroom may use online resource to present content for students to independently learn foundational or knowledge-based content prior to a F2F class meeting. The F2F classroom time is then used to promote application of content, higher-level critical thinking, or psychomotor skill development.7,8 Proponents for the flipped hybrid model believe educators are able to decrease the amount of F2F classroom time, provide exposure to content, and utilize class time more effectively for higher-level learning through active learning strategies.18,19,22
A flipped classroom is a relatively new instructional strategy.20 In a flipped classroom, students are expected to actively engage with peers and professors to solve problems and apply what they learned outside the classroom to new situations.19 Flipping reverses the expectations of the common classroom, requiring students to prepare before a class session and then apply what was learned in F2F class meetings. Students are no longer inactive recipients of the knowledge shared by a faculty member, but instead are active learners who are mentored by a content expert in the field of study.23-25 Many flipped classrooms combine F2F classroom interaction between faculty and students with utilization of new technologies online.
These types of flipped classrooms change the learning environment. There are essentially 2 classrooms: an online classroom and the traditional F2F classroom. The online classroom is the method selected for learners to acquire basic knowledge fact and recall content and is the prerequisite for learning in the classroom.25 The work expectations in the online classroom for a flipped class will depend upon the instructional goals of the course. It is imperative for the learner to receive direction about what facts are important to know in order to apply them at a later date. A common model for a flipped classroom requires students to independently learn basic content through listening to or watching lectures, followed by links to other online resources, and sometimes access to online quizzes that produce immediate feedback.13,20 The availability of online material produced by the educator that highlights the main content of other traditional textbook reading is considered the key ingredient in the flipped class.
The F2F classroom time is then used for active participation. We have termed it an “active learning classroom,” a classroom where the analysis and synthesis of basic content is applied to new situations and evaluated together in group settings. This instructional strategy facilitates a “thinking on your feet” mentality.25 It is a classroom that engages learners in deep learning strategies (ie, critical analysis of new ideas and the linking of new ideas to concepts and principles that the learner is expected to already know), and leads to improved understanding, long-term retention, and the development of problem-solving skills that can be applied in unfamiliar contexts.8
While the flipped classroom may optimize perceived integration of content through student engagement in active learning strategies, assessment of overall learning is essential.8,21 Educators need to assess whether students have learned the required basic content provided online well enough to succeed in application of information and facts while demonstrating critical-thinking skills.26 The US Department of Education,11 in a systematic review of medical and other higher education studies, reported stronger learning outcomes in blended learning environments as compared to purely F2F instruction. Persky and Pollack8 found that pharmacy students were capable of learning basic content using an online method followed by F2F classroom time. In the physical therapy literature, Boucher et al27 described the model utilized to transition to a flipped musculoskeletal curriculum. The authors found both the faculty and the students were satisfied with their experience in the course and course grades, and practical exam scores were noted to be consistent while the curriculum was implemented. However, there was no statistical comparison of learning outcomes between the flipped classroom and a traditional, F2F classroom. The purpose of this study, therefore, was to compare outcomes of physical therapist students who were exposed to the flipped classroom versus traditional F2F classroom lecture in a musculoskeletal clinical science course. We hypothesize that there will be no difference in knowledge retention of basic content between 2 classes, despite the instructional method that was utilized.
Fifty-three physical therapist students participated in this study. The participants successfully completed the third musculoskeletal impairments course in a series of 3 at a small, private university in the Midwest during the academic years 2009-2010 and 2010-2011. This course occurred in the third semester of the curriculum. The course content prepared students for the examination, evaluation, and treatment of patients with musculoskeletal extremity conditions. Content included medical pathology, special tests and measures, and physical therapy interventions. The course objectives focused on examination and evaluation with a secondary focus on interventions.
Multiple-choice written examinations were used to assess knowledge retention and application of course content. Each class cohort completed the same exam questions used in analysis for this study. The exam questions were classified in several categories to determine if student performance differed based on teaching method. First, each exam question was independently coded into 1 of the 6 levels of Bloom's taxonomy28 by 2 authors (L.M. and C.M.). A third reviewer with expertise in test construction (CP) assessed the accuracy of the categorization and resolved any disagreement between the first 2 authors.
The exam questions were then assigned to 1 of 2 cognitive-based categories: lower level (knowledge, comprehension, and application) or higher level (analysis, synthesis, and evaluation). Second, each question was assigned to either an examination/evaluation or intervention category. Finally, questions answered correctly were tallied in aggregate and by cohort based on 5 areas: (1) total exam score (Total), (2) score on examination/evaluation questions (Exam), (3) score on intervention questions (Inter), (4) score on lower-level questions (LL), and (5) score on higher-level questions (HL). The percent correct was then calculated for all areas and were also used for measures of knowledge retention.
The 2010 cohort (n = 23) received content on pathological conditions of the extremities via a traditional F2F lecture format (TR). The TR format entailed standard lecture by the instructor using PowerPointTM in the classroom for either the upper-quarter or lower-quarter content. The regions of the body covered were shoulder, elbow, wrist/hand, hip, knee, and ankle/foot. The content of the lecture included signs and symptoms of various medical pathologies, typical clinical presentation as related to the examination sequence, application of special tests, and any information on applicable surgical procedures related to each region of the body. Students were expected to take notes as needed and were allowed to ask questions throughout the presentations. The length of the presentations ranged from 60 minutes to 150 minutes (out of 300 minutes of class time per day) depending on studentinstructor interaction and region of the body discussed. Case studies were utilized as outside homework due to the extensive amount of lecture time required in the classroom. The case studies involved questions pertaining to the screening process, appropriate tests and measures, diagnosis, prognosis, plan of care development, and treatment based on individual case information. In the next F2F class meeting, a brief (15 minutes or less) class discussion was held to discuss clinical-decision making during the examination, evaluation, and treatment of each case. Students were encouraged to consult with instructors outside of class if questions or clarifications were needed related to the case studies reviewed in class.
The 2011 cohort (n = 30) was exposed to a “flipped” teaching method (FL). The FL format consisted of asynchronous online lectures followed by a F2F class meeting about the same content. The online lectures were identical in content as the TR as described above. Articulate Presenter,TM an e-learning authoring software package that allows instructors to easily add interactivity and narration to PowerPointTM presentations, was used for the online lecture. All lectures were posted in SAKAI,TM the course management system (CMS) available on campus, at the beginning of the course semester. All students had experience with using the online lectures on the CMS during the first 2 semesters of the curriculum. Students were required to view the online materials prior to the next F2F class meeting. The length of these lectures varied from 10 minutes to 25 minutes. The online lectures were shorter than lectures in the TR classroom because of the absence of the instructor-student interaction that typically occurred during traditional lecture scenarios. However, the students had the opportunity to stop, pause, and rewind the lectures as needed. In addition, they were encouraged to take notes and bring questions to class for discussion.
Following an online lecture, the FL classroom began with a 15-minute session dedicated to a brief question and answer period to clarify any information that was unclear from the online lectures. This was followed immediately by a 20-30 minute PowerPointTM presentation on integration of the online lecture content into the examination sequence. This instructor-led presentation allowed the students to apply and discuss how the pathologies would present differently throughout the examination sequence (patient history, range of motion, manual muscle testing, passive accessory motion testing, special tests, soft tissue assessment). Copies of the presentations were available for student access at the beginning of the course via the CMS. The remaining class time (120 minutes to 240 minutes) involved group discussion (2-4 students) of cases with emphasis on clinical decision making. The case studies involved questions pertaining to the screening process, appropriate tests and measures, diagnosis, prognosis, plan of care development, and treatment based on individual case information. These case questions required students to make decisions on what screening to do, for which systems, and about how pathological conditions may present differently related to examination components. Furthermore, students were required to provide rationalization for their decisions based on the case information presented. It was also essential that students assigned a diagnosis, determined a prognosis, and developed a plan of care for the patient based on objective information presented. Extensive discussion sessions led by the instructor were completed in the classroom regarding examination, evaluation, and treatment of each case once the groups completed the case study questions. Students were encouraged to demonstrate understanding of how pathological conditions may present differently in various patient scenarios utilizing whatever technique they preferred (eg, algorithms, mind-mapping, diagnostic tables, etc).
Other components of the course were identical for both cohorts including lab activities for skill development of special tests and treatment techniques (approximately 40% of the total class time) and case studies for application of the course content. The courses were team-taught and all instructors contributed to standard lectures, online lectures, and formation of examination questions. The same instructors were used for both cohorts. Both cohorts utilized the same textbooks as well as supplemental readings, and course times were identical.
Internal consistency of the multiple-choice exam questions was conducted utilizing Cronbach's alpha. Cronbach's alpha of 0.70 or higher is considered acceptable.29 Kappa scores were utilized to calculate inter-rater reliability, or agreement between the 2 reviewers, for categorization into the 6 levels of Bloom's taxonomy. A kappa coefficient κ > 0.6 indicates substantial strength of agreement with a κ = 1 indicating perfect agreement.30 After inter-rater reliability was established, an expert review was conducted to validate categorization.
Prior to statistical analysis using SPSS version 20,31 a factor analysis was conducted to create a new variable based on overall Graduate Records Examination Score (overall GRE), overall undergraduate grade point average (UG GPA), and prerequisite grade point average (PR GPA) for each student. This newly created variable was utilized as a measure of prior academic performance. An analysis of covariance (ANCOVA) was used to compare the means of the percent correct based on teaching method with a level of significance set at P < .05. The variable created was used as a covariate to control for the influence of each students level of academic preparation on exam performance.
Twenty-three students were enrolled in the course in the summer of 2010 and 30 students in the summer of 2011. The 2011 cohort had a slightly higher UG GPA and PR GPA than the 2010 cohort. Overall GRE scores were similar (see Table 1 for academic demographics).
A total of 105 exam questions were used in this study. Internal consistency of the multiple-choice exam questions was acceptable with a Cronbach's alpha of 0.715. Inter-rater reliability was calculated for the categorization of the exam questions into Bloom's taxonomy (κ = 0.79; 95% CI: 0.61,0.90). Seventy-six percent (n = 80) of the exam questions were examination/evaluation based and 24% (n = 25) were intervention based. Sixty-six percent (n = 69) of the questions were lower-level questions compared to 34% (n = 36) that were higher-level questions.
The flipped teaching method did not significantly improve students' performance on the multiple-choice exams in any of the areas identified (Exam, Inter, LL, and HL questions) when compared to TR. However, as indicated in Table 2, the mean percent correct was higher in the cohort that received FL in all assessment areas. Statistical analysis also revealed that student's prior academic performance had a significantly greater impact on knowledge retention in all performance areas than the teaching method utilized in the classroom. (See Table 3 for values associated with statistical analysis comparing teaching methods.)
The primary objective of this study was to compare the outcomes of physical therapist students who were exposed to a flipped classroom versus a traditional F2F classroom teaching method. Our goal was to assess differences in student knowledge development of basic content presented in a course focused on the musculoskeletal system based on teaching method. The findings from this study suggest that physical therapist students can acquire basic content knowledge of musculoskeletal pathology through online instruction outside of class as effectively as traditional F2F education. This result is similar to Persky and Pollack's8 results in the field of pharmacology; however, the findings are specific to the profession of physical therapy.
The study found that utilization of online learning in a hybrid flipped course design resulted in similar exam scores compared to traditional F2F education. In addition, the students who were exposed to online learning performed slightly higher in all knowledge retention categories when compared to the students involved in the traditional format, yet the data did not demonstrate statistical significance. The results are similar to previous studies that suggested there was no significant difference in outcomes based on mode of delivery.14,32-35 Furthermore, because the objectives of the course was focused on examination/evaluation content, these results suggest that the FL method of instruction may produce better outcomes in the category of examination and evaluation.
The hybrid model of instruction, using a flipped classroom in this study, was utilized to promote independent student learning. It was also deliberately developed to increase the amount of time spent in the classroom facilitating application, synthesis, and problem solving of the content in varied situations. The finding of no statistically significant difference between teaching methods in any category may initially seem disappointing, but we view these results as incredibly consequential. It demonstrates that online learning does not result in a detrimental effect on student knowledge retention, as evidenced by written examination performance. Performance was approximately 4-5 points higher in all categories when utilizing online learning prior to the onsite class and did not harm the students academically. Because of these findings, instructors may now feel confident using class time for activities that apply the newly introduced content presented online and begin to foster a higher level of thinking in class rather than for delivery of basic foundational content.
The redesigned course assessed whether learners were able to master basic content at the same level as those learners who were exposed to traditional F2F classroom time format. When questions were classified according to the Bloom's taxonomy28 into lower- and higher-level thinking, there was no significant difference in exam scores in each category between teaching methods. Clark36 reported a need to focus on cognitive efficiency in the classroom, and therefore the mode of delivery of basic content needs to be considered. Our study demonstrated that physical therapist students were able to cognitively learn basic content in an online format with no difference in knowledge outcomes based on written exam performance.
Our rationale for moving lectures on basic content to an online format was to allow more class time to be utilized for activities to promote higher-level thinking. This is different from Bayliss and Warden,21 who decreased F2F class time when online assignments were incorporated. Higher-level learning activities used in the classroom required students to first understand the basic content prior to synthesis and application of the content in unique situations, such as case studies. The transition in the classroom from lower level learning to higher level application may foster a greater understanding of the basic content as well as potentially impact the ability of students to apply the content in clinical settings.8 While our results did not demonstrate a significant difference in student outcomes on the questions categorized as higher-level thinking, the higher mean scores on the multiple-choice exams in all categories suggests that the FL classroom may assist the student in beginning to synthesize and apply the content they are learning compared to the TR classroom. There was potential for students who were more comfortable learning in a TR classroom setting to not fully engage in the FL classroom, which may have affected the results; however, our study chose not to assess student preference. Our primary goal was to assess short-term knowledge retention when basic content was moved to online instruction rather than F2F lectures.
We discovered that the flipped classroom using a hybrid model of instruction was beneficial for students when compared to traditional classroom format. Students did learn the basic content for this course to the same degree and potentially better, as evidenced by higher mean exam scores. It engaged the student through the use of current technology and gave the students the freedom to learn at their own pace. The flipped classroom model required students to be accountable for their learning and come to class prepared to actively engage in the learning process; in addition, students were able to control when and where they learned using the asynchronous online course lectures. The flipped classroom challenged students to understand foundational content prior to higher-level thinking activities in the F2F classroom, which may have prepared them to learn more quickly when placed in the clinical setting; however, this needs to be studied further. Researchers purport that higher-level, active learning methods have potential to cause a paradigm shift in higher education over the next several years13,37; however, our study did not investigate the impact of the active learning strategies used.
There are advantages for faculty when using online lectures to supplement F2F student learning. The online lectures created can assist faculty members in improving consistency regarding basic content that is presented to physical therapist students. Improving consistency may assist in the delivery of information to students from year to year. Because the majority of the content in physical therapist education is lock-step in nature (ie, one course is built upon content received in previous courses), the use of online lectures also may assist faculty members in understanding prerequisite content for courses taught later in a sequence of courses. Online resources may also be helpful for new faculty members. A new instructor could have access to basic content delivered previously and adapt it to a course or learn how to present course material using technology. However, access to course content developed by other faculty members may be dependent on the institutions policy on ownership of academic material.
The F2F lecture is most likely here to stay,38 but educators need to be creative in how they engage the learner while in the classroom. Excessive use of passive learningteaching methods, such as the traditional lecture, may not provide the learner with the most beneficial learning environment. Flipping the classroom environment, where students learn basic content prior to engaging in a F2F classroom, can assist in improving student accountability for learning if the online course materials are designed and implemented correctly.
There are several strengths associated with this study. Much of the previous literature has been focused on the student's perception of the teaching method or the perception of his or her own performance.11 In this study, student performance on written exams was used as the outcome measure. Our results show that students learned required course material comparably regardless of a FL or TR teaching method. As a result, we believe that “no harm was done”26 to academic performance with the change in teaching method from a traditional lecture to one that used online learning material. Previous academic performance, such as UG GPA, and GRE scores, has been shown to have a tremendous effect on success in physical therapist education.11,15 Our study controlled for these factors to ascertain the impact of the teaching methods. Results indicated that previous academic performance might have had a larger impact in exam scores rather than teaching method; however, this should be explored further.
Several limitations were evident in this study. First, other confounding variables were not controlled for such as the alternative learning strategies utilized in the classroom. Similar to Persky and Pollak,8 an underlying tenet in this course was that deep learning of basic content was facilitated through discussion and problem-solving activities; however, we did not investigate the alternative teaching strategies during the first year of using the flipped course design. Because the hybrid model of instruction increased the amount of class time for higher-level learning activities, future research should standardize those learning activities in order to ascertain the true impact of an active learning teaching method on overall learning. Second, exam performance is 1 method of assessing student learning and only captures short-term retention of content. It is possible that knowledge, as measured by an examination score, only demonstrated short-term gains in student learning of foundational content and not long-term retention of material. Therefore, future research should include post class follow-up measures to assess student retention of material and possibly performance in student clinical internships. Third, other creative teaching methods, such as team-based learning, should be compared to the use of online lectures to determine if there is a better method of instruction that will improve student performance. Fourth, there is potential that the limited amount of time that was spent on case study application in the TR classroom may have affected student knowledge and therefore may have influenced exam scores, however, we cannot be certain. In addition, because this course was the final course in the musculoskeletal sequence but still introduced new concepts, the multiple-choice exam needed to incorporate both lower- and higher-level cognitive categories. Therefore, the limited time spent on case studies in the 2010 cohort may not have had as large an impact on knowledge retention in this cohort due to the greater number of lower-level questions on the exams. The end result for both cohorts was successful completion of the clinical internship immediately following these courses. Finally, future studies need to include larger class sizes as most DPT programs have larger cohorts. It is unclear as to how this course design may influence student learning with larger class sizes.
The results of this study suggest that the flipped teaching strategy, used as part of a hybrid course design, produces similar student outcomes in newly presented content as compared to the use of traditional F2F classroom teaching. These results may support physical therapist educators in utilizing class time more efficiently through course redesign that includes learning activities that challenge student thinking at a higher level. Flipping a classroom may also allow students more flexibility in how and when they learn, as well as promote student accountability for learning. Finally, increasing online resources related to course content may improve consistency in delivery of information and threading of content throughout a curriculum.
1. Lake DA. Student performance and perceptions of a lecture-based course compared with the same course utilizing group discussion. Phys Ther
2. Lujan HL, Di Carlo SE. Too much teaching, not enough learning: what is the solution? Adv Physiol Educ
3. Wong CK, Driscoll M. A modified jigsaw method: an active learning strategy to develop the cognitive and affective domains through curricular review. J Phys Ther Educ.
4. APTA responds to The New York Times on physical therapists using the term “doctor” [media release]. Alexandria, VA: American Physical Therapy Association; October 5, 2011. http://www.apta.org/Media/Letters/2011/10/5/
. Accessed May 25, 2012.
5. Maley MA, Havery JR, Bastiaan De Boer W, Scott NW, Arena GE. Addressing current problems in teaching pathology to medical students: blended learning. Med Teach.
6. Blouin RA, Riffee WH, Robinson ET, et al. Roles of innovation in education delivery. Am J Pharm Educ.
7. Howatson-Jones IL. Translating an academic module into an online resource. Nurs Stand.
8. Persky AM, Pollack GM. Transforming a large-class lecture course to a smallergroup interactive course. Am J Pharm Educ.
9. Cook DA, Garside S, Levinson AJ, Dupras DM, Montori VM. What do we mean by web-based learning? A systematic review of the variability of interventions. Med Educ.
10. Cook DA, Hatala R, Brydges R, et al. Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA.
11. US Department of Education, Office of Planning, Evaluation, and Policy Development. Education evaluation of evidence-based practices in online learning
: a meta-analyis and review of online learning
. Revised September 2010. Accessed May 28, 2012.
12. Twomey A. Web-based teaching in nursing: lessons from the literature. Nurs Educ Today.
13. Allen EI, Seaman J. Sizing the opportunity: the quality and extent of online education in the United States, 2002-2003. The Sloan Consortium. http://sloanconsortium.org/publications/survey/sizing_the_opportunity2003
. Published September 2003. Accessed May 28, 2012.
14. Maring J, Costello E, Plack MM. Student outcomes in a pathophysiology course based on mode of delivery: distance versus traditional classroom learning. J Phys Ther Educ.
15. Cook CE, Merrifield HH. Distance learning outcomes for Texas Tech University's physical therapy program. J Phys Ther Educ.
16. Frantz JM, Himalowa S, Karuguti W, Kumurenzi A, Mulenga D, Sakala M. Challenges and opportunities related to postgraduate evidence-based practice module using blended learning. Afr J Health Professions Educ.
17. Lammers WJ, Murphy JJ. A profile of teaching techniques used in the university classroom. Active Learn High Educ.
18. Sankey M, St Hill R. Multimodal design for hybrid learning materials in a second level economics course. Presented at: Proceedings of 11th Australasian Teaching Economics Conference: Innovation for Student Engagement in Economics; July 11-12, 2005; University of Sydney, Australia. http://eprints.usq.edu.au/267/1/Sankey_StHill_05.pdf
. Accessed April 22, 2012.
19. Berrett D. How “flipping” the classroom can improve the traditional lecture. The Chronicle of Higher Education
. February 19, 2012. http://chronicle.com/article/How-Flipping-the-Classroom/130857/
. Accessed April 22, 2012.
20. Educause Learning Initiative. 7 things you should know about flipped classrooms. http://net.educause.edu/ir/library/pdf/ELI7081.pdf
. Accessed April 23, 2012.
21. Bayliss AJ, Warden SJ. A hybrid model of student centered instruction improves physical therapist student performance in cardiopulmonary practice patterns by enhancing performance in higher cognitive domains. J Phys Ther Educ.
22. Perlman C, Weston C, Gisel E. A web-based tutorial to enhance student learning of activity analysis. Can J Occup Ther
23. Petrova K. Teaching differently: a hybrid delivery model. In: Delener N, Chao CN. Facilitating Competiveness and Change in the Global Environment. Proc 2001 GBATA Int Confer
24. Simmons K, DiStasi A. Active learning techniques: engaging students on the road to success. Athl Ther Today.
25. Steinert Y, Snell L. Interactive lecturing; strategies for increasing participation in larger group presentations. Med Teach.
26. Schoenfeld-Tacher R, McConnell S, Graham M. Do no harm—a comparison of the effects of on-line vs. traditional delivery media on a science course. J Sci Educ and Tech.
27. Boucher B, Robertson E, Wainner R, Sanders B. “Flipping” Texas State University's physical therapist musculoskeletal curriculum: implementation of a hybrid learning model. J Phys Ther Educ
28. Bloom, BS. Taxonomy of Educational Objectives, Book 1: Cognitive Domain.
New York, NY: Longman; 1965.
29. Nunally J, Bernstein IH. Psychometric Theory
. 3rd ed. New York, NY: McGraw-Hill; 1994.
30. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics.
31. SPSS Statistic for Windows
[software program]. Version 20.ink. Chicago, Il: IBM; 2013.
32. English T, Harrison A, Hart A. A distance learning model in a physical therapy curriculum. J Allied Health.
33. Hobbs G, Moshinskie J, Roden S, Jarvis J. A comparison of classroom and distance learning techniques for rural EMT-1 instruction. Prehosp Emerg Care.
34. Jedlicka JS, Brown SW, Bunch AE, Jaffe LE. A comparison of distance education instructional methods in occupational therapy. J Allied Health.
35. Leasure AR, Davis L, Thievon SL. Comparison of student outcomes and preferences in a traditional vs. World Wide Web-based baccalaureate nursing research course. J Nurs Educ.
36. Clark RE. What is next in the media and methods debate? In: Clark RE, ed. Learning from Media Arguments, Analysis and Evidence
. Greenwich, CT: Information Age Publishing Inc; 2001;327-338.
37. Young J. “Hybrid” teaching seeks to end the divide between traditional and online instruction. Chron High Educ.
38. Frederick P. The lively lecture: 8 variations. Coll Teach.