Schneider, Joseph R.; Sherman, Heather B.; Prystowsky, Jay B.; Schindler, Nancy; Darosa, Debra A.
Mr. Stephen Markwell of Statistics and Research Consulting at Southern Illinois University School of Medicine provided important advice about data analysis and interpretation. We are grateful for the inspiration and editorial comments provided by Orpha K. Duell, PhD.
Correspondence: Joseph R. Schneider, MD, PhD, Burch 100, Evanston Hospital, 2650 Ridge, Evanston, IL 60201; e-mail: 〈firstname.lastname@example.org〉.
Background. Pauses (wait time) after asking questions in precollege classes result in improved discussion and answer accuracy. The authors hypothesized that this would extend to medical students.
Method. Third-year surgery clerks were randomized to three-second or six-second wait times after questions asked of them during a scripted lecture. Students were randomized within each session to answer 21 scripted questions. Students also completed a post-lecture written examination.
Results. Correct responses ranged from 17% to 100% for oral and 22% to 100% for written questions. Answer accuracy could not be distinguished between three- and six-second wait times for oral or written questions.
Conclusions. The benefit of increasing wait times from three to six seconds appears not to extend to medical students. This may represent evolution of learning or different learning modes in medical students. Alternatively, maximum benefit may be achieved in medical students with shorter wait times.
Faculty development programs have encouraged pausing several seconds after asking a question to allow students to think and provide thoughtful responses.1–3 This is based on observations made more than 30 years ago that high school teachers typically waited less than one second between completing a question and calling for an answer and subsequent research concluding that increasing “wait times” could enhance the accuracy of responses and learning.4–8 Similar experiments have subsequently been performed in college classrooms.9,10
In general, two wait times have been explored.6 Wait time 1 is the time a teacher pauses after asking a question and calling upon a student to answer. Wait time 2 is the time after a student completes an answer until the teacher resumes the presentation or asks another question. Wait time 1 was the focus of this study and the term wait time in this article will hereafter mean wait time 1. Prior investigations cited above suggested that a three-second wait time minimum enhances the accuracy of responses and that prolongation of wait time beyond six seconds does not further improve the accuracy of responses. There is also evidence that extending wait time within these limits is more likely to enhance responses for higher level questions than for low level recall questions.11 We hypothesized first that the benefit of increasing wait time would extend to third year medical students, and second that the benefit would be greater for higher-level questions.
Gagné et al., as cited by Duell,10 have proposed an information-processing model of learning. This model assumes answering a low-level recall question requires location of information in “long-term” memory, retrieval into “working” memory, verification that the information retrieved answers the question and, finally, answering the question. It is implied that some minimum time is required to complete this process effectively. More complex questions require application of known information to unknown situations and assessment of whether the application of the old information to the new situation is correct. If the learner concludes that the application is correct, then she or he is ready to answer the question and, if not, another search for old information is initiated and the process is repeated, hopefully until a satisfactory application of old information to the new situation is achieved. In either of the latter two situations (the more complex question), more steps and more time are required than for simpler recall questions. Given this model, it is hypothesized that more time is required to answer more complex questions.7,10,12 Furthermore, naming a student before asking the questions allows other students to defer and not try to determine the answer since they can assume they will not be called upon.6,10 Therefore, one could further hypothesize that waiting an appropriate period of time for an answer after asking a question may substantially improve the accuracy of answers and the value of the experience for students6,11 and there would appear to be an advantage to asking the question before naming a student to answer so that every student has an incentive to solve the problem.
Increases in wait time have been found to enhance performance of both students and teachers. Students tend to talk more, ask more questions, and increase their interaction with other students, and teachers tend to talk less and increase their expectations of students.6 Tobin11 reviewed the history of research in wait time and its manipulation and found that nearly all prior work had provided evidence that three- to five-second wait times had positive effects on student achievement and teacher performance. Riley also demonstrated a clear benefit of extending wait time from three to six seconds in elementary school students.7 The only study conducted in primary or secondary students known to us that did not observe an advantage to enhanced wait time used wait time 1 of only about one second in elementary students.13
College professors strive to ask higher-level questions and, thus, extended wait time may be even more important in college classrooms.10 Such benefit might be expected to extend to medical students and residents since higher level questions are likely to be the norm for their instructors as well. The observation that wait times of three seconds or more will enhance the accuracy and quantity of responses has rarely been tested in college classrooms. Ellsworth et al.14 noted a preference for longer wait times in undergraduate statistics students. However, those authors did not manipulate wait time, but made observations only. Work by Duell10 that carefully controlled wait time in experiments performed with college undergraduate education students found no difference between three- and six-second wait times. However, the effect of wait time has not to our knowledge been tested previously in a medical school classroom or other medical student presentation.
The experiment was designed to replicate the classic wait time study of Duell.10 A scripted case of abdominal aortic aneurysm was presented after which a lecture approximately 40 minutes in length was given on the same topic using a laptop computer and projected “slides.” The lecture was not “read,” but was controlled for content since the slides prompted the presenter. Twenty-one scripted questions expecting short answers were interspersed throughout the presentation. A black slide was projected as questions were asked verbatim from the script so that students would not be distracted by reading a slide. Questions were developed to include six low-level information “recall” and 15 higher-level questions including four analysis, one comparison, seven inference, and three evaluation questions,15 similar to Duell's strategy. Scripted prompts were used if the student failed to respond or responded incorrectly to the question. The correct answer was provided if not given by the student even after prompting. The lecture was tailored to conform to the appropriate material in the assigned textbook16 and objectives published by the Association for Surgical Education.17
Six sessions including six to 14 third-year medical students each were conducted during their surgery clerkship. All lectures were given by the same presenter. Sessions were randomly assigned to a three-second (three sessions) or six-second (three sessions) wait time. Sessions were conducted at Evanston Hospital (one three-second and two six-second sessions) or Northwestern Memorial Hospital (two three-second and one six-second sessions). One coinvestigator kept time following the completion of a question using a stopwatch and discreetly prompted the presenter from the back of the lecture room with a hand sign to call on a student to answer a question with wait time of either three or six seconds. A total of 32 students participated in three-second sessions and 28 students in six-second sessions. Each student was asked one question in a round of questions until every student had been asked one and only one question in that round, after which a new round of questions would start. Students’ names were ordered randomly prior to each session. Students were not told how they would be selected and would have had to assume that they could be asked any question. Randomization was performed without replacement to assure that every student would be asked at least one question. Since the number of students in these sessions varied from six to 14, there were as few as two and as many as four rounds, and an individual student may have been asked as few as one and as many as four of the 21 oral questions. Student names were also rerandomized for the second and, if necessary, third and fourth rounds.
A brief introduction informed the students only that an experiment was being performed to investigate classroom learning. Students were instructed not to take notes and not to answer unless called upon. Sessions were videotaped to allow subsequent evaluation of the accuracy of answers and to allow confirmation of wait time.
A post-lecture written test was administered with 15 questions similar to those asked during the presentation including four low-level recall (e.g., “Define aneurysm.”) and 11 higher-level questions (four analysis, one comparison, five inference, and one evaluation, e.g., “What are the most common complications after AAA repair?”).15 An announcement about an upcoming educational event was made before the post-lecture test to briefly distract students to reduce the likelihood that recall would involve only transient “short-term memory.”
Answers were scored using answer keys developed by consensus of two coinvestigators who had not written any of the questions. These coinvestigators were not present during presentations and did not review videotapes. Oral examination answers were scored from written transcripts of the videotaped answers. Thus, scorers were blinded to wait time for both oral and written answers. Neither oral test transcripts nor written examinations included any student-identifying information. Oral answers were scored as correct, correct after prompt, or incorrect. Written answers were scored as correct or incorrect. Total points were used to compare results in the three- and six-second wait time groups. A separate analysis was performed for low-level and higher-level questions from both oral and written tests. This study was approved by the Institutional Review Boards of Northwestern University School of Medicine and Evanston Hospital, which waived the requirement for written consent from study subjects because the research was performed “in a commonly accepted educational setting.”
Comparisons between three- and six-second wait times were made using sums of scores from the oral and written tests. Tests of hypothesis were performed using Student's t-test for independent samples. Significance was assumed for all tests if p < .05.
Each oral question was asked six times (once in each of three three-second and three six-second wait time sessions). The number of correct answers for each of the 21 oral questions ranged from one of six (17%) to six of six (100%). Students achieved an average of less than two-thirds of the maximum possible scores on oral questions (Table 1a). Oral question results were slightly better for higher-level than for low-level questions. Students in six-second sessions scored no better than those in three-second sessions. In fact, there was a trend toward better performance with three-second wait time (average difference 2.4 points for low-level and 1.0 point for higher-level oral questions), although this trend was small and statistically insignificant.
All 60 students completed the written test. The number of correct answers for each of the 15 written questions ranged from 13 of 60 (22%) to 60 of 60 (100%) responses. Students scored higher on written than on oral questions, averaging roughly 80% correct for written tests (Table 1b). Written test results were comparable for three and six-second wait time groups. Results were virtually identical in three- and six-second sessions for low-level questions. A trend toward higher scores for higher-level written questions in six-second wait time sessions was small and statistically insignificant.
Medical students benefit most when they are actively involved in discussion and problem solving.3,18 Engaging students by asking questions throughout a presentation enhances learning2 and can be considered the basis for problem-centered instruction. Despite clear evidence that increased wait time is associated with better educational outcomes in primary and secondary school students,6,7,11 Duell10 noted no improvement in written test results in university education students. Indeed, she noted a trend toward poorer results with six-second as compared to three-second wait times for higher-level questions. We were intrigued by the discrepancy between Duell's work in college students and that of other investigators working with younger students, and chose to perform our study accordingly. Our study appears to confirm Duell's work in post–secondary school students. However, Duell scored only the written test. We attempted to determine whether both verbal and written responses could be affected by wait time. We are unaware of any prior wait-time study that has examined the results of questioning during presentations beyond confirming the length of wait time. We are also unaware of any prior study that controlled to the level that our study controlled for lecture content or that used scripted questions so that the exact same questions were asked in all experimental sessions.
One potential explanation for Duell's and our failures to identify the same beneficial effect of extended wait time noted by others in primary and secondary school students is the possible continued evolution of learning methods beyond high school and undergraduate college. A second possible explanation is that medical students may be selected in such a way that they are less impacted by wait times in the range of our experiment. Students selected for achievement in undergraduate course work, generally with increased weighting for science courses, may yield a subpopulation whose learning is affected differently by wait time compared to other postgraduate students. However, Ellsworth et al's14 work with students of statistics, a course that might select for students with interest and demonstrated performance in more scientific subjects and which might be considered similar to medicine in that regard, found that they preferred wait times in the three- to five-second range. A third possibility is an effect of nonrandom assignment of students to the study groups. Students were assigned either to Northwestern Memorial Hospital or Evanston Hospital for one of eight six-week clerkship periods distributed over the academic year. Total students in three-second and six-second groups were balanced and sessions were generally distributed over the academic year. Students may in some cases have been assigned to one or the other hospital or portion of the year to accommodate personal requests, but there is not likely to have been a bias introduced by this assignment. A fourth possibility is that the lecture setting may not have been optimum with respect to motivation for students to participate. For example, they may have perceived this presentation as “low stakes” since no grade would be generated and this may have reduced motivation to verbalize. These findings might not be duplicated in a less formal problem-centered instruction or other small-group setting. Prior knowledge of the subject might also have led to failure to concentrate and participate. The presence of the video camera may also have discouraged participation, despite our attempts to make this as unobtrusive as possible. However, the study was balanced and randomized, the content of the presentation was highly controlled and presented by the same vascular surgeon at every session, and the sample size was adequate to expect that these latter three possibilities would have equal effects in the three- and six-second groups. The requirement for written consent would clearly have significantly altered the experiment, and we are fortunate that our Institutional Review Board provided an exemption for this educational research. A fifth possibility is that we may have asked too few questions. As noted earlier, this study was designed to replicate as much as possible Duell's study, and the number and distribution of questions in our study was identical to those in Duell's study. Furthermore, there was not even a notable trend in our study toward differences between three- and six-second groups as measured by either the oral or written questions.
It is also possible that wait time does affect medical students, but that maximum benefit is reached with three seconds or less wait time 1. Future effort will explore this possibility and experiments with shorter wait times are underway. However, Duell explored this same possibility in her work and found no significant difference between one-second versus longer wait time 1. Wait time 2, the time between the completion of a student's answer and the resumption of speech by the teacher, may also affect the accuracy of responses.11 Wait time 2 was neither evaluated nor manipulated in the present study, but may be the subject of future work.
1. Whitman N, Lawrence P. Surgical Teaching: Practice Makes Perfect. Salt Lake City: University of Utah, 1991.
2. Sachdeva AK. Use of effective questioning to enhance the cognitive abilities of students. J Cancer Educ. 1996;11:17–24.
3. Small P. Consequences for medical education of problem-solving in science and medicine. J Med Educ. 1988;63:848–53.
4. Rowe MB, Hurd P. A study of small group dynamics in the BSCS laboratory block program. J Res Sci Teach. 1966;4:67–73.
5. Gall M. Synthesis of research on teachers questioning. Educ Lead. 1984;42:40–7.
6. Rowe MB. Wait time: slowing down may be a way of speeding up! J Teach Educ 1986;37:43–50.
7. Riley JP. The effects of teachers’ wait time and knowledge comprehension questioning on science achievement. J Res Sci Teach. 1986;23:335–42.
8. Altiere MA, Duell OK. Can teachers predict their students’ wait time preferences? J Res Sci Teach. 1991;28:455–61.
9. Duell O, Lynch D, Ellsworth R, Moore CA. Wait-time in college classes taken by education majors. Res Higher Educ. 1992;33:483–95.
10. Duell OK. Extended wait time and university student achievement. Am Educ Res J. 1994;31:397–414.
11. Tobin K. The role of wait time in higher cognitive level learning. Rev Educ Res. 1987;57:69–95.
12. Pond MR. The effects of extended wait-time on textually explicit and textually implicit comprehension [doctoral dissertation, University of Akron, 1987]. Dissert Abstr Int. 1987;48:618A.
13. Anshutz RJ. An investigation of wait time and questioning techniques as an instructional variable for science methods students microteaching elementary school children [doctoral dissertation, University of Kansas, 1974] Dissert Abstr Int. 1974;A584.
14. Ellsworth R, Duell OK, Velotta CL. Length of wait times used by college students given unlimited wait time intervals. Contemp Educ Psychol. 1991;16:265–71.
15. Stiggins RJ, Rubel E, Quellmalz E. Measuring thinking skills in the classroom. Revised edition. Washington, DC: NEA, 1988.
16. Gewertz BL, Graham A, Lawrence PF, Provan J, Zarins CK. Diseases of the vascular system. In: Lawrence PF (ed). Essentials of General Surgery. 2nd ed. Baltimore: Williams & Wilkins, 1992:328–47.
17. Association for Surgical Education. Manual of Surgical Objectives. 3rd ed. Springfield, IL: ASE, 1990:65.
18. Foley R, Smilansky J, Yonke A. Teacher-student interaction in a medical clerkship. J Med Educ. 1979;54:622–6.