Schwind, Cathy J. RN, MS; Boehler, Margaret L. RN, MS; Markwell, Stephen J. MA; Williams, Reed G. PhD; Brenner, Michael J. MD
Pages are a ubiquitous feature of the intern year,1 and fielding these pages is a time-honored rite of passage for interns.2 Responding effectively to these pages on the inpatient wards requires a combination of knowledge, experience, and judgment. Yet, a majority of interns—despite their relative inexperience and immature clinical acumen—are routinely expected to take call the first day they are on service. There are few responsibilities more difficult or more fraught with the potential for error than responding to pages. For example, an innocent-seeming nursing request for acetaminophen to treat fever may be the first indication of an impending life-threatening complication. Interns often serve as an initial point of physician contact when a patient concern arises, and therefore management of pages is a critical aspect of internship. There is a growing awareness of the need to foster interdisciplinary teamwork between physicians, nurses, and other allied health care professionals involved in patient care.
For the typical new intern, making the transition from medical student to doctor can be a harrowing one with a steep learning curve. The majority of interns feel unprepared for the intern year.3 Surgical residents achieve high stress levels during their junior years, as manifested by tachycardia and increased white blood cell count.4 Internal medicine residents perceive that they have a greater tendency to errors when distressed and fatigued.5 A great deal of interns' stress may be attributed to the lack of thorough preparation.6–8 Prior research on “resident readiness”-type, or “boot camp”-type curricula has shown the usefulness of focused educational interventions.9–18 These curricula are designed to equip students with basic skills required for internship and residency, thereby increasing confidence and decreasing medical errors. Much of the anxiety experienced by the new intern pertains to pages, providing a rationale for innovative training methods that may alleviate pager-related stress for the new intern.
There is also a potential educational benefit to having new interns arrive on the wards reasonably proficient with handling pages. Pages are notoriously disruptive to educational opportunities—be they didactic sessions, conferences, or other structured learning activities.19,20 Interns who can manage their pages well are in a better position to avail themselves of those opportunities. The early didactic exposures—so often surrendered by interns trying to keep their heads above water—are critical for young doctors. Part of the reason that pages are so disruptive to resident education is that they generate a sense of great urgency. The page not only pulls the new intern away from educational activities but also tends to induce a marked sympathetic response4 that alters his or her frame of mind and impairs the concentration required for learning. If some of the learning curve about answering pages can be carried out during medical school, then pages may interfere less with educational programs.
Perhaps most important, improving intern preparedness through a structured program of pages may eventually translate into improved patient safety. The prevalence of medical errors and the need for a safer health care system is now well recognized.5 Early in the academic year, on-call admissions are associated with greater loss of sleep, longer shifts, and increased fatigue for interns.21 Greater proficiency with handling pages could decrease the sleep deprivation associated with on-call duties during these critical months. The impact of sleep deprivation on performance is not negligible,22 and the Accreditation Council for Graduate Medical Education now requires education of residents regarding the effects of sleep deprivation. If greater preparedness even slightly improved efficiency, it might mitigate the fatigue, burnout, and errors to which new interns are highly susceptible.23
We designed this study to investigate simulated patient-management pages as a tool for preparing senior medical students for internship. This approach was based on the notion that learners are generally more receptive to teaching methods that involve active learning.24,25 Our use of simulated pages closely mirrored the clinical setting, incorporating clinical patient scenarios that are commonly encountered by interns on the hospital inpatient wards.
We conducted this study at Southern Illinois University School of Medicine (SIUSOM). Fourth-year medical students were given the option of participating in a four-week resident readiness elective. The paging intervention described herein was embedded into this educational program. Participation in the simulated paging study did not affect the evaluation of the medical students during the elective. The study was performed in strict accordance with the institutional review board guidelines, and approval from this board was received prior to initiation of this project.
Sixteen fourth-year medical student volunteers at SIUSOM participated in the study. The participants provided informed consent for all notes and data collected during simulated medical calls to be used for research purposes. Ten men and six women were enrolled. All 16 students were enrolled in the four-week “resident readiness” elective and had stated interests in general surgery, orthopedics, otolaryngology, or obstetrics–gynecology. The primary elements of the elective were a cadaver dissection, didactic sessions on proper use of blood products, and fluid management. In addition, a substantial portion of the curriculum included development of technical surgical skills such as laparoscopic skills, bowel anastomosis, vascular anastomosis, basic and advanced suturing skills, venous and arterial access, chest tube insertion, and airway management. The portion of the elective pertaining to simulated pages was distributed among these other activities.
Development of simulated pages
Eleven patient case scenarios were developed by a multidisciplinary team consisting of physicians, surgeons, nurse clinical educators, and a surgical education research faculty member. We, the authors of this report, were included as part of this team. The first step was development of a pool of approximately 30 common patient problems that residents encounter while they are on call. This pool was derived from both personal practitioner experience, querying of medical students, interns, and senior residents, and review of an on-call educational reference guide written for surgical interns.26 From this list, 15 topics were selected based on consensus among members of the multidisciplinary team regarding which patient problems were common or of critical importance from a patient safety perspective.
In addition, the experimental design was then vetted by the surgical education and performance group at SIUSOM. This group includes approximately 12 individuals—surgical faculty, skills lab educators, and residents—that helped in vetting scenarios. The overall educational value was also considered, including how well the topic would lend itself to the simulated paging educational intervention. The topics include problems such as nausea/vomiting, pain, hypotension, fever, distended abdomen, and chest pain.
Next, the multidisciplinary team developed a clinical scenario for each of the 11 topics and an evaluation instrument that separately scored participants on assessment and management. The scenario included data such as vital signs, pertinent physical exam findings, and laboratory results. The scoring method was developed by local experts, and group consensus was reached in determining the weight of individual scored items. The diagnostic/therapeutic scoring scheme was based on generally accepted standards of care for these medical situations.27,28
The scoring methodology allowed for differential weighting of various aspects of patient management. For example, a critical maneuver (such as determining appropriate fluid bolus in a hypotensive patient) might be weighted at 50 points, whereas a less critical intervention (asking for estimated blood loss) might only be weighted at 10 points. Furthermore, if a wasteful or dangerous management decision was made (diuresing patient), 50 points would be subtracted. In some cases, the point values assigned did not correlate directly with the importance of the management decision. For example, seeing and evaluating a patient is one of the most important things that a medical student could do in this exercise. However, nearly all of the medical students indicated their intent to see the patient described in nearly all of the scenarios. Therefore, this criterion was not particularly helpful in differentiating among students and was routinely valued at only 10 points. In contrast, a key diagnostic study that was essential in establishing a diagnosis and guiding therapy might be weighted much more heavily to differentiate among students. There were no negative points given for any assessment data, because asking for unnecessary information would not harm a patient, unlike the case with a dangerous management decision. The scoring instrument was designed to have results fall in a roughly 100-point range, but a wider range of scores was possible for evaluation and management scores.
A global assessment was used to capture an overall competence rating. The expert evaluators of the student performance rated medical students as competent (“pass”) on the global assessment if they safely and successfully handled the situation presented. Two of the scoring profiles are shown in Charts 1 and 2.
Immediately before and immediately after the four-week elective, participants were asked to rate their levels of confidence in handling pages concerning specific patient management issues. These confidence ratings were ranked on the following six-point Likert scale:
I can handle pages concerning specific patient management issues—
* 0 = Not at all
* 1 = Maybe with complete supervision walking me through it step by step
* 2 = I still need someone right at my side
* 3 = I think I could do it with someone nearby (maybe in the next room)
* 4 = I think I can do it myself
* 5 = I cannot only do it myself but I can teach others
Simulated page procedure
Students were exposed to the patient scenarios, with clinical nurse educators initiating the pages. There were rare instances where students missed pages because of absences from the elective, such as for residency interviews or illness. In addition, medical students entering residency in obstetrics–gynecology had some variations to certain scenarios, and these alternate scenarios were not included in the final analysis. For example, a few students received a scenario on management of a “boggy fundus,” a scenario that was less relevant to the other students enrolled in the elective.
The simulated medical pages were performed in the inpatient setting to mirror the on-call experience of an intern. The participants received pages throughout the course of their work days, as well as one assigned night of “home” call. These pages were distributed across a four-week period, which minimized the effects of fatigue during the battery of scenarios. The order of presentation of scenarios was randomized for each medical student, to minimize practice-related effects that might otherwise skew scores on any particular scenario. Learners made clinical judgments based on the information communicated during the exercise. Participants were instructed not to communicate with peers regarding the educational scenarios.
The simulated pages all involved a structured presentation to the medical student via phone. The medical students were not presented with specific training on responding to pages prior to the exercise, nor did they undergo didactics specific to the scenarios prior to participating in the simulated paging exercise. Pertinent assessment data were scripted for each case, and in the event that students asked for information not scripted, the expert nurse would ad-lib and then record this information to use in future calls to provide standardization. The medical students received information on a patient care finding; they then were required to ask pertinent questions, assess the problem, and provide a management plan along with verbal orders.
The following is an example of the opening presentation of a simulated page: “Mr. Jones is a 55-year-old male, status post myocardial infarction with a stent placed three days ago. He is now complaining of abdominal pain and diarrhea.” The student would then elicit additional information and proceed with evaluation, including an opportunity to “see” the patient by asking for the nurse to provide assessment data from the virtual bedside. The student was expected to elicit pertinent findings from the patient, develop a plan, and give orders. No feedback on performance was provided until the end of the scenario. The scenarios' average time for completion ranged from 2.5 minutes to 9.8 minutes, not including time for feedback. A total of approximately 20 hours of educator time was required to administer the scenarios to all of the students enrolled in the study.
The scoring was performed in real time by the clinical educators (M.L.B. and C.J.S.) during the simulated paging exercise. Medical student responses were scored with the weighted evaluation instrument. To facilitate scoring, students were instructed to complete all queries and diagnostic maneuvers (assessments) before proceeding to treatment (management). Points were added for appropriate maneuvers and subtracted for inappropriate maneuvers, and these results were tabulated separately for assessment and management. Each medical student had 11 scoring sheets completed by the end of the exercise, one sheet per paging scenario. Charts 1 and 2 are examples of scoring sheets.
After the assessment and management outcomes had been recorded, the two clinical educators would each provide a global rating. Global assessments reflected the two evaluators' overall judgment as to whether the medical student was competent to evaluate and treat the patient in a given scenario. This global assessment was intended to capture nuances of competence that might not be measured well by the quantitative assessment and management scores. For example, a medical student that stumbled onto most of the right diagnostic/treatment maneuvers—despite an obvious lack of knowledge and understanding regarding appropriate workup and management—would be judged “not competent.” The global rating was a binary score of competent/not competent and therefore did not attempt to assess varying degrees of competence. The hard copies of scoring results were maintained in folders, and the data were transferred to an electronic SPSS database spreadsheet for analysis at the conclusion of the experiment.
After completion of each simulated paging scenario, the clinical educators would inform the medical student of the diagnosis and then summarize the recommended diagnostic and therapeutic maneuvers. Notes taken by the clinical educator were used to provide specific feedback regarding areas for improvement. Feedback was provided immediately after each individual scenario so as to optimize educational benefit. The critical importance of the patient exam was emphasized during debriefing. The global assessment regarding whether the medical student was deemed competent for that particular situation was also revealed to the medical student, with subjective feedback based on notes taken during the study.
After completion of the course, medical students were asked to again rate their confidence in managing patient-related pages. Students were also asked to comment on whether they felt the exercise improved their clinical skills and whether they were now more comfortable with handling patient-related pages. Last, students were asked to identify any aspects of the scenarios that were unclear or in which the “gold standard” diagnosis/treatment might be improved.
Data analysis and statistical methods
Descriptive statistics were calculated and inspected for all measurements. Pearson correlation coefficients were used to examine relationships between assessment and management for each of the 11 scenarios. For each scenario, kappa coefficients were computed to assess interrater reliability between the two raters on global assessment. The analysis of correlation in global rating scores was performed to evaluate the interrater consistency when judging global performance. This correlation reflected how likely the two raters were to agree/disagree in their assessments regarding the competence of a given student. A paired t test was used to test for a change in medical student confidence from before to after the intervention. Comments of medical students regarding their feedback on the usefulness and value of the exercise were also reviewed and recorded. Results were considered statistically significant for P < .05.
Data were collected on medical student assessment, management, and global performance for all 11 simulated paging cases. Pre/postintervention self-report measures and comments were collected from all 16 students.
The mean scores on the assessment portions of the 11 scenarios were lowest for hypokalemia (57.9 ± 28.1) and highest for hypertension (86.3 ± 14.6). A wider range of scores was observed in the management portions of the scenarios, with the lowest scores received in wound infection management (−15.6 ± 41.6) and the highest scores received for pain management (95.7 ± 9.4). Mean assessment and management scores by scenario are detailed in Table 1. The scores recorded in this table are the range of scores that students actually received on the test scenarios, rather than the theoretical maximum and minimum scores that were possible for students. The students' ratings regarding their confidence in managing inpatient care increased from 1.87 ± 0.83 prior to the simulated paging exercise to 3.53 ± 0.52 postintervention, using ratings on the six-point scale described earlier (P < .0001).
A similar spread in medical student performance was observed for global assessments across scenarios. The number of medical students rated globally competent ranged from a low of 2 (12%; management of wound infection) to a high of 15 (93%; management of hypertension). No significant correlation was observed between medical student assessment and management scores. Interrater agreement for global assessments was high (mean kappa = 0.88). There was perfect concordance between raters on global assessments for 7 of the 11 scenarios (hypertension, hypotension, atelectasis, nausea/vomiting, oliguria, pain, and wound infection, all with kappa = 1.0). Correlations for the remaining scenarios were as follows: hypoxia 0.865, hypokalemia 0.842, chest pain 0.754, and ischemic bowel 0.194.
For the ischemic bowel scenario, there was poor agreement between the two evaluators on global assessments of medical student competence. In exploring the source of this discrepancy, it became apparent that one evaluator required a higher level of performance than the other to designate competence. The evaluator who was more of a “hawk” maintained that a high index of suspicion was required in order to make this diagnosis promptly. If the medical student did not articulate that there was high mortality risk with ischemic bowel and that urgent surgery was needed, the medical student was judged not competent. In contrast, the evaluator who was more of a “dove” felt that a conscientious evaluation, supportive measures, and making provisions for possible future surgery were adequate at the medical student level because medical students are often unsure of themselves and reluctant to take decisive action. This particular case was brought back to the multidisciplinary team for discussion, and it was agreed that the stricter interpretation/standard was appropriate for this particular scenario.
With respect to medical students' subjective comments and observations, three major themes emerged: First, students uniformly indicated that they found the simulated paging exercise to be of benefit. Several medical students specifically cited the value of experiencing a “preview” of what they might expect when on call as interns. Second, students felt that exposure to simulated pages contributed to their clinical experience and comfort; some students speculated that this experience might even decrease their risk of errors and/or improve patient safety early in their internships. Last, the students felt that the experience was effective in promoting critical thinking skills and would help them be better prepared for the internship. Some students proposed additions to the scoring templates, but there were no significant concerns expressed regarding the “gold standard” recommendations for diagnosis and treatment.
Among the most stressful situations for the new intern is handling urgent patient matters that come to their attention by way of their pager. Our data suggest that a simulated paging intervention may ease this difficult transition, as reflected in highly significant improvements in medical student confidence after the simulated paging exercise. We found that a wide variety of patient scenarios can be effectively simulated using this approach, and students found this approach of high educational value. There are several potential benefits to improving the proficiency of new interns in handling pages, ranging from decreased intern stress to better participation in educational activities to improved patient safety.
We observed a wide range of medical student performance across the 11 scenarios. This finding suggests that the scenarios chosen represented an appropriate mix of low-, intermediate-, and high-difficulty clinical situations. The spread of scores was higher for the management than for the assessment portions of most cases. Although it is possible that the assessment portion of the scenarios was easier than the management portion, a more likely explanation relates to medical student development. Another possible explanation for the smaller spread of scores in the assessment section was that there were no negative values assigned to the assessment section. The decision not to penalize superfluous assessments was a potential weakness of the study design. The fourth-year medical students participating in this study seemed to be more proficient in gathering information than in processing and synthesizing it. This finding is not surprising, because history and exam skills are emphasized early in medical training, whereas clinical judgment often tends to develop more gradually over the course of years.
Although there was excellent agreement among the raters, the global assessment was fairly subjective. The discrepancy in judging students' competence for the ischemic bowel scenario reflects the differences in perceptions of competence. One potential source of confounding in the global assessments was that only one evaluator carried out a given paging scenario for any given student. For the evaluator who did not conduct the exercise with the student, the global assessment was based on notes recorded during the paging exercise. Therefore, the rater who directly interacted with the medical student during the exercise may have unduly influenced the evaluator who was relying on the notes alone. One possible solution would be to use videos or recordings, although this would double the time required for the exercise.
An attractive feature of simulated pages is that they foster interdisciplinary teamwork between nurses and future doctors. Medical students are sometimes reluctant to learn from other members of the health care team, such as nurses, pharmacists, and other allied health professionals. Yet, one of the major goals in medical education is to avoid silos and improve interdisciplinary teamwork.29 The teamwork involved in this exercise may help students to understand the shared nature of patient care. For example, when medical students discover the vital role of their nursing colleagues—that nurses not only bring attention to many patient problems but can also be a key part of the solution—teamwork is improved. This approach improves competencies in patient care, practice-based learning, interpersonal and communication skills, and professionalism in accordance with the recommendations of the Accreditation Council for Graduate Medical Education.30
The urgency of pager-initiated communication makes simulated pages highly effective as a teaching intervention. When responding to pages in this exercise, medical students were forced to commit to a course of action regarding diagnostic or therapeutic interventions. This model allows for teachable moments in which concepts can be emphasized with excellent retention. Surveys show that residents prefer pagers to other mediums for receiving patient information.31 An alternative to a simulated paging exercise might be to have medical students take primary call pages with interns. This latter approach affords medical students experience fielding pages, but it does not allow for a systematic exposure to key patient care issues. Also, such an approach would not allow for the structured feedback provided by our design. Another approach might be to introduce a simulated paging exercise at the start of the internship, although orientation for internships is often very brief, limiting the scope and feasibility of such an endeavor.
The use of simulated pages is also appealing from a patient safety perspective. Interns exhibit a substantially greater incidence of serious errors in ICUs when working longer shifts and fatigued.32 If simulated paging exercises allow for earlier acquisition of patient management skills, such improvement may eventually translate into improvements in patient safety.33,34 Based on the short duration of this exercise, no conclusions are possible regarding the effect of simulated pages on patient safety, but this approach may prove beneficial when included as part of a larger curriculum on preparedness for the internship. We are currently undertaking a multi-institutional study that uses audio taping to allow for multiple ratings.
Some limitations of this study warrant mention. Although we measured the performance of our medical students, we did not evaluate the effect of the simulation exercise on the development of patient management skills. In addition, although our scoring instrument was based on accepted standards of care, the weightings used were subjective. Last, the study did not investigate whether simulated pages resulted in error reduction or in a decrease in adverse patient events during internship. Longitudinal studies tracking the effect of medical school interventions on the performance of new interns may allow for assessment of long-term effects.
Our findings suggest that the use of simulated patient pages fosters confidence among medical students regarding their preparedness for internship. This didactic approach is highly interactive and, once validated, may be used to identify areas of strength and weakness in individual medical students. This exercise is also likely to decrease unnecessary stress on the new intern. Our hope is that a simulated paging exercise will leave medical students more confident, better able to triage care, and more insightful as to the importance of multidisciplinary teamwork. Further studies need to be performed, including pretraining and posttraining assessments, to demonstrate the effectiveness of this intervention in improving patient care skills and optimizing patient safety once students have become interns.
The authors wish to acknowledge the members of the multidisciplinary team who helped in the development of simulated pages, and the Southern Illinois University Surgical Education and Performance Group, which provided expert input regarding the refinement of the scenarios used in this study.
Ethical approval was obtained from the Springfield Committee for Research Involving Human Subjects.
Presented at the second International Conference on Surgical Education and Training, Dublin, Ireland, May 2010.
1 Katz MH, Schroeder SA. The sounds of the hospital. Paging patterns in three teaching hospitals. N Engl J Med. 1988;319:1585–1589.
2 Wentz DK, Ford CV. A brief history of the internship. JAMA. 1984;252:3390–3394.
3 Abuhusain H, Chotirmall SH, Hamid N, O'Neill SJ. Prepared for internship? Ir Med J. 2009;102:82–84.
4 Tendulkar AP, Victorino GP, Chong TJ, Bullard MK, Liu TH, Harken AH. Quantification of surgical resident stress “on call.” J Am Coll Surg. 2005;201:560–564.
5 West CP, Tan AD, Habermann TM, Sloan JA, Shanafelt TD. Association of resident fatigue and distress with perceived medical errors. JAMA. 2009;302:1294–1300.
6 Barone JE. Problems with the fourth-year curriculum of students entering surgical residencies. Am J Surg. 1995;169:334–337.
7 Nakayama DK, Steiber A. Surgery interns' experience with surgical procedures as medical students. Am J Surg. 1990;159:341–343.
8 Sloan DA, Donnelly MB, Johnson SB, Schwartz RW, Strodel WE. Use of an objective structured clinical examination (OSCE) to measure improvement in clinical competence during the surgical internship. Surgery. 1993;114:343–350.
9 Antonoff MB, Shelstad RC, Schmitz C, Chipman J, D'Cunha J. A novel critical skills curriculum for surgical interns incorporating simulation training improves readiness for acute inpatient care. J Surg Educ. 2009;66:248–254.
10 Bansal PK, Saoji VA, Gruppen LD. From a “generalist” medical graduate to a “specialty” resident: Can an entry-level assessment facilitate the transition? Assessing the preparedness level of new surgical trainees. Ann Acad Med Singapore. 2007;36:719–724.
11 Boehler ML, Rogers DA, Schwind CJ, Fortune J, Ketchum J, Dunnington G. A senior elective designed to prepare medical students for surgical residency. Am J Surg. 2004;187:695–697.
12 Brunt LM, Halpin VJ, Klingensmith ME, et al. Accelerated skills preparation and assessment for senior medical students entering surgical internship. J Am Coll Surg. 2008;206:897–904.
13 Esterl RM Jr, Henzi DL, Cohn SM. Senior medical student “boot camp”: Can result in increased self-confidence before starting surgery internships. Curr Surg. 2006;63:264–268.
14 Fisher JW, Thompson BM, Garcia AD. Integrative clinical experience: An innovative program to prepare for internship. Teach Learn Med. 2007;19:302–307.
16 Nishisaki A, Hales R, Biagas K, et al. A multi-institutional high-fidelity simulation “boot camp” orientation and training program for first year pediatric critical care fellows. Pediatr Crit Care Med. 2009;10:157–162.
17 Peyre SE, Peyre CG, Sullivan ME, Towfigh S. A surgical skills elective can improve student confidence prior to internship. J Surg Res. 2006;133:11–15.
18 Pliego JF, Wehbe-Janek H, Rajab MH, Browning JL, Fothergill RE. Ob/gyn boot camp using high-fidelity human simulators: Enhancing residents' perceived competency, confidence in taking a leadership role, and stress hardiness. Simul Healthc. Summer 2008;3:82–89.
19 Blum NJ, Lieu TA. Interrupted care. The effects of paging on pediatric resident activities. Am J Dis Child. 1992;146:806–808.
20 Harvey R, Jarrett PG, Peltekian KM. Patterns of paging medical interns during night calls at two teaching hospitals. CMAJ. 1994;151:307–311.
21 Arora VM, Georgitis E, Siddique J, et al. Association of workload of on-call medical interns with on-call sleep duration, shift duration, and participation in educational activities. JAMA. 2008;300:1146–1153.
22 Dawson D, Reid K. Fatigue, alcohol and performance impairment. Nature. 1997;388:235.
23 Bellini LM, Baime M, Shea JA. Variation of mood and empathy during internship. JAMA. 2002;287:3143–3146.
24 Vygotsky LS. Theories of Development—Concepts and Applications. Englewood Cliffs, NJ: Prentice Hall; 1992.
25 Knowles M. The Adult Learner: A Neglected Species. Houston, Tex: Gulf Publishing; 1990.
26 Lefor A, Gomella L, Wiebke E, Fraker D. Surgery on Call. 4th ed. New York, NY: McGraw-Hill Company; 2005.
27 Fauci AS, Braunwald E, Kasper DL, Hauser SL, Longo DL, Jameson JL. Harrison's Principles of Internal Medicine. 18th ed. New York, NY: McGraw-Hill; 2008.
28 Townsend CM, Beauchamp RD, Evers BM, Mattox KL. Sabiston Textbook of Surgery: The Biological Basis of Modern Surgical Practice. 16th ed. Philadelphia, Pa: Saunders; 2007.
29 Margalit R, Thompson S, Visovsky C, et al. From professional silos to interprofessional education: Campuswide focus on quality of care. Qual Manag Health Care. 2009;18:165–173.
31 Wagner MM, Eisenstadt SA, Hogan WR, Pankaskie MC. Preferences of interns and residents for e-mail, paging, or traditional methods for the delivery of different types of clinical information. Proc AMIA Symp. 1998:140–144.
32 Landrigan CP, Rothschild JM, Cronin JW, et al. Effect of reducing interns' work hours on serious medical errors in intensive care units. N Engl J Med. 2004;351:1838–1848.
33 Marshall RL, Gorman PJ, Verne D, et al. Practical training for postgraduate year 1 surgery residents. Am J Surg. 2000;179:194–196.
34 Qayumi AK, Cheifetz RE, Forward AD, et al. Teaching and evaluation of basic surgical techniques: The University of British Columbia experience. J Invest Surg. 1999;12:341–350.