Emergency physicians (EPs) are being asked to simultaneously care for multiple patients, which requires significant task switching,1 interruptions,2,3 and constant reassessment of task prioritization. Recent literature has shown that increasing emergency department (ED) crowding and increased patient care responsibilities may have an impact on physicians’ decision-making processes, such as decisions to admit.4 Indeed, crowded, multipatient environments such as the ED may lead physicians to think differently than when the department is less crowded.4
Understanding clinical decision making within these complex, multipatient contexts is of great importance for patient safety. Previous work on clinical reasoning and decision making has mainly focused on situations where a physician focuses on diagnosing a single patient’s condition(s).5–9 Limits in attention and memory create challenges for clinicians in multipatient environments, which may in turn increase risk of medical error and harm to patients.10
Researchers have also examined the ED using business or engineering perspectives, describing workflow or processes, and proposing alterations to improve both.11–17 Literature in psychology,18–21 nursing,22,23 and emergency medicine (EM)16,17 presents models of how experts think about and organize themselves at a macrocognitive level. A recent ethnographic study showed that EPs tend to prioritize patient momentum and manage time in the diagnostic, test-ordering, and treatment processes.17
Previous literature has also shed light on how experts tend to engage in higher-order (or macrocognitive) processes to govern their decision making.17,19–21 Insights from these studies provide educators with a general sense of how EPs might broadly organize their work tasks. Although there is emerging work on the phenomenon of task switching (commonly known as “multitasking”),1,24,25 there is a dearth of literature regarding physician reasoning when organizing care for multiple patients, making plans to organize care at an aggregate level, or organizing multiple tasks.26
Managing the care of multiple patients requires a clinician to engage in a number of connected but distinct phenomena, including task prioritization and task switching. To date, the connections between these tasks have not been well described in the literature. To improve the care of patients and support teachers training novice clinicians, it is key to first understand how physicians think and manage multipatient environments.
The aim of this study was to develop a conceptual framework for understanding how EM physicians manage multiplicity by examining how they prioritize patient care tasks in complex, multipatient settings.27 These findings may be useful to teachers who train physicians to function more efficiently in busy clinical environments.
This study was part of a larger program of research examining EP cognition in a simulated multipatient environment, which was conducted from July 2014 to May 2015 at McMaster University in Hamilton, Ontario, Canada.28,29 In this study, participants from both resident and attending physician groups indicated how and why they would prioritize and take actions on hypothetical patients using simulated tracker boards, speaking aloud to explain their thinking as they worked through each simulated scenario.
Initially, we asked the chiefs of the EDs from four local tertiary care academic hospitals in Hamilton, Ontario, Canada, to nominate three attending physicians whom they consider efficient managers of the ED. We then e-mailed the nominated physicians three times, one week apart, to recruit them into our study. The local practice for participants included supervising EPs seeing their own patients, as well as supervising and teaching other providers; meanwhile, residents are charged with seeing patients and presenting their findings to supervising EPs.
Following initial recruitment, we used a snowball sampling technique of peer nomination to optimize the chance for saturation.30,31 Each expert EP was asked to nominate three peers whom they admired for their ED managerial skills or efficiency. Resident–participants were enrolled via a convenience sampling technique.
The Hamilton Integrated Research Ethics Board and the University of Illinois at Chicago approved this study. Informed consent was obtained from each participant.
Simulated tracker board creation.
We developed a simulated tracker board interface using Adobe Dreamweaver CS6 (San Jose, CA). Adobe Photoshop CS6 (San Jose, CA) was used for designing graphical elements. To facilitate easy usage, we designed the tracker board with HTML and Java Script to appear like existing tracker boards used by the local EDs. Wait times of the patients and the number of patients on the board were variables in the tracker boards. We used Microsoft Office Excel 2013 (Redmond, WA) for creation and organization of the simulated patient database and to populate the tracker board interface with simulated profiles. Figure 1 depicts a sample tracker board interface.
Simulated patient database.
To create high-fidelity triage notes, we recruited five Canadian Triage Acuity Scale (CTAS) Triage System trained nurses and two EPs (one resident, one attending) to assist with their development.32,33 Each group was asked to develop triage notes for very sick (CTAS 1), moderately sick (CTAS 2/3), and ambulatory (CTAS 4/5) patients. They developed presenting medical profiles, including chief complaint, past medical/surgical history, medications, and vital signs. Each simulated triage note was reviewed by at least two additional people (T.C., K.V.D.) to ensure content validity and appropriateness, with all edits made as requested.
Participants were asked to think aloud about their prioritization process.34,35 We gave them a set script to explain the technique and reviewed an example case with the investigator if there were questions about the technique. Box 1 details the prompts for the think-aloud exercise.
Box 1Prompt for the Think-Aloud Exercise, From a Study of Emergency Physician Cognitive Process in a Multipatient Environment, McMaster University, Hamilton, Ontario, Canada, 2014–2015a
The following script was presented to participants to support their think-aloud processes:
We will now begin the think-aloud phase. As before, imagine you are about to begin the shift. There will be a number of patients who need to be seen by you on the tracker board. This time, we will ask that you perform the same prioritization task as before, but we would like you to think aloud when you’re prioritizing the patients that you should see next.
Some examples of things you might say:
“The first thing I need to do is….”
“I just thought of ____________, so, I’m going to do ________.”
We will now do a practice think-aloud procedure. I will give you feedback with regard to whether you are thinking aloud enough to let us understand your processes.
aInsights on constructing this script were derived from Van Someren MW, Barnard YF, Sandberg JA. The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes. London, UK: Academic Press; 1994:26.
We allowed the participants to engage and interact with the simulated tracker board in whatever manner they saw fit. We did not instruct the participants to click through, but they were allowed to operate as if practicing in the ED. The simulated tracker board allowed them to click through in a manner similar to the tracker boards used in their natural workplaces.
We randomly chose 4 cases from a complete bank of 30 cases. These cases were used as the scenarios for the think-aloud procedure (see Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/A516 for cases). Participants completed the think-aloud exercise for the same 4 cases. In each case, the participants were asked to sort through multiple patients on these simulated tracker boards, identifying the patient they would see first, those they would see soon, those for whom they would initiate orders, and those whom they would designate as low priority (i.e., would be seen at a later time).
We collected demographic and workplace characteristics for participating individuals, including age, number of years in practice, number of years/rotations of training, type of residency training, and their estimate of the average number of patients seen per hour during a typical shift.
An interpretive descriptive technique was conducted after collecting data from 10 participants in each group (10 residents, 10 attendings).36 The texts generated were constantly reviewed after each interview by a single researcher (T.C.). We analyzed 20 transcripts in clusters of 3 to 4 transcripts at a time in an iterative, constant comparative approach until sufficiency was reached as determined by the analysis team (T.C., M.M., M.L.). We actively sought negative or contradicting examples to ensure the rigor of our analysis as themes emerged. We also conducted axial coding to ensure coherency and decrease redundancy of the analysis. Although we had originally aimed for 15 individuals per grouping, after the third group of transcripts, it was clear that we had reached thematic sufficiency. A member check37 was conducted for a two-week period via e-mail after the analysis was completed. No differing responses were received.
Participant demographic characteristics are summarized in Table 1. Attending EPs and residents were significantly different in age and years of practice (P < .001), but not the proportion of time they spent in an academic health center.
Using the four scenarios, we analyzed the think-aloud transcripts to identify the processes involved in patient prioritization. Our analysis demonstrated that there was no substantial difference in the overall approach of task prioritization between expert and novice physicians in our simulation. Although the process was common, functional story generation by experts was deeper than novices; attendings were more committed to their prioritization decisions, a finding we will explore in the coming paragraphs. See Figure 2 for an overview of the prioritization process.
There were three main phases in setting priorities for managing multiple patients: viewing the big picture, developing a functional patient story, and creating a relative prioritization list.
For each new patient, there was a cyclical process that occurs, with each chart opening leading to a new functional story, which prompted a rejuggling of relative priorities.
Phase 1: Viewing the big picture
The first step was taking in the big picture (i.e., viewing the entire tracker board). During this phase, participants determined the complexity of the simulated scenario. For most participants, this resulted in an emotional reaction (e.g., *sighs* “this is a busy board…”) or a categorization technique (e.g., “Oh, this is a scenario with lots of vaginal bleeding cases”). Mostly, participants opened specific triage notes incorporated into the tracker board in one of four ways (see Table 2):
- Sequentially (top-to-bottom);
- By order of acuity, with higher-acuity Canadian Triage Acuity Score (CTAS) cases being opened first (e.g., CTAS1s before CTAS2s);
- By looking at only triage notes listed as “to be seen” (i.e., not under another physician); or
- By some other strategy (age, chief complaint, time in ED, etc.).
But at times they would combine two of these strategies sequentially (i.e., looked through patients that had not yet been seen by another physician, then in order based on CTAS).
In this phase, there was a divergence in behavior between physicians based on their degree of trust of other physicians. Many attending and resident physicians were more likely to initiate a triage file-opening strategy that would focus on “to be seen” files, and not open the files listed under the care of other physicians. Other physicians would selectively open the files of high-acuity patients under another physician’s care, but not low-acuity files. Those participants who described themselves as less trusting opened every file on the boards and incorporated all relevant patients into their plans regardless of the involvement of other physicians. The following quote is particularly illustrative of the role of trust within a multiprovider and multipatient environment:
I would take into account who the triage nurse is because there are some triage nurses who I give more credence to than others, but we don’t tell anybody about that. They are also based on the emerg[ency] physicians, so if there are some emerg[ency] physicians—not to name names—but there are emerg[ency] physicians who if they are in there seeing the patient, I don’t need to go in and see the patient because I know that they will be well managed. I will walk by, lay eyes on them, but really just so that I know what’s, I can, I can say that I have seen them. There are others who I will go in and chat with the bedside nurse, possibly lay eyes and hands on the patient because I am less sure about what’s going on, because I have certain concerns. I think that is really the process that also comes into play, so today I assumed that everybody was a dunce, because I don’t know any of the doctors that were listed because they were all fictitious, but there is a lot of variability based on who the triage nurse is, who the charge nurse is, who the other doc is in terms of how I go through that list. (Attending 4)
Phase 2: Developing a functional patient story
Participants started this phase with a scan of all available information that they deemed relevant (usually age, chief complaint, CTAS score, triage note, triage vitals, and more rarely the time in ED, past medical history, medications, allergies, and previous medical records). After extracting relevant information, participants began generating a functional patient story. This phase involved several subprocesses, including:
- Generating a tentative diagnosis;
- Accessing recalled patient prototypes (i.e., key memories or examples that capture the core meaning of particular categories)5,38; and
- Linking previously primed responses (i.e., management strategies) to the tentative diagnosis.
Essentially, participants generated ED-context-specific illness scripts for these simulated patients.39,40 The physicians used a filtering process wherein they determined relevant elements within the available information to create a construct that best described the patient (e.g., young, healthy, stable female patient with vaginal bleeding vs. young, healthy, unstable female patient with likely ectopic pregnancy). This procedure was linked to next steps in both the diagnostic process and management planning. Attending physicians were much more certain about their patient stories and tended to quickly label the patients upon reading the available information. Often, attendings would go so far as to make a presumptive diagnosis or the leading differential diagnosis, or prioritize a high-risk diagnosis that they should rule out.
We have three vaginal bleedings, oh that is always hard to keep track of! We would look at the CTAS 1 first. So tachy [sic, as in tachycardic], having heavy period, 34-year-old, irregular periods but we don’t know if she could be pregnant. Okay, um.… So this one is tachycardic, positive preg [sic, as in pregnancy] test, short of breath, and appears in distress with severe pain. So she could be ectopic pregnancy, she’s tachy … so more tachy anyways than the first one whose heart rate was 110. (Attending 4)
Some participants tended to be less certain in some cases or at least less willing to speak aloud about their presumptive diagnoses. These participants would often wonder about multiple diagnoses, and some would even stop short of this, hinging on surface features such as vital signs to make their decisions. The following quote illustrates an attending remarking on the broad differential diagnosis for one case.
And then I will check my CTAS 3 patient. Sounds like an alcoholic pancreatic or gastritis or something with relatively okay vitals, not super worried about that one. My CTAS 4 I am not sure what is going on there but vitals are normal. Definitely not an emergency. (Attending 5)
Phase 3: Creating a relative prioritization list
During this phase, physicians use the available information to make a determination of the sequence of patients to attend to in person. Most participants had four steps within this phase:
- Determine whether they believed a patient was sick or not sick (as previously described in the literature41,42);
- Decide how urgently this patient needed the doctor’s attention, a nurse’s attention, or orders fulfilled, or if they could wait;
- Compare the newest patient to other known patients in the same scenario; and
- Generate a relative prioritization list.
Relative priorities and functional patient stories
The functional patient story was strongly associated with the relative priority-setting process. During this phase, participants leaned heavily on the previously determined prototypical patient stories (e.g., the stable vaginal bleeding case vs. the unstable likely ectopic pregnancy).
This quote more fully illustrates this phenomenon:
Next patient is a 33-year-old with abdominal pain there, is a bit tachycardic, drank a lot of alcohol on their trip but otherwise well. So again, probably not someone who needs to be seen urgently. And then finally a 62-year-old with abdominal pain who is febrile, hypertensive, and tachycardic and although the pain has been there for two months it is worse after eating so I think I am considering whether or not this patient has acute cholecystitis or pancreatitis based on the fact that they are febrile and meeting kind of sepsis criteria I would prioritize them over seeing them over the abdominal pain patient. (Resident 3)
In the above quote the resident is comparing a patient with undifferentiated abdominal pain versus a patient that may have intra-abdominal sepsis. Most participants relied heavily on identifying previous patients by the previously generated prototypical patient stories, referring to patients as “the ectopic” or “the old person with abdominal pain.”
Our study showed that attending and resident EPs had three main phases as depicted: viewing the big picture, developing a functional patient story, and creating a relative prioritization list (see Figure 1). Although the process was common, experts’ functional story generation was deeper than residents’. Attendings were more committed to their prioritization decisions.
Macrocognition and the role of the big picture
Our participants began by viewing the big picture and making sense of their possible tasks. Previous work in this area has been done by Schubert and colleagues,17 which examined the differences in macrocognition between expert and novice EPs with regard to their thinking in the ED. The findings of Schubert et al focused on sensemaking for diagnostic reasoning of the expert EPs versus the novices, noting that the novices tended to rely more heavily on objective measures (e.g., lab tests) when making decisions versus the expert EPs, who use “big picture” processes and more cognitive elasticity (i.e., flexibility and willingness to entertain multiple ideas) when making decisions.17
Pertinent theories on macrocognitive processes arise from the work of Gary Klein19,20 on expert practitioners (e.g., firefighters and military leaders); he described a number of macrocognitive processes that govern their decision making. Klein explained how experienced agents tend to engage in functions (such as problem detection, coordination, sensemaking, planning, adaption, and naturalistic decision making) and processes (developing mental models, mental simulation and story building, maintaining common ground, managing uncertainty/risk, identifying leverage points, and managing attention).19,20
While experts and novices have been shown to have different macrocognition in the ED,17 it is unclear as to whether these differences between experts and novices were due to disparities in thinking or the differences in their roles within the ED. Experts were found to be focusing on the ED within the health system, focusing on throughput, supervising learners, and applying their medical knowledge.17 Novices tended to be more focused on the specific patient care activities (e.g., charting, patient interview), and not on the overarching processes at play.17 If tasked with the same responsibilities as an attending EP, would the interns think differently?
Our simulated study sought to eliminate these workplace-induced differences. Of note, we did not note that there was a substantial difference in their overall thinking process of attendings and junior residents (postgraduate year 1 or 2) when tasked with a similar job of prioritizing multiple patients. Both groups needed to gain a “big picture” perspective on these complicated simulated scenarios before moving on.
The importance of patient stories for prioritization
We found that the second step in managing multiplicity was for our participants to develop individual patient stories, which could then be compared. This is reminiscent of de Groot’s43 work on chess masters, which suggested that experts organize data into “chunks,” or familiar constellations of items that might interact with one another (e.g., prototypical configurations of chess pieces). In nursing, when experts and novices do the same job, there are differences in how they handle complexity.44 Ebright and Paterson described an adaptive strategy called “stacking,” which allows expert nurses to organize and reprioritize tasks when interrupted.22,23 Stacking can be described as parallel “to-do” lists, allowing a nurse to simultaneously initiate work on multiple task streams and rapidly plan/replan.23 The process of creating a functional patient story may be a similar process to the chunking that de Groot observed, and may serve a similar process as stacking, allowing a physician to bundle a number of considerations as a single thought.
“Stacking” may be insufficient in explaining how EM clinicians manage multiplicity, or even to explain effective task prioritization. Burger and colleagues44 found that for experts to function well, “experience is necessary to develop the cognitive skills to prioritize the multiple demands on their time and attention.” Advanced beginners prioritized linearly (i.e., they did one thing at a time), whereas competent nurses employed stacking to initiate parallel processes, thereby increasing efficiency. Expert nurses were able to see and anticipate multiple aspects of care, allowing them to efficiently perform their patient care tasks.44
One could argue that the “stacking” from the nursing literature or the “functional patient story” are simply manifestations of de Groot’s18 “chunking” phenomenon: a grouping of related tasks that are seen within a certain constellation, thereby allowing experts to free up their working memory to better handle new data, such as an interruption that forces them to integrate new tasks into a workflow. Work by Sklar and colleagues15 showed that experienced clinicians demonstrated the use of a flexible strategy of testing to arrive at a diagnosis. Their added experience helps enrich an EP’s mental repository of various prototypes.5,38 These prototypes trigger the previously created cue–response relationships with management and disposition plans (à la Klein’s20 “recognition primed decision making”), and operationalize the prototype and plans into functional patient stories that allow them to anticipate and navigate the complicated ED system.
This explanation is supported by our observation that the attending EPs tended to have richer, more descriptive patient stories after viewing the triage notes of our simulated patients. If attendings can take experience to create richer functional patient stories and then engage in stacking, this would allow them to flip quickly between the patients, just like Ebright and Burger’s experienced nurses.22,23,44
Managing multiplicity by making lists
The development of functional patient stories seemed to be quite important in allowing our participants to flip between multiple patients and engage in the relative prioritization process. Once constructed, individual stories were compared by both groups to determine how they thought they could best prioritize tasks and patients. While previous literature has warned against diagnostic labeling,45 the urge to do so can partially be explained by the phenomenon observed in this study.
The EM literature contains cautionary tales of error caused by diagnostic momentum, anchoring bias, and problems with diagnostic labeling,46–50 but many of these techniques are without strong evidentiary basis.8 The realities of modern EM, however, may require that we acknowledge the need to develop functional patient stories. A requirement of managing multiple patients may be the chunking and the construction of labels (shallow descriptions) and functional stories (richer descriptions) in an effort to economize on thought and perform more efficiently in the setting of complex multipatient environments.
Our model may help educators to anticipate how novice physicians create functional patient stories in multipatient environments. Our proposed conceptual framework may help to diagnose resident physicians who may have difficulty managing multiple patients in an efficient manner. Likewise, since the act of creating functional patient stories may take physicians closer toward a presumptive diagnosis, we may need to be wary of how this desire to chunk and stack multiple patients may predispose EPs to misdiagnosis or error in high-patient-volume situations.
One of the key limitations of our present study is that it was conducted at a single academic institution. Fortunately, we were able to sample from attendings and residents who practice in very different hospital environments and included several attendings who had practiced part-time in other communities. Think-aloud exercises may also not capture all the fast, instinctive thinking (i.e., System 1 processes51), as these processes are not readily available for introspection (and therefore unlikely to be captured by thinking aloud). With qualitative studies, however, the in-depth analysis of a selected group from a single institution occurs frequently.17,52–55 The use of nonclinician investigators was one method we used to ensure that the lead investigator remained reflexive about her experiences.
We have described a framework that conceptualizes how expert physicians and their trainees prioritize patients in complex multipatient environments. Our framework highlights three phases of thinking, beginning with a focus on the general (e.g., a whole group of patients), followed by a focus on the specific (developing a functional story that described an individual patient’s likely course), and an iterative prioritization process as new information is uncovered. Our findings may be useful to teachers who are seeking to teach trainees to function more effectively in multipatient environments.
T.M. Chan would like to thank all her teachers at the University of Illinois at Chicago who have taught her both the art and craft of medical education research.
1. Skaugset LM, Farrell S, Carney M, et alCan you multitask? Evidence and limitations of task switching and multitasking in emergency medicine. Ann Emerg Med. 2016;68:189–195.
2. Chisholm CD, Dornfeld AM, Nelson DR, Cordell WHWork interrupted: A comparison of workplace interruptions in emergency departments and primary care offices. Ann Emerg Med. 2001;38:146–151.
3. Monteiro SD, Sherbino JD, Ilgen JS, et alDisrupting diagnostic reasoning: The effects of interruptions on the diagnostic performance of residents and emergency physicians. Acad Med. 2015;90:511–517.
4. Gorski JK, Batt RJ, Otles E, Shah MN, Hamedani AG, Patterson BWThe impact of emergency department census on the decision to admit. Acad Emerg Med. 2017;24:13–21.
5. Elstein AS, Schwartz A, Schwarz AClinical problem solving and diagnostic decision making: Selective review of the cognitive literature. BMJ. 2002;324:729–732.
6. Norman GR, Eva KWDiagnostic error and clinical reasoning. Med Educ. 2010;44:94–100.
7. Cahan A, Gilon D, Manor O, Paltiel OProbabilistic reasoning and clinical decision-making: Do doctors overestimate diagnostic probabilities? QJM. 2003;96:763–769.
8. Schriger DL, Brown TBDecisions, decisions: Emergency physician evaluation of low probability-high morbidity conditions. Ann Emerg Med. 2005;46:534–535.
9. Croskerry PThe cognitive imperative: Thinking about how we think. Acad Emerg Med. 2000;7:1223–1231.
10. Norman GR, Monteiro SD, Sherbino J, Ilgen JS, Schmidt HG, Mamede SThe causes of errors in clinical reasoning: Cognitive biases, knowledge deficits, and dual process thinking. Acad Med. 2017;92:23–30.
11. Calder LA, Forster AJ, Stiell IG, et alMapping out the emergency department disposition decision for high-acuity patients. Ann Emerg Med. 2012;60:567–576.e4.
12. Marmor YN, Golany B, Israelit S, Mandelbaum ADesigning patient flow in emergency departments. IIE Trans Healthc Syst Eng. 2012;2:233–247.
13. Lucas R, Farley H, Twanmoh J, et alEmergency department patient flow: The influence of hospital census variables on emergency department length of stay. Acad Emerg Med. 2009;16:597–602.
14. King DL, Ben-Tovim DI, Bassham JRedesigning emergency department patient flows: Application of Lean thinking to health care. Emerg Med Australas. 2006;18:391–397.
15. Sklar DP, Hauswald M, Johnson DRMedical problem solving and uncertainty in the emergency department. Ann Emerg Med. 1991;20:987–991.
16. Walter SR, Raban MZ, Dunsmuir WTM, Douglas HE, Westbrook JIEmergency doctors’ strategies to manage competing workload demands in an interruptive environment: An observational workflow time study. Appl Ergon. 2017;58:454–460.
17. Schubert CC, Denmark TK, Crandall B, Grome A, Pappas JCharacterizing novice-expert differences in macrocognition: An exploratory study of cognitive work in the emergency department. Ann Emerg Med. 2013;61:96–109.
18. de Groot AD, Gobet F, Jongman RWPerception and Memory in Chess: Studies in the Heuristics of the Professional Eye. 1996.Chicago, IL: Van Gorcum & Co.;
19. Klein GNaturalistic decision making. Hum Factors. 2008;50:456–460.
20. Klein GAA recognition-primed decision (RPD) model of rapid decision making. Decis Mak Action Model Methods. 1993:138–147.
21. Walter SR, Raban MZ, Dunsmuir WTM, Douglas HE, Westbrook JIEmergency doctors’ strategies to manage competing workload demands in an interruptive environment: An observational workflow time study. Appl Ergon. 2017;58:454–460.
22. Ebright PR, Patterson ES, Chalko BA, Render MLUnderstanding the complexity of registered nurse work in acute care settings. J Nurs Adm. 2003;33:630–638.
23. Patterson ES, Ebright PR, Saleem JJInvestigating stacking: How do registered nurses prioritize their activities in real-time? Int J Ind Ergon. 2011;41:389–393.
24. Laxmisan A, Hakimzada F, Sayan OR, Green RA, Zhang J, Patel VLThe multitasking clinician: Decision-making and cognitive demand during and after team handoffs in emergency care. Int J Med Inform. 2007;76:801–811.
25. Clyne BMultitasking in emergency medicine. Acad Emerg Med. 2012;19:230–231.
26. Croskerry PContext is everything or how could I have been that stupid? Healthc Q. 2009;12:171–177.
27. Pines JMWhat cognitive psychology tells us about ED physician decision-making and how to improve it. Acad Emerg Med. 2017;24:117–119.
28. Chan TMWhat’s next: Cognitive task analysis of emergency physicians’ experience in multi-patient environments [masters thesis]. 2016.Department of Medical Education, College of Medicine, University of Illinois at Chicago;
29. Chan TM, Van Dewark K, Sherbino J, Schwartz A, Norman G, Lineberry MFailure to flow: An exploration of learning and teaching in busy, multi-patient environments using an interpretive description method. Perspect Med Educ. 2017;6:380–387.
30. Goodman LASnowball sampling. Ann Math Stat. 1961;32:148–170.
31. Heckathorn DDSnowball versus respondent-driven sampling. Sociol Methodol. 2011;41:355–366.
32. Beveridge R, Ducharme J, Janes L, Beaulieu S, Walter SReliability of the Canadian Emergency Department Triage and Acuity Scale: Interrater agreement. Ann Emerg Med. 1999;34:155–159.
33. Manos D, Petrie DA, Beveridge RC, Walter S, Ducharme JInter-observer agreement using the Canadian Emergency Department Triage and Acuity Scale. CJEM. 2002;4:16–22.
34. Ericsson KA, Simon HAProtocol Analysis: Verbal Reports as Data. 1993.Revised ed. Cambridge, MA: MIT Press;
35. Ericsson KAEricsson KA, Charness N, Feltovich PJ, Hoffman RRProtocol analysis and expert thought: Concurrent verbalizations of thinking during experts’ performance on representative tasks. In: The Cambridge Handbook of Expertise and Expert Performance. 2006:New York, NY: Cambridge University Press; 223–241.
36. Gillespie MUsing the situated clinical decision-making framework to guide analysis of nurses’ clinical decision-making. Nurse Educ Pract. 2010;10:333–340.
37. Kitto SC, Chesters J, Grbich CQuality in qualitative research. Med J Aust. 2008;188:243–246.
38. Bordage G, Zacks RThe structure of medical knowledge in the memories of medical students and general practitioners: Categories and prototypes. Med Educ. 1984;18:406–416.
39. Feltovich PJ, Barrows HSSchmidt HG, De Volder MLIssues in generality in medical problem solving. In: Tutorials in Problem-Based Learning: A New Direction in Teaching the Health Professions. 1984:Maastricht, the Netherlands: Van Gorcum & Co.; 128–142.
40. Charlin B, Boshuizen HP, Custers EJ, Feltovich PJScripts and clinical reasoning. Med Educ. 2007;41:1178–1184.
41. Wiswell J, Tsao K, Bellolio MF, Hess EP, Cabrera D“Sick” or “not-sick”: Accuracy of System 1 diagnostic reasoning for the prediction of disposition and acuity in patients presenting to an academic ED. Am J Emerg Med. 2013;31:1448–1452.
42. Gruppen LD, Woolliscroft JO, Wolf FMThe contribution of different components of the clinical encounter in generating and eliminating diagnostic hypotheses. Res Med Educ. 1988;27:242–247.
43. de Groot ADThought and Choice in Chess. 1978.The Hague, the Netherlands: Mouton Publishers;
44. Burger JL, Parker K, Cason L, et alResponses to work complexity: The novice to expert effect. West J Nurs Res. 2010;32:497–510.
45. Ilgen JS, Eva KW, Regehr GWhat’s in a label? Is diagnosis the start or the end of clinical reasoning? J Gen Intern Med. 2016;31:435–437.
46. Noon AJThe cognitive processes underpinning clinical decision in triage assessment: A theoretical conundrum? Int Emerg Nurs. 2014;22:40–46.
47. Croskerry PCritical thinking and decisionmaking: Avoiding the perils of thin-slicing. Ann Emerg Med. 2006;48:720–722.
48. Croskerry PHenriksen K, Battles JB, Marks ES, Lewin DIDiagnostic failure: A cognitive and affective approach. In: Advances in Patient Safety: From Research to Implementation. 2005.Rockville, MD: Agency for Healthcare and Quality Research;
49. Croskerry PCognitive forcing strategies in clinical decisionmaking. Ann Emerg Med. 2003;41:110–120.
50. Croskerry PFrom mindless to mindful practice—Cognitive bias and clinical decision making. N Engl J Med. 2013;368:2445–2448.
51. Stanovich KE, Toplak ME, West RFThe development of rational thought: A taxonomy of heuristics and biases. Adv Child Dev Behav. 2008;36:251–285.
52. Moulton CA, Regehr G, Lingard L, Merritt C, Macrae HOperating from the other side of the table: Control dynamics and the surgeon educator. J Am Coll Surg. 2010;210:79–86.
53. Ginsburg S, van der Vleuten C, Eva KW, Lingard LHedging to save face: A linguistic analysis of written comments on in-training evaluation reports. Adv Health Sci Educ Theory Pract. 2016;21:175–188.
54. Watling C, Driessen E, van der Vleuten CP, Lingard LLearning from clinical work: The roles of learning cues and credibility judgements. Med Educ. 2012;46:192–200.
55. Goldszmidt M, Faden L, Dornan T, van Merriënboer J, Bordage G, Lingard LAttending physician variability: A model of four supervisory styles. Acad Med. 2015;90:1541–1546.