Secondary Logo

Journal Logo

Research Reports

How Do Attending Physicians Prepare Residents to Deliver High-Value, Cost-Conscious Care?

Stammen, Lorette A. MD; Driessen, Erik W. PhD; Notermans, Celine C.V.I.; Scheele, Fedde PhD; Stassen, Laurents P.S. PhD; Stalmeijer, Renée E. PhD

Author Information
doi: 10.1097/ACM.0000000000003051

Abstract

When citizens and governments worldwide spend a substantial part of their budgets on health care, it is crucial that these expenditures are considered appropriate and sustainable over time. Unfortunately, estimates indicate that 20% of health care expenditures are wasteful.1,2 Interventions aimed at reducing wasteful health care services often focus on physicians.3,4 Physicians could use their medical expertise to safeguard the quality of care while providing care that is not only cost-conscious but also of high value.3,5 Indeed, physicians are included in high-value, cost-conscious care (HV3C), defined as “care that aims to assess the benefits, harms, and costs of interventions and consequently to provide care that adds value.”6 For the incorporation of HV3C in clinical practice, physicians should fulfill their role as stewards of resources and avoid misuse of health care services that do not benefit patient care.3 There is a rising awareness of the importance of HV3C for current and future physicians, which is partly attributed to the launch of well-known initiatives such as Choosing Wisely,7 the Top Five list,8 and the High Value Care Curriculum.9 To date, competency frameworks such as the Accreditation Council for Graduate Medical Education milestones10 and Canadian Medical Education Directives for Specialists11 have included HV3C, and it is generally accepted that medical education needs to rise to the challenge of training physicians to provide HV3C and incorporate it in curricula worldwide.5,12,13 Stammen et al conducted a systematic review of educational interventions that addressed the topic of HV3C and found that effective educational interventions (1) provide health care workers with knowledge about HV3C, (2) stimulate reflection on care delivery from an HV3C perspective, and (3) create an environment that prioritizes HV3C delivery.14 Creating an environment that prioritizes HV3C delivery relies partly on attending physicians, who play an important role in residency training as supervisors, role models, and members of the supportive environment in which residents work and learn.14–18 Unfortunately, a recent study indicates that residents often feel that HV3C training is inadequate in the workplace.16 Most residency training is conducted in the workplace,19 and lessons from graduate medical education can persist long into practice.13,20–22 This research explored how attending physicians prepare residents to deliver HV3C, a topic that has not been studied previously. Increased understanding could help both residents and attending physicians to optimize the effect of HV3C training in the workplace.

Method

Study design

We invited attending physicians to participate in semistructured interviews to share their experiences preparing future physicians for HV3C. This technique facilitates in-depth conversation, supported by examples, and provides space to discuss related topics in addition to those listed in the interview guide.23 The interview guide was based on a systematic review of the 3 essential elements of teaching HV3C (knowledge about HV3C, reflection on care delivery from an HV3C perspective, and creation of a supportive environment that prioritizes HV3C delivery).14 (The interview guide is provided as Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/A766.) A pilot interview was held with an attending physician familiar with qualitative research, medical education, and HV3C. No modifications were made after the pilot interview. Ethical approval was obtained from the ethical review board associated with the Netherlands Association for Medical Education on June 18, 2015, under file number 547. The research was conducted by researchers from Maastricht University in Maastricht, the Netherlands.

Setting

The context of our study is postgraduate residency training in the Dutch health care system. Dutch health care is funded by a combination of compulsory social insurance and voluntary private insurance. List 1 presents an overview of the financial structure of the Dutch health care system, and Table 1 presents an overview of the Dutch medical education continuum. In line with the increasing interest in HV3C worldwide, awareness of HV3C in the Dutch health care system is visible in government grants24 and in conferences,25 and it has officially been on the national training agenda since 2013.26,27 Despite this increased awareness, none of the medical centers participating in this study had a formal program to train residents in HV3C.

List 1

General Characteristics of Dutch Health Care System (Mixed System of Compulsory Social Insurance and Voluntary Private Insurance)

  • Basic insurance is obligatory for every citizen (compulsory social insurance).
  • Payment is through monthly fee and wage taxes (employer).
  • Additional insurance coverage is voluntary (voluntary private insurance).
  • Primary care is 100% covered.
  • Out-of-pocket payment for basic care is capped at €385 (U.S. $423.50) unless citizens choose a higher threshold (maximum €850 [U.S. $935]) for out-of-pocket payment in return for lower monthly fees.
  • System includes academic and nonacademic hospitals.
Table 1
Table 1:
General Characteristics of Dutch Medical Education Continuum

Sampling and data collection

Between September 2016 and August 2017, we iteratively collected data by interviewing attending physicians. We purposively sampled28 attending physicians from various training programs, both surgical and nonsurgical and hospital and nonhospital based. Six program directors assisted in identifying potential study participants from their training programs. We asked the program directors to select 2 attending physicians from their departments who were representative of the department and differed in their level of involvement (involvement or no involvement) in formal educational roles. We emailed invitations to all potential participants. In the email, we included an informative letter that explained why we were approaching them and what their participation would involve, as well as a form to document their informed consent. Two potential participants (both from 1 discipline) did not respond to the emailed invitation and subsequent reminders, so we asked the director of the training program in question to identify 4 other participants. Again, 2 did not respond. We completed 12 interviews with 9 male (75%) and 3 female (25%) attending physicians from teaching hospitals who work in 6 specialties: dermatology, elderly care, general practice, internal medicine, orthopedic surgery, and surgery. Participants worked in different regions of the Netherlands and were based in 5 different medical centers. Attending physicians who specialized in elderly care and family medicine were classified as nonhospital because they worked in either elderly care institutions or family medicine practices (comparable to outpatient care or ambulatory facilities). Seven participants (58%) held educational roles such as clerkship coordinator or residency program director in their specialty. All participants were involved in supervising residents in daily practice and worked in shifts, as is common in teaching hospitals. The interviews took place at a location convenient to the participant and were scheduled to last 60 minutes. The duration varied from 38 to 58 minutes. The first author, who is a family medicine resident, conducted all the interviews, which were audiotaped and then transcribed verbatim in Dutch. Quotes that were selected for the final manuscript were translated by an English language editor. ATLAS.ti (version 8.3.1., Scientific Software Development GmbH, Berlin, Germany) was used to manage the data.

Data analysis

Data were analyzed iteratively, based on the principles of constructivist grounded theory approach, beginning with line-by-line open coding.29 The second step was axial coding, in which we identified the central codes and the relationships between codes.30 Finally, we used selective coding to develop a provisional coding scheme to better categorize and explore the conditions in which HV3C was addressed. We used the coding scheme, after further refinement by L.A.S., C.C.V.I.N., and R.E.S., to review the transcripts. The whole team discussed the coding scheme and relationships between codes, which resulted in several key themes. After 12 interviews in total, data sufficiency was met, meaning that no new themes emerged from the data, and we achieved an adequate understanding of the themes.28,31 Rigor of the study was ensured by adhering to the following principles throughout the study and writing process: credibility (through investigator triangulation, pilot interview), transferability (through describing context and sampling, relation to literature), dependability (through iterative data collection and analysis and data collection until saturation), and conformability (through reflexivity, analytic notes, and reflective memos).32 We also used the COREQ (consolidated criteria for reporting qualitative research) checklist as a reporting tool.33

In qualitative data analysis, it is important to consider the background of researchers, including their involvement in previous research and the influence of that research on data collection and analysis.34 Our research team contained experts from several disciplines (medical education, gynecology, surgery, and family medicine) and various levels of medical training (medical student, resident, senior physician, and head of residency training). In an effort to maximize the contributions from these different backgrounds, the entire team was closely involved in data analysis.29

Results

Data analysis made apparent that the attending physician’s approach to HV3C influences how he or she prepares residents to deliver HV3C in the workplace. This section discusses the findings on both aspects in turn.

Attending physician’s approach to HV3C

For attending physicians, 3 notable differences became apparent in their approach to their own HV3C practice and to supervision: priority of HV3C training, feedback on HV3C delivery, and obstacles to HV3C delivery.

Priority of HV3C training.

We noted a difference in the priority given to HV3C training in residency. This difference in priority was noted in the words chosen by participants35 that express a low priority (e.g., “the only occasion,” “sometimes”) compared with words that express a high priority (e.g., “drilled into residents,” “always”). Some attending physicians addressed HV3C only when care delivery was perceived as “excessive” by the attending physicians; others aimed to continuously train residents in HV3C delivery: “Um, well actually it’s the core of our profession, you know? I think it comes up in almost every case, in one way or another” (S4). Despite differences in the priority given by participants, all attending physicians aimed to incorporate HV3C both in their own care delivery and in training residents. They described how considering the risks, costs, and harms is essential to high-quality care and thus of the utmost importance for health care delivery. Although the financial cost of health care was also considered important, some participants explicitly rated it as secondary to quality of care or considered it a “dirty word.”

The participants who continuously trained residents in HV3C delivery used a variety of teaching methods and made the training part of daily supervision. Teaching methods such as Socratic questioning, role modeling, and setting limits were used to prepare residents for the delivery of HV3C and will be discussed later in the Results section. During the interviews, these physicians supplied many instances of when they felt they were training residents in HV3C delivery, for example, when discussing a resident’s care services, such as prescribing expensive medication or delivering care that did not meet the patient’s preferences. We also identified physicians who declared that HV3C was important yet applied teaching methods to facilitate HV3C learning only occasionally: “Yes, I do it, but am I really aware of doing it? I wouldn’t know” (S11). A few participants had difficulty providing examples of when they trained residents to deliver HV3C.

Feedback on HV3C delivery.

In general, the participants were interested in receiving feedback on their HV3C behavior from colleagues, residents, interns, and other health care professionals. Additionally, they mentioned their need for supportive data on their own behavior and the opportunity to compare their data with the data of colleagues. Some participants had access to such data (e.g., frequency of complications, frequency of referrals, prescribing data) or had insight into costs (financial overview of own productivity) that were gleaned from feedback tools. Feedback tools gave them the ability to monitor their own behavior with or without the opportunity to compare and discuss these details with local and regional colleagues. At meetings that included the topic of local and regional testing, “ the residents are present, of course. . . .”(S8). Participants who did have access to feedback tools perceived these tools as useful to determine and reflect on HV3C. Participants with no access to feedback tools knew about the use of feedback tools in other specialties, mostly in family medicine and elderly care. In the absence of such tools, participants perceived a lack of insight into their own care delivery, which was considered a real hindrance to critical reflection on HV3C delivery and their ability to train residents in such behavior.

Obstacles to HV3C delivery.

Attending physicians mentioned 2 specific obstacles to HV3C delivery in daily practice: reluctance to discuss HV3C with patients and lack of knowledge about the cost of health care services. The reluctance to discuss HV3C, specifically the costs, occurred mostly during patient consultations. Participants often mentioned the patient–physician relationship as a reason for not discussing health care costs with patients; a few participants felt that it implied prioritizing money over quality. The physicians who did discuss costs openly explained that they wanted to do so because of “societal concerns about rising health care costs” (S9) and “how (expensive) medicine is not always desirable or beneficial for patients” (S1). In light of the patient–physician relationship, these participants could not give examples of conflicts arising when discussing costs with patients, although all emphasized that this delicate topic needed considerate communication. Residents working with these attending physicians were exposed to these conversations and encouraged to discuss costs themselves if deemed necessary.

Participants felt that gaps in knowledge included both inherent knowledge of health care costs and economics in general, as well as an inability to discover the cost of frequently used diagnostics or drugs. As one participant said, “The only difficult thing is that the costs are not very clear to me” (S6). The difficulty became apparent when patients, residents, students, or colleagues asked questions about the price of health care services, departmental health care costs, and coverage. Answering these questions remained difficult, and attempts to unravel financial pathways in their workplace were often unsuccessful. In an attempt to deal with this obstacle, some physicians registered for external courses to help them gain insight in general health care economics, the factors contributing to the composition of price for health care services, and the effects of health care system coverage on health care costs.

How the attending physician prepares residents to deliver HV3C in the workplace

Physicians spoke about aiming to integrate HV3C in residency training in the workplace. Their examples of preparing residents for HV3C delivery can be classified into 3 main teaching methods: Socratic questioning, role modeling, and setting limits.

Socratic questioning.

To ensure that residents pay attention to HV3C, attending physicians asked questions about HV3C while supervising. The questions often focused on the cost of health care: “Do you have any idea of the price of this extra test?” (S2). Or they drew attention to the necessity of a particular service: “Will [the result of] this extra test change our treatment plan?” (S9). Attending physicians aimed to raise residents’ awareness through the use of open-ended questions, without trying to give information on what care would be best or by discussing their personal opinion of HV3C.

A frequently stated training strategy attending physicians used was to deduce with the resident why the resident made a particular treatment decision. These deductions were usually triggered by a case that did not meet the physician’s standards; sometimes the residents proposed a case themselves. In some instances, attending physicians recognized a pattern in residents, such as “defensive medicine,” “conflict-avoidant behavior,” or “excessive testing.” An example of conflict-avoidant behavior given by an attending physician described a resident who always conceded to a patient’s demands, even when the resident had concluded that the requested care was unnecessary or inappropriate. Together, the physician and resident discussed the more global topic of typical behavioral patterns and professional identity: “Is that your style? Why is that? Why do you do that? How do you try to [say no]?” (S9). Other attending physicians discussed professional identity by asking questions such as “Why did you give in to the patient’s wishes? As a physician, is it hard for you to say no?”(S8). These training moments focused on the resident’s attitude toward HV3C, expressed as “What kind of doctor do you want to be?” Data analysis revealed that not all attending physicians asked reflective questions regarding behavioral patterns and professional identity.

Role modeling.

Attending physicians were well aware of their impact on the residents’ delivery of care and aimed to be role models for HV3C delivery in their daily practice. The participants tried to model communication skills, including persuasion and negotiation, which were seen as essential to HV3C delivery. Sometimes residents asked for support if they felt they needed help to convince patients of the inappropriateness of a request but failed to do so alone. On such occasions, the attending physician aimed to demonstrate HV3C by explaining to the patient, in the presence of the resident, why specific diagnostic procedures or checks were not suitable and what the better alternatives were. In so doing, the attending physician aimed to transfer his or her communication skills to the residents. None of the participants explicitly discussed their role modeling with the residents; they assumed that the residents would recognize the teaching moment and its relation to HV3C.

“In terms of communication, I really hope that when they see how I explain things and eh, in my manner and body language, that they will get something out of it. I don’t check that [whether they get something out of it]” (S12).

Role modeling also included leading by example within the department, such as placing restrictions on additional testing, exploring alternatives, and being critical of colleagues’ behavior during handovers, multidisciplinary team meetings, or case discussions.

Setting limits.

Attending physicians characterized junior residents as potential deliverers of wasteful care and in need of guidance by more senior physicians. Some participants provided guidance by closely monitoring care delivery (including additional testing) and limiting resident autonomy, for example, in prescribing specific drugs. As one said: “They’re not in a position to prescribe those expensive drugs on their own. So that’s where they’re often called to a halt” (S12). Some workplaces preselected a list of preferred drugs (a formulary) to set limits on residents’ prescribing patterns; residents were not allowed to prescribe drugs that were not on the list. Attending physicians used the patient perspective (i.e., intervention is harmful) more often than the economic perspective (i.e., intervention is costly) to justify the limits of a particular HV3C treatment to the resident and to advocate the importance of HV3C. For example, extensive laboratory testing was more easily accepted if attending physicians felt it would do the patient no harm. As residents became more senior, these limits would be removed because the residents were deemed capable of dealing with the total spectrum of services.

Discussion

Educational interventions aimed at reducing health care waste by delivering HV3C often focus on the role of the physician. This study sought to understand how attending physicians, who have a central role in the workplace, prepare residents to provide HV3C. Analyzing our data made it apparent that how an attending physician prepared residents was determined by the physician’s own approach to HV3C: the priority he or she gave to HV3C training, which teaching methods he or she used, and which obstacles to HV3C delivery were perceived.

Results indicated that the attending physician’s approach to HV3C shaped how they prepare residents to deliver HV3C. Hence, the priority the participants gave to HV3C training varied. All participants underlined the importance of HV3C to sustainable health care and recognized their personal role in preparing future physicians who are able to balance the costs, risks, and benefits of medical services for individual patients and society at large. This finding is important in the light of how one’s beliefs, personal values, and experiences are known to strongly influence how one teaches.17,36–38 Additionally, it is known that when the training environment, with attending physicians as key figures, pays specific attention to HV3C and has a shared goal in HV3C training, educational interventions are more effective.14,39 Therefore, in educational interventions, it might be wise to consider paying attention to the attending physicians’ beliefs about HV3C and how the training environment is involved in HV3C delivery.

Participants’ personal beliefs about HV3C influenced how they prepared residents to deliver HV3C in the workplace. In their examples, Socratic questioning, role modeling, and setting limits were apparent as teaching methods for HV3C. These teaching methods are suitable for the situations residents encounter in daily health care practice and can be labeled natural teaching moments. Nevertheless, the variety of teaching methods used to prepare residents to deliver HV3C is limited compared with known teaching methods employed for other topics. For example, scaffolding (supporting residents in the delivery of HV3C, tailored to the residents’ needs with gradual reduction in the level of support), reflection (deliberately stimulating residents to think about their weaknesses and strengths in relation to the delivery of HV3C), or coaching (observing residents and providing concrete feedback on their performance) could be interesting teaching methods. Yet these methods were not explicitly mentioned by participants.40 Although participants used the reported teaching methods deliberately, the intended lessons were often implicit and not verbally linked to HV3C. This finding resonates with previous research that revealed that residents struggle to identify and learn HV3C in the workplace.16 Residents indicated needing the support of attending physicians to identify HV3C lessons and build competence.16 Improving HV3C training during residency seems therefore to depend on equipping attending physicians with a broader array of clinical teaching skills so that they can more effectively emphasize HV3C standards and decisions during supervision. From this perspective, the field of medical education has a role in providing attending physicians with practical examples of teaching methods that can support the specific learning of HV3C in the workplace.

Although attending physicians are key figures during residency16,18 as role models and in creating a supportive environment for HV3C delivery,14–17,39 participants in our study voiced how they experienced obstacles to both delivering and demonstrating HV3C. In particular, 2 obstacles were mentioned: their reluctance to discuss HV3C with patients and their lack of knowledge of health care economics in general. The increased awareness regarding the need for sustainable and affordable care is not limited to the medical field7; it also incorporates patients and society and results in an expectation by patients to talk about health care costs with their physicians.41,42 Because it is known that conflicting role-modeling behaviors are present in medical training15 and there is a correlation between practice patterns observed during residency and future care delivery,20,21 dealing with these obstacles is crucial for training and future health care. It is not surprising that today’s attending physicians find it difficult to demonstrate and teach HV3C delivery, considering that they themselves were not trained in these competencies13,16 and curricula focusing on stewardship, cost-consciousness, and patient-centered care are only a recent addition to medical education.11 The absence of any mention of health care costs in the consultation room has been reported previously.17,43,44 Although this may be a delicate topic, it does provide a learning opportunity if residents can observe these conversations.45 Communication skills based on empathic conversation have been proposed to discuss health care costs with patients, yet they are rarely used.44,46 Recent research suggested supporting residents with access to data,17 yet this study demonstrates that not only residents but also attending physicians can be supported with feedback tools, such as benchmark data, referral data, costs of care, and patient satisfaction scores in their specialty. Attending physicians spoke positively about how this information helped them evaluate their own HV3C practice and their ability to train residents in such behavior. Structural use of such data could benefit both attending physicians and residents because reflective practice is known to be an important element of HV3C training14 and influences physician behavior.47 Feedback data also can help improve attending physicians’ understanding of health care economics. Nevertheless, it is advisable that data are not only accessible but also openly discussed in a tolerant environment17,48 to prevent a judgmental setting.

Our results indicate that teaching HV3C is now an individual goal, based on personal beliefs and teaching methods that are considered helpful, with little or no feedback on care delivery or teaching. Therefore, we suggest that interventions focus not only on developing the teaching skills of attending physicians but also on developing an HV3C culture that both teaching staff and residents support. Our findings may prompt department chiefs and deans to review the environment and culture of their departments in light of teaching HV3C. Providing tools for reflective practice and knowledge transmission, together with a shared departmental goal in teaching and delivering HV3C, might be worth considering.

The study has limitations. Because of the attention currently given to HV3C, our participants might have given socially desirable answers. Our results also might have overestimated participants’ attitudes toward HV3C. We tried to address this potential bias by asking participants to give specific examples to substantiate the attention given to HV3C training in the clinical setting. For future research, it might be valuable to observe physician behavior in the workplace to enrich our understanding of how they train residents in HV3C and why ongoing training is challenging in this context. Our research drew from a deliberately chosen group of attending physicians who had been identified by their program directors, and this selection method might have caused unintended sampling bias. Our research aimed to describe how attending physicians prepare residents for the delivery of HV3C in general. Future research might explore the relationship between training approaches and physicians’ specialty because known influencing factors such as patient–physician relationship, scale of institution, and ability to gain insight in practice patterns often differ between specialties.16 Our sample (N = 12) also contained few female participants (n = 3). We do not know if overrepresentation of male participants influenced our findings; further exploration might be valuable.

Conclusions

How attending physicians deal with HV3C themselves influences how they prepare residents in the workplace. To optimize HV3C, it may be important to create an environment that supports attending physicians in HV3C delivery and training by making HV3C a shared goal, providing insight into practice behavior, and minimizing the influence of obstacles. Additionally, attending physicians could use support in the variety of teaching methods available to incorporate HV3C in daily teaching to stimulate continuous learning.

Acknowledgments:

The authors thank the 6 program directors and 12 attending physicians for their time and input in this study.

References

1. Hood VL, Weinberger SE. High value, cost-conscious care: An international imperative. Eur J Intern Med. 2012;23:495–498.
2. Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA. 2012;307:1513–1516.
3. Weinberger SE. Providing high-value, cost-conscious care: A critical seventh general competency for physicians. Ann Intern Med. 2011;155:386–388.
4. Lucey CR. Medical education: Part of the problem and part of the solution. JAMA Intern Med. 2013;173:1639–1643.
5. Sklar DP. How medical education can add value to the health care delivery system. Acad Med. 2016;91:445–447.
6. Owens DK, Qaseem A, Chou R, Shekelle P; Clinical Guidelines Committee of the American College of Physicians. High-value, cost-conscious health care: Concepts for clinicians to evaluate the benefits, harms, and costs of medical interventions. Ann Intern Med. 2011;154:174–180.
7. Choosing Wisely. http://www.choosingwisely.org. Published 2012. Accessed April 26, 2019.
8. Brody H. Medicine’s ethical responsibility for health care reform—The Top Five list. N Engl J Med. 2010;362:283–285.
9. Williams D, Clancy C, Dine J; American College of Physicians. Newly Revised: Curriculum for Educators and Residents (Version 4.0). https://www.acponline.org/clinical-information/high-value-care/medical-educators-resources/newly-revised-curriculum-for-educators-and-residents-version-40. Published 2012. Accessed March 5, 2018.
10. Havyer RD, Norby SM, Leep Hunderfund AN, et al. Science of health care delivery milestones for undergraduate medical education. BMC Med Educ. 2017;17:145.
11. Frank JR, Snell L, Sherbino J. CanMEDS 2015 Physician Competency Framework. 2015.Ottawa, ON, Canada: Royal College of Physicians and Surgeons of Canada.
12. Cooke M. Cost consciousness in patient care—What is medical education’s responsibility? N Engl J Med. 2010;362:1253–1255.
13. Wong BM, Holmboe ES. Transforming the academic faculty perspective in graduate medical education to better align educational and clinical outcomes. Acad Med. 2016;91:473–479.
14. Stammen LA, Stalmeijer RE, Paternotte E, et al. Training physicians to provide high-value, cost-conscious care: A systematic review. JAMA. 2015;314:2384–2400.
15. Leep Hunderfund AN, Dyrbye LN, Starr SR, et al. Role modeling and regional health care intensity: U.S. medical student attitudes toward and experiences with cost-conscious care. Acad Med. 2017;92:694–702.
16. Stammen L, Slootweg I, Stalmeijer R, et al. The struggle is real: How residents learn to provide high-value, cost-conscious care. Teach Learn Med. 2019;31:402–411.
17. Gupta R, Steers N, Moriates C, Wali S, Braddock CH 3rd, Ong M. High-value care culture among the future physician workforce in internal medicine. Acad Med. 2019;94:1347–1354.
18. Eraut M, Alderton J, Cole G, Senker P. Coffield F. The development of knowledge and skills at work. In: Differing Visions of a Learning Society: Research Findings. 2000:Vol 1. Bristol, England: Policy Press; 231–262.
19. Eraut M. Informal learning in the workplace. Stud Cont Education. 2004;26:247–273.
20. Asch DA, Nicholson S, Srinivas S, Herrin J, Epstein AJ. Evaluating obstetrical residency programs using patient outcomes. JAMA. 2009;302:1277–1283.
21. Chen C, Petterson S, Phillips R, Bazemore A, Mullan F. Spending patterns in region of residency training and subsequent expenditures for care provided by practicing physicians for Medicare beneficiaries. JAMA. 2014;312:2385–2393.
22. Teunissen PW, Wilkinson TJ. Dornan T, Mann K, Scherpbier A, Spencer J. Learning and teaching in workplaces. In: Medical Education: Theory and Practice. 2011.London, England: Churchill Livingstone Elsevier.
23. Savin-Baden M, Howell Major C. Savin-Baden M, Howell Major C. Interviews. In: Qualitative Research: The Essential Guide to Theory and Practice. 2013:New York, NY: Routledge; 357–373.
24. Snijders M; ZonMw, high-value, cost-conscious care research (Onderzoek naar doelmatigheid). The Netherlands Organisation for Health Research and Development. https://www.zonmw.nl/nl/onderzoek-resultaten/doelmatigheidsonderzoek. Published 2019. Accessed May 2, 2019.
25. High-Value, Cost-Conscious Care Symposium, April 18, 2018, CORPUS Conference Centre, Leiden, Netherlands. https://www.bewustzijnsproject.nl/agenda-item/symposium2018. Accessed March 10, 2018.
26. Noben C; Training Region, Southeast Netherlands (OORZON). High-value, cost-conscious care. (Doelmatigheid van zorg.) https://www.oorzon.nl. Published 2019. Accessed May 2, 2019.
27. Noben C, Habsati A, Brabant DB; OORZON. High-value, cost-conscious care. (Doelmatigheid van zorg.) www.oorzon.nl/doelmatigheid. 2018. Accessed December 12, 2018.
28. Glaser BG, Strauss AL, Strutzel E. The discovery of grounded theory: Strategies for qualitative research. Nurs Res. 1968;17:364.
29. Charmaz K. Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. 2006.London, England: Sage Publications Ltd..
30. Corbin J, Strauss A. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. 2014.Thousand Oaks, CA: Sage Publications, Inc..
31. Morse JM. The significance of saturation. Qual Health Res. 1995;5:147–148.
32. Frambach JM, van der Vleuten CP, Durning SJ. AM last page. Quality criteria in qualitative and quantitative research. Acad Med. 2013;88:552.
33. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19:349–357.
34. McMillan W. Cleland J, Durning SJ. Theory in healthcare education research: The importance of worldview. In: Researching Medical Education. 2015:Chichester, England: John Wiley & Sons; 15–24.
35. Markee N. Emic and etic in qualitative research. In: The Encyclopedia of Applied Linguistics. 2013.Chichester, England: Wiley-Blackwell.
36. Carter K, Doyle W. Preconceptions in learning to teach. Educ Forum. 1995;59:186–195.
37. Levitt KE. An analysis of elementary teachers’ beliefs regarding the teaching and learning of science. Sci Educ. 2002;86:1–22.
38. Richardson V. Sikula JP. The role of attitudes and beliefs in learning to teach. In: Handbook of Research on Teacher Education: A Project of the Association of Teacher Educators. 1996:2nd ed. New York, NY: MacMillan; 102–119.
39. Sedrak MS, Patel MS, Ziemba JB, et al. Residents’ self-report on why they order perceived unnecessary inpatient laboratory tests. J Hosp Med. 2016;11:869–872.
40. Stalmeijer RE. When I say … cognitive apprenticeship. Med Educ. 2015;49:355–356.
41. Danis M, Sommers R, Logan J, et al. Exploring public attitudes towards approaches to discussing costs in the clinical encounter. J Gen Intern Med. 2014;29:223–229.
42. Bullock AJ, Hofstatter EW, Yushak ML, Buss MK. Understanding patients’ attitudes toward communication about the cost of cancer care. J Oncol Pract. 2012;8:e50–e58.
43. King BC, DiPace J, Naifeh M, Hammad H, Gerber LM, Abramson E. Pediatric training faculty and resident perceptions on teaching high-value, cost-conscious care: A multi-institutional study. Hosp Pediatr. 2017;7:547–552.
44. Alexander GC, Casalino LP, Meltzer DO. Patient-physician communication about out-of-pocket costs. JAMA. 2003;290:953–958.
45. Tilburt JC, Wynia MK, Montori VM, et al. Shared decision-making as a cost-containment strategy: US physician reactions from a cross-sectional survey. BMJ Open. 2014;4:e004027.
46. Hardee JT, Platt FW, Kasper IK. Discussing health care costs with patients: An opportunity for empathic communication. J Gen Intern Med. 2005;20:666–669.
47. Mazmanian PE, Davis DA. Continuing medical education and the physician as a learner: Guide to the evidence. JAMA. 2002;288:1057–1060.
48. von Ferber L, Bausch J, Köster I, Schubert I, Ihle P. Pharmacotherapeutic circles. Results of an 18-month peer-review prescribing-improvement programme for general practitioners. Pharmacoeconomics. 1999;16:273–283.

Supplemental Digital Content

Copyright © 2019 The Author(s). Published by Wolters Kluwer Health, Inc. on behalf of the Association of American Medical Colleges.