Secondary Logo

Journal Logo

Situating Remediation: Accommodating Success and Failure in Medical Education Systems

Ellaway, Rachel, H., PhD; Chou, Calvin, L., MD, PhD; Kalet, Adina, L., MD, MPH

doi: 10.1097/ACM.0000000000001855
Perspectives
Free

There has been a widespread shift to competency-based medical education (CBME) in the United States and Canada. Much of the CBME discourse has focused on the successful learner, with relatively little attention paid to what happens in CBME systems when learners stumble or fail. Emerging issues, such as the well-documented problem of “failure to fail” and concerns about litigious learners, have highlighted a need for well-defined and integrated frameworks to support and guide strategic approaches to the remediation of struggling medical learners.

This Perspective sets out a conceptual review of current practices and an argument for a holistic approach to remediation in the context of their parent medical education systems. The authors propose parameters for integrating remediation into CBME and describe a model based on five zones of practice along with the rules of engagement associated with each zone. The zones are “normal” curriculum, corrective action, remediation, probation, and exclusion.

The authors argue that, by linking and integrating theory and practice in remediation with CBME, a more integrated systems-level response to differing degrees of learner difficulty and failure can be developed. The proposed model demonstrates how educational practice in different zones is based on different rules, roles, responsibilities, and thresholds for moving between zones. A model such as this can help medical educators and medical education leaders take a more integrated approach to learners’ failures as well as their successes by being more explicit about the rules of engagement that apply in different circumstances across the competency continuum.

R.H. Ellaway is professor, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada; ORCID: http://orcid.org/0000-0002-3759-6624.

C.L. Chou is professor, Department of Clinical Medicine, University of California, San Francisco, and staff physician, San Francisco VA Health Care System, San Francisco, California; ORCID: http://orcid.org/0000-0002-2391-4337.

A.L. Kalet is professor, Division of General Internal Medicine and Clinical Innovation, Departments of Medicine and Surgery, New York University, New York, New York; ORCID: http://orcid.org/0000-0003-4855-0223.

Funding/Support: None reported.

Other disclosures: C. Chou and A. Kalet edited the book Remediation in Medical Education (Springer, 2014). Neither receives royalties from sales of the book.

Ethical approval: Reported as not applicable.

Previous presentations: This article was developed from discussions arising from an Emerging Issues session that the authors led at the 2014 AAMC Medical Education Meeting in Chicago, Illinois, November 6–7, 2014. The material presented here has not been previously presented or published.

Correspondence should be addressed to Rachel H. Ellaway, Department of Community Health Sciences, Cumming School of Medicine, TRW Building, Foothills Campus, 3280 Hospital Dr. NW, Calgary, Alberta, T2N 4Z6, Canada; telephone: (403) 220-6076; e-mail: rachel.ellaway@ucalgary.ca.

Written work prepared by employees of the Federal Government as part of their official duties is, under the U.S. Copyright Act, a “work of the United States Government” for which copyright protection under Title 17 of the United States Code is not available. As such, copyright does not extend to the contributions of employees of the Federal Government.

There is growing attention being paid to the design and operations of medical education programs from a systems perspective. In the United States and Canada, this is reflected in the widespread shift to competency-based medical education (CBME), a system of methods and principles around which medical education programs can be constructed.1 The adoption of CBME has focused on conducting more and higher-quality assessments, which has highlighted the need for integrated strategies to remediate struggling medical learners.2 However, remediation practices appear to take place with little attention to broader consequences of the changes to medical education practice associated with CBME.

In this Perspective, we propose parameters for integrating remediation into CBME programs. We describe a model of zones of remediation, whose thresholds are defined by expected milestones of acceptable performance, and the potential rules of engagement for each zone. We present a manifesto for a systems-level approach to integrating emerging models of CBME and remediation. In doing so, we situate remediation as a critical connection between CBME and medical education’s social contract, based on the principle that moral and effective medical education systems should encompass medical education’s obligations to society, resist the argument that learners are entitled to graduate simply on the basis of their personal investment in training, and provide honorable and compassionate exit strategies for learners who are unable to complete their training. A glossary of terms used in this article is provided in Appendix 1.

Back to Top | Article Outline

Remediation in Medical Education

Remediation in medical education has been defined as “the act of facilitating a correction for trainees who started out on the journey toward becoming a physician but have moved off course.”3(p.xvii) Medical learners underperform for a wide range of reasons,4 which can prove challenging for all concerned. Remediation of struggling learners takes different forms according to the particular problems involved, as well as the learners, the faculty who act as their remediators, and the learning contexts in which this takes place. Generally, remediation involves (1) identifying the need or deficit to be addressed; (2) framing it in terms of required learning or performance goals; (3) developing and executing a series of defined and officially sanctioned episodes of additional training and monitoring; and (4) concluding with an assessment of whether the learner has met the predetermined remediation goals.1,5,6

Remediation is (or should be) a focused, time-limited, and highly structured series of episodes within which specific performance deficits must be addressed to the satisfaction of one or more supervisors. Remediation as an undertaking is therefore both reactive and adaptive; it is invoked only when individual learners are at risk of failing, with each instance focusing on the particular issues with which the learner is struggling. Remediation is also a liminal undertaking. It sits outside “mainstream” medical education because a learner’s trajectory may be suspended while the learner is being remediated, or the remediation activity may continue in parallel with the learner’s “normal” studies. Either way, remediation is not the same as regular training: It is more specific, intense, and focused, and the stakes are higher than usual. The learner must unambiguously and reliably demonstrate the required performance improvements to avoid further sanctions. To successfully complete remediation is to be rehabilitated back into “normal” training; to fail is to face probation or exclusion from the program.

Most learners undergo little or no remediation as they progress through their training, but those who do require a disproportionate amount of faculty and administrative time and resources.7 Indeed, a common complaint amongst medical education program leaders is that they spend much of their time helping relatively small numbers of struggling learners. Despite this perception, remediation has been somewhat neglected in the medical education literature to date.8 Existing work has tended to focus on what should happen within an episode of remediation, rather than on how remediation should or can interdigitate with the rest of medical education.1,9

Clearly, there are intersections between remediation and the rest of medical education. We therefore need to situate remediation in the broader context of medical education systems, not just to guide remediation practices but also to address issues of constructive alignment, the hidden curricula of remediation, the transparency and consistency of remediation rules and processes, and the impact of remediation on medical education as a whole.

As educators, we see the costs of failure for learners (time, fees, reputation, stress), their teachers (time, stress), and their programs (coverage, survivor guilt). The difficulties associated with failing learners have led to a chronic “failure to fail” phenomenon,10 where substandard learners are allowed to progress through their training and even into practice because the effort not to allow them to do so is greater than most medical education systems are able to muster.10,11 The failure of the medical education system in this regard becomes a health care system problem. Although “failure to fail” is a worldwide and multidisciplinary problem,12 it seems particularly relevant to the more individualist cultures of Canada and the United States, which have low rates of attrition from medical training. The international rate of attrition for undergraduate medical education programs has been estimated at around 11%,13 but the rate is far lower in the United States (3.4%)14 and an order of magnitude lower still in Canada (0.4%).15 The rapidly increasing cost of medical education to learners in the United States and Canada also seems to have encouraged a sense of entitlement to complete training, exacerbated by institutional concerns regarding the legal implications of dismissing a learner, the reality of heavy financial debt, and a lack of satisfying and reasonable alternatives to the medical profession for individuals who are unable to complete their training.

Back to Top | Article Outline

A Zone-Based Model of Remediation

We can describe normative medical education practice in terms of two intertwined subsystems: one focused on success and completion, the other focused on failure and exclusion. Each has different rules and practices, and each has its own literature and evidence base.9,16–18 We can consider remediation as a bridge between these two subsystems: It functions as both an interface and a distinct rule-bound subsystem in its own right. Key differences between the three subsystems are set out in Table 1.

Table 1

Table 1

We propose reframing these differences in terms of five zones of practice. Zones 1 and 2 reflect the success subsystem. In Zone 1, the learner is performing at or above expected levels, and teaching is focused on supporting the learner’s continued progression toward independent practice. In Zone 2, the learner is performing below expected levels (although not egregiously so), and teaching is intrinsically corrective to enable the learner to return to Zone 1. In Zone 3, the remediation subsystem, the learner is performing below an acceptable minimal standard and is undergoing active remediation; the learner must exit to either the success or failure subsystem. Finally, the failure subsystem is divided into Zones 4 and 5. In Zone 4, the learner has consistently been performing below an acceptable standard and is either suspended, placed on probation, or required to retake a component of the program. In Zone 5, the learner is excluded from the program. Compared with the rules in Zones 1–3, the rules in Zone 4 tend to be more austere, because the learner explicitly participates less in regular programmatic activities and there is a very real possibility of exclusion. Zone 4, like Zone 3, is temporary and must resolve up or down. The rules in Zone 5 are about how to exclude a learner from a program—this may be a brief phase if the exclusion is unchallenged, but it is often drawn out if appeals are involved. These zones along with exemplar learner trajectories are illustrated in Figure 1 and in the vignette in Box 1.

Figure 1

Figure 1

Although a model based on horizontal zone thresholds (as set out in Figure 1) will suffice for relatively short educational episodes, professional training clearly requires a progression toward practice with expected levels of performance rising over time. This reflects one of the core tenets of CBME,16,19 where a learner’s performance is assessed by comparing it with that expected of a generic learner at the same stage of training.20 Extending this principle to our proposed model, we can remap the thresholds between zones as rising over time (Figure 2). A key consequence of this model is that a learner’s performance need not fall in absolute terms for the learner to begin to struggle or fail: Nonprogression within expected parameters can lead to remediation or even exclusion.

Figure 2

Figure 2

We have presented our thesis in terms of an abstract model to demonstrate the principles it embodies. In practice, thresholds between zones would be marked by exams, assignments, clinical skills milestones, or equivalent reference points of expected performance. As such, definitions of the zones would need to be more detailed, the relative sizes of zones may differ, and the thresholds are unlikely to be as linear or parallel as we have presented them. Assessment data, even for the most objective of competencies, are unlikely to ever reach absolute levels of measurement quality; in the case of less tangible aspects of competence, such as professionalism, assessment data are nearly always open to interpretation. Therefore, context will always matter, and clinically experienced experts will continue to need to make informed judgments about which learners should progress, remediate, or fail.

Implementing a zone-based model of a medical education system would involve (1) deciding what dimensions of practice to map (such as CanMEDS19 or Accreditation Council for Graduate Medical Education21 competencies); (2) selecting the threshold events (assignments, exams, observed practice, etc.) to use; (3) ensuring that performance measures can meaningfully map to an integrated model of different levels of performance; (4) anchoring performance thresholds as precisely as possible to differentiate between expected, acceptable, and unacceptable performance; (5) mapping these performance thresholds in terms of what constitutes borderline performance at each stage; and (6) defining the rules to apply in terms of faculty and learner roles and responsibilities when thresholds are approached (formative) or crossed (summative). We can consider the aggregate rules, roles, responsibilities, and thresholds for each zone as its “schema.”

Back to Top | Article Outline

Implications

We have described a systems perspective on remediation that considers five zones of individual learner success and failure in contemporary medical education, and when and how learners move between these zones. Although these zones have different rules of engagement and different standards when crossing their borders, we have argued for a unified model for learner progression in medical education programs. Considering the system as a whole in this way allows for each zone to be adjusted relative to the others and that the zones can all align with the tenets of CBME (which has tended to focus almost exclusively on Zones 1 and 2). In doing so, we seek to shift the CBME discourse from one based on variable kinds of success to one that also acknowledges and responds to failure. This approach flags the importance of supporting learners who are unable, for whatever reason, to flourish in a medical training program, and it speaks to our social contract in terms of seeking to better manage our limited educational resources.

One key implication of our model is that, even in the most idealized competency-based program where time is all but ignored as a factor in learner progression, a learner can still eventually fail and be dismissed. Using specific milestones to define thresholds between all zones would make explicit the stage-specific requirements for all learners, something that we would argue is lacking in existing approaches to CBME implementation. By acknowledging intrinsic practical limits to the effort schools can invest in getting all of their learners through medical training, this model provides a systematic response to managing learner underperformance in the context of CBME.

This approach also allows us to consider more holistic questions regarding medical education systems. For instance, given the asymmetrical intensity of faculty attention required by remediation compared with “normal” learning, most learners (i.e., those who are not remediated) receive less attention than the few learners who underperform. Should we focus more evenly on opportunities for every learner rather than on the acute needs of the less able few? Should we perhaps look to the admissions process to fix the problem of failing learners by selecting more academically or socially resilient learners? Or is it that our learners need better or different forms of nonacademic (pastoral) support or more adaptive and personalized teaching to reduce or reverse underperformance? These questions cannot be resolved purely on psychometric or equity grounds; their answers must also reflect what the medical education system can afford given its limited resources and increasing levels of accountability.

In advancing this argument, we should differentiate between remedial action and remediation. We define remedial action as largely supportive, informal, and short-term events in Zone 2 in which a preceptor facilitates a learner’s progression toward professional mastery and independent practice. Remediation (Zone 3), on the other hand, is a formal response to sustained underperformance, with a different schema (its collected rules, roles, responsibilities, and thresholds) than the ones for the mainstream curriculum (Zones 1 and 2). For example, a learner who has a recurring problem mastering a particular procedural technique can benefit from formative remedial action to identify and “fix” the issues she has in achieving mastery. If the same student consistently underperforms across a broader range of issues (procedural or otherwise) that cannot be (or at least are not being) resolved, she is likely to enter a period of formal remediation, which may have serious consequences if not successfully completed. While both remedial action and remediation are course corrections, they differ in terms of the depth and duration of underperformance that triggers the event and the rules of engagement involved—in particular, the consequences of failure.

Back to Top | Article Outline

Box 1A Practical Example of a Learner’s Trajectory Through Different Zones: Mo’s Story

Mo is an internal medicine trainee who intermittently struggles to meet the expected levels of performance set out in the program milestones. Sometimes her performance is acceptable and she receives minimal guidance (Zone 1); at other times, she needs more intensive help with diagnostic accuracy (Zone 2). However, she has been having increasing problems both with the quality of her diagnostic decisions and in her interactions with her interprofessional colleagues. In consultation with the residency program’s competency committee, the program director decides that Mo needs remediation in both clinical reasoning and professionalism (Zone 3—see Figure 1).

Two preceptors are assigned to Mo to remediate these issues with her. The preceptor assigned to work with her on clinical reasoning is one of her existing preceptors, who is a highly respected clinician. He has Mo observe him and engage in extensive discussion of patient cases. While others notice Mo’s improved clinical reasoning skills, she becomes consistently argumentative, defensive, and irritable with this remediating preceptor because of the formality of remediation; as a result, she is cited for professionalism issues (wrong schema).

The preceptor assigned to work with Mo on professionalism issues, on the other hand, is very explicit in his expectations for engagement in remediation. As part of providing a supportive, learner-centered coaching relationship, this preceptor negotiates a remediation plan with Mo involving written assignments and exercises to practice increasingly complex communication challenges (right schema). Mo successfully completes this side of her remediation.

However, because Mo failed one of the two remediation topics, she is placed on probation with significant scrutiny of her performance (Zone 4) and with an explicit expectation that she will make substantial improvements or face exclusion from the program (Zone 5). Finally realizing the seriousness of the situation, Mo addresses her performance problems, demonstrates significant improvement in her interprofessional communication in her daily interactions, and is then reinstated to normal participation within the program (Zones 1 and 2).

This difference is important because well-meaning remediators will too often keep trying to help a learner well beyond the point where improvement is likely to occur. This, in turn, contributes to the “failure to fail” phenomenon.10 For instance, individuals who are unlikely to thrive as medical professionals may have fewer and fewer options and more financial debt the longer they languish unsuccessfully in medical training. Adopting a zone-based model would more clearly set out the rules of engagement and expectations for all concerned (what the different schemas are and when they apply) and could thereby be used to reduce the punitive aspect of responses to failure.

In proposing this model, we argue that remediation should be explicitly structured as part of medical education systems, and not as an afterthought or an “outsider” activity as too often seems to be the case. Moreover, remediation should be a shared responsibility for a community of educators rather than being left to a select few.22 When engaging with learners in the remediation zone, faculty typically need to take more time, expend more effort, and possess more advanced skills than when they teach in the routine curriculum. In addition, learners may have emotional issues, not least because remediation is conducted in that liminal space where dismissal from the program is a very real possibility. Institutions should therefore provide additional remediation resources, such as professionalism assessments that require scoring and interpretation by consultants, and standardized patients and other learning specialists able to give expert feedback.

Our model of zones is a systems-level response to integrating remediation with the rest of medical education, and as such it can be linked to a number of more tactical solutions. One example, proposed for medical learners with “moderate” professionalism lapses (e.g., see the trajectory of Learner B in Figures 1 and 2), involves a council that hears the learner’s perspective and provides a guided reflection opportunity for that learner.23 Another technique is to involve learners in designing and implementing their own remedial interventions—a low-cost model that can promote autonomy and self-regulated learning.24 A more integrated approach may help to correct learners before full remediation is needed, which may in turn reduce stress and make better use of faculty resources.18,25,26

In advancing this model, we should also acknowledge that different problems in different competencies present different challenges in terms of both seriousness and remediability. For instance, professionalism problems tend to be more serious and less tractable than lapses or gaps in medical knowledge. Learners may also be failing in a number of areas but to different degrees. Indeed, rarely is a remediation plan focused narrowly on improving performance in a single competency. Learners requiring remediation often present several simultaneous struggles involving personal issues, problems with institutional cultures, extraprogram challenges (e.g., mental health), learning-related issues (e.g., learning disabilities), and/or professionalism-related issues. Because remediation requires attention to the unique needs of individual struggling learners, translating our model into practice will clearly need to accommodate the particular circumstances of different educational programs.

As medical educators, we should try to incorporate the same humanistic approach in remediation situations that we ideally bring to patient care. Fully understanding the multiple facets that bring a learner to remediation requires patience, active listening, and reflection. Yet, we cannot allow our acceptance and compassion to cloud our ultimate judgment when a learner enters the failure subsystem. A clinical analogy could be the consideration of and transition to palliative care. Initially, there are strong efforts to provide interventions that could bring learners to an acceptable level of academic function. At times, actively managed learners require repeated interventions, or remediation, over time. Those interventions may result in a learner’s return to the success subsystem. However, with insufficient return on those investments, or with an acute decompensation in function, a decision to transition to a palliative approach (i.e., providing a dignified exit rather than fighting over whether the learner can remain in the program) may become necessary. Greater clarity regarding what zone we are working in with any given learner (and which schema should therefore be applied) should help to manage and align expectations, options, and actions, and thereby support a more compassionate stance in medical education as a whole.

We acknowledge a number of limitations in this article. First, we developed this model inductively from our own experiences and deductively from general principles of remediation and CBME, but we have not validated its use in practice, nor have we defined specific thresholds for the zones; these are beyond the scope of the current article. Further work is needed to explore how the model can inform the work of different stakeholders, such as program directors and competency committees. Second, we have concentrated on those medical education systems that prepare future physicians, but we acknowledge that there are likely to be applications in other medical education systems, such as continuing medical education. Although the zones, their schemas, and the rules for passing between zones may differ, the general principles would still seem to apply. However, future work must validate this assertion. Third, we did not focus on “best practices” in specific episodes of remediation, nor did we consider the specific assessment practices that identify whether learners may need to be remediated; this article is intrinsically strategic and system-wide in scope. Subsequent studies will be required to address these issues. Finally, we have presented an ideal model without factoring in issues such as difficulties in acquiring performance data, data gaps, and other process challenges.27 We acknowledge that even the most carefully planned system will not function optimally, and that systems resilience and sustainability will also need to be considered in future work.

Although we present a simple model for these five zones, the schemas that define them, and possible phenotypes and educational responses for the learners who traverse them, we acknowledge that reality is more complex and that the model is perforce abstract and idealized. Most learners will likely take an uneven path in developing different competencies. In building on concepts of CBME, we inherit their common challenge: that measurements of competencies need to be practical and fit for purpose. Ultimately, medical school leaders must take responsibility for making high-stakes decisions in the face of uncertainty and complexity. We hope that, by using this model, they will be better able to do so both systematically and consistently.

Back to Top | Article Outline

Conclusions

The need for individualized remediation for learners who stumble along the way has been a relatively neglected aspect of CBME. By making theories of remediation explicit and integrating them into the emerging practices of CBME, we have sought to clarify systems-level responses to degrees of learner difficulty and failure. Much of the discourse around CBME has emphasized a success-focused approach. In this Perspective, we have sought to expand this thinking to encompass the realities of suboptimal learning outcomes. This is an important development because of the burden struggling learners place on medical education systems, and because these learners deserve to be treated compassionately throughout the remediation process. We hope that this model may provide a framework for further research on developing medical competence, as well as helping to better define the expertise needed to conduct effective remediation, better manage educational resources, and better embody compassion for all our learners.

Acknowledgments: The authors wish to thank Dr. Muriel Bebeau and Dr. Catherine Cervin for their contributions to the initial stages of this work.

Back to Top | Article Outline

References

1. Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: Theory to practice. Med Teach. 2010;32:638–645.
2. Kalet A, Chou C. Remediation in Medical Education. 2014.New York, NY: Springer.
3. Kalet A, Chou C. Kalet A, Chou C. Preface. In: Remediation in Medical Education. 2014:New York, NY: Springer; xvii–xxi.
4. Guerrasio J. Kalet A, Chou C. “She needs to read more”: Helping trainees who struggle with medical knowledge. In: Remediation in Medical Education. 2014:New York, NY: Springer; 39–54.
5. Hauer KE, Teherani A, Irby DM, Kerr KM, O’Sullivan PS. Approaches to medical student remediation after a comprehensive clinical skills examination. Med Educ. 2008;42:104–112.
6. Guerrasio J, Garrity MJ, Aagaard EM. Learner deficits and academic outcomes of medical students, residents, fellows, and attending physicians referred to a remediation program, 2006–2012. Acad Med. 2014;89:352–358.
7. Steinert Y. The “problem” learner: Whose problem is it? AMEE Guide No. 76. Med Teach. 2013;35:e1035–e1045.
8. Hauer KE, Ciccone A, Henzel TR, et al. Remediation of the deficiencies of physicians across the continuum from medical school to practice: A thematic review of the literature. Acad Med. 2009;84:1822–1832.
9. Guerrasio J. Remediation of the Struggling Medical Learner. 2013.Irwin, PA: Association for Hospital Medical Education.
10. Dudek NL, Marks MB, Regehr G. Failure to fail: The perspectives of clinical supervisors. Acad Med. 2005;80(10 suppl):S84–S87.
11. Guerrasio J, Furfari KA, Rosenthal LD, Nogar CL, Wray KW, Aagaard EM. Failure to fail: The institutional perspective. Med Teach. 2014;36:799–803.
12. Luhanga FL, Larocque S, MacEwan L, Gwekwerere YN, Danyluk P. Exploring the issue of failure to fail in professional education programs: A multidisciplinary study. J Univ Teach Learn Pract. 2014;11(2):article 3.
13. O’Neill LD, Wallstedt B, Eika B, Hartvigsen J. Factors associated with dropout in medical education: A literature review. Med Educ. 2011;45:440–454.
14. Association of American Medical Colleges. Graduation rates and attrition factors for U.S. medical school students. AAMC Analysis in Brief. May 2014;14(5). https://www.aamc.org/download/379220/data/may2014aib-graduationratesandattritionfactorsforusmedschools.pdf. Accessed May 17, 2017.
15. Association of Faculties of Medicine of Canada. Canadian Medical Education Statistics 2015. 2015.Volume 37. Ottawa, Ontario, Canada: Association of Faculties of Medicine of Canada.
16. Ten Cate O. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013;5:157–158.
17. Veloski J, Boex JR, Grasberger MJ, Evans A, Wolfson DB. Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Med Teach. 2006;28:117–128.
18. Winston KA, van der Vleuten CP, Scherpbier AJ. Prediction and prevention of failure: An early intervention to assist at-risk medical students. Med Teach. 2014;36:25–31.
19. Frank JR, Snell L, Sherbino J. CanMEDS 2015 Physician Competency Framework. 2015. Ottawa, Ontario, Canada: Royal College of Physicians and Surgeons of Canada; http://canmeds.royalcollege.ca/en/framework. Accessed May 17, 2017.
20. Pusic MV, Boutis K, Hatala R, Cook DA. Learning curves in health professions education. Acad Med. 2015;90:1034–1042.
21. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system—Rationale and benefits. N Engl J Med. 2012;366:1051–1056.
22. Kalet A, Guerrasio J, Chou CL. Twelve tips for developing and maintaining a remediation program in medical education. Med Teach. 2016;38:787–792.
23. Gill AC, Nelson EA, Mian AI, Raphael JL, Rowley DR, Mcguire AL. Responding to moderate breaches in professionalism: An intervention for medical students. Med Teach. 2015;37:136–139.
24. Bierer SB, Dannefer EF, Tetzlaff JE. Time to loosen the apron strings: Cohort-based evaluation of a learner-driven remediation model at one medical school. J Gen Intern Med. 2015;30:1339–1343.
25. Chang A, Boscardin C, Chou CL, Loeser H, Hauer KE. Predicting failing performance on a standardized patient clinical performance examination: The importance of communication and professionalism skills deficits. Acad Med. 2009;84(10 suppl):S101–S104.
26. O’Neill LD, Morcke AM, Eika B. The validity of student tutors’ judgments in early detection of struggling in medical school. A prospective cohort study. Adv Health Sci Educ Theory Pract. 2016;21:1061–1079.
27. McConnell M, Sherbino J, Chan TM. Mind the gap: The prospects of missing data. J Grad Med Educ. 2016;8:708–712.
Appendix 1

Appendix 1

© 2018 by the Association of American Medical Colleges