Share this article on:

The Incorporation of Patient Safety into Board Certification Examinations

Kachalia, Allen MD, JD; Johnson, Julie K. MSPH, PhD; Miller, Stephen MD, MPH; Brennan, Troyen MD, JD, MPH

Patient Safety

In response to growing recognition of the importance of patient safety in the practice of medicine, the American Board of Medical Specialties (ABMS) is encouraging its individual specialty member boards to introduce material on patient safety into certification examinations. Certification and recertification candidates, medical schools, and residency programs will likely need to prepare for the possibility that upcoming examinations will include patient safety. Due to the relative youth of the patient safety movement, some may ask whether the field is sufficiently developed and, if so, ask what would be fair to ask candidates to know. To determine what subject matter could be reliably tested, the ABMS and the Council of Medical Specialty Societies convened a panel of patient safety experts in 2003. Based on the panel’s recommendations, the authors provide a brief summary of the panel’s findings and an overview of current concepts in patient safety. The authors also explore which current concepts of patient safety candidates may reasonably expect to find on upcoming examinations.

Dr. Kachalia is an academic hospitalist, Division of General Medicine, Brigham and Women’s Hospital, Boston, Massachusetts.

Dr. Johnson is assistant professor, Department of Medicine, University of Chicago and Director of Research, American Board of Medical Specialties, Evanston, Illinois.

Dr. Miller is president and CEO, American Board of Medical Specialties, Evanston, Illinois.

Dr. Brennan is a member of the Division of General Medicine, Brigham and Women’s Hospital, Boston, Massachusetts.

Address correspondence to Dr. Kachalia, Division of General Medicine, Brigham and Women’s Hospital, 75 Francis Street, Boston, MA 02115.

The issue of patient safety caught the attention of both the medical profession and the public when the Institute of Medicine (IOM) released its landmark report To Err is Human: Building a Safer System in 1999.1 More than five years later, scientific, organizational, and public interest in reducing error in medicine remains strong.2 As a result, the American Board of Medical Specialties (ABMS) decided in 2003 to encourage its member boards to introduce subject matter specific to patient safety into their member boards’ respective certification examinations.

This decision raises significant implications for educators and trainees because it increases the knowledge and skills that must be imparted to and possessed by potential certification candidates. In addition, questions arise regarding whether the field of patient safety is sufficiently developed so that principles may be reliably tested and, if so, what types of questions could candidates reasonably be expected to answer. To address these questions, the ABMS and Council of Medical Speciality Societies (CMSS) convened an expert panel in 2003. In this article, we provide the findings of that panel and an overview of current concepts in patient safety. Further, we explore what candidates for certification may be expected to know about patient safety.

Back to Top | Article Outline

The Specialty Boards and the ABMS

One indicator frequently used to measure the whether a physician possesses the ability to provide quality care is the attainment of board certification status.3 Today, many physicians seek to obtain board certification, either as evidence of professional qualification, or because their employers or affiliated hospitals require it.4 The importance of certification is perhaps best illustrated by the fact that in 2002, more than 85% of licensed physicians in the United States were certified by at least one board.5 As a result, the influential role that certification can play in ensuring that physicians possess necessary knowledge and skills has received increasing attention.5–7

Collectively, the 24 individual specialty boards are members of the ABMS, which serves as their umbrella organization.8 The ABMS was established in 1933 out of concern for the quality of medical care delivered by doctors and maintains its mission to improve the quality of medical care in the United States.8 The ABMS can make recommendations, but does not mandate the actions that boards must take. Individual medical specialty boards set the standards required to achieve certification status. Nevertheless, the ABMS assists its member boards in developing professional and educational standards for the evaluation and certification of physicians.

Throughout the continuum of medical education, examinations are used during different stages and phases, spanning the career of every physician. Tests include course examinations and the United States Medical Licensing Examination™ “Steps” for medical students, in-training examinations for residents and fellows, continuing education self-assessment examinations for practicing physicians, and the processes of initial certification, recertification, and Maintenance of Certification™. Even though there may be many differences in purpose and intent across the various examinations, basically all seek to assess the cognitive knowledge (in some instances, the analytic reasoning) of physicians, be they trainees or practicing clinicians. While cognitive knowledge is but a single competency among several others, it is of particular importance because it is the basis of many others. Satisfactory performance in each stage requires a broad, in-depth, and current level of knowledge and competence in the diagnosis and treatment of important common and less common conditions, which span the broad range and scope of each specialty.

The boards keep examinations and expectations current by continually updating subject material to reflect the ever-changing body of knowledge in medicine. Basic research and clinical developments in all medical specialties are routinely incorporated into upcoming examinations. Furthermore, recent times have brought the inclusion of “newer” topics that span across traditional specialty boundaries—such as epidemiology, women’s health, critical care medicine, ethics, and environmental medicine—because of the importance each of these topics has attained in medical practice.

The ABMS and member boards have determined that patient safety has reached a significant and visible level of importance in medical practice today. In particular, the ability of physicians to practice medicine with a broad understanding of the complex system of medical care delivery continues to gain import. Requisite familiarity with and basic knowledge of the intricacies of the health care system are frequently linked with the ability to determine where and how medical errors occur. Without an appreciation of how, where, and why patient injuries occur, it may become difficult for physicians to deliver the highest-quality care. This is one of the main reasons that the ABMS is encouraging its member boards to include material on medical error and patient safety in their certification and maintenance of certification (or recertification) examinations.

The potential inclusion of patient safety questions into board examinations raises implications for prospective candidates for certification, current diplomates seeking to maintain certification, and for those providing instruction and training, including medical schools and residency programs. Candidates will need to be prepared for potential examination questions, and medical education curricula may need modification to provide the necessary teaching. Because the field of patient safety is arguably less developed than more traditional topics, questions as to what type of material would be appropriate for testing arise. In order to provide a basis for what concepts or material the boards may opt to include as they start the process of designing and testing potential questions.

Back to Top | Article Outline

Patient Safety: Overview

Patient safety has been defined as the freedom from accidental or preventable injuries produced by medical care.9 This includes freedom from harm caused by the omission of appropriate care. Conceptually, safety is a subset of quality. The highest quality of care should be free from any accidental or preventable injury. However, not all quality-of-care issues automatically trigger patient safety concerns. Quality measures can include dimensions—such as patient satisfaction, comfort, access to care and medical information, and utilization of resources—that may not ordinarily be associated with significant physical harm or injury.

Due to the need to have a shared, common language when describing and addressing medical error and patient safety, the IOM developed definitions that are often employed.

  • An error is defined as “the failure to complete a planned action as intended or the use of a wrong plan to achieve an aim.”1,9
  • An adverse event is defined as “an injury caused by medical management rather than by the underlying disease or condition of the patient.”1,9
  • Accidental or preventable injuries (also known as preventable adverse events) occur when an adverse event results from medical error.1,9

Patient safety principles are not focused on reducing all adverse events, but rather on reducing all errors. To illustrate, when a patient is appropriately prescribed and properly administered chemotherapy but develops heart failure, an adverse event has occurred. However, because there is no underlying error in the decision to prescribe and in the administration of the chemotherapy, the injury is not deemed to be a preventable adverse event. In contrast, if chemotherapy is incorrectly prescribed instead of necessary surgery, an error has occurred. Two results can follow from this error. If the patient develops an injury such as heart failure, a preventable adverse event has now occurred. However, if the patient does not sustain any harm from the inappropriate chemotherapy, a near miss (or a “close call”) has occurred. A near miss is an event or situation in which an error occurs but does not produce injury because of some intrinsic characteristic of the patient.10 A near miss can also occur due to a fortuitous and timely intervention (e.g., the chemotherapy is inappropriately prescribed, but before it is infused, someone recognizes the error and cancels the infusion).

Patient safety experts often, but not universally, advance the notion that a higher yield of data on errors can be garnered through specifically studying near misses.11–13 The advantages of studying near misses include the fact that near misses occur in greater numbers while still demonstrating exactly what types of mistakes happen and how harm may result in the future. In addition, near misses do not carry the same barriers to reporting that are associated with preventable adverse events: shame, fear of blame, and fear of lawsuit are all diminished in the context of near misses.14

It is important to note that under the IOM definition of error, all errors do not necessarily result from negligence. Negligence (legally compensable error) exists when there has been a failure to adhere to a standard set by an expectation of reasonable behavior in the surrounding circumstances.15 On the other hand, the IOM and patient safety advocates accept the notion that humans will “err” even when acting reasonably (i.e., not negligently). Therefore, a systems approach should be adopted in which systems of care should be designed to catch such errors and prevent them from causing harm.16–18

The use of a systems approach to reducing error is not new and has been employed in other disciplines, notably in aviation and engineering.11,19,20 Fundamental to this approach is the premise that all participants intend for a good outcome and act accordingly, in terms of effort and due care. Assigning blame to those who err is not a priority.21 The goal is to standardize processes, simplify systems, and implement protocols to remove unnecessary complexity, variations in treatment, and errors in execution and cognition. Ideally, care delivery systems are designed and redesigned with these principles in mind. Errors, though, will inevitably occur. Various frameworks for analyzing adverse events have thus been proposed.13,16,19,22

Factors that may underlie an error, known as systems factors or contributing factors, have been organized into the following categories: institutional context (the economic and regulatory context), organization and management (financial resources and constraints, policies, and culture), work environment (staffing, design, and availability or maintenance of equipment), team factors (communication, supervision, and team structure), individual staff factors (knowledge, skills), task factors (availability of protocols), and patient characteristics (complexity of illness, language, and personality).16,23,24 Methods and tools for analyzing adverse events to prevent further errors from occurring are available. One means is to conduct a root cause analysis (RCA), which involves a thorough retrospective investigation to flesh out what factors contributed to the occurrence of an error.10,25

Organizations can also take a proactive approach by conducting a failure mode and effects analysis (FMEA) to identify potential contributing factors to potential adverse events.10,26 With this approach, the likelihood of particular processes’ failing is combined with an estimate of the consequences of failure to create a criticality index, which allows for prioritization of what processes need “error proofing.” Either via an RCA or FMEA, the goal is to determine at the organizational level how to prevent errors from occurring in the future.

Complementing the growing knowledge on how to reduce injury at the level of individual institutions, there has been a substantial amount of research activity aimed at reducing errors across the healthcare system.27 The impetus for such work lies in the relative infrequency with which many types of errors can occur at one institution. By pooling data across institutions, global trends and data can theoretically be gathered.

Garnering exact figures on the extent to which medical errors and adverse events occur has not been without challenges. Limitations arise from the unknown level of underreporting of errors and adverse events that occurs.28 The difficulty of determining retrospectively whether an error actually occurred (reasonable disagreement on whether an error was made may frequently exist) compounds this situation.13,29 Notwithstanding such limitations, the study of patient safety has created a body of evidence reasonably estimating which types of errors frequently occur, the necessary first step in identifying a problem so that it can be addressed.1,30–32

Research has uncovered information on both errors of commission and omission, whether they are errors of plan or of execution. In addition, the efficacy (in terms of the extent by which preventable adverse events are reduced) of selected patient safety practices or interventions has been demonstrated.27 However, this efficacy does not appear to be the only factor considered when assessing an intervention’s prioritization for widespread adoption. Organizations leading the charge of patient safety have endorsed overlapping but different sets of practices as places to start making health care safer, based on how they have weighed additional factors.27,33–35

The differences in choice of factors are likely attributable to the missions and goals of the respective organizations. When prioritizing patient safety interventions, these organizations have given varying degrees of consideration to the frequency with which the error in question occurs, the frequency with which harm results from the error, the extent of the resulting harm, the feasibility of implementing an intervention, and the cost of implementing that intervention. The higher the level of evidentiary support any of these factors enjoys, the more weight it tends to be given. Table 1 provides a list of four organizations (and their goals) that have been leading participants in the patient safety movement and the factors that each of these organizations weighed to determine its set of endorsed patient safety practices. Table 2 displays a list of the practices these organizations have endorsed and allows for comparison across the organizations.

Table 1

Table 1

Table 2

Table 2

Table 2

Table 2

Many of the patient safety practices promulgated by leading organizations are based on a strong level of evidentiary support. However, some advocates argue that requiring the strictest of evidentiary support for any practice before widespread implementation has the potential to be counterproductive.36,37 They advance that evidentiary support should clearly be sought, but sometimes it may be difficult to find a benefit from implementing a certain practice that actually exists; detecting preventable adverse events can be a challenge because specific types of events can be relatively infrequent and troubles in identification can be compounded by a lack of reporting and difficulty with measurement. Therefore, they argue that the absence of evidentiary support should not be a reason to disregard practices and improvements based on common sense. Much of the patient safety agenda today thus continues to call for research while pushing for implementation of patient safety practices.

Back to Top | Article Outline

Including Patient Safety Questions on the Certification Exam

How does the present knowledge base in patient safety translate to what certification candidates should ideally be expected to know? The boards could opt to simply rephrase questions that have traditionally tested basic knowledge as patient safety questions. For example, when a patient presents with severe chest pain radiating to the back, the practitioner must generate a differential diagnosis that includes aortic dissection and should know the mandated diagnostic evaluation and treatment. A failure for a physician to know any of these steps is clearly a concern for patient safety; for physicians to deliver safe care, they must first know what safe care is. However, even though phrasing such exam questions in the context of patient safety may raise awareness of the critical need of accurate diagnostic evaluation and treatment, this approach would likely not be adding significant new material to the certification exam. In other words, candidates would not need to learn more than the diagnostic and treatment plan for a chest pain radiating to the back.

To help the boards determine what material would be appropriate for inclusion, the ABMS convened its Patient Safety Working Group Advisory Panel. Table 3 lists the areas that were identified by the panel as ripe for inclusion on upcoming examinations. The table illustrates that the member boards have been provided a broad range of topics and approaches available for selection. Topics may include general overview types of questions such as those related to the epidemiology of medical error and injury. Due to the relative newness of the field, boards may opt to ask about the basic definitions or framework employed. Alternatively, they may move into the methodology of patient safety.

Table 3

Table 3

Specifically, boards may test on how the errors in care could have been prevented and what to do once they happen. In this context, candidates may be expected to appreciate the complexity of systems of care and where the breakdowns occur. To provide a case example, a patient presents with a stroke within 30 minutes of the onset of symptoms and meets the clinical criteria for lytic therapy. The patient, however, is not administered lytic therapy within three hours. With the traditional clinical testing approach, candidates should know the proper window for administering lytic therapy had passed. Further analyzing this case specifically from a patient safety viewpoint, candidates should be also able to appreciate that an adverse event has occurred here (even though this is a case of omission) and that it was likely preventable.

Candidates may then be expected to identify why the failure occurred. Was it a failure to recognize stroke symptoms by a provider? Did the failure occur because obtaining an interpretation of the CT scan took two hours because there was no on-call radiologist for reading emergent studies? Was it the unavailability of the lytic medication? In addition, the ability to detect contributing factors such as those involved in training, staffing, and organizational structure may be examined. The boards may also choose to ask about what types of tools can be employed to learn why this error occurred, such as an RCA.

The boards may also expect candidates to possess knowledge regarding ethical and legal obligations related to patient safety once an error or preventable adverse event has occurred. Topics could include appropriate actions to be taken with regard to disclosure of the error to patients and families, understanding the goals of reporting systems, and what types of reporting obligations exist for providers.

The member boards may also decide to test on what can be done to reduce error in the future. This could include querying about appropriate actions in response to information learned from an RCA or about patient safety practices that have been promulgated. With regard to the latter, practices that have evidentiary support in the literature demonstrating their efficacy are possible candidates. For example, a vignette in which a patient sustains a ventilator-acquired pneumonia could be provided. A question could then be phrased to ascertain if a candidate knows what practice could have prevented it (asking for an evidence-based answer). Candidates may also be queried about the rationale behind leading patient safety practices.

The ABMS recommendation to include patient safety in board exams thus raises relatively significant, but straightforward, implications for individuals seeking certification or recertification in the future. Medical schools, training programs, and continuing medical education programs would benefit their trainees by providing patient safety material in curricula and in-training exams. The topics include general patient safety concepts, the epidemiology of medical error, the concept of a systems-based approach, specific system factors such as communication and teamwork, specific patient safety practices, and obligations related to disclosure and reporting. Regardless of the questions asked, candidates must have an understanding that treatments and procedures are not always delivered correctly in the clinical arena, what causes the breakdowns, and what can be done from a systems viewpoint to correct them.

Back to Top | Article Outline

Summing Up

As part of their mission, the ABMS and its member boards seek to maintain and enhance the quality and safety of patient care. Certification examinations are a key component of the methods used to attain these goals. The advent of patient safety and its growing importance in patient care has prompted the ABMS to encourage member boards to include patient safety principles as a requirement for certification. Even though the study of patient safety is arguably young, a range of topics and types of questions a candidate may reasonably be expected to know can be estimated. Potential candidates will need to ensure that they possess the necessary knowledge and skills and educators will need to redesign curricula to ensure that they teach what the candidates need to know. In fact, by including patient safety material on their examinations, the member boards may be further fostering the culture of safety in medicine that is much needed.

Back to Top | Article Outline


The ABMS/CMSS Patient Safety Working Group Advisory Panel was generously supported by the ABIM Foundation. Troyen Brennan, MD, JD, MPH served as the chair, and the participants were David Bates, MD; Paul Friedmann, MD; Tejal Gandhi, MD; Walter J McDonald, MD; Ellison Pierce, MD; Bruce Spivey, MD; Eric Thomas, MD; Robert Wachter, MD; Daniel Wolfson; Michael Zinner, MD; and Stephen Miller, MD, MPH.

Back to Top | Article Outline


1 To Err is Human: Building a Safer System. Washington, D.C.: National Academy Press, 2000.
2 Altman DE, Clancy C, Blendon RJ. Improving patient safety—five years after the IOM report. N Engl J Med. 2004;351:2041–43.
3 Sharp LK, Bashook PG, Lipsky MS, Horowitz SD, Miller SH. Specialty board certification and clinical outcomes: the missing link. Acad Med. 2002;77:534–42.
4 Kassirer JP. The new surrogates for board certification—what should the standards be? N Engl J Med. 1997;337:43–44.
5 Horowitz SD, Miller SH, Miles PV. Board certification and physician quality. Med Educ. 2004;38:10–11.
6 Brennan TA, Horwitz RI, Duffy FD, Cassel CK, Goode LD, Lipner RS. The role of physician specialty board certification status in the quality movement. JAMA. 2004;292:1038–43.
7 Weinberger SE, Duffy FD, Cassel CK. “Practice makes perfect.” Or does it? Ann Intern Med. 2005;142:302–3.
8 American Board of Medical Specialties: About ABMS 〈〉. Accessed 6 January 2006.
9 AHRQ Quality Indicators—Guide to Patient Safety Indicators. Report No. 03-R203. Rockville, MD: Agency for Healthcare Research and Quality, 2003.
10 Aspden P, Corrigan JM, Wolcott J, Erickson SM (eds). Patient Safety: Achieving a New Standard for Care. Washington, D.C.: National Academy Press, 2004.
11 Barach P, Small SD. Reporting and preventing medical mishaps: lessons from non-medical near miss reporting systems. BMJ. 2000;320:759–63.
12 Layde PM, Maas LA, Teret SP, et al. Patient safety efforts should focus on medical injuries. JAMA. 2002;287:1993–97.
13 McNutt RA, Abrams R, Aron DC, for the Patient Safety Committee. Patient safety efforts should focus on medical errors. JAMA. 2002;287:1997–2001.
14 Aspden P, Corrigan JM, Wolcott J, Erickson SM. Patient Safety: Achieving a New Standard for Care. Washington, D.C.: National Academy Press, 2003.
15 Studdert DM, Mello MM, Brennan TA. Medical malpractice. N Engl J Med. 2004;350:283–92.
16 Vincent C. Understanding and responding to adverse events. N Engl J Med. 2003;348:1051–56.
17 Nolan TW. System changes to improve patient safety. BMJ. 2000;320:771–73.
18 Reason J. Human error: models and management. BMJ. 2000;320:768–70.
19 Leape LL. Error in medicine. JAMA. 1994;272:1851–77.
20 Helmreich RL. On error management: lessons from aviation. BMJ. 2000;320:781–85.
21 Runciman WB, Merry AF, Tito F. Error, blame, and the law in health care—an antipodean perspective. Ann Intern Med. 2003;138:974–79.
22 Bogner MS. Human Error in Medicine. Bethesda, MD: Lawrence Erlbaum Associates, 1994.
23 Vincent C, Taylor-Adams S, Chapman EJ, et al. How to investigate and analyse clinical incidents: clinical risk unit and association of litigation and risk management protocol. BMJ. 2000;320:777–81.
24 Vincent C, Taylor-Adams S, Stanhope N. Framework for analysing risk and safety in clinical medicine. BMJ. 1998;316:1154–57.
25 Wald H, Shojania KG. Root Cause Analysis. In: Shojania KG, Duncan BW, McDonald KM, Wachter RW (eds). Making Health Care Safer: A Critical Analysis of Patient Safety Practices. Report No. 43. Rockville, MD: Agency for Health Care Research and Quality, 2001:51–56.
27 Shojania KG, Duncan BW, McDonald KM, Wachter RW (eds). Making Health Care Safer: A Critical Analysis of Patient Safety Practices. Report No. 43. Rockville, MD:Agency for Health Care Research and Quality, 2001.
28 Pietro DA, Shyavitz LJ, Smith RA, Auerbach BS. Detecting and reporting medical errors: why the dilemma? BMJ. 2000;320:794–96.
29 Hayward RA, Hofer TP. Estimating hospital deaths due to medical errors: preventability is in the eye of the reviewer. JAMA. 2001;286:415–20.
30 Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. results of the Harvard Medical Practice Study I. N Engl J Med. 1991;324:370–76.
31 Thomas EJ, Studdert DM, Burstin H, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care. 2000;38:261–71.
32 Classen DC, Pestotnik SL, Evans RS, LloydJF, Burke JP. Adverse drug events in hospitalized patients: excess length of stay, extra costs, and attributable mortality. JAMA. 1997;277:301–6.
33 The National Quality Forum: Safe Practices for Better Healthcare 〈〉. Accessed 6 January 2006.
34 Leapfrog Fact Sheet 〈〉. Accessed 6 January 2006.
35 JCAHO: 2005 Hospitals’ National Patient Safety Goals 〈〉. Accessed 6 January 2006.
36 Shojania KG, Duncan BW, McDonald KM, Wachter RM. Safe but sound: patient safety meets evidence-based medicine. JAMA. 2002;288:508–13.
37 Leape LL, Berwick DM, Bates DW. What practices will most improve safety? evidence-based medicine meets patient safety. JAMA. 2002;288:501–7.
Back to Top | Article Outline

References Used in Table 1 Only

38 Facts About the Joint Commission on Accreditation of Healthcare Organizations 〈〉. Accessed 6 January 2006.
    39 JCAHO: Introduction to the National Patient Safety Goals 〈〉. Accessed 6 January 2006.
      © 2006 Association of American Medical Colleges