Defining Diagnostic Error: A Scoping Review to Assess the Impact of the National Academies’ Report Improving Diagnosis in Health Care : Journal of Patient Safety

Secondary Logo

Journal Logo

The Health Care Manager

Defining Diagnostic Error: A Scoping Review to Assess the Impact of the National Academies’ Report Improving Diagnosis in Health Care

Giardina, Traber D. PhD∗,†; Hunte, Haslyn PhD‡,§; Hill, Mary A. BS‡,§; Heimlich, S. Layla MLIS; Singh, Hardeep MD, MPH∗,†; Smith, Kelly M. PhD‡,§,¶,#

Author Information
Journal of Patient Safety 18(8):p 770-778, December 2022. | DOI: 10.1097/PTS.0000000000000999

Abstract

Diagnostic errors are major contributors to patient harm.1 Although exact numbers are unknown, about 5% of U.S. adults are estimated to experience a diagnostic error every year in the ambulatory setting, with about half being potentially harmful.2 Diagnostic errors often involve common conditions and result from breakdowns in information gathering or interpretation, or follow-up of abnormal diagnostic test results.3–7 Standards for accurate and timely diagnosis are ill-defined,8 and clinicians must constantly balance diagnostic accuracy against judicious use of diagnostic tests or procedures. Diagnosis also involves uncertainty and evolves over time.8 All of these factors make diagnostic errors difficult to define.

In 2005, Dr. Mark Graber, one of the pioneers in the field, and colleagues defined diagnostic error using the Australian Patient Safety Foundation classification of error as “unintendedly delayed, wrong, or missed as judged from the eventual appreciation of more definitive information.”9 Although this definition has persevered, thought leaders have emerged with conceptually similar but competing definitions.4,8,10–13 Use of different definitions can make it difficult to compare outcomes across studies and introduces ambiguity in measurement.14

In 2015, the Institute of Medicine, now the National Academies of Sciences, Engineering, and Medicine (NASEM), published Improving Diagnosis in Health Care highlighting the imperative to improve the diagnostic process to reduce errors.1 The NASEM committee defined diagnostic error as, “the failure to (a) establish an accurate and timely explanation of the patient’s health problem(s) or (b) communicate that explanation to the patient.”1 Although the NASEM definition was informed by existing definitions and definitional frameworks of diagnostic error,9–13 the committee did not operationalize accuracy, timeliness, or communication. Almost 6 years after its publication, it is unclear how the new definition has impacted the study of diagnostic errors. The objective of our scoping review was to explore how researchers have operationalized the NASEM committee’s definition of diagnostic error in peer-reviewed published literature and establish its impact on this growing field.

METHODS

Design

We used Arksey and O’Malley’s framework15 for scoping reviews to explore how authors have operationalized the NASEM definition.16–19 The framework recommends a comprehensive search strategy across several literature sources, including electronic databases, reference lists of relevant literature, hand-searching key journals, networks, and relevant organizations.15–17 For the purpose of this study, we focused on peer-reviewed literature and consultation with subject matter experts (SMEs) in the field of diagnostic safety. The study received an exempt determination by the Single IRB for the project (MHRI: 00001338).

Research Question: “How are authors operationalizing the NASEM committee’s definition of diagnostic error in published peer-reviewed diagnostic error research?”

Electronic Literature Database Searching

A clinical library scientist codesigned the search strategy and supported electronic database searches. We performed the final search using Medline and Google Scholar around a broad list of terms for defining diagnostic errors (Appendix A, https://links.lww.com/JPS/A472). The strategy was tested against core readings in the field (Appendix B, https://links.lww.com/JPS/A472). Medical subject heading terms from these articles were indexed and informed the final search strategy. The medical subject heading terms along with text word terms and phrases were then mapped to Google Scholar. The nonindexed portion of Medline was searched separately with a text-only variant of the search.

We modified the initial search strategy in Medline to eliminate literature on medical error, preventable harms, or patient safety issues unrelated to diagnostic error, and to remove subheadings under the Diagnostic Error subject heading that related to errors in laboratory results, such as false-positive and false-negative results (Appendix C, https://links.lww.com/JPS/A472). Database searches were limited to English and non-English language articles with abstracts in English, published between October 1, 2015, and November 1, 2020. This date range was informed by the NASEM report publication date.1 An additional informal PubMed and Google Scholar (key terms: diagnostic error, diagnostic safety) search was conducted to capture any new published literature between November 2020 and February 2021. Finally, the library scientist conducted an initial deduplication process. This list of candidate articles was screened for inclusion.

Selecting Studies

Our initial search yielded a large number of abstracts for screening (see PRISMA diagram, Fig. 1). Articles were screened for inclusion using an initial set of criteria tested by 3 investigators and refined to yield a final set of criteria outlined in Table 1. A team of abstractors experienced in scoping review methodology then applied the screening criteria to identify articles to undergo full-text screening and subsequent review.

F1
FIGURE 1:
PRISMA flow diagram.
TABLE 1 - Systematic Review Inclusion and Exclusion Criteria
Inclusion Criteria Exclusion Criteria
• Formal research study related to diagnostic error
• Study references the 2015 NASEM report
• 2015 NASEM committee definition for diagnostic error
• Study using mixed methods, quantitative or qualitative methods, including formal review methods (e.g., systematic review, scoping review, meta-analysis)
• Non-English language articles
• Studies not conducted on humans
• Articles without descriptions of methods used
• Articles that focus on patient safety without addressing diagnostic error
• Articles that focus on diagnostic safety but not specifically the concept of diagnostic error (e.g., diagnostic accuracy, uncertainty)
• Opinion, editorials, perspectives, or viewpoints
• Case studies, case series, or case reports
• Books, book chapters, news articles
• Conference articles or abstracts
• Methods-only articles
• Articles that focus on a specific disease with limited potential for generalizability to other diagnoses
• Narrative review articles
• Articles accepted for publication pre-2016
• Studies initiated before the NASEM report

Data Abstraction

We designed a broad abstraction tool to collect descriptive characteristics such as general citation information (e.g., title, abstract, universal record locator, unique record identifier, resource type) to support the initial screen. Secondary screening required review of the full-text articles. Our team made an a priori decision that operationalization of the definition would require 2 things: (1) referencing of the NASEM report and (2) reference to the NASEM committee’s definition of diagnostic error. Articles that did not meet these criteria were excluded. The secondary and tertiary screening abstraction forms were designed to elicit information on if and where the NASEM definition was reported. Tertiary abstraction forms were designed to summarize if and how the definitions were operationalized for use and to assess article quality for literature undergoing full-text review. Consensus for consideration of each article was completed during a final team meeting.

SME Interviews

The first author (T.D.G.), a qualitative methodologist, conducted semistructured interviews with 9 SMEs selected based on scholarship (e.g., number of publications, national/international reputation) or diversity in research topic (e.g., clinical education, reporting, patient perspective, cognitive psychology). Subject matter experts were asked to discuss (1) perception of the definition, (2) experience operationalizing the definition, and (3) impact of the NASEM definition on research. Interviews lasted between 30 and 60 minutes, and SMEs were provided a $150 stipend for their time.

Interviews were audio recorded and transcribed verbatim and coded using an inductive/deductive content analysis. The first author (T.D.G.) became familiar with all transcripts and created an initial codebook. Initial codes were created deductively based on the questions asked (e.g., opinion, impact on field, operationalizing). Inductive codes were open coded and added to the codebook as they emerged. Coding was discussed with the team, and any discrepancies were resolved through consensus.

RESULTS

Initially, 1077 unique peer-reviewed articles were screened after exclusions and deduplication. Abstract screening yielded further exclusions including publication type (n = 536), being published or accepted for publication before NASEM report publication (n = 51), and for including nonhuman subjects (n = 109). The most common publication type excluded was case study/case reports (n = 258 [48.1%]) followed by commentary or editorials (n = 84 [15.7%]). These exclusions were not mutually exclusive, resulting in 469 articles requiring full-text secondary screening. Of these, 241 articles cited the NASEM report and 53 mentioned the committee’s definition for diagnostic error. Most articles referenced the NASEM definition in the introduction (61.5%), methods (15.4%), or discussion (19.2%). Upon full-text screening, additional articles were excluded based on publication type (n = 66; e.g., narrative reviews, opinion, viewpoints, or editorials), research not focused on diagnostic error (n = 112), or not referencing the NASEM definition (n = 29). Full-text review of the remaining 34 articles that directly referenced the NASEM committee’s definition was completed independently by 2 authors (T.D.G., K.M.S.). Upon full-text review, an additional 8 articles were excluded where the NASEM diagnostic error definition was included, but the concept of diagnostic error was not used or was not under evaluation in their study (e.g., diagnostic safety, uncertainty) and another 10 articles were excluded for publication type.

Sixteen studies were included in the final analysis and abstracted to determine how diagnostic error was operationalized (i.e., what is being measured; Table 2). Of those included, 13 were U.S. studies, 2 were in Japan, and 1 was in Germany. Nine studies20,23,24,26,29,30,32,33,35 indicated using the NASEM definition, 5 of those20,23,24,30,32 operationalized it using a definition proposed before the NASEM report (see Table 3 for list of definitions). Three studies21,31,34 operationalized error using existing definitions, and 4 studies22,25,27,28 operationalized components of the NASEM definition (i.e., accuracy, timeliness, communication) for the purpose of the study and did not cite existing definitions. To capture content focus, we grouped studies according to the area of focus for which the definition was used: epidemiology, patient perspectives, measurement/surveillance, and clinician perspectives.

TABLE 2 - Articles Included in the Final Analysis With Corresponding Definition of Diagnostic Error Used
Articles Country Operationalized Diagnostic Error Type of Study NASEM Key Concepts
Newman-Toker et al 20 United States The authors used the NASEM definition and misdiagnosis-related harm (e.g., the delay or failure to treat a condition actually present, when the working diagnosis was wrong or unknown [delayed or missed diagnosis], or from treatment provided for a condition not actually present [wrong diagnosis]). Malpractice claims Timeliness, accuracy
Watari et al 21 Japan To minimize bias during the review, the authors selected the widely used definitions of a diagnostic error: “delay in diagnosis,” “misdiagnosis,” and “wrong diagnosis.” Judgments were deemed final if made by the Supreme Court, high courts, or local district courts. Malpractice claims Timeliness, accuracy
Gupta et al 22 United States Failure to diagnose, delay in diagnoses, wrong diagnosis, and other Malpractice claims Timeliness, accuracy
Lee et al 23 Japan Diagnosis that is “missed, wrong, or delayed as detected by some subsequent definitive test or finding.” Did not distinguish “missed” from “delayed” diagnoses Retrospective medical record review Timeliness, accuracy
Rinke et al 24 United States The authors separated the concept of diagnostic error and MOD. For diagnostic errors, the authors used the NASEM definition and operationalized using clinical guidelines (e.g., children/adolescent BP levels). MOD was adapted from previous work and defined to occur when evaluation for a diagnosis was not pursued despite a clear need to do so. The diagnostic concepts chosen involved failures at different stages of the diagnostic process: evaluation of symptoms (adolescent depression), evaluation of signs (elevated BP), and follow-up of diagnostic tests (abnormal laboratory values). Retrospective medical record review Timeliness, accuracy
Dadlez et al 25 United States Failure to document or exclude concerns for depression during a health supervision visit in patients 11 years or older. Missed elevated BP occurred when a provider failed to document an appropriate action for a patient with an elevated BP. Missed or delayed response to abnormal laboratory values was limited to patients with specific abnormal results that are often received by pediatric practices but can cause harm if missed. “Mini-root cause analysis” Timeliness, accuracy
Sacco et al 26 United States Accuracy: “In the past 5 years, has your provider given you the wrong explanation for your health care problem(s)?” Communication: “In the past 5 years, have you left the hospital, the emergency department, or your provider’s office confused about the explanation of your health care problem(s)?” Timeliness: “In the past 5 years, has it taken too long to receive an explanation for your health care problem(s)?” Patient reports of diagnostic error Timeliness, accuracy, communication
Aoki and Watanuki 27 Japan Patient-reported diagnostic errors were identified based on response to the question “In the past 10 years, has a doctor made a wrong diagnosis or misdiagnosed you?” Participants were asked to answer on a binary scale. Patient reports of diagnostic error Accuracy
Bontempo and Mikesell 28 United States Patient-reported misdiagnosis of a mental and/or other physical health problem was measured using 2 single-item questions: “Has a physician ever misdiagnosed your endometriosis as a physical health problem (before you were diagnosed with endometriosis)?” and “Has a physician ever misdiagnosed your endometriosis as a mental health problem (before you were diagnosed with endometriosis)?” to which participants could respond with either “yes” or “no.” Diagnostic delay was measured by subtracting the number of years ago patients reported their diagnosis was received from the number of years ago since symptom onset. Patient survey reporting diagnostic error Timeliness, accuracy
Giardina et al 29 United States NASEM definition was used to identify diagnostic error narratives. The authors did not operationalize. Secondary analysis of patient reports of adverse events Timeliness, accuracy, communication
Giardina et al 30 United States Used the NASEM definition to identity potential diagnostic errors. Cases were included as “concerning” if summary statements included one or more of the following: (a) any language about a diagnosis, (b) any mention of a potential patient safety issue, and (c) any clinician behaviors related to communication. To confirm the presence of diagnostic error, defined as a missed opportunity in making a correct or timely diagnosis. Patient complaints Timeliness, accuracy
Soleimani et al 31 United States Initial diagnostic criteria: new diagnostic label within 24 h after rapid response team. Time: features >6 h before initial presentation of new diagnosis. In cases of disagreement about diagnostic error, a secondary EHR chart review strategy was applied using Schiff taxonomy. 12 Medical record review Timeliness, accuracy
Jayaprakash et al 32 United States Diagnostic error is defined as a failure to establish an accurate diagnosis or failure to communicate the diagnosis in medical records. Retrospective medical record review Timeliness, accuracy
Diagnostic delay is the failure to establish a timely explanation of the patient’s health problem and communicate it in the medical records. However, the authors’ operationalized the definition using Schiff taxonomy. 12
Perry et al 33 United States Operationalized: Given inherent ambiguity in defining specific measures for “accurate” and “timely,” the QI team focused on determining if the error was related to deviation from generally accepted local or national performance standards, if the diagnosis could have reasonably been made based on available information at the time of presentation, and if any diagnostic uncertainty was discussed with the patient or family. Review of the medical encounter documentation, including the provider’s medical decision making and patient’s discharge instructions, helped determine communication of an uncertain diagnosis. Retrospective medical record review Accuracy, communication
Donner-Banzhoff et al 34 Germany The authors defined diagnostic error as the originally assumed diagnosis later turned out to be wrong (i.e., in which an undesirable diagnostic outcome had occurred). Clinician survey on diagnostic error Accuracy
Matulis et al 35 United States The authors did not operationalize. Full NASEM definition was included on the survey for the respondent to refer to. Clinician survey on diagnostic error Timeliness, accuracy, communication
EHR, electronic health record; MOD, missed opportunities in diagnosis.

TABLE 3 - List of Definitions of Diagnostic Error Most Commonly Cited
Authors Definition
Graber 36 A simple working definition of diagnostic error is those diagnoses that are missed, wrong, or delayed, as detected by some subsequent definitive test or finding. The origins of these errors can be classified by considering the provider-specific elements, the system-related contributions, and “no fault” elements reflecting diseases that present atypically or involve excessive patient noncompliance.
Graber et al 9 Diagnostic error is operationally defined as a diagnosis that was unintentionally delayed (sufficient information was available earlier), wrong (a different diagnosis was made before the correct one), or missed (no diagnosis was ever made), as judged from the eventual appreciation of more definitive information.
Newman-Toker 10 The author distinguishes between diagnostic process failures and diagnostic labeling failures. Diagnostic process failures are problems in the diagnostic workup. Diagnosis label failures are an incorrect diagnosis or no attempt at a diagnosis. Preventable diagnostic error is the overlap between diagnostic process failures and diagnostic label failures. Unavoidable misdiagnosis is a diagnostic labeling failure without a diagnostic process failure. 1
Singh 11 Identified 3 criteria for defining diagnostic errors:
 1. Case analysis reveals evidence of a missed opportunity to make a correct or timely diagnosis. The concept of a missed opportunity implies that something different could have been done to make the correct diagnosis earlier. The missed opportunity may result from cognitive and/or system factors or may be attributable to more blatant factors, such as lapses in accountability or clear evidence of liability or negligence.
 2. Missed opportunity is framed within the context of an “evolving” diagnostic process. The determination of error depends on the temporal or sequential context of events. Evidence of omission (failure to do the right thing) or commission (doing something wrong) exists at the particular point in time at which the “error” occurred.
 3. The opportunity could be missed by the provider, care team, system, and/or patient. A preventable error or delay in diagnosis may occur because of factors outside the clinician’s immediate control or when a clinician’s performance is not contributory. This criterion suggests a system-centric versus physician-centric approach to diagnostic error.
Schiff et al 12 Delayed, missed, or misdiagnosis is [that can be related *] to errors in the diagnostic process. These include any failure in timely access to care; elicitation or interpretation of symptoms, signs, or laboratory results; formulation and weighing of differential diagnosis; and timely follow-up and specialty referral or evaluation.
Olson et al 37 The authors define undesirable diagnostic events as specific, measurable, and actionable clinical situations likely to denote the presence of diagnostic error.
NASEM 1 The failure to (a) establish an accurate and timely explanation of the patient’s health problem(s) or (b) communicate that explanation to the patient.
*Indicates an adapted definition after correspondence from the author.

Epidemiological Focus

Six studies focused on epidemiology—incidence, frequency, outcome, and contributory factors—and explored accuracy and timeliness concepts. Of the studies in this category, 3 used malpractice claims,20–22 2 used retrospective data,23,24 and 1 used root cause analysis reports.25 Both Newman-Toker et al20 and Lee et al23 indicated using the NASEM definition of error. However, both studies operationalized error using a combination of pre-NASEM definitions—wrong and missed/delayed diagnoses.36 Missed and delayed diagnoses were not considered distinct concepts.10 Watari et al,21 looking at clinical outcomes, operationalized errors using the concept of delayed, wrong, missed diagnosis by Graber et al.9 Gupta et al22 operationalized diagnostic error by selecting from existing categories of malpractice claims: failure to diagnose, delay in diagnoses, wrong diagnosis, and other—consistent with the definition used by Graber et al.9 Rinke et al24 examined the frequency of missed elevated blood pressure (BP), abnormal laboratory values, and adolescent depression in primary care pediatrics using the NASEM definition. They operationalized the definition through use of children/adolescent clinical guidelines for diagnosing high BP and by evaluating follow-up of abnormal test results. For depression, Rinke et al24 used the Singh11 missed opportunity definition (e.g., missed opportunity in making a correct or timely diagnosis) that implied that an evaluation for a diagnosis of adolescent depression was not pursued. Finally, Dadlez et al,25 looking at diagnostic process errors (i.e., failure points and contributing factors), evaluated missed diagnosis of adolescent depression, missed elevated BP, and missed actionable laboratory values in the pediatric ambulatory setting. None of the studies in this category focused on the concept of communication to patient.

Patient Focus

Five studies explored patient reports of diagnostic errors. Aoki and Watanuki,27 and Bontempo and Mikesell28 asked patients specifically about their error experiences related to accuracy. Aoki and Watanuki27 asked primary care patients whether a doctor had made a wrong diagnosis or misdiagnosis. Bontempo and Mikesell28 focused on diagnosis of endometriosis, asking patients whether their endometriosis had ever been misdiagnosed by another physician and calculated diagnostic delay. Only Sacco et al26 operationalized each component of the NASEM definition from the patient perspective: (1) accuracy, wrong explanation for a health care problem; (2) communication, confusion about the explanation of the health care problem; and (3) timeliness, it took too long to receive an explanation for a health care problem. The remaining 2 studies (Giardina et al29,30) analyzed existing data sets of patient reports of adverse events and patient complaints, respectively. Giardina et al29 conducted a secondary data analysis of a subset of patient-reported errors and analyzed open-ended narratives using the NASEM definition for inclusion without further operationalization of accuracy, timeliness, and communication. Giardina et al30 analyzed a subset of patient complaint categories and used the NASEM definition to identify complaints “concerning” for diagnostic error—operationalized as any language about a diagnosis, mention of a potential patient safety issue, and/or any clinician behaviors related to communication. These cases were further reviewed using the Singh11 missed opportunity definition to determine the presence/absence of diagnostic error using the lens of both accuracy and timeliness.

Measurement/Surveillance Focus

Three studies focused on measurement and surveillance of diagnostic errors. The prospective observational study by Soleimani et al,31 focused on accuracy and timeliness, used medical record review that applied diagnostic criteria (e.g., a new diagnostic label within 24 hours after rapid response team and whether any features, indicative of that diagnosis, were present for greater than 6 hours before the first documentation of that new diagnosis). Where there was disagreement, a second review was conducted using the taxonomy delineating stages in the diagnostic process used in the study by Schiff et al12: history, physical exam, testing, assessment, referral/consultation, and follow-up. Jayaprakash et al,32 also using a retrospective medical record review of patients who experienced a rapid response team call, used the NASEM report to operationalize error into 2 parts called diagnostic fidelity—diagnostic error and diagnostic delay—to capture accuracy and timeliness. Diagnostic error was defined as failure to establish an accurate diagnosis or failure to communicate the diagnosis in medical records, and diagnostic delay was the failure to establish a timely explanation of the patient’s health problem and communicate it in the medical records. Perry et al33 conducted a quality improvement project to implement a methodology to identify and measure diagnostic error using the concepts of accuracy and communication across a single pediatric academic center using medical record review. The authors determined the presence of an error when there was a deviation from generally accepted performance standards, if the diagnosis could have reasonably been made based on available information at the time of presentation, and if any diagnostic uncertainty was discussed with the patient or family.

Clinician Focus

Two surveys on clinicians’ perceptions of diagnostic errors were identified. Matulis et al35 conducted a survey of perceptions of outpatient internal medicine clinicians and included the NASEM definition on the survey itself, presumably including accuracy, timeliness, and communication. The primary care physicians’ survey by Donner-Banzhoff et al’s34 operationalized error as cases where the original diagnosis later turned out to be wrong (e.g., a case in which an undesirable diagnostic outcome had occurred as defined by Olson et al37) and treatment was delayed.

SME Interviews

Qualitative analysis of the SME interviews revealed an agreement that the NASEM definition resulted in a fundamental shift in diagnostic safety to prioritize patient and family perspectives. “There is no diagnostic error work without it being patient centered…the whole point of diagnostic error work is to improve the care of patients.” (Participant [P] 1004). However, not all SMEs agreed that lack of communication to the patient should be considered a diagnostic error. Some suggested that it may alternately be an indicator of diagnostic safety. “I do think it’s an important part of the diagnostic process. I think it’s part of diagnostic quality, but I don’t see it…as part of the definition.” (P1001).

Overall SMEs indicated that the NASEM definition functions as a baseline for researchers to conceptualize diagnostic error. Furthermore, the publication of the NASEM definition legitimized and has drawn attention to diagnostic error as a distinct issue within the larger context of patient safety. The act of citing the report may highlight agreement across researchers about the relevance of this change and its impact on the field. “Now…we use the National Academy of Medicine definition and then people leave us alone. So, in that sense for researchers around the definition, for those of us who like the definition and use it, that’s a change. I don’t know that it’s a substantive change from the field standpoint, but it’s an important change…that facilitates a certain amount of cohesion, which I think was the intent of the definition.” (P1008). There was no consensus on operationalizing the definition among SMEs. Most acknowledged that researchers cite NASEM but continue to use their preferred definition. “I would say I actually have a much harder time operationalizing that definition into some of the work that I have done.” (P1005). Although most of the SMEs expressed positive views of the definition, there was some frustration conveyed about ambiguity of concepts, especially timeliness and communication, which may limit the ability to compare outcomes across studies.

DISCUSSION

This scoping review yielded several important findings to advance research related to measurement and reduction of diagnostic error. First, although many of the research articles we reviewed referenced the NASEM report as an anchor for their work, we only identified 16 studies that focused on diagnostic error and that both cited the NASEM report and included the NASEM definition in the text of the article. Second, the NASEM report has significantly influenced patient-centeredness concepts in discussions of diagnostic error definitions. Much of this progress has been from the inclusion of the patient perspective and the concept of communication to the patient within the definition. Third, many researchers reference the NASEM report and definition, yet continue to use pre-NASEM or thought leaders’ definitions to operationalize accuracy (misdiagnosis, wrong diagnosis) and timeliness (missed/delayed diagnosis); few address communication at all. This is likely because these definitions more applicable to their research context or the disease under study and the concept of communication of a diagnosis to a patient was not necessarily considered a diagnostic error before the NASEM report publication.

The review identifies a small but growing body of literature focused on inclusion of patients’ perspectives. Although only 1 article operationalized all 3 components of the NASEM definition,1 the increasing focus on patients’ experiences reflects the SMEs’ consensus that the NASEM report legitimized the patient’s role in the diagnostic process. However, SMEs did not all agree that failure to communicate a diagnosis to a patient should be considered an error. That ambivalence is also reflected in the literature. For example, we did not find concepts related to communication of a diagnosis to patients well established in our review, and only 1 study26 specifically attempted to measure diagnostic communication (e.g., confusion about the explanation of the health care problem). Patient-physician communication has been well studied,38 including how communication contributes to pitfalls in the diagnostic process.28,29,39 Although concepts related to communication of a diagnosis to the patient comparatively have not been as well studied or operationalized, there is a growing body of literature related to timely communication of abnormal test results. For instance, a large number of studies have identified lack of timely communication of abnormal test results to patients and provided specific metrics with which to measure and improve the process.40–48 Additional work is needed to study how factors related to information, timing, method, and behaviors play an integral role in communication in the context of the diagnostic process.28,29 An increased emphasis on patient-partnered or patient-oriented diagnostic research and the application of interdisciplinary research from fields outside of medicine can provide knowledge on how to measure and improve diagnostic communication.

Given that the components of accuracy (e.g., missed, wrong, misdiagnosis) and timeliness (e.g., delayed diagnosis) were often cited and debated pre-NASEM definition (as early as 2005), it is not surprising that these concepts were addressed most often in our review (n = 16 and n = 13, respectively). Researchers may be more comfortable focusing on these more well-established components and tend to select pragmatic definitions that have been applied in prior work. Use of such existing definitions may allow for greater generalizability of results and easier comparisons between studies, particularly in a field where science is nascent. Another area ripe for exploration is how definitions can account for concepts related to both diagnostic processes49 such as missed opportunities11 and outcomes such as clinical endpoints (e.g., harm, morbidity, and mortality50), given that diagnosis is an intermediate outcome. This will enable better generalization of results and comparison across studies, and advance the field of diagnostic safety. Although the NASEM report highlighted the role of patients in the diagnostic process, our review shows that more paradigm shifting results are still awaited.

More than one-third of the studies (n = 6) operationalized diagnostic error, whether the authors indicated using the NASEM definition or not, in ways that were specific to their research questions rather than using an existing operationalized definition (Table 3). Because some of these studies included citations from leaders in the field, it is less likely to be an issue of awareness of existing definitions but an issue of fit. Uncertainty is ubiquitous in diagnosis, and definitional concepts related to timeliness and accuracy may also vary by conditions being studied. Presumably, the NASEM definition is intentionally broad, allowing researchers more flexibility within the core components to be creative and thoughtful about how errors are identified and measured—especially considering the significance placed on patients’ experiences. Given the inherent difficulty in creating an absolute definition to identify error, perhaps the field may need to acknowledge and encourage diversity and innovation in safety measurement as long as the goal is to reduce patient harm.14 The publication of Improving Diagnosis in Health Care consolidated existing diagnostic error knowledge and, in doing so, has articulated the importance of accuracy, timeliness, and communication. These concepts were chosen based on pioneering research published between 2005 and 2015, providing a foundation upon which future researchers can build their work. This approach should accelerate research and enhance generalizability of research findings.

Of the 240 articles that underwent full-text screening, we identified 29 articles that referenced the NASEM report and focused on diagnostic error, but that did not include or use the NASEM definition. Although these articles fell outside the scope of our research question, this may be indicative of the difficulty in operationalizing the NASEM definition. It may also be in line with our SME’s suggestion that researchers cite the report to legitimize the study of diagnostic safety. Perhaps citing the report implies agreement with the definition—especially because much of the literature used the pre-NASEM definitional concepts (e.g., wrong, missed/delayed) that were foundational for the report. In the articles that did include the NASEM definition (n = 52), most mentions occurred in the introduction or discussion sections of the paper suggesting that, although authors believed the definition to be important and relevant, few directly attempted to operationalize the definition or even apply it to their own research.

The institutional authority of NASEM has provided credibility to the diagnostic safety movement, and diagnostic errors have rapidly emerged as a leading patient safety issue in the United States.51 While acknowledging the disagreement among thought leaders, NASEM has effectively produced a common understanding of diagnostic error—accuracy, timeliness, and communication. In our interviews, SMEs acknowledged this and suggested that citation of the NASEM report provides credibility and helps to move toward consensus. Subject matter experts also indicated that, despite any remaining disagreements about measurement of the concepts, the definition unified the field. Even when researchers do not specifically operationalize the NASEM definition and/or cite the report while using previous definitions, the work often falls within the core concepts of the NASEM definition. Application of definitional concepts may also vary by the context or disease or setting in which the specific study is being undertaken. For example, timeliness of diagnosis in cancer would be operationalized differently from that in appendicitis. Thus, additional work can help inform a set of common approaches for operationalizing each component within the NASEM definition.

Our review has several limitations. Although we attempted to conduct an exhaustive search of the diagnostic error literature, it is possible that some studies were not included. We attempted to address this by conducting a broad search in multiple databases by a librarian scientist as well as including purposive hand searching of reference lists, table of contents of domain-specific journals, and key author searches. In addition, it is possible that some of the studies were conceptualized and designed before the report publication. We attempted to address this by excluding articles accepted before NASEM report publication or that indicated within the methods section that the study was initiated before the NASEM report. We cannot fully eliminate bias in our analysis because our team itself is made up of diagnostic safety researchers. Methodological limitations may also exist because only English language studies were included. Finally, our study did not address other important conceptual issues such as process versus outcome aspect of diagnosis and validity and usefulness of the various components of the NASEM definition.10,11,49

CONCLUSIONS

We found that, in the 6 years since the publication of the NASEM committee’s report, Improving Diagnosis in Healthcare, many studies mention the NASEM committee’s definition of diagnostic error, whereas few studies actually operationalize the definition. Most authors focused on accuracy and timeliness, with only 3 studies exploring communication to the patient. Future efforts should bring together established experts and emerging scientists in the field of diagnostic safety to formulate a set of common approaches for operationalizing each component within the NASEM report definition in various contexts. In line with the patient-centered focus of the NASEM report, patients and families should also take a lead role in defining the construct of “communication to the patient.”

REFERENCES

1. National Academy of Medicine. Improving Diagnosis in Health Care. Balogh EP, Miller BT, Ball JR, eds. Washington, DC: National Academies Press; 2015. doi:10.17226/21794.
2. Singh H, Meyer AND, Thomas EJ. The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations. BMJ Qual Saf. 2014;23:727–731.
3. Medford-Davis L, Park E, Shlamovitz G, et al. Diagnostic errors related to acute abdominal pain in the emergency department. Emerg Med J. 2016;33:253–259.
4. Singh H, Giardina TD, Meyer AND, et al. Types and origins of diagnostic errors in primary care settings. JAMA Intern Med. 2013;173:418–425.
5. Singh H, Thomas EJ, Mani S, et al. Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential?Arch Intern Med. 2010;169:1578–1586.
6. Singh H, Thomas EJ, Sittig DF, et al. Notification of abnormal lab test results in an electronic medical record: do any safety concerns remain?Am J Med. 2010;123:238–244.
7. Graber M. Diagnostic Error in Medicine. Arch Intern Med. 2005;165:1493–1499.
8. Zwaan L, Singh H. The challenges in defining and measuring diagnostic error. Diagnosis (Berl). 2015;2:97–103.
9. Graber ML, Franklin N, Gordon R. Diagnostic error in internal medicine. Arch Intern Med. 2005;165:1493.
10. Newman-Toker DE. A unified conceptual model for diagnostic errors: underdiagnosis, overdiagnosis, and misdiagnosis. Diagnosis (Berl). 2014;1:43–48.
11. Singh H. Editorial: helping health care organizations to define diagnostic errors as missed opportunities in diagnosis. Jt Comm J Qual Patient Saf. 2014;40:99–101.
12. Schiff GD, Kim S, Abrams R, et al. Diagnosing diagnosis errors: lessons from a multi-institutional collaborative project. In: Henriksen K, Battles JB, Marks ES, et al., eds. Advances in Patient Safety: From Research to Implementation. Vol 2: Concepts and Methodology. Rockville, MD: Agency for Healthcare Research and Quality; 2005:255–278. Available at: https://apps.dtic.mil/sti/citations/ADA434037. Accessed March 6, 2021.
13. Schiff GD. Diagnostic error in medicine: analysis of 583 physician-reported errors. Arch Intern Med. 2009;169:1881.
14. Singh H, Bradford A, Goeschel C. Operational measurement of diagnostic safety: state of the science. Diagnosis (Berl). 2020;8:51–65. doi:10.1515/dx-2020-0045.
15. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8:19–32.
16. Brien SE, Lorenzetti DL, Lewis S, et al. Overview of a formal scoping review on health system report cards. Implement Sci IS. 2010;5:2.
17. Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci IS. 2010;5:69.
18. Daudt HML, van Mossel C, Scott SJ. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. BMC Med Res Methodol. 2013;13:48.
19. McDonald J, McKinlay E, Keeling S, et al. How family carers engage with technical health procedures in the home: a grounded theory study. BMJ Open. 2015;5:e007761.
20. Newman-Toker DE, Schaffer AC, Yu-Moe CW, et al. Serious misdiagnosis-related harms in malpractice claims: the “big three”—vascular events, infections, and cancers. Diagnosis (Berl). 2019;6:227–240.
21. Watari T, Tokuda Y, Mitsuhashi S, et al. Factors and impact of physicians’ diagnostic errors in malpractice claims in Japan. PLoS One. 2020;15:e0237145.
22. Gupta A, Snyder A, Kachalia A, et al. Malpractice claims related to diagnostic errors in the hospital. BMJ Qual Saf. 2017;27. doi:10.1136/bmjqs-2017-006774.
23. Lee SH, Stanton V, Rothman RE, et al. Misdiagnosis of cerebellar hemorrhage—features of ‘pseudo-gastroenteritis’ clinical presentations to the ED and primary care. Diagnosis (Berl). 2017;4:27–33.
24. Rinke ML, Singh H, Heo M, et al. Diagnostic errors in primary care pediatrics: project RedDE. Acad Pediatr. 2018;18:220–227.
25. Dadlez NM, Adelman J, Bundy DG, et al. Contributing factors for pediatric ambulatory diagnostic process errors: project RedDE. Pediatr Qual Saf. 2020;5:e299.
26. Sacco AY, Self QR, Worswick EL, et al. Patients’ perspectives of diagnostic error: a qualitative study. J Patient Saf. 2020;17:e1759–e1764. doi:10.1097/PTS.0000000000000642.
27. Aoki T, Watanuki S. Multimorbidity and patient-reported diagnostic errors in the primary care setting: multicentre cross-sectional study in Japan. BMJ Open. 2020;10:e039040.
28. Bontempo AC, Mikesell L. Patient perceptions of misdiagnosis of endometriosis: results from an online national survey. Diagnosis (Berl). 2020;7:97–106. doi:10.1515/dx-2019-0020.
29. Giardina TD, Haskell H, Menon S, et al. Learning from patients’ experiences related to diagnostic errors is essential for progress in patient safety. Health Aff (Millwood). 2018;37:1821–1827.
30. Giardina TD, Korukonda S, Shahid U, et al. Use of patient complaints to identify diagnosis-related safety concerns: a mixed-method evaluation. BMJ Qual Saf. 2021;30:996–1001. doi:10.1136/bmjqs-2020-011593.
31. Soleimani J, Pinevich Y, Barwise AK, et al. Feasibility and reliability testing of manual electronic health record reviews as a tool for timely identification of diagnostic error in patients at risk. Appl Clin Inform. 2020;11:474–482.
32. Jayaprakash N, Chae J, Sabov M, et al. Improving diagnostic fidelity: an approach to standardizing the process in patients with emerging critical illness. Mayo Clin Proc Innov Qual Outcomes. 2019;3:327–334.
33. Perry MF, Melvin JE, Kasick RT, et al. The diagnostic error index: a quality improvement initiative to identify and measure diagnostic errors. J Pediatr. 2020;232:257–263. doi:10.1016/j.jpeds.2020.11.065.
34. Donner-Banzhoff N, Müller B, Beyer M, et al. Thresholds, rules and defensive strategies: how physicians learn from their prior diagnosis-related experiences. Diagn Berl Ger. 2020;7:115–121.
35. Matulis JC, Kok SN, Dankbar EC, et al. A survey of outpatient internal medicine clinician perceptions of diagnostic error. Diagn Berl Ger. 2020;7:107–114.
36. Graber M. Diagnostic errors in medicine: a case of neglect. Jt Comm J Qual Patient Saf. 2005;31:106–113.
37. Olson APJ, Graber ML, Singh H. Tracking Progress in improving diagnosis: a framework for defining undesirable diagnostic events. J Gen Intern Med. 2018;33:1187–1191.
38. Matusitz J, Spear J. Effective doctor-patient communication: an updated examination. Soc Work Public Health. 2014;29:252–266.
39. Davis Giardina T, King BJ, Ignaczak AP, et al. Root cause analysis reports help identify common factors in delayed diagnosis and treatment of outpatients. Health Aff (Millwood). 2013;32:1368–1375.
40. Georgiou A, Li J, Thomas J, et al. The impact of health information technology on the management and follow-up of test results—a systematic review. J Am Med Inform Assoc. 2019;26:678–688.
41. Dahm MR, Georgiou A, Herkes R, et al. Patient groups, clinicians and healthcare professionals agree—all test results need to be seen, understood and followed up. Diagnosis (Berl). 2018;5:215–222.
42. Thomas J, Dahm MR, Li J, et al. Variation in electronic test results management and its implications for patient safety: a multisite investigation. J Am Med Inform Assoc. 2020;27:1214–1224.
43. Miao M, Dahm MR, Li J, et al. Managing uncertainty during the communication of diagnostic test information between patients and clinicians in Australian emergency care. Qual Health Res. 2020;30:1287–1300.
44. Giardina TD, Baldwin J, Nystrom DT, et al. Patient perceptions of receiving test results via online portals: a mixed-methods study. J Am Med Inform Assoc. 2018;25:440–446.
45. Nystrom DT, Singh H, Baldwin J, et al. Methods for patient-centered interface design of test result display in online portals. EGEMS (Wash DC). 2018;6:15. doi:10.5334/egems.255.
46. Murphy DR, Thomas EJ, Meyer AND, et al. Development and validation of electronic health record–based triggers to detect delays in follow-up of abnormal lung imaging findings. Radiology. 2015;277:81–87.
47. Murphy DR, Meyer AND, Vaghani V, et al. Development and validation of trigger algorithms to identify delays in diagnostic evaluation of gastroenterological cancer. Clin Gastroenterol Hepatol. 2018;16:90–98.
48. Murphy DR, Meyer AND, Vaghani V, et al. Electronic triggers to identify delays in follow-up of mammography: harnessing the power of big data in health care. J Am Coll Radiol. 2018;15:287–295.
49. Schiff GD, Leape LL. Commentary: how can we make diagnosis safer?Acad Med. 2012;87:135–138.
50. Singh H, Schiff GD, Graber ML, et al. The global burden of diagnostic errors in primary care. BMJ Qual Saf. 2017;26:484–494.
51. ECRI Institute. 2019 Top 10 Patient Safety Concerns: Executive Brief. Plymouth Meeting, PA; 2019.
Keywords:

diagnostic errors; patient-centered care; delivery of health care; medicine

Supplemental Digital Content

Copyright © 2022 The Author(s). Published by Wolters Kluwer Health, Inc.