Medical overuse is a universal phenomenon present in rich and developing countries and across all health professions and specializations.1–4 5–7 1 8
Several initiatives have been launched to draw attention to this problem, to overcoming the challenge of how to measure overuse, and to reduce its frequency. The do-not-do recommendations (DNDs; National Institute for Health and Care Excellence) has contributed to focusing on low-value practices that should be eradicated. “Choosing Wisely” (Advancing Medical Professionalism to Improve Health Care, American Board of Internal Medicine) has pursued the same goal. Although initially carried out in the United States and Canada, it has likely been the most wide campaign throughout the world.9–12 13
A majority of studies have focused on reductions in utilization rates rather than on a change in the appropriateness of services (such as outcome, cost, or patient experience).14–17 18,19
Adverse event has been defined as the unintentional harm suffered by patients throughout health care that is related to the care received and not the patient’s underlying illness.20 21 22
Primary care in Spain includes the practices of general practitioners (GPs) and pediatricians. Spain has a total of 17 autonomous communities, each having the capacity to manage and organize the health resources (hospitals and primary care) located in its territory. In the Spanish health care model, primary care physicians (GPs and pediatricians) play the role of gatekeepers to the health system. Each patient has their electronic medical record with all their health information entered. Primary care physicians can review information on patient hospitalizations and vice versa for hospital-based physicians. Not all health services use the same diagnostic classification. Currently, the International Classification of Diseases, Ninth Edition (ICD-9 ), ICD-10 , and International Classification of Primary Care, Second Edition (ICPC-2) are being used to code diagnoses, according to the different health services in each autonomous community.
The aim of this study was to measure the frequency and severity of AAEs associated with doing what should not be done (overuse related to ignoring DNDs) in primary care in Spain.
METHODS
A retrospective cohort study analyzing the frequency and severity of AAEs related to DNDs (7 from family medicine and 3 from pediatrics) was conducted.
The DNDs considered were a selection of practices included in the lists of consensus-based recommendations made by scientific societies of the “Commitment to the Quality of Scientific Societies.” The DNDs included in this study were those that meet the following criterion: first, there was a broad consensus that they represented very low value for the patients, and in some cases, they could be harmful. Second, they were present relatively frequently in practice and could be reliably identified. Third, they could be the cause of an AAE. These DNDs were defined in a previous study involving 60 GPs and pediatricians. A set of 35 DNDs was assessed, considering their estimated frequency in primary care and the likelihood of causing an AAE. In addition, the capacity to be detected when seeking information in the medical records was assessed. Ten DNDs fulfilling criteria were chosen (Box 1).23
Box 1 Do-not-do recommendations included in this study.
Do-not-do recommendations during the treatment for adults’ patients
Do not use benzodiazepines (ATC : NO5B-BA, BB, BE; NO5CD; N05CF; N03AE) for the treatment of insomnia (ICPC-2 : P06) in people older than 65 years.
Do not prescribe nonsteroidal anti-inflammatory drugs (CTAs: M01A, M01AC, M01AE, M01AG, M01B) to patients with high blood pressure, heart failure, chronic kidney disease, or liver cirrhosis (ICPC-2 : K86, K87, K77, U99, D97).
Do not starter kit paracetamol (ATC : N02BE01) in doses of 1 g.
Do not indicate antibiotic treatment in acute bronchitis when the patient does not have COPD (CIAP—2 R95), heart failure, diabetes, pneumonia, tonsillitis, Bordetella infection, or kidney disease or is undergoing active chemotherapy (ICPC-2 : K77, T89, T90, R72, R76, U99).
Do not prescribe lipid-lowering agents (ATC : C10AA, C10AB) in patients older than 75 years without previous cardiovascular events.
Do not screen asymptomatic patients older than 50 years for prostate cancer using prostate-specific antigen.
Do not perform imaging tests in nonspecific low back pain (ICPC-2 : L03) without warning signs before 6 weeks.
Do-not-do recommendations during the treatment for pediatrician patients
Do not prescribe antibiotics (ATC : J01CA, J01D, J01AA) for pharyngitis or tonsillitis, even if the patient does not test positive for streptococcus (ICPC-2 : R72).
Do not prescribe mucolytics (ATC antitussives: R05CB, R05D) or antibiotics (ATC : J01CA, J01D, J01AA) for respiratory tract infections (ICPC-2 : R72, R74, R76).
Do not combine or alternate treatment with ibuprofen and acetaminophen (ATC : M01AE).
ATC , Anatomical Therapeutic Chemical Classification System ; ICPC-2 , International Classification of Primary Care .
Procedure
This was a coordinated, multicenter, national research project. Data (AAEs related to DNDs) were randomly extracted from computerized databases of primary care medical records of patients who had used health services between February 2018 and September 2019 in Andalusia, Aragon, Castile La Mancha, Catalonia, Valencian Community, Madrid, and Navarre. This period was defined after conducting a preliminary trial for extracting data to ensure the feasibility of the procedure and to obtain reliable data. Medical charts were retrieved through a simple randomized procedure, by reviewing 1 of every 3 medical records of the total number of patients assigned to a doctor. The research includes the steps outlined in Figure 1 .
FIGURE 1: Flowchart of study phases and main outcomes.
The review of these computerized medical records was conducted by a total of 20 GPs and 5 pediatricians acting as reviewers. Reviewers decided whether to review between 77 and 125 stories each according to their work duties. They were independent of the physicians who indicated treatment to the patients during the period in which medical records were reviewed. The review of medical records was conducted between April and September 2019.
Depending on the convention in the health services of each autonomous community, Anatomical Therapeutic Chemical Classification System (ATC ), ICD-9 , ICD-10 , and ICPC-2 codes were used to correctly identify the patient records that were included in the study (Supplementary File 1 https://links.lww.com/JPS/A392
The reviews´ template and triggers were implemented using a computer application for managing the data to reduce variability between reviewers in the way data were recorded and to avoid coding errors. Confidentiality was ensured using a blind registration system that was developed for the study.
We used as triggers diagnosis, treatments, or services in a record that could indicate that a safety event had occurred. The presence of a trigger did not indicate the presence of AAEs. For each identified trigger, the reviewer assessed whether or not it reflected the presence of a safety event (AAE).
These triggers were designed specifically for this study, involving 25 primary care professionals in a previous study.23
The harm severity of each incident was assessed on a 0–7 scale (0, no harm; 1, emotional harm; 2, insignificant harm; 3, minor temporary harm; 4, major temporary harm; 5, minor permanent harm; 6, major permanent harm; and 7, death). This assessment was adapted from the scale used previously by Woods et al24
Reviewers assessed if the AAE was related to ignoring DNDs using a 0–7 scale (0, no relationship between ignoring DNDs and AAE; 1, ignoring DND is unlikely to have contributed to the AAE; 2, ignoring DND could have contributed to the AAE to some extent; 3, ignoring DND could be the cause of some of the harm associated with the AAE; 4, ignoring DND is probably one of the causes of the AAE; 5, ignoring DND probably caused the AAE; 6, complete certainty that ignoring DND is one of the direct causes of the AAE; and 7, complete certainty that ignoring DND is the only direct cause of the AAE). Ratings higher than or equal to 4 were deemed to indicate a positive relationship between the AAE and ignoring DND.
The sample was stratified, considering the annual number of appointments with GPs and pediatricians, by age groups as follows: 30 to 45 years (11.0%), 46 to 64 years (13.0%), 65 to 74 years (52.6%), 75 to 80 years (14.3%), and >80 years (9.1%) in adults and 3 to 12 months (35.2%), 2 to 5 years (35.2%), and 6 to 12 (29.6%) in minors. In all age groups, the numbers of males’ and females’ charts were equal. For a 95% confidence level and an accuracy (expected error) of 2%, a review of a minimum of 2397 medical records was considered necessary.
Interrater Agreement
Before starting the field study, 15 medical records randomly chosen were assessed by pairs of reviewers to determine the degree of agreement identifying AAEs and whether ignoring DNDs was linked to them using the predefined scale. The interrater agreement between reviewers was calculated using Cohen κ coefficient.25
During the study, the reviewers referred to a consultant who, in doubtful cases, advised the reviewer how to code the information. These consultants were physicians who participated in the APEAS study, designed to assess the frequency of AEs in primary care in Spain.26
Study Variables
In this study, continuing to do what should not be done (DNDs) was used as a measure of overuse. The frequency and severity of AAEs related to ignoring them were also measured. This study only registered AAEs that originated in primary care.
Data Analysis
The frequency of AAEs was calculated as the number of patients with at least one event that resulted in unintentional harm. The crude DND and AAE rates were calculated; considering the differences in frequencies between male and female, the adjusted rate by age group and sex was determined by referring to the 2018 frequency data provided by the interactive application of the Primary Care Information System of the Ministry of Health, Consumption and Social Welfare. χ2 Tests with Yates correction were conducted to compare the frequency of ignoring DNDs and AAEs between male and female, and the Cochran-Mantel-Haenszel test was used to analyze the differences in the adjusted rate between both sexes. The DND screening by prostate-specific antigen (PSA) was not considered in this analysis. Statistical significance was set at P < 0.05 (2-tailed) for all the tests used, and the analyses were conducted using the Statistical Package for the Social Sciences (SPSS) V. 24.0 (SPSS, Chicago, Illinois).
Ethics
The study protocol was approved by the Ethics Committee of Primary Care Clinical Research of the Valencian Community (reference: DGFPS/SDGPOI/PLG/JMV/BP/PS/RI; September 28, 2017), Navarre Government Ethics Committee of Drugs Research (reference number: 432; July 26, 2019), and Research Centre of the Welfare Management of Primary Care in Madrid (reference number: 26/18; January 9, 2019).
RESULTS
A total of 2557 medical records were randomly reviewed (adult population: 1928/2557; 75.4%).
Frequency of Ignoring DNDs
In the period analyzed, there were 1859 (72.7%) of 2557 (95% confidence interval [CI], 71.0%–74.4%) cases of ignoring DND in 1307 patients. Of those, 1507 were performed by GPs and 352 by pediatricians. More than one case of ignoring DND was identified in 443 patients (Table 1 ). Furthermore, ignoring DNDs was identified in 901 (62.1%) of 1452 (95% CI, 59.6%–64.5%) patient records in adult patients 65 years or older.
TABLE 1 -
Do Not Dos and AAEs Frequencies During the Last 2 Years
GP (n = 1928)
%
PED (n = 629)
%
No. medical records in which at least one DND was identified
Females
583/1004*
58.1
122/317
38.5
Males
479/924
51.8
123/312
39.4
Total
1062/1928
55.1
245/629
39.0
No. medical records with more than one DND
Females
211/1004*
21.0
42/317
13.2
Males
145/924
15.7
45/312
14.4
Total
356/1928
18.5
87/629
13.8
No. patients suffering from at least one AAE related to ignoring DNDs
Females
32/583
5.5
7/122
5.7
Males
22/479
4.6
7/123
5.7
Total
54/1062
5.1
14/245
5.7
No. patients who suffered more than one ignoring DND-related AAEs
Females
0
0
0
0
Males
0
0
1/123
0.8
Total
0
0
1/245
0.4
n = 2557. These data include AAEs related to ignoring DND recommendations about screening of prostate cancer in asymptomatic PSA patients older than 50 years.
*P < 0.05.
Frequency of Ignoring DNDs and Sex Differences
Do-not-do recommendations were ignored more often by female adult patients (crude rate, 583/1004 [58.1%; 95% CI, 55.0–61.1]; adjusted rate, 49.4% [95% CI, 48.5–50.3]) than by male adult patients (crude rate, 430/924 [46.5%; 95% CI, 43.3–49.8]; adjusted rate, 41.8% [95% CI, 40.8–42.9]; crude rate, χ2 = 25.2 [P < 0.0001]; adjusted rate, χ2 = 112.6 [P < 0.0001]). In pediatric patients, no differences were found between males and females in the frequency of ignoring DNDs (Table 1 ).
Frequency of AAEs Related to Ignoring DNDs
Sixty-nine AAEs were recorded related to ignoring DNDs (69/1307 [5.3%]; 95% CI, 4.1%–6.5%). Of those, 54 (5.1%) of 1062 were in adult patients (95% CI, 3.8%–6.4%) and 15 (6.1%) of 245 in pediatric patients (95% CI, 3.1%–9.1%). Reviewers considered that 18 (33.3%) of 54 AAE were caused mainly by ignoring DNDs (Table 2 ). In adult patients, the majority of AAEs (51/901; crude rate, 5.7% [95% CI, 4.2%–7.2%]) occurred in patients 65 years or older and 3 in patients younger than 65 years (3/161; crude rate, 1.9% [95% CI, 0%–4.0%]; χ2 = 4.1, P = 0.04).
TABLE 2 -
Extent to Which the AE Was Related to Ignoring DNDs
GP
%
PED
%
Ignoring DNDs is probably one of the causes of the AAE.
21
38.9 (25.9–51.9)
6
40.0 (15.2–64.8)
Ignoring DNDs has probably caused this AAE.
15
27.8 (15.8–39.7)
9
60.0 (35.2–84.8)
Complete assurance that Ignoring DNDs is one of the causes of this AAE.
14
25.9 (14.2–37.6)
0
0
Complete assurance that Ignoring DNDs is the only cause of this AAE.
4
7.4 (0.4–14.4)
0
0
The degree of interevaluator agreement ranged from substantial (0.64–0.76) to a perfect (0.80–1) agreement. During the study, the reviewers shared and agreed on 17 cases with the consultants.
Frequency of AAEs Related to Ignoring DNDs and Sex Differences
Female patients suffered from a greater number of AAEs because of ignoring DNDs than did male patients (crude rate female, 32/583 [5.5%; 95% CI, 3.6–7.3]; crude rate male, 22/430 [5.1%; 95% CI, 3.0–7.2]; χ2 = 0.01, P = 0.9; adjusted rate female, 4.9% [95% CI, 4.4–5.5]; adjusted rate male, 4.0% [95% CI, 3.4–4.7]; χ2 = 3.9, P = 0.047).
Severity of AAEs
Most AAEs were characterized by temporary minor harm, in both, adult patients (n = 28/54 [51.9%]; 95% CI, 38.5%–65.2%) and pediatric patients (n = 15/15 [100%]). No deaths were observed in relation to ignoring DNDs (Table 3 ). The severity of AAEs suffered by male and female patients was similar both in the adult and pediatric populations.
TABLE 3 -
Severity of AAEs Related to Ignoring DNDs
GP (n = 54)
%
PED (n = 15)
%
Minor temporary harm
Females
16/32
57.1
7/7
100
Males
12/22
54.5
8/8
100
Total
28
51.9
15
100
Major temporary harm
Females
6/32
54.5
0
0
Males
5/22
22.7
Total
11
20.4
0
0
Minor permanent harm
Females
9/32
64.3
0
0
Males
5/22
22.7
Total
14
25.9
0
0
Major permanent harm
Females
1/32
100
0
0
Males
0
0
Total
1
1.9
0
0
In adult patients, AAEs with the greatest severity were produced when ignoring the following DNDs: avoiding starting paracetamol treatment in doses of 1 g (4 AAEs with temporary severe harm) and prescribing lipid-lowering agents in patients older than 75 years without previous cardiovascular events (1 AAE with temporary severe harm). There were 11 mild AAEs with permanent harm to the patients associated with using benzodiazepines for the treatment of insomnia in people older than 65 years; 14 mild AAEs with temporary harm associated with prescribing nonsteroidal anti-inflammatory drugs to patients with high blood pressure, heart failure, chronic kidney disease, or liver cirrhosis; and 6 other AAEs associated with using an antibiotic treatment in acute bronchitis when the patient did not suffer from chronic obstructive pulmonary disease (COPD), heart failure, diabetes, pneumonia, tonsillitis, Bordetella infection, or kidney disease.
In pediatric patients, 13 AAEs were identified with mild temporary harm to the minor associated with prescribing antibiotics for pharyngitis or tonsillitis when the patient did not test positive for streptococcus and 2 AAEs associated with ignoring DNDs to not prescribe mucolytics or antibiotics for respiratory tract infections.
The number of AAEs was higher in females than in males in 4 of the 7 DNDs studied (Table 4 ). Statistically significant differences were not observed in any of the cases between the 2 subsamples. Furthermore, in the pediatric subsample, there were no differences in the number of AAEs between males and females (Table 5 ).
TABLE 4 -
Number of AAEs Related for Every DND Comparing Total, Males, and Females (Adult Patients/GPs)
No. Medical Records in Which This DND Was Identified, n/N (%)
No. Patients Suffering From AAE Related to This DND, n/N (%)
Use benzodiazepines for the treatment of insomnia in people older than 65 y (n = 1401)
Females
175/254 (68.9)
13/21 (61.9)
Males
79/245 (32.2)
8/21 (38.1)
Total
254/1401 (18.1)
21/254 (8.3)
Prescribe nonsteroidal anti-inflammatory drugs to patients with high blood pressure, heart failure, chronic kidney disease, or liver cirrhosis (n = 1928)
Females
179/314 (57.0)
11 (64.7)
Males
135/314 (43.0)
6/11 (54.5)
Total
314/1928 (16.3)
17/314 (5.4)
Starter kit paracetamol in doses of 1 g (n = 1928)
Females
343/602 (57.0)
2/6 (33.3)
Males
259/602 (43.0)
4/6 (66.7)
Total
602/1928 (31.2)
6/602 (1.0)
Indicate antibiotic treatment in acute bronchitis when the patient does not have COPD, heart failure, diabetes, pneumonia, tonsillitis, Bordetella infection, or kidney disease or is undergoing active chemotherapy (n = 1928)
Females
71/117 (60.7)
4/6 (66.7)
Males
46/117 (39.3)
2/6 (33.3)
Total
117/1928 (6.1)
6/117 (5.1)
Prescribe lipid-lowering agents in patients older than 75 y without previous cardiovascular events (n = 437)
Females
47/69 (68.1)
1/1 (100)
Males
22/69 (31.9)
0 (0)
Total
69/437 (15.8)
1/69 (1.4)
Screening asymptomatic patients older than 50 y for prostate cancer using PSA (n = 1648)
Males
99/1648 (6.0)
0 (0)
Perform imaging tests in nonspecific low back pain without warning signs before 6 wks (n = 1928)
Females
29/52 (55.8)
1/3 (33.3)
Males
23/52 (44.2)
2/3 (66.7)
Total
52/1928 (2.7)
3/52 (5.8)
COPD, chronic obstructive pulmonary disease.
TABLE 5 -
Number of AAEs for Every DND Comparing Total, Boys, and Girls (Pediatric Population)
No. Medical Records in Which This DND Was Identified, n/N (%)
No. Patients Suffering From AAE Related to This DND, n/N (%)
Do not prescribe antibiotics for pharyngitis or tonsillitis, even if the patient does not test positive for streptococcus
Females
67/141 (47.5)
7/13 (53.8)
Males
74/141 (52.5)
6/13 (46.2)
Total
141/629 (22.4)
13/141 (9.2)
Do not prescribe mucolytics. or antibiotics for respiratory tract infections
Females
57/114 (50.0)
0 (0)
Males
57/114 (50.0)
2/2 (100)
Total
114/629 (18.1)
2 (1.8)
Do not combine or alternate treatment with ibuprofen and acetaminophen
Females
48/97 (49.5)
0 (0)
Males
49/97 (50.5)
0 (0)
Total
97/629 (15.4)
0 (0)
DISCUSSION
Much of the studies have focused on the frequency of the low-value practices measurement and have proved interventions to reduce them. These findings provide a new perspective about the consequences of low-value practices for the patients and health care. Ignoring DNDs could place patients at unnecessary risk. This study is in addition to others that pursue to reduce patient harm as a consequence of inefficiency care due to overuse5,9,10,12,16
These results are suggesting that approximately half of the patients included in this study received treatment or diagnostic test that should not have been indicated to them. Approximately, 1 in 40 of the total patients who were attended in primary care could suffer from an AAE related to such low-value care. Among the pediatric patients, the AEs were slightly higher in number but less severe than those suffered by the adult patients.
These data, in conjunction with those of an Australian study that analyzed the harm related to the selection of low-value care in 225 public hospitals in New South Wales,19
Low-value practices are not only a burden in economic terms. Reducing practices such as those studied here has also the potential to improve outcomes for the patients. Because GPs are located at the starting point of the care process, their decisions can determine the quality of care received.
The potential impact of low-value care on patient safety can be ruled out in light of these findings. Decisions made by doctors in primary care that ignore the principles of the right care could have a negative effect on patients. In some cases, if the patient perceives that the harm could have been avoided, it may contribute to a loss of trust in the physician.27
The DNDs analyzed in this study are similar to those selected in other studies and are recognized as targets for enhancing the quality of care either to analyze the frequency of overuse28 29
To reduce low-value care, various deimplementation strategies have been designed30,31
Limitations and Potential Sources of Bias
This study aimed to identify the frequency and severity of, but not the causes of, overuse related to the DNDs, which may not always be clear-cut cases of overuse. The study design, based on the review of medical records, did not permit us to establish that doing what must not be done by GPs or pediatricians is a direct cause for the occurrence of AEs. However, it is conceivable that if the DND had not been ignored, the AAE would have been less likely to occur.
Our analysis is based on patients’ clinical records; however, the quality of the registered information can vary across physicians. The magnitude of harm related to not adhering to DNDs can vary from mild to severe, thereby introducing a detection bias. It is difficult to determine some of the consequences of AEs (e.g., unnecessary antimicrobial treatment) for only one patient. This study adjudicates whether AEs were avoidable using only chart review. The figure of patients receiving treatments included in the DNDs was limited to the 10 recommendations analyzed. The current data could be worse if additional DNDs were also considered.
Practical Implications
These findings suggest that one measure of the impact of interventions aiming to reduce overuse may lie in the extent of damage related to low-value care. Low-value care can be classified into 3 categories32 30
Future Studies
Female patients underwent a greater number of treatments that ignored DNDs, a finding that requires further attention. The safety events considered in this study have exclusively been focused on physical consequences for patients in the short-term. Other negative consequences from overused services, such as psychological, social, financial, treatment burden, and dissatisfaction with health care,33 34
ACKNOWLEDGMENTS
Technical personnel in charge of the health information systems of the health services that have participated in the study have collaborated selflessly and made this study possible.
REFERENCES
1. Brownlee S, Chalkidou K, Doust J, et al. Evidence for overuse of medical services around the world.
Lancet . 2017;390:156–168.
2. Otoom SA, Sequeira RP. Health care providers’ perceptions of the problems and causes of irrational use of drugs in two Middle East countries.
Int J Clin Pract . 2006;60:565–570.
3. Find NL, Terlizzi R, Munksgaard SB, et al. Medication overuse headache in Europe and Latin America: general demographic and clinical characteristics, referral pathways and national distribution of painkillers in a descriptive, multinational, multicenter study.
J Headache Pain . 2015;17:20.
4. Verkerk EW, Huisman-de Waal G, Vermeulen H, et al. Low-value care in nursing: a systematic assessment of clinical practice guidelines.
Int J Nurs Stud . 2018;87:34–39.
5. Ellen ME, Wilson MG, Vélez M, et al. Addressing overuse of health services in health systems: a critical interpretive synthesis.
Health Res Policy Syst . 2018;16:48.
6. Chassin MR, Galvin RW. The urgent need to improve health care quality. Institute of Medicine National Roundtable on Health Care Quality.
JAMA . 1998;280:1000–1005.
7. Levinson W, Kallewaard M, Bhatia RS, et al. ‘Choosing Wisely’: a growing international campaign.
BMJ Qual Saf . 2015;24:167–174.
8. Morgan DJ, Dhruva SS, Coon ER, et al. 2019 Update on medical overuse: a review.
JAMA Intern Med . 2019;179:1568–1574.
9. Born KB, Levinson W. Choosing Wisely campaigns globally: a shared approach to tackling the problem of overuse in healthcare.
J Gen Fam Med . 2018;20:9–12.
10. Vernero S, Domenighetti G, Bonaldi A. Italy’s “Doing more does not mean doing better” campaign.
BMJ . 2014;349:g4703.
11. Garner S, Littlejohns P. Disinvestment from low value clinical interventions: NICEly done?
BMJ . 2011;343:d4519.
12. Gaspoz JM. Smarter medicine: do physicians need political pressure to eliminate useless interventions?
Swiss Med Wkly . 2015;145:w14125.
13. García-Alegría J, Vázquez-Fernández Del Pozo S, Salcedo-Fernández F, et al. Commitment to quality of the Spanish scientific societies.
Rev Clin Esp . 2017;217:212–221.
14. NICE. Savings and productivity collection. Do not do recommendations [Internet]. 2007. Available at:
https://www.nice.org.uk/savingsandproductivity/collection?page=1&pagesize=2000&type=do%20not%20do . Accessed October 31, 2019.
15. Elshaug AG, Watt AM, Mundy L, et al. Over 150 potentially low-value health care practices: an Australian study.
Med J Aust . 2012;197:556–560.
16. Wammes JJ, van den Akker-van Marle ME, Verkerk EW, et al. Identifying and prioritizing lower value services from Dutch specialist guidelines and a comparison with the UK do-not-do list.
BMC Med . 2016;14:196.
17. Colla CH, Mainor AJ, Hargreaves C, et al. Interventions aimed at reducing use of low-value health services: a systematic review.
Med Care Res Rev . 2017;74:507–550.
18. Maratt JK, Kerr EA, Klamerus ML, et al. Measures used to assess the impact of interventions to reduce low-value care: a systematic review.
J Gen Intern Med . 2019;34:1857–1864.
19. Badgery-Parker T, Pearson SA, Dunn S, et al. Measuring hospital-acquired complications associated with low-value care.
JAMA Intern Med . 2019;179:499–505.
20. Word Health Organization (WHO).
More Than Words. Conceptual Framework for the International Classification for Patient Safety. Final Technical Report January 2009. Geneva, Switzerland: WHO; 2009.
21. Nabhan M, Elraiyah T, Brown DR, et al. What is preventable harm in healthcare? A systematic review of definitions.
BMC Health Serv Res . 2012;12:128.
22. Kost A, Genao I, Lee JW, et al. Clinical decisions made in primary care clinics before and after choosing wisely.
J Am Board Fam Med . 2015;28:471–474.
23. Mira JJ, Caro Mendivelso J, Carrillo I, et al. Low-value clinical practices and harm caused by non-adherence to ‘do not do’ recommendations in primary care in Spain: a Delphi study.
Int J Qual Health Care . 2019;31:519–526.
24. Woods DM, Thomas EJ, Holl JL, et al. Ambulatory care adverse events and preventable adverse events leading to a hospital admission.
Qual Saf Health Care . 2007;16:127–131.
25. Landis JR, Koch GG. The measurement of observer agreement for categorical data.
Biometrics . 1977;33:159–174.
26. Aranaz-Andrés JM, Aibar C, Limón R, et al. A study of the prevalence of adverse events in primary healthcare in Spain.
Eur J Public Health . 2012;22:921–925.
27. Mira JJ, Carrillo I, Silvestre C, et al. Drivers and strategies for avoiding overuse. A cross-sectional study to explore the experience of Spanish primary care providers handling uncertainty and patients’ requests.
BMJ Open . 2018;8:e021339.
28. Kullgren JT, Krupka E, Schachter A, et al. Precommitting to choose wisely about low-value services: a stepped wedge cluster randomised trial.
BMJ Qual Saf . 2018;27:355–364.
29. García-Mochón L, Olry de Labry A, Bermúdez-Tamayo L. Priorización de actividades clínicas no recomendadas en Atención Primaria.
An Sist Sanit Navar . 2017;40:401–412.
30. Alber K, Kuehlein T, Schedlbauer A, et al. Medical overuse and quaternary prevention in primary care—a qualitative study with general practitioners.
BMC Fam Pract . 2017;18:99.
31. Pruskowski JA, Springer S, Thorpe CT, et al. Does deprescribing improve quality of life? A systematic review of the literature.
Drugs Aging . 2019;36:1097–1110.
32. Verkerk EW, Tanke MAC, Kool RB, et al. Limit, lean or listen? A typology of low-value care that gives direction in de-implementation.
Int J Qual Health Care . 2018;30:736–739.
33. Korenstein D, Chimonas S, Barrow B, et al. Development of a conceptual map of negative consequences for patients of overuse of medical tests and treatments.
JAMA Intern Med . 2018;178:1401–1407.
34. Harrison R, Walton M, Manias E, et al. The missing evidence: a systematic review of patients’ experiences of adverse events in health care.
Int J Qual Health Care . 2015;27:424–442.
