Figure 2 shows the results from the segmented regression where the intervention period is compared to baseline while adjusting for any preintervention trends. Separate plots were created for each of the order types. Confirming the results from Table 1, the steepest decrease was seen for joint serum folate/vitamin B12 order set ([INCREMENT]slope = −4.2 orders/month; p < .05), whereas a significant increase was seen for single serum vitamin B12 orders ([INCREMENT]slope = 5.6 orders/month; p < .05), suggesting that serum vitamin B12 was often ordered using the combined order set when just vitamin B12 levels were needed. Patterns of ordering single folate levels and simultaneous single orders of vitamin B12 and folate did not change significantly over the course of the intervention.
To calculate cost savings from the project, we compared average monthly costs before the intervention ($4,123.54) to the average monthly costs during the third phase of the intervention ($1,896.91). This extrapolated to savings of $2,226.63 per month or $26,719.56 per year.
Of the 1,550 folate tests ordered in both the preintervention and intervention periods, the folate level of 21 patients was below our laboratories normal limit (<3.5 ng/ml)—11 during the preintervention period and 10 during the intervention. During the preintervention period, 136 folate tests were ordered for every one abnormal result, whereas 42 tests were ordered for an abnormal result during the intervention period. Of these patients with low folate levels, 5 of the 21 patients were admitted for alcohol withdrawal, 7 had elevated mean corpuscular volume, and 12 patients were subsequently treated with folate supplementation.
Our project has several limitations. We acknowledge that our cost analysis is limited because it is only based on Medicare and Medicaid charges. It is challenging to determine labor costs for one laboratory test, and the effect of one test's ordering pattern on patient care and the length of stay. Accordingly, our financial calculations likely do not reflect the entire savings rendered by this change in ordering pattern.
One of the drawbacks of our multiphase intervention was our inability to isolate the impact and sustainability of the individual strategies. The phases built on each other and ultimately all contributed to the intervention's success. We also acknowledge that our scope was limited by the single-institution, single-division design. Finally, although a few institutions were able to achieve institutional buy-in to eliminate front-line clinicians' routine ability to order folate levels,16 we were unable to gain approval. Forcing functions, or eliminating the ability to test, is undoubtedly the most effective for reducing low-value care. However, we sense that most other institutional workgroups are in a similar situation to ours, and we hope that this multifaceted intervention will serve as a more practical model to reduce testing.
Our intervention successfully decreased folate testing by 87%. Although other interventions aimed at folate ordering reduction have been reported,17,18 we demonstrate a more dramatic reduction compared to the studies in Turkey and Israel, which showed reductions of 68% and 58%, respectively. Additionally, the recent U.S. study by Eaton et al19 did not demonstrate a significant change from their intervention compared to their previous downward trend. A recent Canadian study targeted red blood cell folate testing rather than serum folate testing, described as the more prevalent folate testing. A 94.4% reduction was achieved with a forcing function EMR intervention, where only hematologists and gastroenterologists could order the red blood cell folate test.16
A distinguishing feature of our intervention was how it galvanized the insight and enthusiasm of medical students and house staff. The student-led education and publicity campaign was impressively effective and facilitated engagement of front-line clinicians through a “bottom-up” process—a phenomenon that has instrumental in implementing high-value care.20 Students passing out small cereal boxes in the resident room with the slogan, “Don't be a cereal folate orderer,” was an unintimidating approach of quality improvement education at time when burnout and administrative burden remains high.
Understanding the impact of EMR order set on ordering behavior was a key factor in the success of our intervention. After the uncoupling of the order set, single ordering of vitamin B12 levels without concurrent folate levels increased significantly, and simultaneous ordering of individual vitamin B12 and folate levels did not change. In effect, the EMR change significantly reduced folate testing without any change in vitamin B12 testing.
The uncoupling process took extensive administrative effort and several months to achieve, but it was integral for achieving sustainability. Data collected 1 year following the end of our intervention has confirmed sustainability. An average of 12.3 folate laboratory tests were ordered per month from January 2017 to December 2017, representing a similar number of average monthly folate laboratory tests compared to when the intervention was completed. Still, we do not believe uncoupling in isolation is an effective approach. Our education with targeted feedback laid the groundwork for uncoupling to be effective. It solidified clinician understanding of the lack of evidence of folate testing to avoid workarounds in EMR.21 This combined strategy was effective and may be generalizable to other settings (nonacademic, outpatient) and to other discipline teams (i.e., gastroenterology, neurology).
Keeping in mind that the folate test itself is prone to inaccurately reflecting the actual folate stores in the body,10–15 our incidence of low folate values was higher than previously reported.7 Our laboratory reports low folate as “<3.5 ng/ml” and does not specify lower values. There is no standard normal value, in part because levels measured by this test often do not reflect the body's actual folate stores.10 The normal value cutoff reported in other institutions range from <3.0 to <1.5 ng/ml, which are lower.14 Additionally, in our analysis, one-third of low folate levels did not have macrocytosis, and approximately one-fourth of low levels were associated with hospitalizations for alcohol withdrawal, wherein falsely lower levels have been reported with high alcohol intake.10
Unnecessary folate laboratory testing is widespread and an important target for reduction. Our project successfully demonstrated 87% reduction in inpatient folate laboratory testing and $26,719.56 cost savings per year among hospitalist teams. We utilized a multifaceted intervention including education, targeted feedback, and EMR change. This was achieved through a bottom-up, student-led process, with engagement of front-line clinicians. Given the prevalence of unnecessary folate testing and its associated costs, further models for reducing overtesting are needed.
Education, clinician-targeted feedback, and EMR change is a practical approach in curbing clinician folate ordering, particularly in settings where it is difficult to obtain institutional buy-in to prohibit front-line clinicians' ability to order. Order sets may be a strong driver of overtesting in many institutions, but without additional education and feedback to change clinicians' understanding, EMR changes alone may not be as effective. Further research is needed to examine this intervention in other settings and disciplines, as well as the impact and sustainability of each phase alone.
The authors would like to thank Nicole Zubizarreta, MPH for her assistance with the statistical analysis.
2. Emanuel EJ, Fuchs VR. The perfect storm of overutilization. JAMA. 2008;299(23):2789–2791.
3. Grady D, Redberg RF. Less is more: How less health care can result in better health. Arch Intern Med. 2010;170(9):749–750.
5. Rosenberg A, Agiro A, Gottlieb M, et al. Early trends among seven recommendations from the choosing wisely campaign. JAMA Intern Med. 2015;175(12):1913.
6. Theisen-Toupal J, Horowitz G, Breu A. Low yield of outpatient serum folate
testing: Eleven years of experience. JAMA Intern Med. 2014;174(10):1696–1697.
7. Theisen-Toupal J, Horowitz GL, Breu AC. Utility, charge, and cost of inpatient and emergency department serum folate
testing. J Hosp Med. 2013;8(2):91–95.
9. Pfeiffer CM, Hughes JP, Lacher DA, et al. Estimation of trends in serum and RBC folate
in the U.S. population from pre- to postfortification using assay-adjusted data from the NHANES 1988–2010. J Nutr. 2012;142(5):886–893.
10. Breu AC, Theisen-Toupal J, Feldman LS. Serum and red blood cell folate
testing on hospitalized patients. J Hosp Med. 2015;10(11):753–755.
11. Gilfix BM. Utility of measuring serum or red blood cell folate
in the era of folate
fortification of flour. Clin Biochem. 2014;47(7-8):533–538.
12. Ashraf MJ, Cook JR, Rothberg MB. Clinical utility of folic acid testing for patients with anemia or dementia. J Gen Intern Med. 2008;23(6):824–826.
13. Latif T, Hsi ED, Rybicki LA, Adelstein DJ. Is there a role for folate
determinations in current clinical practice in the USA? Clin Lab Haematol. 2004;26(6):379–383.
14. Robinson AR, Mladenovic J. Lack of clinical utility of folate
levels in the evaluation of macrocytosis or anemia. Am J Med. 2001;110(2):88–90.
15. Shojania AM, von Kuster K. Ordering folate
assays is no longer justified for investigation of anemias, in folic acid fortified countries. BMC Res Notes. 2010;3:22.
16. MacMillan TE, Gudgeon P, Yip PM, Cavalcanti RB. Reduction in unnecessary red blood cell folate
testing by restricting computerized physician order entry in the electronic health record. Am J Med. 2018;131:939–944.
17. Yilmaz FM, Kahveci R, Aksoy A, et al. Impact of laboratory test use strategies in a Turkish hospital. PLoS One. 2016;11(4):e0153693.
18. Kahan NR, Waitman DA, Vardy DA. Curtailing laboratory test ordering in a managed care setting through redesign of a computerized order form. Am J Manag Care. 2009;15:173–176.
19. Eaton KP, Chida N, Apfel A, et al. Impact of nonintrusive clinical decision support systems on laboratory test utilization in a large academic centre. J Eval Clin Pract. 2018;24(3):474–479.
20. Moriates C, Wong BM. High-value care
programmes from the bottom-up… and the top-down. BMJ Qual Saf. 2016;25(11):821–823.
21. Blijleven V, Koelemeijer K, Wetzels M, Jaspers M. Workarounds emerging from electronic health record system usage: Consequences for patient safety, effectiveness of care, and efficiency of care. JMIR Hum Factors. 2017;4(4):e27.
Celine Goetz, MD, is an Assistant Professor of Internal Medicine and Director of Education for the Division of Hospital Medicine at the Rush University Medical Center in Chicago, IL.
John Di Capua, MHS, is a medical student at the Icahn School of Medicine at Mount Sinai in New York, NY.
Irene Lee, BA, is a medical student at the Icahn School of Medicine at Mount Sinai in New York, NY.
Rena Mei, BS, is a medical student at the Icahn School of Medicine at Mount Sinai in New York, NY.
Sukrit Narula, BS, is a medical student at the Icahn School of Medicine at Mount Sinai in New York, NY.
Sarah Zarrin, BA, is a medical student at the Icahn School of Medicine at Mount Sinai in New York, NY.
Jashvant Poeran, MD, PhD, is an Assistant Professor of Population Health Science and Policy, Assistant Professor of Medicine, and Assistant Professor of Orthopedics at the Icahn School of Medicine at Mount Sinai in New York, NY.
Hyung J. Cho, MD, is Director of Quality, Safety and Value for Division of Hospital Medicine at the Icahn School of Medicine at Mount Sinai in New York, NY. He is also Senior Fellow at the Lown Institute in Brookline, MA.
Keywords:© 2019 National Association for Healthcare Quality
folate; overuse; high-value care; laboratory testing