Secondary Logo

Journal Logo

A Student-Led, Multifaceted Intervention to Decrease Unnecessary Folate Ordering in the Inpatient Setting

Goetz, Celine; Di Capua, John; Lee, Irene; Mei, Rena; Narula, Sukrit; Zarrin, Sarah; Poeran, Jashvant; Cho, Hyung J.

The Journal for Healthcare Quality (JHQ): September/October 2019 - Volume 41 - Issue 5 - p e54–e60
doi: 10.1097/JHQ.0000000000000177
Original Article

ABSTRACT To reduce unnecessary laboratory testing, a three-phase intervention was designed by students to decrease serum folate laboratory testing in the inpatient setting. These included an educational phase, a personalized feedback phase, and the uncoupling of orders in the electronic medical record. Average monthly serum folate ordering decreased by 87% over the course of the intervention, from 98.4 orders per month at baseline to 12.7 per month in the last phase of the intervention. In the segmented regression analysis, joint ordering of folate and vitamin B12 significantly decreased during the intervention ([INCREMENT]slope = −4.22 tests/month, p = .0089), whereas single ordering of vitamin B12 significantly increased ([INCREMENT]slope = +5.6 tests/month; p < .001). Our intervention was successful in modifying ordering patterns to decrease testing for a deficiency that is rare in the U.S. population.

For more information on this article, contact Hyung J. Cho at

The authors declare no conflicts of interest.

All authors had access to the data and played a role in writing this manuscript.

Back to Top | Article Outline


Low-value healthcare is increasingly recognized as a target to reduce costs and patient harm, as signified by the Choosing Wisely campaign.1–4 To curb overutilization, the Choosing Wisely campaign has highlighted unnecessary laboratory testing designated by professional medical societies.1 However, implementation of Choosing Wisely recommendations has been challenging.5

An often-unnecessary laboratory test is serum folate, primarily measured when evaluating a patient for macrocytic anemia.6,7 Before mandated grain fortification in the United States, approximately 16% of the U.S. population had low folate levels. In 1998, folate fortification of enriched grains in the United States was mandated to decrease the risk of neural tube defects,8 and subsequently, the prevalence of low folate levels decreased to <1%.9 Given the low prevalence in the population, the utility and appropriateness of testing patients for folate deficiency has been questioned. Moreover, the test itself is a poor marker of a patient's actual folate stores.10–15 It has been suggested that empirically treating patients with a high clinical suspicion of folate deficiency with folate supplementation (few cents per pill) or simply ensuring that the patient has a folate-rich diet are more cost-effective strategies than testing.10

With a robust literature suggesting that testing folate levels is largely unnecessary, we aimed to reduce serum folate testing among hospitalists through a multifaceted project. Our goal was to develop an effective strategy for modifying physician ordering behavior with the hope that this could be applied more widely to curb unnecessary laboratory testing.

Back to Top | Article Outline


This intervention took place at an urban, 1,134-bed, academic medical center under the Student High Value Care Initiative. The project team consisted of five medical students and one faculty mentor. It was implemented from December 2015 to December 2016 on all patients under the care of hospitalists. Hospitalists are the attending physicians for the general medicine teaching services and for attending directed services that may include nurse practitioners and physician assistants. House staff (with hospitalist oversight) place orders on the general medicine teaching services, and both hospitalists and advanced practitioners place orders on the attending directed service. All folate and vitamin B12 orders are ordered through the electronic medical record (EMR), Epic Systems Corporation (Verona, WI).

Practitioners could order vitamin B12 and folate levels in the following ways at the beginning of the intervention. In the EMR, there was a “joint order” that appeared as the default when a practitioner typed in orders for either vitamin B12 or folate. Both tests would be simultaneously ordered in this case. Vitamin B12 and folate tests could be ordered as separate orders. We analyzed whether practitioners placed orders through the joint order set, whether they ordered the separate vitamin B12 and folate tests simultaneously, or whether they ordered just one test at a specific time point.

The intervention was rolled out in three phases, the first of which was educational. In December 2015, informative didactics were integrated into educational conferences for internal medicine house staff, nurse practitioners, physician assistants, and attending physicians for the first 7 months of the intervention. Teaching points focused on the low prevalence of folate deficiency in the U.S. population and data available for specific disease states previously thought to be prone to deficiency (i.e., dementia, inflammatory bowel disease, alcoholism, etc.). The key points were reinforced with posters in the house staff workroom and with information cards attached to boxes of popular brands of cereal (fortified with folate) handed out during resident sign-out in the morning. The slogan “Don't be a cereal folate orderer” was promoted, relaying information with humor to prompt behavior change.

During the second phase of the intervention, 7 months after Phase 1 of our educational intervention began, we evaluated the frequency of testing of individual clinicians. We then crafted targeted feedback e-mails to hospitalists (n = 21) and house staff (n = 59) and disclosed their individual ordering patterns in comparison to the entire division.

In developing the third phase of the intervention, we analyzed ordering patterns and noted that a high percentage of folate tests were ordered as part of a joint order set in the hospital's EMR described above. We worked with our institution's laboratory utilization director, medicine quality leadership, and our clinician informaticist to remove the joint order. The individual folate order remained in the EMR. Collaborating with hematology colleagues and colleagues at a peer institution, we drafted an advisory warning on the folate laboratory order. When folate was ordered in the EMR, the following language appeared: “The High Value Care Committee recommends no folate testing. Folate deficiency occurs in <1% of the population. The only indication to test folate levels is when MCV >130. Consider empirically treating high risk patients with folic acid.” These EMR integrations went into effect at the intervention's 10-month mark on October 1, 2016, at which time the joint folate–vitamin B12 order was removed from the EMR.

Electronic medical record ordering data were gathered from Epic. All folate and vitamin B12 orders completed under hospitalists attendings from February 2015 to December 2016 were included in the analysis. The following trends in monthly folate and vitamin B12 orders were evaluated: (1) joint folate–vitamin B12 ordering via the order set, (2) simultaneous folate and vitamin B12 level ordering (i.e., two single orders at the same time), (3) single serum folate ordering without concomitant vitamin B12 ordering, and (4) serum vitamin B12 testing without concomitant folate ordering. After all phases of the intervention, we applied a quasiexperimental interrupted time series design with segmented regression analysis to estimate any changes in ordering patterns compared to the preintervention period (February 2015 to November 2015; post-intervention December 2015 to December 2016), while adjusting for preintervention trends.

To calculate cost comparisons, we used the 2016 Clinical Diagnostic Laboratory Fee Schedule from Centers for Medicare and Medicaid Services, which quotes serum folate assay at $20.03 and vitamin B12 at $20.54. We applied these charges to the number of orders in each phase of the intervention. This project was submitted to the Department of Medicine Quality Improvement Committee and determined to be a quality improvement project rather than research, and thus an institutional review board submission was not required.

Back to Top | Article Outline


Over the course of the preintervention and postintervention periods, 1,582 orders for folate and 2,056 orders for vitamin B12 were ordered on 1,781 patients (54.2% women and 45.8% men; mean age, 65.8 years). No changes in the length of stay (mean, 10.8 days) were observed during the study period. Sixty-one percent of all orders were from residents on general medicine teaching services, and 39% of all orders were performed by attendings or advanced practitioners on the attending directed service.

Figure 1 shows the decrease in folate orders at each stage of the intervention, whereas Table 1 describes the average testing per month and percent changes for the order categories at each stage of the intervention. The greatest relative reductions were seen for the joint serum folate/vitamin B12 order set: −38%, −50%, and −100% in Phases 1, 2, and 3, respectively (all changes p < .05 when using t-tests and not taking into account preintervention trends). Single serum vitamin B12 orders (i.e., without accompanying folate orders) had the steepest increase in Phase 3.

Table 1

Table 1

Figure 2 shows the results from the segmented regression where the intervention period is compared to baseline while adjusting for any preintervention trends. Separate plots were created for each of the order types. Confirming the results from Table 1, the steepest decrease was seen for joint serum folate/vitamin B12 order set ([INCREMENT]slope = −4.2 orders/month; p < .05), whereas a significant increase was seen for single serum vitamin B12 orders ([INCREMENT]slope = 5.6 orders/month; p < .05), suggesting that serum vitamin B12 was often ordered using the combined order set when just vitamin B12 levels were needed. Patterns of ordering single folate levels and simultaneous single orders of vitamin B12 and folate did not change significantly over the course of the intervention.

To calculate cost savings from the project, we compared average monthly costs before the intervention ($4,123.54) to the average monthly costs during the third phase of the intervention ($1,896.91). This extrapolated to savings of $2,226.63 per month or $26,719.56 per year.

Of the 1,550 folate tests ordered in both the preintervention and intervention periods, the folate level of 21 patients was below our laboratories normal limit (<3.5 ng/ml)—11 during the preintervention period and 10 during the intervention. During the preintervention period, 136 folate tests were ordered for every one abnormal result, whereas 42 tests were ordered for an abnormal result during the intervention period. Of these patients with low folate levels, 5 of the 21 patients were admitted for alcohol withdrawal, 7 had elevated mean corpuscular volume, and 12 patients were subsequently treated with folate supplementation.

Back to Top | Article Outline


Our project has several limitations. We acknowledge that our cost analysis is limited because it is only based on Medicare and Medicaid charges. It is challenging to determine labor costs for one laboratory test, and the effect of one test's ordering pattern on patient care and the length of stay. Accordingly, our financial calculations likely do not reflect the entire savings rendered by this change in ordering pattern.

One of the drawbacks of our multiphase intervention was our inability to isolate the impact and sustainability of the individual strategies. The phases built on each other and ultimately all contributed to the intervention's success. We also acknowledge that our scope was limited by the single-institution, single-division design. Finally, although a few institutions were able to achieve institutional buy-in to eliminate front-line clinicians' routine ability to order folate levels,16 we were unable to gain approval. Forcing functions, or eliminating the ability to test, is undoubtedly the most effective for reducing low-value care. However, we sense that most other institutional workgroups are in a similar situation to ours, and we hope that this multifaceted intervention will serve as a more practical model to reduce testing.

Back to Top | Article Outline


Our intervention successfully decreased folate testing by 87%. Although other interventions aimed at folate ordering reduction have been reported,17,18 we demonstrate a more dramatic reduction compared to the studies in Turkey and Israel, which showed reductions of 68% and 58%, respectively. Additionally, the recent U.S. study by Eaton et al19 did not demonstrate a significant change from their intervention compared to their previous downward trend. A recent Canadian study targeted red blood cell folate testing rather than serum folate testing, described as the more prevalent folate testing. A 94.4% reduction was achieved with a forcing function EMR intervention, where only hematologists and gastroenterologists could order the red blood cell folate test.16

A distinguishing feature of our intervention was how it galvanized the insight and enthusiasm of medical students and house staff. The student-led education and publicity campaign was impressively effective and facilitated engagement of front-line clinicians through a “bottom-up” process—a phenomenon that has instrumental in implementing high-value care.20 Students passing out small cereal boxes in the resident room with the slogan, “Don't be a cereal folate orderer,” was an unintimidating approach of quality improvement education at time when burnout and administrative burden remains high.

Understanding the impact of EMR order set on ordering behavior was a key factor in the success of our intervention. After the uncoupling of the order set, single ordering of vitamin B12 levels without concurrent folate levels increased significantly, and simultaneous ordering of individual vitamin B12 and folate levels did not change. In effect, the EMR change significantly reduced folate testing without any change in vitamin B12 testing.

The uncoupling process took extensive administrative effort and several months to achieve, but it was integral for achieving sustainability. Data collected 1 year following the end of our intervention has confirmed sustainability. An average of 12.3 folate laboratory tests were ordered per month from January 2017 to December 2017, representing a similar number of average monthly folate laboratory tests compared to when the intervention was completed. Still, we do not believe uncoupling in isolation is an effective approach. Our education with targeted feedback laid the groundwork for uncoupling to be effective. It solidified clinician understanding of the lack of evidence of folate testing to avoid workarounds in EMR.21 This combined strategy was effective and may be generalizable to other settings (nonacademic, outpatient) and to other discipline teams (i.e., gastroenterology, neurology).

Keeping in mind that the folate test itself is prone to inaccurately reflecting the actual folate stores in the body,10–15 our incidence of low folate values was higher than previously reported.7 Our laboratory reports low folate as “<3.5 ng/ml” and does not specify lower values. There is no standard normal value, in part because levels measured by this test often do not reflect the body's actual folate stores.10 The normal value cutoff reported in other institutions range from <3.0 to <1.5 ng/ml, which are lower.14 Additionally, in our analysis, one-third of low folate levels did not have macrocytosis, and approximately one-fourth of low levels were associated with hospitalizations for alcohol withdrawal, wherein falsely lower levels have been reported with high alcohol intake.10

Back to Top | Article Outline


Unnecessary folate laboratory testing is widespread and an important target for reduction. Our project successfully demonstrated 87% reduction in inpatient folate laboratory testing and $26,719.56 cost savings per year among hospitalist teams. We utilized a multifaceted intervention including education, targeted feedback, and EMR change. This was achieved through a bottom-up, student-led process, with engagement of front-line clinicians. Given the prevalence of unnecessary folate testing and its associated costs, further models for reducing overtesting are needed.

Back to Top | Article Outline


Education, clinician-targeted feedback, and EMR change is a practical approach in curbing clinician folate ordering, particularly in settings where it is difficult to obtain institutional buy-in to prohibit front-line clinicians' ability to order. Order sets may be a strong driver of overtesting in many institutions, but without additional education and feedback to change clinicians' understanding, EMR changes alone may not be as effective. Further research is needed to examine this intervention in other settings and disciplines, as well as the impact and sustainability of each phase alone.

Back to Top | Article Outline


The authors would like to thank Nicole Zubizarreta, MPH for her assistance with the statistical analysis.

Back to Top | Article Outline


1. American Board of Internal Medicine Foundation. Choosing wisely. Accessed March 7, 2017.
2. Emanuel EJ, Fuchs VR. The perfect storm of overutilization. JAMA. 2008;299(23):2789–2791.
3. Grady D, Redberg RF. Less is more: How less health care can result in better health. Arch Intern Med. 2010;170(9):749–750.
4. McCarthy D, How SKH, Fryer AK, Radley D, Schoen C. Why not the best? Results from the National Scorecard on U.S. Health System Performance. 2011. Accessed March 7, 2017.
5. Rosenberg A, Agiro A, Gottlieb M, et al. Early trends among seven recommendations from the choosing wisely campaign. JAMA Intern Med. 2015;175(12):1913.
6. Theisen-Toupal J, Horowitz G, Breu A. Low yield of outpatient serum folate testing: Eleven years of experience. JAMA Intern Med. 2014;174(10):1696–1697.
7. Theisen-Toupal J, Horowitz GL, Breu AC. Utility, charge, and cost of inpatient and emergency department serum folate testing. J Hosp Med. 2013;8(2):91–95.
8. Food and Drug Administration. Food Standards: Amendment of Standards of Identity for Enriched Grain Products to Require Addition of Folic Acid: Food and Drug Administration; 1996:8781–8789.
9. Pfeiffer CM, Hughes JP, Lacher DA, et al. Estimation of trends in serum and RBC folate in the U.S. population from pre- to postfortification using assay-adjusted data from the NHANES 1988–2010. J Nutr. 2012;142(5):886–893.
10. Breu AC, Theisen-Toupal J, Feldman LS. Serum and red blood cell folate testing on hospitalized patients. J Hosp Med. 2015;10(11):753–755.
11. Gilfix BM. Utility of measuring serum or red blood cell folate in the era of folate fortification of flour. Clin Biochem. 2014;47(7-8):533–538.
12. Ashraf MJ, Cook JR, Rothberg MB. Clinical utility of folic acid testing for patients with anemia or dementia. J Gen Intern Med. 2008;23(6):824–826.
13. Latif T, Hsi ED, Rybicki LA, Adelstein DJ. Is there a role for folate determinations in current clinical practice in the USA? Clin Lab Haematol. 2004;26(6):379–383.
14. Robinson AR, Mladenovic J. Lack of clinical utility of folate levels in the evaluation of macrocytosis or anemia. Am J Med. 2001;110(2):88–90.
15. Shojania AM, von Kuster K. Ordering folate assays is no longer justified for investigation of anemias, in folic acid fortified countries. BMC Res Notes. 2010;3:22.
16. MacMillan TE, Gudgeon P, Yip PM, Cavalcanti RB. Reduction in unnecessary red blood cell folate testing by restricting computerized physician order entry in the electronic health record. Am J Med. 2018;131:939–944.
17. Yilmaz FM, Kahveci R, Aksoy A, et al. Impact of laboratory test use strategies in a Turkish hospital. PLoS One. 2016;11(4):e0153693.
18. Kahan NR, Waitman DA, Vardy DA. Curtailing laboratory test ordering in a managed care setting through redesign of a computerized order form. Am J Manag Care. 2009;15:173–176.
19. Eaton KP, Chida N, Apfel A, et al. Impact of nonintrusive clinical decision support systems on laboratory test utilization in a large academic centre. J Eval Clin Pract. 2018;24(3):474–479.
20. Moriates C, Wong BM. High-value care programmes from the bottom-up… and the top-down. BMJ Qual Saf. 2016;25(11):821–823.
21. Blijleven V, Koelemeijer K, Wetzels M, Jaspers M. Workarounds emerging from electronic health record system usage: Consequences for patient safety, effectiveness of care, and efficiency of care. JMIR Hum Factors. 2017;4(4):e27.
Back to Top | Article Outline

Authors' Biographies

Celine Goetz, MD, is an Assistant Professor of Internal Medicine and Director of Education for the Division of Hospital Medicine at the Rush University Medical Center in Chicago, IL.

John Di Capua, MHS, is a medical student at the Icahn School of Medicine at Mount Sinai in New York, NY.

Irene Lee, BA, is a medical student at the Icahn School of Medicine at Mount Sinai in New York, NY.

Rena Mei, BS, is a medical student at the Icahn School of Medicine at Mount Sinai in New York, NY.

Sukrit Narula, BS, is a medical student at the Icahn School of Medicine at Mount Sinai in New York, NY.

Sarah Zarrin, BA, is a medical student at the Icahn School of Medicine at Mount Sinai in New York, NY.

Jashvant Poeran, MD, PhD, is an Assistant Professor of Population Health Science and Policy, Assistant Professor of Medicine, and Assistant Professor of Orthopedics at the Icahn School of Medicine at Mount Sinai in New York, NY.

Hyung J. Cho, MD, is Director of Quality, Safety and Value for Division of Hospital Medicine at the Icahn School of Medicine at Mount Sinai in New York, NY. He is also Senior Fellow at the Lown Institute in Brookline, MA.


folate; overuse; high-value care; laboratory testing

© 2019 National Association for Healthcare Quality