The Health Information Technology for Economic and Clinical Health (HITECH) Act, enacted as part of the American Recovery and Reinvestment Act of 2009, became law in February 2009 as a means to incentivize hospitals to use information technology (IT).1 The goal of the act is to promote the adoption and meaningful use of a certified electronic health record (EHR) by hospitals and other providers. In May 2011, hospitals began receiving incentive payments from the Centers for Medicare and Medicaid Services (CMS) for meeting the criteria of meaningful use of a certified EHR.2 There is some evidence that use of health IT is associated with increased quality of care3,4 and efficiency.5 Although the meaningful use of EHRs has the potential to reduce the overall cost of health care in the United States by improving efficiencies in the availability of information, improving the coordination of care, and reducing medical errors, this has not yet been rigorously evaluated.6,7
Additionally, the Patient Protection and Affordable Care Act of 2010 required the CMS to initiate a Hospital Value-Based Purchasing (Hospital VBP) Program in an effort to move from a passive payer of hospital claims to an active purchaser of high-quality health care.8 The Hospital VBP Program placed 1% of base Medicare Inpatient Prospective Payment System (IPPS) reimbursement at risk starting with October 2012 discharges. The CMS designed the program to be budget-neutral, and the program distributes the available amount of value-based incentive payments to hospitals, based on total performance scores on the Hospital VBP measures. In federal fiscal year (FFY) 2013, the Hospital VBP Program measured hospital total performance in two domains: clinical process of care, comprising 12 clinical process of care measures; and patient experience of care, comprising 8 Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey measures. The relative performance of these two scores was used to calculate the proportion of the at-risk Medicare payments received under the Hospital VBP Program.9
The enactment of these two laws within the span of two years means that hospitals are working to simultaneously implement and integrate advanced EHRs into the day-to-day workflows of patient care and improve their quality and patient satisfaction scores. Although there is some evidence that EHR use improves hospital quality of care,3 critics claim that EHRs decrease patient satisfaction in the hospital, thus offsetting the potentially positive impact on clinical quality. Furthermore, the extent to which hospitals use the full capabilities of EHRs varies significantly. To our knowledge, there are no studies that have concurrently evaluated the impact of hospitals’ advanced use of EHRs with quality of care and the patient experience as measured by the CMS’s Hospital VBP Program. The objectives of this study were to evaluate the association between advanced EHR use with hospital quality of care and patient satisfaction for U.S. hospitals as measured by the CMS Hospital VBP Program composite process and experience of care scores, evaluate whether the level of sophistication of EHR use was independently associated with hospital quality of care and patient satisfaction, and determine whether the relationships between advanced EHR use with quality and patient satisfaction were similar for teaching hospitals.
We conducted a retrospective, cross-sectional study in 2012 that compared estimated hospital scores used for calculating the at-risk portion of the base Medicare IPPS payment under the FFY 2013 Hospital VBP Program. The estimated clinical process of care and patient experience of care scores were compared between hospitals that did and that did not use an advanced EHR.
Sample and setting
We included all hospitals in the United States that were eligible to participate in the CMS Hospital VBP Program for FFY 2013.10,11 Hospitals that were subject to payment reductions under the Hospital Inpatient Quality Reporting Program were excluded from the 2013 Hospital VBP Program and, therefore, had no estimated Hospital VBP Program scores. For example, if a hospital was cited for deficiencies during the performance period of the program that posed immediate jeopardy to the health or safety of patients, it was excluded from the Hospital VBP Program. Hospitals were also excluded from the program if they did not have a minimum number of cases (10 cases for each of at least four applicable clinical process measures) or surveys (100 completed HCAHPS surveys) during the performance period. In addition, hospitals with insufficient data for calculating both an achievement and improvement score for an individual measure’s score were excluded from the Hospital VBP Program because they did not have Hospital VBP scores.
The Healthcare Information Management Systems Society (HIMSS) Analytics launched an Electronic Medical Record Adoption Model (EMRAM) to track adoption of electronic medical record applications within hospitals and health systems. The EMRAM scores hospitals on their progress in completing the eight stages to create a paperless patient record environment. We used the EMRAM adoption level, which we retrieved from the HIMSS Analytics Web site,12 as a proxy for advanced EHR use.12
Definition of measures
Estimated clinical process of care and patient experience of care scores were calculated by the American Hospital Association (AHA) using data from Hospital Compare for April 2008 to March 2010 and the formulas publicly available from the CMS.9 The clinical process of care domain scores was calculated by summing the higher of the achievement or improvement score for each process of care measure that the hospital reported. Each hospital reported performance for up to 12 process of care measures, depending on the specific services provided by the hospital. The patient experience of care domain score was calculated by summing the higher of the achievement or improvement scores for each of the eight HCAHPS dimensions and a consistency score. The consistency score is based on the distance that the lowest HCAHPS score is from the achievement threshold for that particular HCAHPS dimension. The clinical process of care domain does not have a consistency component, because hospitals report performance only for services that they provide, and this results in variation in the number of measures reported per hospital. Therefore, a consistency measure could unevenly affect hospitals reporting different numbers of measures.13 Whereas the Hospital VBP Program uses the clinical process of care and patient experience of care domain performance scores to calculate a total performance score on which payment determinations are made, we used the individual domain scores, so that we could evaluate the impact of advanced EHR use on clinical quality and patient experience separately.
The patient experience of care score was a composite score calculated using the eight required experience of care dimensions from HCAHPS. For each domain (process of care and patient experience), hospitals are given points for both achievement and improvement in each measure, and the greater set of points is used in calculating the scores. The patient experience domain also includes consistency of performance to calculate the score.9 The AHA estimated the clinical process of care and experience of care scores for the first implementation year of the CMS Hospital VBP Program based on data available from Hospital Compare and the published final rule for program implementation.9 Estimated scores were calculated using a baseline period of April 2008 to March 2009 (December 2009 release of Hospital Compare data) and a performance period of April 2009 to March 2010 (December 2010 release of Hospital Compare data).14
We defined advanced EHR use in two ways using the HIMSS Analytics EMRAM levels.12 The first was a dichotomous variable that indicated whether or not the hospital had a Stage 6 or 7 EHR. Stage 6 EHR use is defined as the use of physician documentation including structured templates, full clinical decision support systems with variance and compliance reporting capabilities, and full radiology picture archive and communication systems. Stage 7 EHR use is defined as a complete EHR with clinical information being readily shared via standardized electronic transactions, data warehousing, and data continuity with the emergency department and other ambulatory care departments. The second variable was a categorical variable that indicated the level of EHR use (nonadvanced EHR, Stage 6 EHR, or Stage 7 EHR).
Hospital characteristics included total bed size, total annual facility admissions, teaching status (teaching hospital or not), health care system status (member of a system or not), type of organization control (government, for-profit, or not-for-profit), and geographic region. These variables were gathered from the 2009 AHA Annual Survey.15
We used descriptive statistics (means and standard deviations, frequency distributions) to describe each variable. We then used independent-samples t tests and one-way analysis of variance to test the bivariate relationships between process and experience of care scores with each of the independent variables. We fit generalized linear regression models with a log link function and gamma distribution to test the relationships between process and experience of care scores with advanced EHR use, controlling for hospital characteristics. We used a modified Park test to select the appropriate mean-variance relationship for the models.16 We used Microsoft Excel for Mac 2011, version 14.1.2, for data management; and SAS version 9.2 (SAS Inc., Cary, North Carolina) for statistical analysis.
Of the 2,988 hospitals with estimated VBP scores, 248 (8.3%) were classified as advanced EHR users (HIMSS Stage 6 or Stage 7), whereas the remaining hospitals had less advanced EHRs. Table 1 describes the sample of hospitals with and without advanced EHR use. Hospitals with advanced EHRs were larger, with more beds and more admissions per year (P < .001). In addition, a larger proportion of hospitals with advanced EHR use were part of a health care system (P = .002) and were classified as not-for-profit (P < .001). There was significant variation in the proportion of hospitals classified as advanced EHR users by geographic region, with the Pacific region having the highest proportion and the Mountain and East South Central regions having the lowest proportions of advanced users. The mean AHA-estimated clinical process of care score was higher for hospitals with advanced EHRs than for hospitals with nonadvanced EHRs (41.8 versus 38.3, P = .004), whereas the mean AHA-estimated experience of care score was lower for hospitals with advanced EHRs than for hospitals with nonadvanced EHRs (29.4 versus 35.1, P < .001) (see Table 1).
Table 2 reports the mean AHA-estimated process of care and experience of care scores for each hospital characteristic. In the bivariate analyses, process of care and experience of care scores were associated with each of the hospital characteristics. Table 3 reports results of the generalized linear regression models for process of care and experience of care. After controlling for hospital characteristics, hospitals classified as advanced EHR users had 4.21-point-higher estimated process of care scores compared with nonadvanced users (Model 1). Furthermore, system hospitals were associated with 5.17-point-higher process of care scores, and for-profit hospitals were associated with 9.72-point-higher process of care scores. Advanced EHR use was not associated with estimated experience of care scores, after controlling for hospital characteristics. System membership was associated with 2.09-point-lower experience of care scores. Government ownership was associated with 1.93-point-lower experience of care scores, whereas for-profit ownership was associated with 3.01-point-higher experience of care scores compared with not-for-profit ownership.
Table 4 reports the characteristics of hospitals classified as HIMSS Stage 6 EHR users compared with HIMSS Stage 7 EHR users. A larger proportion of Stage 7 EHR users were members of a hospital system; however, there were no other differences in hospital characteristics between the two groups. Before adjusting for hospital characteristics, HIMSS Stage 7 users had significantly higher clinical process of care scores (49.4 versus 40.2, P = .005) and significantly lower experience of care scores (25.2 versus 30.2, P = .035) compared with HIMSS Stage 6 users. After controlling for hospital characteristics, Stage 7 advanced EHR use was associated with significantly higher process of care scores than both Stage 6 advanced users and nonadvanced users (see Table 3, Model 2). There was no difference in process of care scores between Stage 6 advanced use and nonadvanced use. After adjusting for hospital characteristics, there was no difference in experience of care scores by level of advanced use.
Table 5 reports the results of the secondary analysis that evaluated the characteristics of advanced versus nonadvanced EHR use for the 268 teaching hospitals in the sample. There were no significant differences by advanced versus nonadvanced EHR use.
Our findings suggest that advanced EHR use is associated with higher clinical process quality of care in U.S. hospitals. Hospitals with advanced EHRs had estimated process of care scores that were 4.21 points higher on average than those hospitals not equipped with advanced EHRs. This difference was relatively small, however, translating into less than a one-standard-deviation difference in the average scores. Overall, institutions with advanced EHRs had lower unadjusted patient experience scores, explaining why some critics claim that EHRs have a detrimental influence on patient satisfaction. However, once hospital characteristics were controlled for, there were no differences in patient experience scores. Although we did not test for causation, these results, taken together, suggest that advanced EHR use might facilitate improved clinical processes of care, without negative effects on the patient experience.
Our findings add to the current body of knowledge in several important ways. The majority of work related to EHR use with quality and satisfaction has been conducted in the ambulatory setting, and these results have been mixed.5,17–19 Furthermore, few studies have evaluated the association between EHR use and patient satisfaction. A systematic review of the use of EHRs in the exam room and patient satisfaction found mostly neutral or positive effects; however, the review was not specific to the hospital inpatient setting.20
The most closely related study by Kazley and colleagues21 examined the association between EHR use and patient satisfaction in the hospital setting. They found that only 3 out of 10 measures of patient satisfaction were associated with EHR use. Our study differed from theirs in several important ways. First, we used composite scores for clinical process of care and patient experience that aggregated scores across the individual measures. Kazley and colleagues measured performance for 10 separate patient satisfaction items using the percentage of patients who ranked their experience in the highest category for each item. Our composite patient experience measure took into account both the percentage of patients scoring in the highest category for each HCAHPS item and the achievement threshold (50th percentile of all hospitals), rather than an array of individual scores. Second, our approach took into account consistency and improvement in scores over time, whereas Kazley’s study evaluated patient satisfaction at a single point in time. In addition, their definition of EHR use differed from ours. Kazley and colleagues defined EHR use as a dichotomous measure that indicated whether or not a hospital used an EHR. We defined advanced EHR use as using, at minimum, physician documentation with a full clinical decision support system and full radiology picture archive. Less than 10% of hospitals had achieved advanced EHR use in our study, whereas Kazley reported that approximately 17% of hospitals had an EHR during their time period of study (study time period not reported).
Although there was a small but significant difference in process of care scores for advanced versus nonadvanced EHR users, the difference increased substantially for hospitals with Stage 7 EHRs compared with those with Stage 6 or less advanced EHRs. Stage 7 users had estimated process of care scores that were 11.7 points higher than nonadvanced users, compared with Stage 6 users with only 2.6-point-higher scores than nonadvanced users. To achieve Stage 7 EHR use, a hospital must have a complete electronic medical record with the ability to readily share clinical information for the continuum of care through standardized electronic transactions, data warehousing capabilities to analyze clinical data to improve quality, and data continuity with the emergency department, ambulatory, and outpatient services. As of December 2012, Stage 7 users represented 1.9% of total U.S. hospitals.12 Our findings suggest that the greatest gains from EHR adoption will be achieved only when hospitals have implemented a full, integrated EHR. Poon and colleagues4 evaluated the relationship between individual EHR features (e.g., using medication lists, using problem lists, ordering laboratory tests, and reviewing laboratory and radiology results) and quality measures in the ambulatory setting. They found that the relationship between quality and individual EHR components varied across the outcomes measures evaluated, but was most consistent for using problem lists, visit notes, and radiology study results. These results suggest that it is critical to carefully evaluate the specific features and sophistication of EHR use and their impact on outcomes.
Although our findings provide early evidence of the potential of meaningful EHR use for hospital quality, there are several limitations. First, the HIMSS Analytics database was not designed for research purposes; however, it was the best available source of information for stage of EHR implementation at the time we conducted this study. Both HIMSS Analytics and the Medicare EHR Incentive Program rely on self-reported data. The Medicare EHR Incentive Program requires providers and hospitals to demonstrate meaningful use of EHR technologies via attestation in a CMS database. Although the CMS plans to audit some hospitals, the program predominantly relies on self-reported data for payment. HIMSS Analytics relies on self-reported information for Stages 1 to 5, but validates the stage of EHR capabilities for Stages 6 and 7, which we have defined as advanced EHR use. HIMSS conducts telephone interviews to validate EHR capabilities to qualify for Stage 6 and conducts on-site visits with providers to evaluate Stage 7 criteria.22 Additionally, our definition of advanced EHR use was based on the HIMSS EMRAM, and therefore, hospitals must participate in the HIMSS Analytics Annual Study to receive an EMRAM score. It is possible, therefore, that hospitals that were operating at a Stage 6 or Stage 7 level had not yet applied for this level of HIMSS EMR performance at the time of this study. However, HIMSS EMRAM is currently one of the most comprehensive proxy measures available for the level of sophistication of EHR use by U.S. hospitals.
Furthermore, although we hypothesized that advanced EHR use drives process and experience of care scores, it is also plausible that high-performing hospitals are more likely to implement advanced EHRs. In addition, data from April 2008 to March 2010 were used to calculate the Hospital VBP scores, whereas the level of EHR use from HIMSS reflected information current through December 2011. In practice, the CMS 2013 Hospital VBP Program results will be based on baseline data from July 2009 to March 2010 and performance data from July 2011 to March 2012. Future work should focus on disentangling the relationship between hospital quality and meaningful adoption of EHRs.
Moving forward, more work is needed to evaluate whether other hospital characteristics influence the association between advanced EHR use and quality and patient satisfaction. Further research should examine quality and patient satisfaction between hospitals using advanced EHRs classified as HIMSS Stage 6 or 7, and hospitals at the other extreme with EHRs classified as HIMSS Stage 0 or 1. As a preliminary study, we examined only Stage 6 and 7 hospitals versus lower levels of EHR use; however, there is potentially a large difference between hospitals with lower levels of EHR use. Additional studies should also be undertaken that use discrete measures of quality and patient satisfaction rather than the estimated Hospital VBP scores that we examined. Finally, as more hospitals receive funding under the Medicare EHR Incentive Program, more research could be done examining the quality and patient satisfaction of those hospitals, especially as the criteria for receiving funding evolve over the coming years.
Our findings provide early evidence of the effectiveness of the government’s investment in hospital IT infrastructure and the potential payoff of meaningful use incentive payments. Findings suggest that investment in advanced EHR adoption is associated with higher clinical process of care scores, without significantly degrading the patient experience in hospitals across United States. Importantly, in the years ahead, the CMS is expected to further refine the Hospital VBP Program and expand it to include variables such as Medicare spending per beneficiary. Furthermore, penalties for high readmission rates have begun, and payment reductions for additional hospital-acquired conditions will be implemented in FFY 2013.23 With this increasing focus on pay for performance, one would hope to learn much more about the extent to which the CMS’s incentivized investment in EHR adoption measurably affects patient outcomes. Such research will serve the government, providers, and patients well in designing and implementing future incentives for adoption of advanced IT.
1. . U.S. Department of Health and Human Services. HITECH Act enforcement interim final rule. http://www.hhs.gov/ocr/privacy/hipaa/administrative/enforcementrule/hitechenforcementifr.html
. Accessed June 5, 2013
2. . Centers for Medicare and Medicaid Services. Medicare and Medicaid EHR Incentive Program: Meaningful Use and Stage 1 Requirements Overview. http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/Downloads/MU_Stage1_ReqOverview.pdf
. Accessed June 5, 2013
3. McCullough JS, Casey M, Moscovice I, Prasad S. The effect of health information technology on quality in U.S. hospitals. Health Aff (Millwood). 2010;29:647–654
4. Poon EG, Wright A, Simon SR, et al. Relationship between use of electronic health record features and health care quality: Results of a statewide survey. Med Care. 2010;48:203–209
5. Chaudhry B, Wang J, Wu S, et al. Systematic review: Impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144:742–752
6. Hillestad R, Bigelow J, Bower A, et al. Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Aff (Millwood). 2005;24:1103–1117
7. . Congressional Budget Office. Evidence on the Costs and Benefits of Health Information Technology. http://www.cbo.gov/sites/default/files/cbofiles/ftpdocs/91xx/doc9168/05-20-healthit.pdf
. Accessed June 5, 2013
8. . One Hundred Eleventh Congress of the United States of America. The Patient Protection and Affordability Act. http://www.gpo.gov/fdsys/pkg/BILLS-111hr3590enr/pdf/BILLS-111hr3590enr.pdf
. Accessed June 5, 2013
9. Centers for Medicare & Medicaid Services (CMS), HHS. . Medicare program; hospital inpatient value-based purchasing program. Final rule. Fed Regist. 2011;76:26490–26547
10. . Centers for Medicare and Medicaid Services. Open Door Forum: Hospital Value-Based Purchasing. Fiscal Year 2013 Overview for Beneficiaries, Providers, and Stakeholders. July 27, 2011. http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/hospital-value-based-purchasing/Downloads/HospVBP_ODF_072711.pdf
. Accessed June 13, 2013
11. . Social Security Act. Payment to hospitals for inpatient hospital service. 1987;1886(d)(1)(B) http://www.ssa.gov/OP_Home/ssact/title18/1886.htm
. Accessed June 5, 2013
12. . HIMS Analytics. Electronic Medical Record Adoption Model (EMRAM). http://www.himssanalytics.org/emram/
. Accessed June 5, 2013
14. American Hospital Association. . Value based purchasing calculator, estimated summary scores. May 2011 Washington, DC American Hospital Association
15. American Hospital Association. FY2009 American Hospital Association Annual Survey Database. 2010 Chicago, Ill Health Forum, LLC
16. Manning WG, Mullahy J. Estimating log models: To transform or not to transform? J Health Econ. 2001;20:461–494
17. Linder JA, Ma J, Bates DW, Middleton B, Stafford RS. Electronic health record use and the quality of ambulatory care in the United States. Arch Intern Med. 2007;167:1400–1405
18. Cebul RD, Love TE, Jain AK, Hebert CJ. Electronic health records and quality of diabetes care. N Engl J Med. 2011;365:825–833
19. Zhou L, Soran CS, Jenter CA, et al. The relationship between electronic health record use and quality of care over time. J Am Med Inform Assoc. 2009;16:457–464
20. Irani JS, Middleton JL, Marfatia R, Omana ET, D’Amico F. The use of electronic health records in the exam room and patient satisfaction: A systematic review. J Am Board Fam Med. 2009;22:553–562
21. Kazley AS, Diana ML, Ford EW, Menachemi N. Is electronic health record use associated with patient satisfaction in hospitals? Health Care Manage Rev. 2012;37:23–30
22. . HIMSS Analytics. Stage 7 award: Shared patient data, without paper. http://www.himssanalytics.org/emram/stage7award.aspx
. Accessed June 5, 2013
23. Centers for Medicare and Medicaid Services (CMS), HHS. . Medicare program; hospital inpatient prospective payment systems for acute care hospitals and the long-term care hospital prospective payment system and fiscal year 2013 rates; hospitals’ resident caps for graduate medical education payment purposes; quality reporting requirements for specific providers and for ambulatory surgical centers. Final rule. Fed Regist. 2012;77:53257–53750