Background: Few tools exist to quantify the performance of a hospital's information system from a user perspective.
Objectives: Our objective was to develop and evaluate a survey-based metric that assesses the automation and usability of a hospital's information system.
Research Design and Methods: This is a cross-sectional study of 117 physicians and 3 chief information officers (CIOs) working in 2 community hospitals with historically low investment in IT (Hospitals A and B), an academic hospital with an advanced IT system (Hospital C), or a major Veterans Affairs hospital (Hospital D). Respondents completed a survey assessing their institution's information system. The mean of 90 summed responses yields the clinical information technology (CIT) index, a global measure of a hospital's information system performance on a 100-point scale.
Results: On the global CIT index, mean physician scores were significantly higher for hospitals with advanced IT (61.1 and 64.3 for C and D) compared with those with low investment in IT (32.6 and 29.4 for A and B, P < 0.001). These differences also were observed for each of 7 separate subdomains. The CIO scores, 74.7, 78.0 for Hospitals C and D, and 44.5 for Hospitals A and B, paralleled the mean physician scores for these hospitals. All measures exhibited low variance for each hospital (eg, standard deviations for the CIT index ranged from 5.9 to 8.1) and intraclass correlation was high (Chronbach's alpha >.70).
Conclusions: This assessment tool demonstrates initial evidence of validity and reliability.
From the *Robert Wood Johnson Clinical Scholars Program and †Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland; ‡Department of Biostatistics, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland; §Indiana University Center for Aging Research and Regenstrief Institute, Inc., Indianapolis, Indiana; ¶Division of Health Sciences Informatics, Johns Hopkins University School of Medicine, Baltimore, Maryland; ∥Office of the Inspector General, Department of Veterans Affairs, Washington, DC; and **Departments of Epidemiology and Health Policy & Management, Bloomberg School of Public Health and Welch Center for Prevention, Epidemiology, and Clinical Research, Johns Hopkins University School of Medicine, Baltimore, Maryland.
Supported by the Robert Wood Johnson Foundation, Princeton, New Jersey.
Reprints: Ruben Amarasingham, MD, Welch Center for Prevention, Epidemiology, and Clinical Research, Johns Hopkins University School of Medicine, 2024 E. Monument Street, Suite 2-600, Baltimore, MD 21205-2223. E-mail: email@example.com