According to the Institute of Medicine,1 the development and implementation of more sophisticated information systems are essential not only to enhance quality and efficiency of patient care but also to support clinical decision making. Clinical decision support becomes more and more a core function of health information systems to eliminate preventable medical errors,2 and the investments in decision support technologies targeted at nursing practice have increased.3 A computerized clinical decision support system (CDSS) refers to any electronic system designed to aid directly in clinical decision making. To generate patient-specific recommendations, CDSSs use the characteristics of individual patients; these recommendations are then presented to nurses for consideration.4,5 The knowledge base embedded in CDSSs contains the rules and logic statements that encapsulate knowledge required for clinical decisions so that it generates tailored recommendations for individual patients.6 With this, CDSSs assist nurses in completing the knowledge base rule–driven decision making or standardized rule-driven decision making,7 instead of using their own biases and intuition.8–10 On the one hand, CDSSs applied to nursing care are an expansion of the CDSS prototype defined above. For example, CDSSs for nursing care provide prebuilt forms for data entry of patient assessment, care plans, or outcome evaluation on given nursing interventions.8 Although it is not the case of recommendations automatically generated by the algorithm, the predesigned forms help decision making for nurses because these present the full scope of components that should be included for related nursing care activities. Thus, CDSSs for nursing care in this study include all the CDSS prototypes and the expanded versions.
Because using CDSSs to support nurses’ decision making is widespread, it is worth capturing which features of CDSSs were empirically effective for optimum decision support for frontline nurses. Currently, there are studies on CDSSs used to improve the clinical practice of nurses; however, system features addressing particular nursing care activities have been dispersed in individual reports. Nursing does not have the well-organized knowledge base on the features of nursing practice–oriented CDSSs in real practice settings. The purpose of this study was to organize the features of CDSSs useful for nursing practice through a literature review, especially using the categories of assessment, problem identification (ie, diagnosis), care plans, implementation, and outcome evaluation. The current decision support technologies typically operate in these five stages. A certain CDSS helps decision making in a single stage, while other CDSSs help decision making in two or more stages. However, because of a lack of empirical investigations, it has not been clear whether a CDSS providing decision support in all the stages from assessment to outcome evaluation was more clinically useful than a CDSS operating, for example, in only a single stage of assessment. If there are evidential data to answer this question, the evidence should be included as an important feature for better decision support. As a preliminary to conducting an empirical study to address the question above, the first priority was in conducting a literature review to identify to the extent of sequential decision support provided by CDSSs in the stages from assessment to outcome evaluation. In this study, the sequential decision support, which is another important concept, is one of the CDSS features.
Studies Eligible for Review
To obtain the most relevant studies, studies eligible for inclusion were primary studies on CDSSs used for nursing practice and designed to contain at least two aspects of assessment, problem identification, care plans, implementation, and outcome evaluation. Studies published in peer-reviewed journals and in English were included. On the other hand, studies were excluded if they were studies on a nonelectronic decision support system such as a paper-based system, studies not providing a description on a CDSS, and studies providing only a technical description of a CDSS application (ie, testing algorithms of an application). Review studies on CDSSs were also excluded.
Databases of MEDLINE, CINAHL, and EMBASE were searched up to 2012 by using the search terms computer-assisted decision support system, automated decision support, computerized evidence-based decision making, computerized evidence-based practice, and evidence, decision support system, having nursing in common. Conference proceedings and the reference lists of all included articles were reviewed to identify additional primary studies.
The author reviewed titles and abstracts of identified references and rated each article as “potentially relevant” or “not relevant” by using the inclusion and exclusion criteria. The author reviewed the full texts of potentially relevant primary studies and again rated each article as “potentially relevant” or “not relevant” using a screening checklist. Thus, the final selection of studies for review was made. A screening checklist was to check the presence or absence of and appropriateness of data that should be extracted from studies. Its content is identical to a data extraction form for double-checking (see “Data Extraction” section). Use of the checklist prevented important data from inadvertently being omitted. Before actual use of the checklist, the author piloted it on a sample of three articles to address the issues of arranging the checklist items in user-friendly sequence and completing the checklist.11
The author extracted necessary information from each of the finally selected articles by using a data extraction form. The form was to record study purpose, study design, data collection methods, study settings and participants, nursing care areas addressed by the use of a CDSS, functions of a CDSS, study results, and features of a CDSS. The functions of a CDSS were categorized into assessment, problem identification, care plans, implementation, and outcome evaluation. A CDSS was considered having the functions of the stages from assessment to outcome evaluation: when a CDSS had preformulated forms for data entry that are embedding evidence to support clinical decision making relating from assessment to outcome evaluation, when the rule engine of a CDSS automatically generated recommendations or instructions for a next action based on data entered in a prior step, or when the sections from assessment to outcome evaluation were automatically linked to each other for a logical continuity of clinical decision making and then relevant data have to be entered in a prebuilt form or selected from a prebuilt list. For example, if an assessment entry form existed, the CDSS had the function for patient assessment. If care plans were automatically generated based on assessment data entered, the CDSS had the functions of assessment and care plans. When a set of care plans was linked to patient outcome evaluation and then an outcome measurement form should be filled out, the CDSS had the functions of care plans and outcome evaluation. Study results are any changes by the use of a CDSS. These would include improvement or nonimprovement in terms of, but not limited to, nurses’ decision making, nurse performance, and patient outcomes.
As the features of CDSSs, components of CDSSs that improved nurses’ decision making, nurse performance, or patient outcomes were extracted. If some components deteriorated them (eg, “the need to devise care plans made nurses spend much time”), the author treated the logically opposite component as a potential improvement component (eg, “removing the need to devise care plans made nurses save time”).12,13 In addition, if authors of studies mentioned important features of their CDSS, the features were also included here. The functions of CDSSs mentioned above were integrated as part of the features of CDSSs. The author recorded extracted information on the data extraction form and also double-checked extracted information with original articles for accuracy.
The extracted data, including study purpose, design, data collection methods, settings and participants, nursing care areas addressed by the use of a CDSS, functions of a CDSS, and study results, were organized in tables. To synthesize CDSS features across the reviewed studies, the author carefully read and compared the features extracted from each study and divided them into meaning units. The meaning units were assessment, problem identification, care plans, implementation, and outcome evaluation. The author integrated or separately organized the features into key words and phrases capturing core content of each unit. The synthesized results were organized in a separate table.
Of 681 potentially relevant studies published from 1990 to 2012, 27 studies met the eligibility criteria and the items on the screening checklist. The study description in Table 1 combines study purpose, design, data collection methods, settings, and participants. Table 2 presents a summary of Table 1, which includes study purpose, design, data collection methods, CDSS-applied nursing care areas, and sequential decision support functions of CDSSs. Of the 27 studies reviewed, 17 were system development, and eight of the 17 studies piloted their system immediately after system development (Table 2). In the study purpose of Table 2, others included two studies examining barriers to use of computerized advice6,26 and a study evaluating completeness of nursing documentation.19
The designs of 20 studies that conducted system evaluation or pilot test, except for seven studies of system development only, varied (Table 2). When considering the presence of a CDSS as the given intervention, 15 studies, which were mostly pilot tests, were posttest studies without a control group. Two pretest-posttest studies used different groups for comparison before and after system use. Four studies used a one-group pretest-posttest format. Also included were a quasi-experimental study with two nonrandomized control groups and a randomized controlled trial. Three studies used two different designs for their system evaluation or pilot test7,25,34; thus, they were counted twice in the design. Data collection methods used in the 20 studies for system evaluation or pilot test were individual interviews, focus group interviews, observations, chart review, analysis of screen usage, questionnaires for nurses and other healthcare providers, and questionnaires for patients. Eight studies collected data by mixed methods; three studies, by quantitative methods; and nine studies, by qualitative methods.
Nursing care areas addressed by the use of a CDSS varied; however, fall, pressure ulcer, pain, blood glucose control, and patient referral overlapped, as shown in Tables 1 and 2. Eighteen studies targeted a single area of nursing care, while nine studies covered multiple areas of nursing care. Two mobile-based decision support systems targeted multiple areas of nursing care (Table 2).
Table 1 presents the functions of CDSSs that provided decision support in the stages available from assessment to outcome evaluation. The reviewed CDSSs showed the diverse ranges of sequential decision support functions. Sequential decision support for patient assessment and care plans existed in all of the reviewed CDSSs (Table 2). With reference to the sequence, movement to a next stage such as from assessment to problem identification or to care plans occurred as a next screen automatically showed up or was clicked after completion of a prior stage; a nurse was forced to implement the movement. Two studies’ assessment entry forms were to assess patients’ responses to treatments (ie, patient outcomes),34,35 instead of initial assessment for patients (Table 1). Most CDSSs started their function for patient assessment with a nurse’s entry in an electronic assessment form (Table 2). Five CDSSs started their function as they automatically retrieved necessary data from hospital databases or other connected information systems and a nurse inputs additional information. Three CDSSs were a real-time system for patient assessment,23,29,37 and two of them were tele-advice systems.29,37 Two CDSSs automatically assessed patients without input of a nurse (Table 2).23,29 For the details of CDSS functions from problem identification to outcome evaluation, see Table 1.
Table 1 presents the study results on patient outcomes, nurse performance, and nurses’ decision making by the use of CDSSs. The CDSSs were of benefit to patients and nurses as they improved patient status in the CDSS-applied nursing care areas,7,14,15,20,34 improved nurses’ work,7,13,17,19,22,23,25,27,28,30,31 simplified nurses’ work,13,28,37 and complemented nurses’ knowledge.18,30,31,37 However, there were still problems in integration with nurses’ workflow,6,17,33 system flexibility,17 user interface,26 learning computer skills, and implementing new guidelines.18 Other problems were malfunctioning computer system issues, lack of administrative leadership,18 and disagreement on system advice.6,13,37
In the studies of system development, because it was common that an interdisciplinary team participated in their system development, it was not described as study subjects in Table 1. As sources of knowledge embedded in decision support systems, all of the studies reviewed basically used scientific evidence such as nationally recognized clinical practice guidelines, randomized controlled trials, systematic review studies, literature review of other study designs, and topic-specific, valid assessment tools. The patterns and types of evidence used were similar among the studies.
The features of CDSSs across the studies are synthesized and organized in Table 3. The system features collected represented the characteristics of each category of the five stages from patient assessment to outcome evaluation. However, there were differences in the numbers of the system features extracted for each category. The features separately grouped as “others” in Table 3 were associated with the five stages. Certain features, such as being available at the point of care and being used in a clinical routine, were common among all of the CDSSs reviewed. The first feature in the others of Table 3, “providing automatic links between CDSS functions,” means the sequential decision support of CDSSs provided in the stages available from assessment to outcome evaluation.
This study aimed to organize the features of CDSSs useful for nursing practice into assessment, problem identification, care plans, implementation, and outcome evaluation. As a part of the CDSS features, the study identified the diverse ranges of sequential decision support of CDSSs that operated in the stages from assessment to outcome evaluation.
The CDSS features related to patient assessment and care plans comparatively varied, whereas the features related to implementation and outcome evaluation did not (Table 3). This indicates that a small number of related studies limited the number of features to be extracted. In fact, there were only three CDSSs providing decision support in an implementation stage and four CDSSs operating in an outcome evaluation stage (Table 2). Eleven of the reviewed CDSSs operated in the stage of problem identification and two features for it were identified. In a single area of nursing care addressed by CDSSs, the step of problem identification by CDSSs would be skipped because the CDSSs were developed and implemented to address the targeted nursing care area. For example, in the study by Gunningberg et al,19 problem identification by a CDSS was not needed because the target area of nursing care was pressure ulcer and the CDSS was used to address the identified problem. However, CDSSs, which operated in multiple areas of nursing care, needed to have useful features for problem identification. In the study by Lee et al,13 nurses had to select nursing diagnoses from a list from the North American Nursing Diagnosis Association (NANDA) that are consistent with patient assessment data. However, there was no consensus among nurses about the diagnoses selected by them. In the implementation step of care plans (Table 3), the CDSSs provided three features about checking the completion of care activities. Unlike other categories with prebuilt formats embedding evidence from literature, decision support in the implementation step was grounded on the performance of nurses. The CDSSs in four studies provided decision support in an outcome evaluation stage (Table 2). Outcome evaluation is a very important stage that should not be omitted for quality patient care. Outcome evaluation allows nurses to determine relationships between patients’ outcome achievement and nursing interventions. After the effectiveness of care plans and intervention is evaluated, the results are fed back into nursing practice.35 Outcome evaluation is an ongoing activity to conduct reassessment of patient status, reordering of priorities, new goal-setting, and revision of care plans. However, most CDSSs reviewed in the study, except the four studies, did not include the function of outcome evaluation on the given nursing care. In two studies, outcome evaluations were implemented outside their CDSS function.13,19 In the case that patient outcome evaluation is not a routine, nurses need to search for appropriate measurements or evidence for patient outcome evaluation; however, such a search may not be carried out for many reasons including a lack of time based on workload, difficulty accessing computers, and/or difficulty finding proper materials. A CDSS needs to provide a prepackaged measurement form or evidence-based recommendations for outcome evaluation. On the other hand, Table 3 shows the common features provided by all of the CDSSs reviewed. Through the organized system features, a comprehensive picture of nursing practice–oriented CDSSs that were attempted up to now was identified.
All of the CDSSs reviewed provided sequential decision support in at least two steps; nine CDSSs, in three stages; three CDSSs, in four stages; and a CDSS, in five steps (Table 2). The important thing to which we have to pay attention is the decision support provided in the full scope from initial assessment to outcome evaluation. As grounded in this review, the key steps of a CDSS for sequential decision support were initial patient assessment, problem identification, care plan, and outcome evaluation. It is to provide decision support at the most effective level of nursing care. If such a CDSS is used in a clinical routine, it allows for safe and continuous decision support from the initial stage of patient assessment to the outcome evaluation. Such decision support must be an indispensable part of the CDSS features for quality patient care.
There were limitations, although various studies were included in this review to extract the features of CDSSs useful for nursing practice. As most of the studies reviewed were in the stage of system development immediately followed by pilot test or evaluation, one limitation would be that the CDSS features were extracted from such studies, instead of rigorous study designs such as randomized controlled trials. Regardless, the types of the reviewed studies became an advantage in discerning the features of each CDSS because they focused on CDSS functionality. Of the 27 studies reviewed, three studies developed a CDSS as a tool to implement evidence-based practice in nursing, as carefully reviewed and selected evidence was embedded in a CDSS.16,18,36 One study developed a CDSS as a tool to increase the completeness and quality of nursing documentation.19 Therefore, there was a limitation to extracting the features of CDSSs because these studies focused on compliance with evidence-based recommendations and nursing documentation. Lastly, as one study lacked information on system function31 and one study lacked information on outcome evaluation,32 there was difficulty describing the system functions from those studies.
For nursing practice and research, the development of a guideline toward an optimum CDSS that best supports nursing practice will have to go beyond the scope of system features identified from a literature review. The steps of sequential decision support by a CDSS were identified, and its importance was emphasized. On the other hand, for empirical support, there is the need to conduct a study to examine clinical effectiveness of CDSSs providing decision support in sequence from initial assessment to outcome feedback. Two suggestions for further research to mitigate the weakness of the reviewed studies are the following: that more nursing care areas become targets of CDSSs and that the effectiveness of CDSSs on decision support for nurses, nurse performance, and patient outcomes be evaluated by rigorous study designs, to have stronger nursing practice-oriented CDSSs.
This study organized the features of CDSSs useful for nursing practice into the categories of assessment, problem identification, care plans, implementation, and outcome evaluation, and identified the diverse ranges of the five category-related sequential decision supports that CDSSs provided. This review added the evidence-based knowledge regarding the features of nursing practice-oriented CDSSs. To design the optimum CDSS for nursing practice, a wider range of evidence-based knowledge is needed. Furthermore, providing continuous decision support from the initial stage of patient assessment to outcome evaluation cannot be overemphasized.
I specially thank Dr Jane White at the College of Nursing and Public Health for her assistance with editing.
1. Institute of Medicine Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy press; 2001.
2. Cho I, Kim JA, Kim JH, Kim H, Kim Y. Design and implementation of a standards-based interoperable clinical decision support architecture in the context of the Korean EHR. Int J Med Inform. 2010; 79: 611–622. doi:10.1016/j.ijmedinf.2010.06.002.
3. Dowding D, Mitchell N, Randell R, et al. Nurses’ use of computerized clinical decision support systems: a case site analysis. J Clin Nurs. 2009; 18: 1159–1167.
4. Grag AX, Adhikari NKJ, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005; 293 (10): 1223–1238.
5. Randell R, Mitchell N, Dowding D, Cullum N, Thompson C. Effects of computerized decision support systems on nursing performance and patient outcomes: a systematic review. J Health Serv Res Policy. 2007: 12 (4): 242–249.
6. Sward K, Orme J Jr, Sorenson D, et al. Reasons for declining computerized insulin protocol recommendations: application of a framework. J Biomed Inform. 2008; 41 (3): 488–497.
7. Hoekstra M, Vogelzang M, Drost JT, et al. Implementation and evaluation of a nurse-centered computerized potassium regulation protocol in the intensive care unit—a before and after analysis. BMC Med Inform Decis Mak. 2010; 10: 5. http://www.biomedcentral.com/1472-6947/10/5
. Accessed June 1, 2012.
8. Brokel JM. Infusing clinical decision support interventions into electronic health records. Urol Nurs. 2009; 29 (5): 345–352.
9. Coopmans VC, Biddle C. CRNA performance using a handheld, computerized, decision-making aid during critical events in a simulated environment: a methodologic inquiry. AANA J. 2008; 76 (1): 29–35.
10. Cho I, Staggers N, Park I. Nurses’ responses to differing amounts and information content in a diagnostic computer-based decision support application. Comput Inform Nurs. 2010; 28 (2): 95–102.
12. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features crucial to success. BMJ. 2005; 330 (7494): 765. doi:10.1136/bmj.38398.500764.8F
13. Lee TT, Yeh CH, Ho LH. Application of a computerized nursing care plan system in one hospital: experiences of ICU nurses in Taiwan. J Adv Nurs. 2002; 39 (1): 61–67.
14. Fick DM, Steis MR, Mion LC, Walls JL. Computerized decision support for delirium superimposed on dementia in older adults. J Gerontol Nurs. 2011; 37 (4): 39–47. doi:10.3928/00989134-20100930-01.
15. Browne JA, Covington BG, Davila Y. Using information technology to assist in redesign of a fall prevention program. J Nurs Care Qual. 2004; 19 (3): 218–225.
16. Bakken S, Currie LM, Lee NJ, et al. Integrating evidence into clinical information systems for nursing decision support. Int J Med Inform. 2007; 77 (6): 413–420.
17. Quaglini S, Grandi M, Baiardi P, et al. A computerized guideline for pressure ulcer prevention. Int J Med Inform. 2000; 58–59: 207–217.
18. Clarke HF, Bradley C, Whytock S, et al. Pressure ulcers: implementation of evidence-based nursing practice. J Adv Nurs. 2005; 49 (6): 578–590.
19. Gunningberg L, Fogelberg-Dahm M, Ehrenberg A. Improved quality and comprehensiveness in nursing documentation of pressure ulcers after implementing an electronic health record in hospital care. J Clin Nurs. 2009; 18 (11): 1557–1564.
20. Fossum M, Alexander GL, Ehnfors M, Ehrenberg A. Effects of a computerized decision support system on pressure ulcers and malnutrition in nursing homes for the elderly. Int J Med Inform. 2011; 80 (9): 607–617.
21. Im EO, Chee WS. Decision support computer program for cancer pain management. Comput Inform Nurs. 2003; 21 (1): 12–21.
22. Huang HY, Wilkie DJ, Zong SP, et al. Developing computerized data collection and decision support system for cancer pain management. Comput Inform Nurs. 2003; 21 (4): 206–217.
23. Kroth PJ, Dexter PR, Overhage JM, et al. A computerized decision support system improves the accuracy of temperature capture from nursing personnel at the bedside. AMIA Annu Symp Proc. 2006: 444–448.
24. Henry SB, Douglas K, Galzagorry G, Lahey A, Holzemer WL. A template-based approach to support utilization of clinical practice guidelines within an electronic health record. J Am Med Inform Assoc. 1998; 5 (3): 237–244.
25. Vogelzang M, Zijlstra F, Nijsten MWN. Design and implementation of GRIP: a computerized glucose control system at a surgical intensive care unit. BMC Med Inform Decis Mak. 2005; 5: 38. doi:10.1186/1472-6947-5-38.
26. Campion TR Jr, Waitman LR, Lorenzi NM, May AK, Gadd CS. Barriers and facilitators to the use of computer-based intensive insulin therapy. Int J Med Inform. 2011; 80 (12): 863–871.
27. Heermann LK, Thompson CB. Prototype expert system to assist with the stabilization of neonates prior to transport. Proc AMIA Annu Fall Symp. 1997: 213–217.
28. Guite J, Lang M, McCartan P, Miller J. Nursing admissions process redesigned to leverage EHR. J Healthc Inf Manag. 2006; 20 (2): 55–64.
29. Adams WG, Fuhlbrigge AL, Miller CW, et al. TLC-Asthma: an integrated information system for patient-centered monitoring, case management, and point-of-care decision support. AMIA Annu Symp Proc. 2003: 1–5.
30. Jirapaet V. A computer expert system prototype for mechanically ventilated neonates. Comput Nurs. 2001; 19 (5): 194–203.
31. Keenan GM, Stocker JR, Geo-Thomas AT, et al. The HANDS project: studying and refining the automated collection of a cross-setting clinical data set. Comput Inform Nurs. 2002; 20 (3): 89–100.
32. Kim Y, An M, Park J, et al. New method of realization of nursing diagnosis based on 3N in an electronic medical record system. Stud Health Technol Inform. 2007: 19 (pt 1): 364–366.
33. Kim T, Lang NM, Berg K, et al. Clinician adoption patterns and patient outcomes results in use of evidence-based nursing plans of care. AMIA Annu Symp Proc. 2007; 11: 423–427.
34. Doran D, Paterson J, Clark C, et al. A pilot study of an electronic interprofessional evidence-based care planning tool for clients with mental health problems and addictions. Worldviews Evid Based Nurs. 2010; 7 (3): 174–184.
35. Doran DM, Mylopoulos J, Kushniruk A, et al. Evidence in the palm of your hand: development of an outcomes-focused knowledge translation intervention. Worldviews Evid Based Nurs. 2007; 4 (2): 69–77.
36. Lee N, Bakken S. Development of a prototype personal digital assistant-decision support system for the management of adult obesity. Int J Med Inform. 2007; 76 (Suppl 2): S281–S292.
37. Ernesäter A, Holmström I, Engström M. Telenurses’ experiences of working with computerized decision support: supporting, inhibiting and quality improving. J Adv Nurs. 2009; 65 (5): 1074–1083.
© 2013 Wolters Kluwer Health | Lippincott Williams & Wilkins.