Secondary Logo

Journal Logo

Original Article

User acceptance of an anaesthesia information management system

Quinzio, L.*; Junger, A.*; Gottwald, B.; Benson, M.*; Hartmann, B.*; Jost, A.*; Banzhaf, A.*; Hempelmann, G.*

Author Information
European Journal of Anaesthesiology: December 2003 - Volume 20 - Issue 12 - p 967-972

Abstract

Along with increasing cost consciousness in healthcare and the need for detailed documentation of the care provided, there is also increased demand for computerized record keeping. The integration of an anaesthesia information management system (AIMS) into the hospital medical information system allows existing data to be used and can reduce redundant data entry. The advantages of electronic record keeping over conventional methods have been shown in previous studies [1-7]. It is well known that the introduction of computer systems in healthcare is a far-reaching event that influences the structure of the information and its processing by medical personnel [8]. However, even in the era of information and Internet technology, one cannot automatically assume that all staff members have sufficient computer knowledge. Hence, the needs of the users will have to be of major importance. Ultimately, the quality of collected data depends to a large extent on the users' acceptance and satisfaction [9].

We conducted a survey after an AIMS had been in routine use for more than 5 yr in order to investigate the attitude of the anaesthesia staff towards the increasing use of electronic data processing in their clinical work and to obtain information on their acceptance of the AIMS. The objective of the study was to assess the attitude towards hospital information systems in general, determine user satisfaction with the software and reveal possible weaknesses of the system.

The critical appraisal of the results should not only reflect the system's acceptance among staff members but should also reveal problems and criticism that inevitably accompany the introduction of such a system and that can lead to recommendations for the implementation process.

Methods

User survey

The survey questionnaire for the computerized anaesthesia record keeping system was based on a pre-existing questionnaire for a patient data management system (PDMS) in intensive care [10]. It had a total of 75 statements in five sections eliciting information on the following topics: user characteristics, satisfaction with data management systems, general attitude towards data management systems, problems with hardware implementation and special programme features. Participants responded on a five-point Likert scale ranging from 'no' to 'partly no', 'half and half', 'partly yes' and 'yes'. The responses 'no' and 'partly no' were rated as negative, 'half and half' as neutral and 'partly yes' and 'yes' as positive answers. These categorical data were assigned numerical values for rank-sum analysis: one for the negative responses, and the values two and three for neutral and positive responses, respectively. The questionnaires were distributed in March 2001 to the physicians and nurses of the Department of Anaesthesiology. Completed questionnaires were to be deposited anonymously in a centrally located box within 2 weeks.

Statistical analyses (frequencies, U-tests) were performed using SPSS® for Windows® (Release 9.0, SPSS Inc., Chicago, ILL, USA).

The AIMS at the University Hospital of Giessen

The University Hospital of the Justus-Liebig-University Giessen is a 1300 bed tertiary care facility where approximately 20 000 anaesthetic procedures are performed every year in more than 40 operating rooms. Documentation is carried out by the anaesthesiologists with the support of the anaesthetic nurses. Until 1995, anaesthesia data were documented manually at over 100 decentralized workstations. The present version of the documentation software (NarkoData®; IMESO GmbH, Hüttenberg, Germany) has been in service since 1996. All staff members received initial training at that time and new members (approximately 10 a year) were introduced to the routine use of the system by experienced colleagues. Further individual training or special sessions were not provided, but special training sessions were offered for important programme updates or changes. Support was always available from a medical system administrator by telephone or by remote control software. The architecture and functionality of the system are described in detail elsewhere [11].

Administrative patient data and laboratory data imported from the hospital information system, vital signs such as automatically recorded monitor and lung ventilator data, and manually recorded data such as adverse events and drug administration are recorded. The programme runs as a local application and two independent back-up copies of the anaesthesia protocol are recorded every 2 min, one on the local hard disk and the other on a back-up server. The system uses a relational database for permanent data storage (Oracle 7®; Oracle Corporation, USA). Data from the hospital information system is imported through a network (HL 7) and medical devices are connected via an RS232 interface.

Preoperative data can be entered into a portable computer at the patient's bedside on the ward or directly in the operating room. The 'worksheet' with its graphic user interface (Fig. 1) corresponds to the conventional paper form giving a continuously visible documentation of the entire anaesthetic procedure including the recovery period. Data are entered with mouse, keyboard or barcode scanner. At the end of the case, the record is subjected to an automatic integrity check before it is stored in the database and becomes 'write protected' and a paper copy of the record is printed out for the patient's chart.

Figure 1
Figure 1:
Graphical user interface of the online anaesthesia record keeping software NarkoData®.

More than 20 000 anaesthetic procedures are recorded annually at 115 decentralized workstations throughout the hospital [12]. Data still had to be recorded manually on the standard anaesthesia form at six workplaces (approximately 6-8% of the annual anaesthesia records). Since these were commonly used workplaces one could assume that all members of the department were familiar with this conventional form of documentation.

Results

Forty-four physicians (28 residents and 16 consultants) and 24 anaesthetic nurses returned their questionnaires. None had more than five missing items and all were included in the analysis. The return rate of 60% was the same for physicians (44 of 73) and nurses (24 of 40). An evaluation of the patient characteristics data of responders and non-responders to determine a possible sampling bias revealed no significant differences in gender, profession or professional experience.

The majority of the respondents were male with previous computer knowledge and prior experience with an automated data management system (Table 1).

Table 1
Table 1:
Gender, computer skills and experience with AIMS of anaesthesiologists and anaesthetic nurses in relation to professional experience (n = 68).

The majority was satisfied with this particular system without, of course, having had an opportunity for comparison. They considered that the data management system made their work easier without causing additional work (52.3%). Most of the users felt that it improved their quality of work and was beneficial for the patient (70.5% and 59.1%). Only one-third of the participants thought that the system saved time.

The majority of the participants thought that a hospital information system facilitated a more precise documentation (85.3%) and about half thought that there were fewer errors (47.1% positive rating). One-third considered themselves poorly trained, but a majority considered themselves familiar with data entry (76.5%) and felt that using the automated data management system did not pose any problem (80.9%). Only 35.3% thought that data protection was guaranteed by the system. One-half of the users stated that the electronic documentation was well integrated into their daily routine (51.5%) and stated that it did not bother them to call for external support when the computer was down (50.0%). The majority did not want to switch back to manual documentation on paper (76.5%). The position of the monitor and the keyboard was a controversial issue (36.8% negative rating, 29.4% neutral and 32.4% positive rating) with one-half of the users stating that they had poor visual contact with the patient while working on the personal computer. In addition, the position of the workstation caused physical discomfort in many users (58.8%).

Software stability and speed were important issues: 57.4% were satisfied with the stability, but satisfaction with speed varied (36.8% positive, 26.5% neutral and 32.4% negative). Visual alarms were deemed useful by two-thirds of the participants (79.4%) but only 50% wanted acoustic alarms.

Data entry by keyboard (80.9%) or by mouse (75.0%) was preferred over a barcode scanner (42.6%). Entry of data relevant to anaesthesia (preoperative data, catheters, drugs, vital parameters, etc.) was generally considered easy, but 77.3% found coding of diagnosis and type of surgery difficult. Data import from monitors (100%), lung ventilator (90.9%), syringe pumps (54.5%), laboratory data (86.4%) and other data management systems (administration, radiology, previous anaesthesia records, etc.) was deemed important. Slightly more than half of the participants (58.8%) stated that the computerized record keeping system generally met their expectations, but 11.8% were not satisfied. Initial training was important for acceptance of the system. Users satisfied with their training (response to item 20) exhibited higher satisfaction with the system than those who perceived their initial training as inadequate (Table 2).

Table 2
Table 2:
Acceptance of AIMS as factor of perceived quality of training (n = 57).

Discussion

More and more anaesthesiology departments are replacing their manual documentation with computerized systems. The results of our survey of 5 yr use of a computerized system provide interesting information on the attitude and wishes of the users pertaining to hardware and software. Users considered that automated data management led to better documentation and they did not want to switch back to manual documentation on paper. This subjective feeling that computerized record keeping is superior is supported by the results of other studies [2,3,7,13,14]. The question of patient benefit was rated higher in this study than in that of Wang and colleagues [13].

Users had differing opinions on error reduction using electronic data processing. More than half of them were concerned about adequate data protection and they disagreed on the efficiency of the programme training.

The survey uncovered a sense of training deficit that shows that it is necessary to re-evaluate the existing training concept, especially as other authors assign a key role to user training for a good user acceptance of PDMSs [15,16].

Our data show the strong influence of adequate training on many aspects of user acceptance. Users who felt that they had received appropriate training not only assessed themselves as better skilled in using the system compared with the group who perceived a lack of training but also rated the quality of the system in their workplace higher and thought that it was more user-friendly than did their colleagues with a perceived lack of adequate training. Altogether, the well-trained users were significantly more satisfied with the AIMS in routine use.

The majority of the responders did not feel impaired by computer workstations in their clinical routine and they did not think that computers disturbed their workflow. However, half of them stated that they had poor visual contact with the patient during computer work. One-third did not like the current position of monitor and keyboard whereas other one-third appreciated the position of the workstation.

Since the AIMS had to be implemented into existing anaesthesia workstations, only a limited number of suitable positions were available for monitors and data entry devices when the system was installed. For budget reasons, compromises had to be found that might have been disadvantageous to functionality. In many cases, there was no room to install conventional monitors next to the patient, and data entry has to be done with the back turned towards the patient [13]. The introduction of new anaesthesia equipment and flat monitor screens can help remedy this shortcoming.

Contrary to other studies that report savings in data entry time, this advantage was not evident in our survey [13,14]. One possible reason for the observed lack of time saving might be the incomplete integration of the patient data systems into other clinical subsystems. The urgency of the implementation of such connections is expressed by the high rating given to automatic data collection from medical devices and data import from other computer systems. Additionally, participation in typing classes may improve keyboard skills of the users and thus reduce the time for data entry.

Automatic data transfer is certainly one of the most important aspects when installing patient data systems in the near future. This means that only systems with an integrated structure will gain user acceptance, an important aspect for software manufacturers. Another aspect is that systems administrators and users have different points of view. Administrators focus primarily on clinical data processing of which documentation is only a small part. In the context of a more efficient processing of data for statistical purposes the time saved by online data collection compared to conventional manual recording is most important. For the documenting medical staff this is of secondary importance, since they have to bear the additional burden of data entry.

The visual alarms for pathological, incomplete and non-logical values that received positive ratings are already programme features. The installation of additional acoustic signals was discontinued following the negative response in the survey. Due to an already high noise level [17] at the workplace from the vital data monitor and ventilator alarms, an additional signal from the recording system could increase work stress rather than facilitate the clinical work process.

The majority of the anaesthetic staff who answered the questionnaire was satisfied with the range of the programme functions and had no problems with data entry or data import except for the coding of main diagnosis and type of surgery.

At the time the survey was conducted, the operating room management system was not linked to the AIMS. It was not possible to import diagnoses and type of procedure coded during surgery into the AIMS and this data had to be entered a second time. Despite the fact that one can show the advantages of computerized coding of diagnoses compared with conventional methods [18,19], it is not surprising that this redundant task was given a negative rating by the users.

The survey was conducted 5 yr after the system had been installed and the users' answers might be of limited significance since they were accustomed to using the AIMS. However, 19% of the users had less than 1 yr experience with the system. Since all staff members had at least some experience with manual record keeping on paper, one can assume that they had an impression of the alternative 'standard method'. A comparison to other AIMSs was not possible at that time due to a lack of similar installations of comparable size running in Germany.

A 60% response rate is satisfactory for an anonymous survey and other studies quote similar or lower response rates [20-22]. We distributed the questionnaires to the staff and the response rate might have been better if they had been given to the participants personally. In summary, our clinical user survey revealed a predominantly positive attitude towards new information technologies in anaesthesiology and a good acceptance of the current AIMS. But relevant deficits of the system were also detected. These pertain mainly to reducing recording time, improved connections with other data management systems and position of the hardware. These aspects will have to be considered for further improvement.

Our results emphasize the strong influence of adequate training on the acceptance of an AIMS. We recommend a three-step training strategy. The employees should be extensively informed of planned programme installations before starting to implement an AIMS. A second phase should include general on-job training, followed by a third phase with the opportunity to work on the individual problems of each user. New colleagues should have an initial short, standardized introduction that is followed up with more thorough on-the-job training.

This structured user survey turned out to be a useful instrument for gathering information for further development and implementation of an AIMS and for analysing and evaluating relevant programme deficits. We recommend that user surveys should be conducted repeatedly during the implementation of similar systems.

Acknowledgements

Declaration of interest: Dr Matthias Benson is a partner of IMESO GmbH. Complete questionnaire and details on responses to each item are available from the authors on request.

References

1. Sanborn KV, Castro J, Kuroda M, Thys DM. Detection of intraoperative incidents by electronic scanning of computerized anesthesia records. Comparison with voluntary reporting. Anesthesiology 1996; 85: 977-987.
2. Reich DL, Wood RK, Mattar R, et al. Arterial blood pressure and heart rate discrepancies between handwritten and computerized anesthesia records. Anesth Analg 2000; 91: 612-616.
3. Benson M, Junger A, Fuchs C, et al. Using an anesthesia information management system to prove a deficit in voluntary reporting of adverse events in a quality assurance program. J Clin Monit 2000; 16: 211-217.
4. Junger A, Klasen J, Benson M, et al. Factors determining length of stay of surgical day-case patients. Eur J Anaesthesiol 2001; 18: 314-321.
5. Junger A, Hartmann B, Benson M, et al. The use of an anesthesia information management system for prediction of antiemetic rescue treatment at the postanaesthesia care unit. Anesth Analg 2001; 92: 1203-1209.
6. Lubarsky DA, Sanderson IC, Gilbert WC, et al. Using an anesthesia information management system as a cost containment tool. Description and validation. Anesthesiology 1997; 86: 1161-1169.
7. Thrush DN. Are automated anesthesia records better? J Clin Anesth 1992; 4: 386-389.
8. Patel VL, Kushniruk AW, Yang S, Yale JF. Impact of a computer-based patient record system on data collection, knowledge organization, and reasoning. J Am Med Inform Assoc 2000; 7: 569-585.
9. Ammenwerth E, Eichstadter R, Haux R, Pohl U, Rebel S, Ziegler S. A randomized evaluation of a computer-based nursing documentation system. Methods Inf Med 2001; 40: 61-68.
10. Junger A, Benson M, Quinzio L, et al. Anwenderzufriedenheit von Patienten-Daten-Management-Systemen (PDMS) in der Intensivmedizin. Stud Health Technol Inform 2000; 77: 513-517.
11. Benson M, Junger A, Quinzio L, et al. Clinical and practical requirements of online software for anesthesia documentation an experience report. Int J Med Inf 2000; 57: 155-164.
12. Benson M, Junger A, Quinzio L, et al. Data processing at the anesthesia workstation: from data entry to data presentation. Methods Inf Med 2000; 39: 319-324.
13. Wang X, Gardner RM, Seager PR. Integrating computerized anesthesia charting into a hospital information system. Int J Clin Monit Comput 1995; 12: 61-70.
14. Edsall DW, Deshane P, Giles C, Dick D, Sloan B, Farrow J. Computerized patient anesthesia records: less time and better quality than manually produced anesthesia records. J Clin Anesth 1993; 5: 275-283.
15. Wager KA, Lee FW, White AW, Ward DM, Ornstein SM. Impact of an electronic medical record system on community-based primary care practices. J Am Board Fam Pract 2000; 13: 338-348.
16. Gardner RM, Prakash O. Challenges and opportunities for computerizing the anesthesia record. J Clin Anesth 1994; 6: 333-341.
17. Hodge B, Thompson JF. Noise pollution in the operating theatre. Lancet 1990; 335: 891-894.
18. Hohnloser JH, Kadlec P, Puerner F. Coding clinical information: analysis of clinicians using computerized coding. Method Inf Med 1996; 35: 104-107.
19. Krall MA, Chin H, Dworkin L, Gabriel K, Wong R. Improving clinician acceptance and use of computerized documentation of coded diagnosis. Am J Manag Care 1997; 3: 597-601.
20. Boy O, Ohmann C, Aust B, et al. Systematische Evaluierung der Anwenderzufriedenheit von Ärzten mit einem Krankenhausinformationssystem - Erste Ergebnisse. Stud Health Technol Inform 2000; 77: 518-522.
21. Cork RD, Detmer WM, Friedman CP. Development and initial validation of an instrument to measure physicians' use of, knowledge about, and attitudes toward computers. J Am Med Inform Assoc 1998; 5: 164-176.
22. Dumont R, van der Loo R, van Merode F, Tange H. User needs and demands of a computer-based patient record. Medinfo 1998; 9: 64-69.
Keywords:

DATA COLLECTION, questionnaires; MANAGEMENT INFORMATION SYSTEMS, hospital information system; MEDICAL RECORD SYSTEMS, COMPUTERIZED; TEACHING; computer user training

© 2003 European Academy of Anaesthesiology