Secondary Logo

Journal Logo

Estimating and reporting error rates, and detecting improvements

Webster, Craig S.

European Journal of Anaesthesiology (EJA): January 2018 - Volume 35 - Issue 1 - p 60–61
doi: 10.1097/EJA.0000000000000602

From the Centre for Medical and Health Sciences Education, University of Auckland, Auckland, New Zealand

Correspondence to Craig S. Webster, Centre for Medical and Health Sciences Education, University of Auckland, Private Bag 92-019, Auckland, New Zealand Tel: +649 923 6525; e-mail:


I was interested to read in your journal the latest and very useful recommendations on medication safety from the European Board of Anaesthesthiology.1 However, I found one statement in the justification for the development of the new guidelines somewhat misleading — it states that early retrospective studies put the medication error rate in anaesthesia at one in 133, but that a recent observational study has estimated the rate to be as high as one in 20.1 Let us deal with the issue of nonobservational studies first. The rate of drug error per anaesthetic of one in 133, actually comes from a prospective incident monitoring study conducted by our group, and in fact, despite this study being published in 2001, to my knowledge remains the highest, and we believe the most accurate, estimate of its kind in the world.2 Many studies of drug error in anaesthesia do not allow the calculation of an error rate because they have not captured a denominator. It is also well known that ‘retrospective’ studies tend to underestimate adverse events even when they are large and include a denominator — often reporting rates a magnitude lower than one in 133. However, all prospective studies are not created equal. Prospective incident monitoring studies which are based on incident data that were collected as part of a wider incident monitoring scheme tend to underestimate particular error types such as drug error. Our prospective study used what we called facilitated incident monitoring in which we exclusively studied drug error in anaesthesia and requested the anonymous return of a study form for every anaesthetic conducted, the vast majority of which indicated that no incident had occurred.2 A dedicated staff member also followed up on anaesthetic records that were filed without an accompanying incident form. Using the same technique, this study has now been replicated in the United States, South Africa and China, yielding similar estimates of drug error in three of four studies (Table 1).2–5

Table 1

Table 1

The estimate of drug error during anaesthesia of one in 20, based on an observation study by Nanji et al.6 has recently been the centre of some debate and criticism.7 This study used very broad definitions of events, which it is believed overinflated the estimated error rate, and authors failed to include sufficient event data in their study to allow readers to verify the origin of the rate for themselves.7 Observation studies do typically identify error rates around a magnitude higher than those from even the best incident monitoring studies — this is presumed to be largely because of the fact that many events are not reported simply because the clinician is not aware that they have occurred. However, analysing data in a comparable way, Nanji et al.'s6 results appear to be 10 times higher than even other comparable observation studies estimating drug error during anaesthesia, suggesting that this study may not represent a reliable estimate of the error rate.7

So why does all this matter? Accurately estimating baseline data is the important first step in detecting improvements because of safety initiatives. It is important to understand the relative reliability of data and the strengths and weaknesses of different study techniques. For example, the recent recommendations statement from the European Board of Anaesthesiology mentions that between-class drug error swaps are typically more dangerous than within-class swaps.1 We recently showed evidence for the value of the international colour-code standard for anaesthetic labels in preventing between-class swap errors. Using a facilitated incident monitoring study of 74 478 anaesthetic cases, syringe swap errors between differently-coloured drug classes were reduced by 66%, from 47 between-class swaps in an estimated 550 105 bolus drug administrations (0.009%) to 5 in 183 852 bolus drug administrations (0.003%), P = 0.01.8 I believe this result represents the first evidence of significant safety benefits with the use of the international colour-code standard for anaesthetic labels, but it relies on accurate data.1

Back to Top | Article Outline

Acknowledgements related to this article

Assistance with the letter: none

Financial support and sponsorship: none.

Conflict of interest: CSW owns a small number of shares in a company called SaferSleep Ltd, which aims to improve safety in healthcare.

Back to Top | Article Outline


1. Whitaker D, Brattebo G, Trankler S, et al. European Section and Board of Anaesthesiology of the UEMS. The European Board of Anaesthesiology recommendations for safe medication practice: first update. Eur J Anaesthesiol 2017; 34:4–7.
2. Webster CS, Merry AF, Larsson L, et al. The frequency and nature of drug administration error during anaesthesia. Anaesth Intensive Care 2001; 29:494–500.
3. Bowdle A, Kruger C, Grieve R, et al. Anesthesia drug administration error in a university hospital. Anesthesiology 2003; 99:A1358.
4. Llewellyn RL, Gordon PC, Wheatcroft D, et al. Drug administration error: a prospective survey from three South African teaching hospitals. Anaesth Intensive Care 2009; 37:93–98.
5. Zhang Y, Dong YJ, Webster CS, et al. The frequency and nature of drug administration error during anaesthesia in a Chinese hospital. Acta Anaesthesiol Scand 2013; 57:158–164.
6. Nanji KC, Patel A, Shaikh S, Seger DL, Bates DW. Evaluation of perioperative medication errors and adverse drug events. Anesthesiology 2016; 124:25–34.
7. Bowdle TA, Jelacic S, Nair B. Evaluation of perioperative medication errors. Anesthesiology 2016; 125:429–431.
8. Webster CS, Larsson L, Frampton CM, et al. Clinical assessment of a new anaesthetic drug administration system: a prospective, controlled, longitudinal incident monitoring study. Anaesthesia 2010; 65:490–499.
© 2018 European Society of Anaesthesiology