A ROC curve analyzed for %COHb ≥10% as significant carboxyhemoglobinemia maximized sensitivity and specificity at an SpCO of 6.6%, with an area under the curve (AUC) of 0.84 (95% CI, 0.79–0.89) (Fig. 5A). Sensitivity, specificity, and predictive values for this cutoff are summarized in Table 5. The ROC curve for %COHb ≥5% as “positive” carboxyhemoglobinemia had an AUC of 0.88 (95% CI, 0.84–0.92) (Fig. 5B).
MetHb levels were in the low range of normal for all subjects during the study. SaO2 had a slight effect on SpMet bias (P = 0.008) with a slightly more positive bias as lower SaO2, but this was only 0.006% for every 1% of desaturation (Fig. 6A). COHb produced a small but statistically significant effect on SpMet bias (P = 0.018, Fig. 6B).
Women had significantly lower PI than men, with a mean (95% CI) of 1.7% (0.4%–3.0%) vs 5.5% (3.9%–7.1%) (P = 0.0014). Despite efforts to warm hands, 1 female subject had a PI <1%. Overall, PI had no significant effect on the Rainbow SpO2 bias (P = 0.086, Fig. 7A) or SpCO bias (P = 0.95, Fig. 7B). SpO2 bias had lower precision (higher SD) at PI <2% (P < 0.0001), although SpCO bias did not (P = 0.93).
The primary purpose of this study was to assess the accuracy of pulse CO-oximetry measurement of COHb in the presence of mild to moderate hypoxemia. We found that, in the presence of 10% to 12% COHb, accuracy for detecting hypoxemia was not degraded, with an Arms still <3%. Mild to moderate hypoxemia did not appreciably degrade the accuracy of COHb measurement as indicated by the bias, but slightly degraded the precision and Arms. However, when the SaO2 was <85%, the devices read “low signal IQ” and would not report SpCO values.
The usefulness of a noninvasive measurement of COHb has been demonstrated in numerous case reports,18–20 and studies of occult CO poisoning.4,5 The ability to diagnose suspected cases of CO exposure in a timely manner and to avoid unnecessary invasive testing requires good positive and negative predictive values. Detecting elevated COHb levels to enable rapid initiation of appropriate treatment, including normobaric and hyperbaric oxygen, may improve outcomes.21 Detecting elevated COHb levels, even at levels that may not be clinically important, may identify sources of CO exposure at home or at work that could cause more serious harm in the future or lead to testing of others exposed to the event. Determining whether COHb levels are improving in patients requires more frequent measurements.
No prior studies involving pulse CO-oximetry in patients mention simultaneous evaluation in the presence of hypoxemia. Previous reports of pulse CO-oximetry in emergency room patients probably involved supplemental oxygen administration.5,12 Simultaneous hypoxemia would be likely in cases of smoke inhalation and loss of consciousness. However, studies comparing SpCO and blood values did not include such data in the field because of the lack of blood analyzers. The manufacturer reports that the Rainbow Radical-7 is not accurate with simultaneous methemoglobinemia. We reported erroneous SpCO reading in an earlier study with induced methemoglobinemia,7 but we did not test the combination of MetHb and COHb in the current study.
Our results concerning the detection of COHb during normoxia were similar to 2 studies in normal volunteers in laboratory settings at normal levels of oxygen that found good accuracy and precision.9,10 In contrast, studies in emergency room patients have shown larger bias, from −3% to +4%, and wider limits of agreement, spanning from lower to upper limits of agreement of 15% to 25%.4,11–13,22 A study of 139 patients in a pulmonary function laboratory found a low bias, but fairly wide limits of agreement.23 In some studies, significant delays occurred between the SpCO reading and blood sampling for COHb measurements, making accurate assessment of bias difficult.4,12
Skin color and gender alter pulse-oximeter performance.15,16 Many studies have not reported the gender and skin pigmentation of study subjects, making direct comparison of the results difficult, although Touger et al.13 attempted some analysis excluding dark-skinned patients. Our subjects were intentionally of different genders, ethnicities, and skin color because it is important that the results apply broadly and is also required by the FDA for studies of pulse-oximeter accuracy. Our study did not have the power to resolve differences in performance related to skin pigmentation.
The sensitivity, specificity, and positive and negative predictive values for detecting COHb in the presence of mild hypoxia were acceptable (Table 5 and Fig. 5). However, our study was not optimally designed to measure these variables because most of our data were clustered around subjects’ baseline values and at the target of 10% to 12% COHb. Testing in volunteers at these higher levels would not be appropriate, especially if combined with hypoxia. ROC analysis of our data (Fig. 5 and Table 5) indicated that maximal sensitivity and specificity for detecting COHb levels ≥10% was an SpCO of 6.6%. Similarly, Roth et al.12 found that an SpCO of 6.6% was 94% sensitive in identifying the 17 patients with CO poisoning, with 77% specificity. Decreasing the SpCO thresholds to maximize sensitivity in order to prevent missing anyone with potential serious CO poisoning might be better for initial screening. For our data, a threshold of 5% SpCO is >90% sensitive in detecting all subjects with COHb ≥10%. While lowering the threshold decreases specificity, this may be desirable as a screening test for the presence of COHb. In a study of emergency room patients, Suner et al.4 reported 94% sensitivity and 54% specificity for the Rad-57 from 64 data points. However, Touger et al.13 found lower sensitivity (48%) but good specificity and positive and negative predictive values for a 15% COHb cutoff. The exact clinical threshold indicating treatment necessity is not clear, although a level of COHb of 25% has been suggested for hyperbaric oxygen treatment.22
We studied 2 devices of each type, randomly placing them on different hands and different fingers. This increased the total number of data points for analysis. The oximeters behaved similarly, although slight differences, typically only 1% to 2%, were apparent at times. We have previously found small difference in probes types.16 Limitations on the reasonable number of probes we can study in volunteers mean that we do not have data on all probe types in this study.
Defining acceptable performance and accuracy is somewhat arbitrary, but depends on the clinical purpose of the device. Clearly, measurement of SpCO is less accurate than for SpO2 and SpMet, being reported only to a whole number. Piatkowski et al.11 concluded that the bias of 3.15% (precision 2.36%) represented acceptable accuracy. Roth et al.12 concluded that accuracy was acceptable at a bias of 2.32 ± 4.01. Touger et al.13 defined ±5% as acceptable accuracy, reporting that 33% of data was outside this range, which was discussed in an editorial by Maisel and Lewis24 in the Annals of Emergency Medicine.
MetHb readings from the Rad-7 CO-oximeters showed excellent stability with changes in COHb and SaO2. Although a repeated-measures analysis is extremely robust in detecting small changes in bias with a large number of measurements, the changes in SpMet bias as shown in Figure 6 are not clinically important. Changes in SpMet bias might not be expected from induced carboxyhemoglobinemia and hypoxemia; however, early versions of the pulse CO-oximeter could not discriminate multiple different hemoglobin species.7 This was corrected in subsequent versions.8 Cyanide toxicity can occur in fires because of combustion of nitrogen-containing compounds. Treatment is with sodium nitrite, which produces methemoglobinemia.25 This creates a clinical scenario in which MetHb and COHb would be present concurrently.
Within the range of carboxyhemoglobinemia studied, both the Rainbow and the conventional pulse oximeters were able to detect hypoxemia even in the presence of elevated COHb, with an Arms of 1.70% and 2.05% being well below the acceptable FDA threshold of 3%. Confusion of COHb and oxyhemoglobin might lead the oximeters to read a higher SpO2 (positive bias) even at low oxygen saturation. Data on standard pulse-oximeter accuracy with carboxyhemoglobinemia have shown a slightly negative bias at high COHb levels, with an obvious “gap” in measuring “fractional” oxygen saturation.6,26
Because of the similarity of the absorption spectra of oxyhemoglobin and COHb, measurement may be intrinsically more difficult than for MetHb, which has greater spectral separation from oxyhemoglobin. The “pulse-oximeter gap” describes the difference between SpO2 and fractional oxygen saturation with elevated %COHb, and implied that pulse oximeters were reading COHb as if it were oxyhemoglobin.26–28 Current pulse oximeters are calibrated with functional oxygen saturation, so accuracy should properly be considered only for functional SaO2. The ability of the pulse oximeters to detect oxygen desaturation in the presence of elevated COHb suggests there is no clinically relevant “confusion.”
Similar to findings with other parameters from pulse oximetry,29–31 accuracy and precision were degraded slightly at lower PI. The effect was not dramatic, and it should be noted that we were actively warming subjects’ hands.
A volunteer study has both limitations and advantages over other study designs. In volunteers, we are limited as to the degree of carboxyhemoglobinemia and hypoxemia that we can safely produce. We have set the upper limit to 15%, with a target of 12% COHb in the setting of hypoxemia. Twelve subjects may not be adequate to produce robust data on sensitivity, specificity, and predictive performance, although the study provided a total >150 data points for SpCO and nearly 300 data points for SpMet and SpO2 with simultaneous arterial blood measurements. However, the repeated-measures design is very robust for determining interaction between low SaO2 and elevated COHb. A laboratory setting also provides excellent coordination of blood draws and noninvasive measurement. Our step changes in COHb, rather than continuous breathing of CO, provides better stability for the coordination of SpCO and blood measurements. Continued updates and changes to hardware and software make comparisons between our studies and other past or future studies difficult. We treated laboratory measurements as a gold standard, although even such devices may have inaccuracies. Our population of study subjects may not represent all patient populations, but did have intentional variability of gender and ethnicity. Performance in a controlled laboratory environment may still differ from the clinical setting in patients with multiple comorbidities.
Accuracy of the Masimo pulse CO-oximeter for measuring COHb was not affected by hypoxemia to a clinically significant degree. However, hypoxemia did result in a significant increase in device-reported low signal errors and blank COHb readings. COHb increases up to 12% minimally affected measurements of SpO2 and SpMet. Sensitivity, specificity, and predictive value at up to 12% COHb were good (AUC of the ROC curve >0.8), but more data at higher COHb levels would be useful in helping clinicians define appropriate thresholds for optimizing screening for potential CO poisoning. Given the history of pulse-oximeter development, further investments in multiwavelength pulse-oximeter technology are likely to improve accuracy and performance.
Name: John R. Feiner, MD.
Contribution: This author helped design the study, conduct the study, record the data, analyze the data, and prepare the manuscript.
Attestation: John Feiner approved the final manuscript. John Feiner attests to the integrity of the original data and the analysis reported. John Feiner is the archival author.
Name: Mark D. Rollins, MD, PhD.
Contribution: This author helped perform the study and revise the manuscript.
Attestation: Mark Rollins read and approved the final manuscript.
Name: Jeffrey Sall, MD, PhD.
Contribution: This author helped perform the study and revise the manuscript.
Attestation: Jeffrey W. Sall read and approved the final manuscript.
Name: Helge Eilers, MD.
Contribution: This author helped perform the study and revise the manuscript.
Attestation: Helge Eilers read and approved the final manuscript.
Name: Paul Au, BS.
Contribution: This author helped recruit the study patients and collect the data.
Attestation: Paul Au read and approved the final manuscript.
Name: Philip E. Bickler, MD, PhD.
Contribution: This author helped design the study, conduct the study, and revise the manuscript.
Attestation: Philip Bickler read and approved the final manuscript. Philip Bickler attests to the integrity of the original data and the analysis reported.
This manuscript was handled by: Dwayne R. Westenskow, PhD.
1. Centers for Disease Control and Prevention. . Carbon monoxide exposures: United States, 2000–2009. MMWR Morb Mortal Wkly Rep. 2011;60:1014–7
2. Hampson NB, Weaver LK. Carbon monoxide poisoning: a new incidence for an old disease. Undersea Hyperb Med. 2007;34:163–8
3. Hampson NB, Scott KL, Zmaeff JL. Carboxyhemoglobin measurement by hospitals: implications for the diagnosis of carbon monoxide poisoning. J Emerg Med. 2006;31:13–6
4. Suner S, Partridge R, Sucov A, Valente J, Chee K, Hughes A, Jay G. Non-invasive pulse CO-oximetry screening in the emergency department identifies occult carbon monoxide toxicity. J Emerg Med. 2008;34:441–50
5. Chee KJ, Nilson D, Partridge R, Hughes A, Suner S, Sucov A, Jay G. Finding needles in a haystack: a case series of carbon monoxide poisoning detected using new technology in the emergency department. Clin Toxicol (Phila). 2008;46:461–9
6. Hampson NB. Pulse oximetry in severe carbon monoxide poisoning. Chest. 1998;114:1036–41
7. Feiner JR, Bickler PE, Mannheimer PD. Accuracy of methemoglobin detection by pulse CO-oximetry during hypoxia. Anesth Analg. 2010;111:143–8
8. Feiner JR, Bickler PE. Improved accuracy of methemoglobin detection by pulse CO-oximetry during hypoxia. Anesth Analg. 2010;111:1160–7
9. Barker SJ, Curry J, Redford D, Morgan S. Measurement of carboxyhemoglobin and methemoglobin by pulse oximetry: a human volunteer study. Anesthesiology. 2006;105:892–7
10. Zaouter C, Zavorsky GS. The measurement of carboxyhemoglobin and methemoglobin using a non-invasive pulse CO-oximeter. Respir Physiol Neurobiol. 2012;182:88–92
11. Piatkowski A, Ulrich D, Grieb G, Pallua N. A new tool for the early diagnosis of carbon monoxide intoxication. Inhal Toxicol. 2009;21:1144–7
12. Roth D, Herkner H, Schreiber W, Hubmann N, Gamper G, Laggner AN, Havel C. Accuracy of noninvasive multiwave pulse oximetry compared with carboxyhemoglobin from blood gas analysis in unselected emergency department patients. Ann Emerg Med. 2011;58:74–9
13. Touger M, Birnbaum A, Wang J, Chou K, Pearson D, Bijur P. Performance of the RAD-57 pulse CO-oximeter compared with standard laboratory carboxyhemoglobin measurement. Ann Emerg Med. 2010;56:382–8
14. Lee AS, Mellins RB. Lung injury from smoke inhalation. Paediatr Respir Rev. 2006;7:123–8
15. Bickler PE, Feiner JR, Severinghaus JW. Effects of skin pigmentation on pulse oximeter accuracy at low saturation. Anesthesiology. 2005;102:715–9
16. Feiner JR, Severinghaus JW, Bickler PE. Dark skin decreases the accuracy of pulse oximeters at low oxygen saturation: the effects of oximeter probe type and gender. Anesth Analg. 2007;105:S18–23
17. Bland JM, Altman DG. Agreement between methods of measurement with multiple observations per individual. J Biopharm Stat. 2007;17:571–82
18. Roth D, Hubmann N, Havel C, Herkner H, Schreiber W, Laggner A. Victim of carbon monoxide poisoning identified by carbon monoxide oximetry. J Emerg Med. 2011;40:640–2
19. Crawford DM, Hampson NB. Fire and ice: diagnosis of carbon monoxide poisoning in a remote environment. Emerg Med J. 2008;25:235–6
20. Bledsoe BE, Nowicki K, Creel JH Jr, Carrison D, Severance HW. Use of pulse CO-oximetry as a screening and monitoring tool in mass carbon monoxide poisoning. Prehosp Emerg Care. 2010;14:131–3
21. Hampson NB. Noninvasive pulse CO-oximetry expedites evaluation and management of patients with carbon monoxide poisoning. Am J Emerg Med. 2012;30:2021–4
22. Coulange M, Barthelemy A, Hug F, Thierry AL, De Haro L. Reliability of new pulse CO-oximeter in victims of carbon monoxide poisoning. Undersea Hyperb Med. 2008;35:107–11
23. Ruppel GL, Wilson HA, Gall VK, Hempkens JA. Multi-wavelength pulse oximeter is not suitable for adjusting D(LCO) measurements. Respir Care. 2011;56:1115–21
24. Maisel WH, Lewis RJ. Noninvasive measurement of carboxyhemoglobin: how accurate is accurate enough? Ann Emerg Med. 2010;56:389–91
25. Cummings TF. The treatment of cyanide poisoning. Occup Med (Lond). 2004;54:82–5
26. Barker SJ, Tremper KK. The effect of carbon monoxide inhalation on pulse oximetry and transcutaneous PO2. Anesthesiology. 1987;66:677–9
27. Buckley RG, Aks SE, Eshom JL, Rydman R, Schaider J, Shayne P. The pulse oximetry gap in carbon monoxide intoxication. Ann Emerg Med. 1994;24:252–5
28. Toffaletti J, Zijlstra WG. Misconceptions in reporting oxygen saturation. Anesth Analg. 2007;105:S5–9
29. Miller RD, Ward TA, Shiboski SC, Cohen NH. A comparison of three methods of hemoglobin monitoring in patients undergoing spine surgery. Anesth Analg. 2011;112:858–63
30. Gehring H, Hornberger C, Matz H, Konecny E, Schmucker P. The effects of motion artifact and low perfusion on the performance of a new generation of pulse oximeters in volunteers undergoing hypoxemia. Respir Care. 2002;47:48–60
© 2013 International Anesthesia Research Society
31. Broch O, Bein B, Gruenewald M, Höcker J, Schöttler J, Meybohm P, Steinfath M, Renner J. Accuracy of the pleth variability index to predict fluid responsiveness depends on the perfusion index. Acta Anaesthesiol Scand. 2011;55:686–93