Journal Logo

Feature Articles

Comparative Performance of Pulmonary Ultrasound, Chest Radiograph, and CT Among Patients With Acute Respiratory Failure*

Tierney, David M. MD, FACP1; Huelster, Joshua S. MD2; Overgaard, Josh D. MD1; Plunkett, Michael B. MD3; Boland, Lori L. MPH4; St. Hill, Catherine A. DVM, PhD4; Agboto, Vincent K. PhD, MS4; Smith, Claire S. MS4; Mikel, Bryce F. MD1; Weise, Brynn E. MD1; Madigan, Katelyn E. MD5; Doshi, Ameet P. MD1; Melamed, Roman R. MD, FCCP2

Author Information
doi: 10.1097/CCM.0000000000004124

Abstract

Acute respiratory failure (ARF) requiring mechanical ventilatory support is the most common reason for ICU admission and has high mortality (1,2). Initial misdiagnosis of ARF etiology is common and increases mortality (3). With the broad differential of diseases causing ARF, efficiency of diagnosis is not only related to accuracy of the chest imaging modality, but also its ability to anatomically localize a process and monitor for change with treatment.

Portable chest radiograph (pCXR) lacks the sensitivity and specificity of CT (4–6), yet may be favored in some settings due to speed, lower cost and radiation, and risk associated with ICU patient transport especially with technologically complex support modes such as extracorporeal membrane oxygenation (ECMO) (7,8). Pulmonary ultrasound (PU) has demonstrated the benefits of pCXR’s portability, speed, and lower cost while approaching accuracy of chest CT for pneumonia, interstitial processes, acute respiratory distress syndrome (ARDS), pleural effusion, and pneumothorax (9–15). The Society of Critical Care Medicine and international guidelines support incorporation of point-of-care ultrasound (POCUS) in diagnosis of these conditions (16,17). In addition to its initial diagnostic ability, POCUS offers clinicians a radiation-free tool to repeatedly reassess for progression or regression of pulmonary findings in real-time at the bedside which can further aid in tailoring treatment and differential diagnosis.

The majority of research reporting diagnostic accuracy of PU has defined agreement as compatible ultrasound findings anywhere within the correct lung as identified by CT. However, two small studies (n = 32 and 20) conducted in patients with ARDS have demonstrated a refined ability of PU to accurately localize findings beyond the correct lung to distinct lung zones on CT defined not by anatomic lung lobe (i.e., right lower lobe, right middle lobe), but by zones corresponding to contemporary protocolized ultrasound examination regions (18,19).

The goal of this study was to concurrently evaluate the accuracy of both PU and pCXR with CT not only for agreement of findings within the ipsilateral lung or a correlating CT zone, but within the specific anatomic lobe using a previously published, quick, pragmatic, 9-zone PU protocol (20) among diverse medical/surgical, cardiac, and neurologic ICU patient populations with a wide array of ARF diagnoses.

MATERIALS AND METHODS

Setting and Study Design

This prospective cohort study took place in a 670-bed, 62 ICU bed, quaternary care, teaching hospital. Patients included in this study were a subset from a larger (n = 250), previously published study evaluating a protocolized 9-location PU examination in patients with ARF requiring mechanical ventilation (20). Patients were included in the current study if, in addition to their PU at the time of intubation, they underwent pCXR and chest CT within 24 hours of the PU. The protocol was approved by the institutional review board (Schulman Associates IRB, number 4080-2E).

Imaging and Classification

Details of the full PU examination protocol have been described previously (20). Presence of normal lung, quantified B-line category (B1–B3), atelectasis/consolidation, and pleural effusion was registered in nine anatomically chosen (to approximate lobar anatomy) and clinically applicable PU zones (Fig. 1). The surface area examined within each zone was limited to 7.5 × 5 cm (the size of our hospital ID badge). A comparison of this 9-point examination protocol to other frequently used PU protocols (11,21,22) can be found in Supplemental Content 1 (http://links.lww.com/CCM/F166).

Figure 1.
Figure 1.:
Ultrasound and chest radiograph zone location. AAx = anterior axillary line, LL = left lower, LU = left upper, MC = mid-clavicular line, PAx = posterior axillary line, RL = right lower, RU = right upper.

All PU examinations were performed by one of four study physicians with 3+ years of PU experience who participated in a 1-hour pre-study teaching session to ensure uniformity of PU classification. They independently scored archived video clips representing the spectrum of classifications from 64 patients, and inter-rater agreement for pulmonary classification was calculated. All examinations were performed with SonoSite EDGE portable ultrasound systems and a P21 (1–5 MHz) phased-array transducer (FUJIFILM SonoSite, Bothell, WA) (see additional methods, Pulmonary Ultrasound Exam Detail, Supplemental Content 2, http://links.lww.com/CCM/F167).

Ultrasound zones were classified as normal when the region contained A-lines and lung sliding in the absence of B-lines or consolidation/atelectasis. Presence of B-lines was subclassified by B1, B2, or B3 based on quantity (B1 = 1–3 discrete B-lines present per intercostal space; B3 = confluent B-lines occupying > 50% of an intercostal space; or B2 = quantity of B-lines between B1 and B3). An isolated B1 area in the postero-caudal tip of the lung was classified as normal. Areas of nonaerated lung greater than 3 cm in its axis perpendicular to pleura with dynamic air bronchograms and lack of compressive reason for aeration loss (e.g., pleural effusion, elevated diaphragm) were classified as non-atelectatic consolidation. Zones with small, focal, less than 3 cm areas, of peripheral consolidation were subclassified as “small” consolidations. Areas of nonaerated lung greater than 3 cm in its axis perpendicular to pleura with distinct features suggesting volume loss atelectasis such as presence of a pleural effusion size consistent with volume of nonaerated lung, lack of dynamic air bronchograms, typical atelectatic distribution in the inferior or lateral tip of lung with a smooth re-aeration interface, were classified as atelectasis. A combined classification of atelectasis/consolidation was assigned when the area examined consisted of tissue-like density and the physician was unable to see discriminating features indicative of either specific entity. Pleural effusion was classified as fluid in the pleural space whether simple or complicated with a minimum visceral-parietal pleural dimension greater than or equal to 0.5 cm at any location. Individual classifications were also grouped into the categories of parenchymal abnormality (normal, interstitial, ground glass, atelectasis/consolidation) or pleural effusion for subanalyses.

A single, expert chest radiologist without PU training, blinded to PU results and clinical information, reviewed all pCXR and chest CT studies to classify each anatomical zone (pCXR) and lobe (CT) as normal, interstitial, ground glass (CT only), atelectasis, consolidation, or pleural effusion. CXR zones were defined as shown in Figure 1. CT regions were defined by anatomical lobe.

Agreement for lung findings and zones/lobes across modalities are presented in Table 1. “Lobe-specific” agreement was defined as identification of an agreeing PU/pCXR finding in an agreeing PU/pCXR zone with the CT finding and specific anatomic lobe. “Lung-specific” agreement was defined as identification of a PU/pCXR finding to an agreeing finding anywhere in the ipsilateral lung on CT (not restricted to the specific anatomic lobe mapping in Table 1) (see additional methods, Pulmonary Ultrasound Agreement Details and Rationale, Supplemental Content 2, http://links.lww.com/CCM/F167).

TABLE 1.
TABLE 1.:
Anatomical Correspondence and Definition of Finding Agreement Across CT, Pulmonary Ultrasound, and Portable Single-View Chest Radiograph

Final clinical diagnoses for ARF were adjudicated by two study physicians blinded to, and not having performed, the PU examination (see additional methods, Final Clinical Diagnosis Assignment, Supplemental Content 2, http://links.lww.com/CCM/F167). They then reviewed the PU examination collectively and determined whether it was consistent with the clinical diagnosis.

Data Analysis

Descriptive statistics were used to describe study sample characteristics. With CT findings considered gold standard, the proportion of patients with agreement between PU and CT, and pCXR and CT was compared using two-sample tests of proportions. Level of agreement was examined for both the combination of correct finding in the correct lobar correlate (lobe-specific agreement) and for the correct finding anywhere in the ipsilateral lung (lung-specific agreement). The sensitivity of PU and pCXR to detect CT findings was evaluated overall, by body mass index (BMI), and duration of time between CT and PU/pCXR (dichotomized at sample medians) using z tests for difference in proportions. Fleiss’s kappa coefficient was used to assess inter-rater agreement in the pre-study training set. A sample size of 67 patients achieved 83% power to detect an overall difference of 22% between the agreement of ultrasound and pCXR, using the two-sided z test with pooled variance. Statistical analyses were conducted at 5% significance level. Analyses were conducted using Stata 14.1 (StataCorp, LP, College Station, TX).

RESULTS

From the broader 250 patient cohort (20), 67 patients (Table 2) met inclusion criteria. The most common ARF diagnosis was pneumonia (30%), followed by aspiration, cardiac arrest, sepsis, congestive heart failure (CHF), and ARDS. Median partial pressure of arterial oxygen/percentage of inspired oxygen (P/F) ratio was 214 (interquartile range [IQR], 135–336) and overall mortality was 28%.

TABLE 2.
TABLE 2.:
Patient Characteristics

Inter-rater agreement was “very good” overall (kappa = 0.83), “almost perfect” for assignment of normal lung (kappa = 0.96), “very good” for atelectasis/consolidation (kappa = 0.84), and “substantial” for specific B1, B2, and B3 classifications (kappa = 0.77, 0.61, 0.74, respectively).

PU had significantly better overall lobe-specific agreement with CT than pCXR, when agreement was examined by right versus left lung, and within each zone/lobe individually. Lung-specific agreement of PU with CT for parenchymal findings (excludes pleural effusion) was lower in the left lung compared with right (83.6% vs 92.5%; p = 0.109), and PU performed worst (82.1%) in the left lower lobe (LLL) (Table 3).

TABLE 3.
TABLE 3.:
Agreement of Pulmonary Ultrasound and Portable Chest Radiograph Findings With CT

Analysis by specific CT finding (Table 3) demonstrated that PU consistently had better agreement with CT than pCXR, with the greatest difference for interstitial findings (86.2% vs 28.6%; p < 0.001). Patient BMI and time between PU and CT did not significantly affect agreement of PU overall (BMI < 29 = 88.5% vs BMI ≥ 29 = 89.4%, p = 0.734; < 12.2 hr = 90.3% vs ≥ 12.2 hr = 87.6%, p = 0.295) or for any individual finding (e.g., interstitial, consolidation).

The overall PU examination was determined to be consistent with final diagnosis in 62 of 67 patients. Four cases where PU did not agree are described below with corresponding images (Fig. 2) as illustrations of potential PU pitfalls.

Figure 2.
Figure 2.:
Discrepant case images. A and B, Case 1 chest CT; arrows = pleural-based nodules. C, Case 2 chest CT; arrows = bullae. D, Case 3 chest CT; arrow = right upper lobe mass. E, Case 3 portable chest radiograph without visible right upper lobe mass. F, Case 4 chest CT; arrow = posterior left lower lobe consolidation missed by ultrasound; *stomach. L = left, R = right. See high resolution image versions of the figures (Supplemental Content 3, http://links.lww.com/CCM/F168).
  • 1) A patient with metastatic pleural nodules from breast cancer and LLL pneumonia/empyema. CT showed diffuse pleural nodules in the right thorax and consolidation/effusion in the LLL (Fig. 2, A and B). Classification by PU showed interstitial findings in the right middle and upper lobes and consolidation in the right lower lobe. These interstitial and consolidative findings on PU were likely the pleural-based nodules. The PU correctly identified LLL pneumonia and pleural effusion.
  • 2) A patient with bullous emphysema and bilateral scarring on CT (Fig. 2C) was classified by PU as diffuse interstitial process. Without knowledge of the patient’s prior bilateral scarring, this scenario could result in an obstructive lung disease exacerbation being interpreted as hydrostatic pulmonary edema or influenza pneumonia if PU findings are not integrated with other clinical and POCUS data. Although PU findings are consistent with the diagnosis, it demonstrates the low specificity of B-lines in isolation.
  • 3) A patient with a superior/anterior right upper lobe mass on CT, not visualized on pCXR (Fig. 2, D and E), had a normal PU examination due to the superior and nonpleural-based location of the mass.
  • 4) In addition to a right lower lobe pneumonia visualized with PU, the posteromedial LLL consolidation on CT scan was missed with PU of left thorax. The large, superiorly displaced stomach bubble/diaphragm impeded visualization in this recumbent patient because the transducer was not far enough posterior (Fig. 2F).

DISCUSSION

This study adds to the literature on the agreement of PU with CT and provides new evidence that a clinically feasible 9-point PU protocol can accurately localize findings not only to a specific lung or region but to anatomic lobe. The lobe-specific agreement in this study adds further nuance to the lung zone localizing ability of PU demonstrated in the prior smaller studies of ARDS patients by Lichtenstein et al (19) and Chiumello et al (18). By prospectively and concurrently evaluating both lung- and lobe-specific agreement for pCXR and PU across a broad set of ARF diagnoses, a wide range of illness severity (P/F IQR, 135–336), and within medical/surgical, neurologic, and cardiac ICUs, our study provides more generalizable evidence regarding the accuracy of PU for localization of specific findings and adds support for the clinical and diagnostic value of the 9-point PU protocol and scoring system previously shown to correlate with mortality, length of stay, and P/F ratio (20).

Localization of findings to the correct lung with PU is often clinically adequate, and the higher “lung-specific” (compared to “lobe-specific”) agreement found in our study (overall right lung parenchymal and pleural effusion agreement = 92.5% and 98.5%, respectively; left lung = 83.6% and 98.5%, respectively) is consistent with the majority of existing PU literature evaluating agreement by lung and is indicative of the accuracy of PU as a diagnostic tool. However, “lobe-specific” localization also has clinical utility. It can support diagnoses such as aspiration by localizing findings to commonly involved lobes such as the superior segments of the lower lobe or posterior segment of the upper lobe (23), or assist in differentiating acute findings in patients with previously known fibrosis/scarring in a specific lobe. It can guide bronchoscopic intervention, especially in patients difficult to transport for CT (e.g., ECMO patients), to a lobe with presumed airway obstruction for suctioning or to a lobar parenchymal consolidation for culture. Evaluation of lobe-specific agreement also allows for examination of differences in PU agreement within specific lobes such as the LLL which can be difficult to visualize due to recumbent patient positioning and positioning of the heart in the left hemithorax. Lower agreement of PU in the LLL (82.1%) reveals a potential pitfall of PU in mechanically ventilated patients that would likely improve with examination of posterior zones. Basal regions, especially the left retrocardiac region can also be difficult to evaluate with pCXR and is a potential shortcoming of both modalities (24).

The PU lobe-specific agreement in our study (87%) is similar to that reported by Lichtenstein et al (19) where overall concordance of PU with CT regions (not anatomic lobes) among a cohort of 32 ARDS patients was 83%. Both studies report very good inter-rater agreement across categories and show significantly greater accuracy of PU compared with CXR. The main differences between our study and the study by Lichtenstein et al (19) include as follows: 1) size of study (67 vs 32 patients, respectively), 2) a wide spectrum of ARF diagnoses versus ARDS alone, 3) a 9-point examination protocol limited to a 7.5 × 5 cm area at each point versus a 12 zone examination with full exploration of each zone, and 4) PU examination point agreement with CT anatomic lobes versus CT zones chosen to approximate the PU examination areas.

PU had better agreement with CT than pCXR across a variety of findings but outperformed pCXR most strongly for interstitial process identification (Table 3). The lower agreement of PU for the normal CT classification (78.8%) may in fact reflect superior sensitivity of PU compared with CT for subtle interstitial processes seen as 1–3 B-lines in an interspace. One might argue 1–3 B-lines (B1 classification) is a normal finding, especially in dependent lung locations. However, in our protocol, the B1 classification was assigned to agree with the “interstitial” rather than “normal” (except when isolated to the postero-caudal tip of the lung). Our decision to map the B1 PU classification in this manner reflects the clinical experience that occasional B-lines in a nondependent location often represents pulmonary pathology but when isolated to the postero-caudal lung tip are often insignificant. Prior studies evaluating a greater surface area of the chest have used similar approaches to grouping B-line quantity but have categorized 1–2 B-lines or an “isolated B-line” as “normal” (25,26). A comparatively smaller chest surface area was examined in our study, increasing the chance that 1–3 B-lines in the examined area represent the penumbra of a more significant process and was a factor in our decision to not map the B1 classification to normal lung on CT. Due to the divergence from previous classification schemes, a sensitivity analysis was performed evaluating agreement if all B1 zones were reclassified as normal and showed a 5–10% reduction in lobe-specific and overall agreement between PU and CT (Supplemental Table 1, Supplemental Digital Content 4, http://links.lww.com/CCM/F169). This supports that the B1 classification (except when isolated to the postero-caudal tip of the lung) correlates better with an interstitial process on CT than with normal lung using this 9-point protocol among this patient population.

Our study has limitations, some inherent to clinical POCUS workflow. First, study physicians did not discuss patients with caregivers prior to or during the examination. However, clinical clues in the room, such as antibiotic or diuretic infusions, may have influenced zone interpretation. Second, the inclusion requirement of CT and pCXR within 24 hours of PU examination may have introduced a bias for more severe illness in this subgroup. Although not detracting from the findings, this may impact generalizability. However, comparison of the original 250 patient cohort (20) and the 67 patient subgroup showed strong similarities in clinical markers (mortality: 24% vs 28%, median ventilator days: 4.1 vs 4.1, mean P/F ratio: 223 vs 234, respectively) and diagnoses (pneumonia: 30% vs 30%, aspiration: 14% vs 15%, sepsis: 12% vs 9%, CHF 11% vs 8%, ARDS and chronic obstructive pulmonary disease/asthma: 7% vs 6%, respectively), thus supporting greater generalizability of our subgroup results. Third, to preserve PU protocol efficiency, each zone’s examination area was restricted to the size of our hospital ID badge potentially resulting in a lower sensitivity for processes just outside the scanned area. In clinical use when suspicion is high and a limited PU examination is normal, a more detailed examination would occur and likely improve sensitivity. Fourth, duration of time between imaging modalities may have positively or negatively impacted agreement, but we did not observe any meaningful impact on agreement with increasing elapsed time. This may be in part due to the severity of illness and predominance of slowly resolving radiographic processes (pneumonia and aspiration) comprising 45% of diagnoses. We conjecture that in healthier patients with more rapidly changing ARF etiologies, time between imaging studies may have greater impact on agreement. Finally, examination of whether PU examination summaries matched adjudicated clinical diagnoses had the potential for significant bias. Therefore, these data are included only to illustrate potential pitfalls of the PU examination, and to use available, correlated CT data to better understand the limitations of our PU protocol in certain scenarios.

CONCLUSIONS

The results of this study, conducted within a clinical ICU workflow and in a population with diverse ARF diagnoses and severity, demonstrate strong support for the role of PU in not only pulmonary pathology identification but also its lobar localization; in this regard, PU significantly outperformed pCXR. The study also illustrates potential scenarios where PU has limitations as compared with CT. In the hands of well-trained providers with an appreciation for its limitations, PU can be an invaluable, cost-saving, risk-reducing imaging modality as an adjunct or replacement for pCXR and CT in patients with ARF.

REFERENCES

1. Barrett ML, Smith MW, Elixhauser A, et al. Utilization of Intensive Care Services, 2011: Statistical Brief #185. 2014Rockville, MD, Healthcare Cost and Utilization Project.
2. Stefan MS, Shieh MS, Pekow PS, et al. Epidemiology and outcomes of acute respiratory failure in the United States, 2001 to 2009: A national survey. J Hosp Med 2013; 8:76–82
3. Ray P, Birolleau S, Lefort Y, et al. Acute respiratory failure in the elderly: Etiology, emergency diagnosis and prognosis. Crit Care 2006; 10:R82
4. Self WH, Courtney DM, McNaughton CD, et al. High discordance of chest x-ray and computed tomography for detection of pulmonary opacities in ED patients: Implications for diagnosing pneumonia. Am J Emerg Med 2013; 31:401–405
5. Syrjälä H, Broas M, Suramo I, et al. High-resolution computed tomography for the diagnosis of community-acquired pneumonia. Clin Infect Dis 1998; 27:358–363
6. Claessens YE, Debray MP, Tubach F, et al. Early chest computed tomography scan to assist diagnosis and guide treatment decision for suspected community-acquired pneumonia. Am J Respir Crit Care Med 2015; 192:974–982
7. Brenner DJ, Hall EJ. Computed tomography–an increasing source of radiation exposure. N Engl J Med 2007; 357:2277–2284
8. Beckmann U, Gillies DM, Berenholtz SM, et al. Incidents relating to the intra-hospital transfer of critically ill patients. An analysis of the reports submitted to the Australian Incident Monitoring Study in Intensive Care. Intensive Care Med 2004; 30:1579–1585
9. Hew M, Corcoran JP, Harriss EK, et al. The diagnostic accuracy of chest ultrasound for CT-detected radiographic consolidation in hospitalised adults with acute respiratory failure: A systematic review. BMJ Open 2015; 5:e007838
10. Llamas-Álvarez AM, Tenza-Lozano EM, Latour-Pérez J. Accuracy of lung ultrasonography in the diagnosis of pneumonia in adults: Systematic review and meta-analysis. Chest 2017; 151:374–382
11. Lichtenstein DA, Mezière GA. Relevance of lung ultrasound in the diagnosis of acute respiratory failure: The BLUE protocol. Chest 2008; 134:117–125
12. Lichtenstein D, Mezière G, Biderman P, et al. The “lung point”: An ultrasound sign specific to pneumothorax. Intensive Care Med 2000; 26:1434–1440
13. Volpicelli G, Boero E, Sverzellati N, et al. Semi-quantification of pneumothorax volume by lung ultrasound. Intensive Care Med 2014; 40:1460–1467
14. Lichtenstein D, Mezière G. A lung ultrasound sign allowing bedside distinction between pulmonary edema and COPD: The comet-tail artifact. Intensive Care Med 1998; 24:1331–1334
15. Copetti R, Soldati G, Copetti P. Chest sonography: A useful tool to differentiate acute cardiogenic pulmonary edema from acute respiratory distress syndrome. Cardiovasc Ultrasound 2008; 6:16
16. Frankel HL, Kirkpatrick AW, Elbarbary M, et al. Guidelines for the appropriate use of bedside general and cardiac ultrasonography in the evaluation of critically ill patients-part I: General ultrasonography. Crit Care Med 2015; 43:2479–2502
17. Volpicelli G, Elbarbary M, Blaivas M, et al.; International Liaison Committee on Lung Ultrasound (ILC-LUS) for International Consensus Conference on Lung Ultrasound (ICC-LUS): International evidence-based recommendations for point-of-care lung ultrasound. Intensive Care Med 2012; 38:577–591
18. Chiumello D, Mongodi S, Algieri I, et al. Assessment of lung aeration and recruitment by CT scan and ultrasound in acute respiratory distress syndrome patients. Crit Care Med 2018; 46:1761–1768
19. Lichtenstein D, Goldstein I, Mourgeon E, et al. Comparative diagnostic performances of auscultation, chest radiography, and lung ultrasonography in acute respiratory distress syndrome. Anesthesiology 2004; 100:9–15
20. Tierney DM, Boland LL, Overgaard JD, et al. Pulmonary ultrasound scoring system for intubated critically ill patients and its association with clinical metrics and mortality: A prospective cohort study. J Clin Ultrasound 2018; 46:14–22
21. Volpicelli G, Mussa A, Garofalo G, et al. Bedside lung ultrasound in the assessment of alveolar-interstitial syndrome. Am J Emerg Med 2006; 24:689–696
22. Baldi G, Gargani L, Abramo A, et al. Lung water assessment by lung ultrasonography in intensive care: A pilot study. Intensive Care Med 2013; 39:74–84
23. Kim M, Lee KY, Lee KW, et al. MDCT evaluation of foreign bodies and liquid aspiration pneumonia in adults. AJR Am J Roentgenol 2008; 190:907–915
24. Beydon L, Saada M, Liu N, et al. Can portable chest x-ray examination accurately diagnose lung consolidation after major abdominal surgery? A comparison with computed tomography scan. Chest 1992; 102:1697–1703
25. Mongodi S, Bouhemad B, Orlando A, et al. Modified lung ultrasound score for assessing and monitoring pulmonary aeration. Ultraschall Med 2017; 38:530–537
26. Bouhemad B, Liu ZH, Arbelot C, et al. Ultrasound assessment of antibiotic-induced pulmonary reaeration in ventilator-associated pneumonia. Crit Care Med 2010; 38:84–92
Keywords:

critical care; lung; point-of-care; radiography; respiratory insufficiency; ultrasonography

Supplemental Digital Content

Copyright © 2019 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.