Secondary Logo

Journal Logo

Review Article

Assessing left ventricular systolic function by emergency physician using point of care echocardiography compared to expert: systematic review and meta-analysis

Albaroudi, Bilala; Haddad, Mahmouda; Albaroudi, Omara; Abdel-Rahman, Manar E.b; Jarman, Robertc,d; Harris, Tima,e

Author Information
European Journal of Emergency Medicine: February 2022 - Volume 29 - Issue 1 - p 18-32
doi: 10.1097/MEJ.0000000000000866

Abstract

Introduction

Point-of-care (POC) echocardiography is widely used in the evaluation and resuscitation of patients in the emergency department (ED) [1,2]. Acute breathlessness, cardiac arrest and shock are common presentations to the ED where POC echocardiography performed by clinicians provides rapid and focused findings to assist in the initial assessment. Data suggest that emergency physicians’ clinical assessment fails to accurately estimate the patient’s cardiac function and hemodynamic status [3], potentially leading to incorrect diagnosis and treatment with associated increases in length of stay and mortality [4,5]. A recent randomised control trial showed that using point-of-care ultrasound (POCUS) did not improve outcomes for patients presenting the ED with undifferentiated shock [6]. However, this trial excluded patients with conditions that had evidence of rapid diagnosis with POCUS, the interventions in both groups were similar and the trial evaluated a diagnostic test as therapy. Studies, including one systematic view and meta-analysis, have shown that rapid assessment with POC echocardiography early in the patient’s ED journey reduces time to diagnosis, improves diagnostic accuracy and changes treatment potentially improving outcomes in critical illness, shock and acute heart failure [7–16]. Consequently, there is an increasing interest in emergency physicians performing POC echocardiography with more detailed studies performed urgently on selected patients or following admission by an expert.

Assessment of left ventricular systolic function (LVSF) is one of the most common indications for echocardiography [17]. In 2013, the American Society of Echocardiography published a recommendation to consider POC echocardiography as a bedside adjunct to physical examination to assist in diagnosis and guide treatment [18]. The American Society of Echocardiography and American College of Emergency Physicians 2010 consensus statement for use of POC echocardiography in the ED stated that POC echocardiography had a role in the assessment and diagnosis of pericardial effusion, right ventricle dilatation, intravascular volume status and left ventricular performance [1]. The latter was suggested as visually assessed and categorised as normal (ejection fraction >50%), reduced (ejection fraction 30–50%), or severely reduced (ejection fraction <30%). In 2014, the American College of Emergency Medicine published a revised policy statement [19] stating that the assessment of LVSF is one of three primary indications for POC echocardiography. In 2014, the International Evidence-Based Recommendations for Focused Cardiac Ultrasound stated that POC echocardiography can accurately assess left ventricle global function, narrow differential diagnosis and improve outcome in the setting of shock and is superior to physical examination alone [20].

The ability of emergency physicians to evaluate LVSF has been previously assessed in observational studies and one narrative review [21]. The authors were unable to identify any previous systematic review and meta-analysis. The aim of this review was to assess the level of agreement between emergency department-based clinician sonographers as compared to expert sonographers for the assessment of LVSF using POC echocardiography. The PICO question is depicted in Table 1.

Table 1 - Review question details
Review question (PICO)
Population Adult patients (18 years or over) in the emergency department (ED), no limits on presenting complaint or physiology
Intervention POC echocardiography performed in ED by clinician sonographer working in the ED as emergency physician who had limited ultrasound training defined as:
 Level 1 (core training) or level 2 (advanced training).
 Clinicians who had completed a defined local training program instituted for study.
 AND who do not meet the criteria below for expert sonographers.
Comparison Echocardiogram performed or reported by expert sonographer defined as:
 Graduated sonographer.
 Graduated board-certified cardiologist.
 Clinician with completed fellowships in echocardiography (the training standards are defined by country of training and not standardised worldwide, so we did not define experts by hours in training or number of scans performed).
Outcome Level of agreement between the clinician sonographer and the expert sonographer for the assessment of left ventricular systolic function (LVSF) where clinician sonographer uses one of the following three methods:
 Visual Estimation.
 E-Point Septal Separation (EPSS).
 Velocity Time Integral (VTI).
The method for expert sonographers to assess LVSF was not defined a priori.

Methods

This systematic review was designed in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analysis statement and was registered on the International Prospective Register of Systematic Reviews (CRD42020179209). All authors reviewed the article and T.H. acts as a guarantor.

Data sources and search strategy

The medical literature was searched using PubMed (MEDLINE), OvidSP (EMBASE), Cochrane library and ClinicalTrials.gov. The grey literature was searched using Google Scholar and TRIP database. A combination of Medical Subject Headings terms and search terms was used and are listed in Appendix 1 in Supplemental Digital Content 1, https://links.lww.com/EJEM/A318. In addition to the database search, the bibliographies of the selected articles were reviewed for further references. Supplementary searching was performed using snowballing and hand searching of key medical journals. The publication period was defined by the database searched. The literature search was performed independently by a medical librarian, B.A. and M.H. in April 2020. A repeat search was performed in February 2021.

Study selection

The search was restricted to studies on humans and published in English. Studies that recruited adult patients where POC echocardiography was performed in the ED were included, regardless of physiological parameters, diagnoses or presenting complaints. Clinician sonographer was defined by the study authors as a trainee or specialist in emergency medicine with level 1 (core) or level 2 (advanced training or local training). A priori studies that focused on patients outside the ED were excluded (i.e. prehospital, ICU or ward) as well as studies using sonographers other than emergency physicians (e.g. intensivist or medical student). We defined expert sonographers as sonographers who had completed a training program, cardiologists who had completed training or ultrasound fellowship-trained clinicians who had completed a fellowship in echocardiography. We did not assess the experience of expert sonographers as this varies between countries and was not described in detail in any article under review.

We included studies that assessed LVSF using transthoracic echocardiography by visual estimation, E-point septal separation (EPSS), or an assessment of stroke volume/cardiac output using velocity time integral (VTI). We excluded studies where the clinician sonographer used fractional shortening to assess left ventricle ejection fraction (LVEF), reflecting its inherent inaccuracies. We imposed no restrictions on the method used by expert sonographers or machine used by either clinician or expert sonographer. We included studies performed using hand-held and free-standing ultrasound machines and any transducer type but excluded studies using transoesophageal echo. We included studies that used the review of recorded videoclips or stored images by expert sonographers as the reference standard and those where the expert sonographer’s echocardiogram was performed using either the same or a different machine to the clinician sonographer. Where there was a lack of clarity in the article, we wrote to the corresponding authors.

Three reviewers (T.H., B.A. and M.H.) worked independently in two groups to review all eligible titles and abstracts using inclusion criteria defined a priori. We included studies regardless of their designs. Randomised trials and observational studies (prospective and retrospective cohort studies) were included. Conference abstracts with sufficient methodological description for quality assessment and data for analysis were also included. Case studies, case reports, guidelines, editorials, letters and review articles were excluded. The full text was then read, and consensus was achieved for inclusion. When consensus agreement could not be reached between the authors, an independent third-party adjudicated (B.J.).

Data extraction and quality assessment

Two reviewers (B.A. and M.H.) independently extracted data using a standardised form (Table 2). The revised Quality Assessment Tool for Diagnostic Accuracy Studies-2 (QUADAS-2) was used to evaluate the overall quality of the included studies [22–33]. The tool was applied by the two reviewers independently, and disagreements were resolved by a third reviewer (T.H.). The QUADAS-2 tool allows modification of the signalling questions to customise to the topic under review. Our signalling questions are described in Appendix 2 with details for the process of rating as low, high or unclear risk of bias in Supplemental Digital Content 1, https://links.lww.com/EJEM/A318.

Table 2 - Summary of included studies
Author
Year
Country
Designs
Title
Study characteristics Clinician sonographer/training Participant characteristics Intervention and expert sonographer Outcome (95% confidence interval)
Moore et al. [22]
2002
USA
POS
Determination of Left Ventricular Function by Emergency Physician Echocardiography of Hypotensive Patients
Machine type: Index: Apogee CX-100 Floor-based US Machine Transducer PA Reference: Recorded images and clips Windows: PSLA, PSSA, A4C, A2C, SC. Method: Index: visual estimation Reference: visual estimation Categorization: (>50%) normal function, (30–50%) reduced function, (<30%) severe dysfunction Clinician sonographer: four EPs Level of experience: highly experienced with various levels. Must have completed 10 h of basic US instruction and 100 documented non-cardiac ultrasound examinations prior to echocardiography training. Additional training:
Theoretical: 6-h consist of video clips review
Practical: 10-h observing and performing Echo Supervised by cardiologist.
Convenience sample, n = 51 ED adult patients Inclusion: symptomatic hypotension, systolic Blood pressure <100 mmHg for two times and 15 min apart, no trauma
Exclusion: Asymptomatic hypotension, CPR, ECG suggestive of myocardial infarction
Intervention: POC echocardiography by clinician sonographer, then the scan repeated immediately by another clinician sonographer in convenience sample n = 8 (blinded) Expert sonographer videoclips reviewed by 1# cardiologist then 2# cardiologist reviewed some scans in convenience sample n = 20 (blinded) Interval: N/A Agreement of expert sonographer with clinician sonographer, n = 50 (1 patient excluded due to poor image quality):
•kw = 0.61 (0.39–0.83)
•Weighted percent agreement = 84%
•Overall agreement = 68%
•Pearson’s correlation, r = 0.86 (+-8%)
•Modified Bland–Altman plot showed the difference falls within 1 SE of the estimate 96% of the patients.
Correlation between the clinician sonographer, n = 8
•Pearson’s correlation, r = 0.94.
Interobserver reliability between the expert sonographer, n = 20
•Pearson’s correlation, r = 0.84
•κ = 0.7
•Modified Bland–Altman plot showed agreement within 1 SE of the estimate in 95% of patients
Randazzo et al. [23]
2003
USA
POS
Accuracy of Emergency Physician Assessment of Left Ventricular Ejection Fraction and central venous pressure Using Echocardiography
Machine type: Index: Hewlett-Packard Sonos 1000 Floor-based US Machine
Reference: Sonos 5500 Floor-based US Machine Transducer PA Windows: SC, PSSA, and A4C Method: Index: visual estimation
Reference: Not Described Categorization:
(>55%) normal function, (30%–55%) reduced function, (<30%) severe dysfunction.
Clinician sonographer: 7 EPs, (three residents, 4 attendings) and 1 physician assistant. Level of experience: completed an emergency medicine US workshop (didactic and practical sessions, with a minimum of 150 examinations of non-echo scans) Additional training:
Theoretical: 3 h session of series review of normal and abnormal.
Practical: >5 scans.
Convenience sample, n = 115 ED adult patients Inclusion The formal echocardiogram should be performed within 4 h of the ED echocardiogram.
Most reason for echo: chest pain 45.1%, CHF 38.1%, SOB 5.7%, suspected endocarditis 10.6% Exclusion: >4 h delay between echocardiograms, or if the investigator was aware of the result of previous Echo.
Intervention: POC echocardiography by clinician sonographer Expert sonographer: TTE by certified cardiac sonographer and interpreted by cardiologist Interval: <4 h. clinician sonographer first then expert sonographer Categorical LVSF agreement for clinician sonographer with expert sonographer n = 115
•κw = 0.71 (CI = 53%–89%)
•Overall agreement = 78.3%
Calculated LVSF correlation for clinician sonographer visual estimation with expert sonographer n = 102 (calculated ejection fraction was available for 102 patients. Thirteen cases had no formal ejection fraction reported on the final Echo report by cardiology)
•Pearson’s Correlation, r 2 = 0.72
Raw agreement for each category of LVSF
•Normal 92.3%
•Poor 70.4%
•Moderate 47.8%
•Overall agreement = 86.1%
Secko et al. [24]
2011
USA
POS
Can Junior Emergency Physicians Use E-Point Septal Separation to Accurately Estimate Left Ventricular Function in Acutely Dyspneic Patients?
Machine type:
Index: Philips HD11XE Floor-based US machine
Transducer PA. Reference: Recorded images and clips.
Windows: SC, PSSA, PSLA, A4C. Method: Index: EPSS Reference: visual estimation
Categorizations: normal or low
Clinician sonographer: 12 emergency medicine residents (PGY3 and PGY4) Level of experience: 70–150 total ED US scans. And <25 Echo and had been instructed on EPSS through informal teaching during their day-long Introduction to emergency ultrasound course. Additional training: No Convenience sample, n = 70 ED adult patients Inclusion: dyspneoa, ED length of stay <2 h, no trauma, and normal mental status. Exclusion: known history of mitral valve repair or replacement, aortic insufficiency, or mitral stenosis. Intervention: POC echocardiography by clinician sonographer Expert sonographer: videoclips. All clips reviewed by 2 EP US fellows at bedside but were not blinded to the EPSS measurements by clinician sonographer.
Half of the clips reviewed by two cardiologists.
Both used visual estimation method
Interval: N/A
Correlation between measurements of EPSS by resident EPs and visual estimations of LVSF by the EP US fellows, n = 58:
•Spearman’s correlation, ρ = −0.844
The agreement between EP US fellow and cardiologists for visual estimation (expert to expert), n = 58:
•κ = 0.75 (0.48–1.00)
Weekes et al. [25]
2011
USA
POS
Comparison of Serial Qualitative and Quantitative Assessments of Caval Index and Left Ventricular Systolic Function During Early Fluid Resuscitation of Hypotensive Emergency Department Patients
Machine type:
Index: Philips CX50 Floor-based US machine
Transducer PA
Reference: Recorded images and clips Windows: Not Described Method: Index: visual estimation and fractional shortening
Reference: Not Described
Categorization:
increased function, normal function, reduced function, or severe dysfunction
Clinician sonographer: EPs number not Described Level of experience: not well described Emergency US division physicians Additional training: no standardized training. training on EPSS and the fractional shortening methods and on the details of image acquisition, technique, details of scoring criteria, and the sequence of measurements to standardize the protocol. Convenience sample, n = 24 ED adult patients Inclusion: symptomatic hypotension systolic Blood pressure ≤100 mm Hg, sinus rhythm, the EP had intent to administer IV fluid challenges.
Exclusion: unstable arrhythmia or CPR, suspected CHF, inability to obtain adequate quality US images, pregnancy, trauma, inability to tolerate the positioning, predictable ED stay of <45min.
Intervention: POC echocardiography by clinician sonographer Expert sonographer: videoclips. US EP fellowship
Interval: N/A
Agreement between clinician sonographer and expert sonographer scan for visual estimation, n = 24 patients (72 scans):
•κ = 0.46 (0.29–0.63)
•κw = 0.57 (0.41–0.72)
•Overall agreement = 65.3%
US scoring and measurements were performed at three defined points per enrolment. Time 0 was defined as the start of the fluid challenges. Time 1 was defined as immediately after the first fluid challenge. Time 2 was defined as 15–20 minutes after Time 1. Seventy-two videos of LVSF and caval index were obtained.
Dinh et al. [26]
2012
USA
POS
Measuring cardiac index with a focused cardiac ultrasound examination in the ED
Machine type:
Index: Z. One Ultra Floor-based US machine
Reference:
1.Sonographer: same machine.
2.Cardiologist: stored images.
Transducer PA
Windows: PSLA, A4C Method: Index and Reference: cardiac index by VTI and LVOT diameter and HR.
Categorization: normal Cardiac index was defined as 2.5–4.0 L/min/m2
Rating for the images by cardiologist: Optimal, suboptimal, and unobtainable images
Clinician sonographer: two EPs Level of experience: Not Described Additional training: Practical: 20 h by a certified sonographer of acquiring adequate images with PSLA, A4C. Convenience sample, n = 100 ED adult patients Inclusion: 5 days per month when the study team was available, no described indication for the Echo, cardiac indication in (30%) of sample. Exclusion: congenital heart disease, aortic valvular disease, inability to lie supine or inability to lie in the left lateral decubitus during the scan. Intervention: POC echocardiography by clinician sonographer. Expert sonographer:
Echo by sonographer (expert sonographer #1) for 23/100 patients
All US images by the expert sonographer and clinician sonographer were stored and reviewed by a cardiologist (expert sonographer #2) Interval: clinician sonographer first then expert sonographer <10 min.
Agreement between clinician sonographer and sonographer (expert sonographer #1) (n = 23):
Agreement for VTI:
•κ = 0.40 (0.04–0.76)
•Pearson’s correlation, r = 0.87 (0.70–0.94)
•Bland-Altman −0.17 (−4.08 to 18.01)
Agreement for cardiac index:
•κ = 0.56 (0.23–0.90)
•Pearson’s correlation, r = 0.82 (0.60–0.92)
•Bland–Altman −0.11 (−1.06 to 0.83)
Cardiologist (expert sonographer #2) rating of cardiac US images for Optimal VTI measurements with:
•clinician sonographer: Overall agreement = 78.4% of subjects (n = 97)
•Sonographer: Overall agreement = 78.3% of subjects (n = 23)
Bustam et al. [27]
2014
Malaysia
POS
Performance of emergency physicians in point-of-care Echocardiography following limited training
Machine type: Index and reference: Logiq-e portable on a trolley US
Transducer PA Windows: PSLA, PSSA, A4C, SC Method: Index and Reference: visual estimation, Teichholz method Categorization:
(>50%) normal function, (30–50%) reduced function, (<30%) severe dysfunction
Clinician sonographer: nine emergency medicine Residents (PGY1–PGY2) Level of experience: novice with the least experience in Echo, all had < 10 informal Echo Additional training:
Theoretical: web-based learning which included images and videos. Practical: 3 h delivered at the bedside by a cardiologist.
Convenience sample, n = 50 ED adult patients Inclusion: random sample. no described indication for the scan. Stable. Exclusion: if they were unstable (not described) or were still receiving acute treatment. Intervention: POC echocardiography by clinician sonographer
Expert sonographer: Echo by cardiologist.
Interval: expert sonographer first then clinician sonographer, directly.
Agreement between clinician sonographer and expert sonographer for visual estimation, n = 100 scan:
•κ = 0.79 (0.773–0.842)
•Overall agreement of 93% of scans
Agreement between clinician sonographer and expert sonographer for quantitative LVSF measurements, n = 92 scan:
•Pearson’s correlation, r = 0.82 (0.734-0.881).
•Bland–Altman: 2.1 (−9.5% to 13.7%)
Agreement between clinician sonographer quantitative LVSF and expert sonographer visual estimation n = 84
•κ = 0.80 (0.636–0.882)
•Overall agreement of 92.9% of scans
McKaigney et al. [28]
2014
Canada/USA
POS
E-point septal separation: a bedside tool for emergency physician assessment of left ventricular ejection fraction
Machine type: Index: Logic P6 Floor-based US Reference: Phillips iE33 Floor-based US
Transducer PA
Windows: SC, PSSA, PSLA, A4C. Method: Index: visual estimation, EPSS. Reference: Teichholz method
Categorization:
(>55%) Normal function, (30%–55%) reduced function, (<30%) severe dysfunction.
Clinician sonographer: 3 EPs (EP US fellows – 7 months into a 1-year fellowship). Level of experience: 100 bedside echocardiograms. And had training in both visual and calculated LVSF estimation as part of their fellowship Additional training:
Theoretical: 10-min presentation on EPSS
Practical: three separates supervised EPSS measurements with the principal investigator.
Convenience sample, n = 80 ED adult patients
7 (8.8%) subjects were enrolled in the ED, 11 (13.8%) in ICU, and 62 (77.5%) in ward. Inclusion: treating physician had ordered TTE. Exclusion: pregnancy.
Intervention: POC echocardiography by clinician sonographer in the ED, ICU, inpatient ward. Expert sonographer: TTE by certified cardiac sonographer and interpreted by cardiologist Interval: expert sonographer then clinician sonographer <24 h.
n = 23 patients were randomly selected to have a second bedside ultrasound study performed independently byanother clinician sonographer, blinded to the results of the first examination.
Agreement between clinician sonographer visual estimation and the calculated LVSF in TTE (expert sonographer)
•Overall agreement 69%
•κw = 0.58.
•for subjects with severe systolic dysfunction (n=8) overall agreement 100%
Correlation of EPSS by clinician sonographer to the calculated LVSF by expert sonographer:
•Pearson’s correlation, r = 0.73
EPSS measurement of >7 mm for LVSF ≤30% n = 71
•Sensitivity = 100.0% (62.9–100.0)
•Specificity = 51.6% (38.6–64.5)
•Positive LR = 2.07
•Negative LR = 0.00
EPSS measurement of >8 mm for LVSF <55%, n = 71
•Sensitivity 83.3% (62.6–95.2)
•Specificity 50.0% (29.2–70.9)
•Positive LR = 1.67
•Negative LR = 0.33
Interobserver correlation for EPSS measurements, n = 23
•Spearman’s correlation, ρ = 0.87
Interobserver agreement for visual estimation, n = 23
•κw = 0.49
Ünlüer et al. [29]
2014
Turkey
POS
Visual Estimation of Bedside Echocardiographic Ejection Fraction by Emergency Physicians
Machine type: Index and reference: M7R model ultrasound portable on a trolley US, Transducer PA Windows: PSLA Method: Index: visual estimation
Reference: SBT Categorization: normal or low threshold of 55%
Clinician sonographer: two EPs attending with 1-year experience. Level of experience: certified on focused abdominal sonography for trauma by Emergency Radiology Association in Turkey. Additional training: 3 h of didactic training; performed in the presence of an experienced echocardiographer. Convenience sample, n = 133 ED adult patients Inclusion: dyspneoa. Exclusion: intubated, elevated cardiac biomarkers, pregnancy, atrial fibrillation, valvular pathology or surgery, if technical limitations. Intervention: POC echocardiography by clinician sonographer Expert sonographer: Formal Echo by experienced cardiologist Interval: clinician sonographer first then expert sonographer within 2 h Correlation between clinician sonographer to expert sonographer:
•Pearson’s correlation, r = 0.77 for clinician sonographer #1 and 0.78 for clinician sonographer #2
•Spearman’s correlation, ρ = 0.87 (0.82–0.90) clinician sonographer #1 and 0.88 (0.84–0.91) clinician sonographer #2
•Intra-class correlation coefficient of both clinician sonographer = 0.952 (0.934, 0.966)
Clinician sonographer POC echocardiography scans (cardiologist reports as reference standard):
•Sensitivity 98.7%
•Specificity 86% (86.2–87.9%)
•Positive PV = 0.902
•Negative PV = 0.980
•Negative LR = 0.015
•Positive LR = 7.153
Farsi et al. [30]
2017
Iran
POS
Focused cardiac ultrasound (FOCUS) by emergency medicine residents in patients with suspected cardiovascular diseases
Machine type: Index: SonoAce X8, Samsung Medison Co, Floor-based US Reference: EKO 7 device, Samsung Madison Co, Floor-based US Transducer PA Windows: PSLA, PSSA, A4C, SC Method: Index: EPSS or Quinones equation, or both Reference: visual estimation Categorization: (≥50%) normal function, (30–49%) reduced function, (<30%) severe dysfunction Clinician sonographer: 17 emergency medicine Residents Level of experience: Not Described Additional training: Theoretical: 2 h for movies and pictures Practical: 2 h.
Workshop designed by the American Society of Echocardiography
Convenience sample, n = 205 ED adult patient Inclusion: cardiac emergency requiring POC echocardiography (chest pain 45%, SOB 33%, syncope 10%) Exclusion: if interfere with treatment. Intervention: POC echocardiography by clinician sonographer Expert sonographer: cardiologist. Interval: clinician sonographer first then expert sonographer, as soon as possible. The agreement between clinician sonographer and expert sonographer, n = 205:
•κ = 0.85 (0.79–0.91)
•Overall agreement = 91%
Clinician sonographer POC echocardiography scans (cardiologist reports as reference standard):
•Sensitivity 89% (81–99)
•Specificity 96% (90–99)
•Postive PV 96% (89–99)
•Negative PV 89% (81–99)
•Accuracy 92% (85–99)
•Negative LR 0.12 (0.07–0.2)
•Positive LR 22 (8–58)
For wall motion abnormality
κ = 0.83 (0.76–0.90)
Balderston et al. [31]
2019
USA
ROS
Diagnostic Yield and Accuracy of Bedside Echocardiography in the Emergency Department in Hemodynamically Stable Patients
Machine type: Index: Fujifilm Sonosite X Porte US machine, Floor-based US
Reference: not described
Transducer PA Windows: Not Described Method: Index: visual estimation
Reference: SBT
Categorization: (≥50%) normal function, (30-49%) reduced function, (<30%) severe dysfunction
For the analysis: normal ≥50% and reduced <50%
Clinician sonographer: 90 EPs from all grades (40 residents, fellows, 50 attendings). Level of experience: US certificates in accordance with American College of Emergency Physicians guidelines.
Additional training: no
Consecutive sample, n = 276 ED adult Inclusion: stable patients, had TTE < 2 days of POC echocardiography indications were mainly for chest pain or SOB, or both and syncope and lower limps oedema. Exclusion: unstable systolic Blood pressure <90 mm Hg, need for positive pressure ventilation, or cardiac arrest, Paediatric. Intervention: POC echocardiography by clinician sonographer Expert sonographer: TTE by certified cardiac sonographer and interpreted by cardiologist Interval: clinician sonographer first then expert sonographer, <2 days Clinician sonographer to expert sonographer for the reduced LVSF and pericardial effusion, n = 270
•Overall agreement of 77%
•κ = 0.53
Clinician sonographer POC echocardiography scan (cardiologist reports as reference standard), n = 224:
•Overall agreement 78.1% Sensitivity 80% (66.6–89.1)
•Specificity 77.5% (70.3–83.4)
•Positive PV 53.7% (42.4–64.6)
•Negative PV 92.3% (86.2–95.9)
•Negative LR 0.26 (0.15–0.44)
•Positive LR 3.65 (2.61–4.85)
Monsomboon et al. [32] 2019
Thailand Clinician sonographer S
Agreement between Emergency Physicians and a Cardiologist on Cardiac Function Evaluation after Short Training
Machine type: Index: Philips HD15 PureWave US, Floor-based US
Transducer PA Reference: Recorded images and clips Windows: PSLA, PSSA, A4C, A5C, SC Method: Index: visual estimation Reference: Not described Categorization:
(>55%) Normal function, (30%–55%) reduced function, (<30%) severe dysfunction.
Clinician sonographer: seven emergency medicine Residents (PGY2 - PGY3) Level of experience: The residents had been working in the ED for less than 3 years. They had not participated in any formal US training courses because emergency ultrasound was not part of the emergency medicine curriculum.
Additional training: Theoretical: 2 h lecture
Practical: 1-h hands-on workshop conducted by cardiologist.
Consecutive sample n = 101 ED adult Inclusion: patient needed a volume status or LV function assessment 1) shock (systolic blood pressure < 90 mmHg or diastolic blood pressure < 60 mmHg); and 2) suspected heart failure, defined as having at least 1 of the Framingham criteria.
Exclusion: uncooperative. STEMI. Intubated. Patients for whom POC echocardiography might delay their standard treatment.
Intervention: POC echocardiography by clinician sonographer Expert sonographer: videoclips review by one cardiologist Interval: N/A
After the course, each resident had to practice on at least 1 patient in the ED. All recorded images and videos were subsequently sent to the cardiologist to assess the residents’ ability to perform POC echocardiography. The cardiologist had to approve each participant’s performance before he or she was permitted to participate
Agreement between clinician sonographer visual estimation and expert sonographer review, n = 92.
•Overall agreement = 79.4%
•κw = 0.73 (0.58–0.89)
The agreement between clinician sonographer and expert sonographer on the pericardial effusion diagnoses = 100%.
Lafon et al. [32]
2019
France
Comparative Early Hemodynamic Profiles in Patients Presenting to the Emergency Department with Septic and Nonseptic Acute Circulatory Failure Using Focused Echocardiography
Machine type: Index: CX 50, Philips Healthcare
Transducer PA Reference: CX 50, Philips Healthcare
Windows: PSLA, PSSA, A4C, SC Method: Index: visual estimation Reference: visual estimation Categorization:
(30%–50%) reduced function (<30%) severe dysfunction.
Clinician sonographer: five EPs
Level of experience: no previous experience in US
Additional training:
validated curriculum was fully conducted and supervised by two experienced intensivists who were board-certified
in echocardiography (level 3 competence)
Theoretical: total 9-h (6-h of didactics, 3-h of interactive clinical cases)
Practical: 3-h of hands-on
Consecutive sample n = 114 ED adult
Inclusion: Acute respiratory failure, hypotension (systolic blood pressure <90 mmHg or mean blood pressure <65 mmHg), altered tissue perfusion (skin: cold, clammy, mottling; kidneys: urine output <0.5 mL/kg/h; brain: altered mental status), and cellular dysoxia (lactate >2 mmol/L).
Exclusion: moribund status or pregnancy.
Intervention: POC echocardiography by clinician sonographer Expert sonographer: expert in critical care echocardiography
Interval: <1 h, in random order, depending on the availability of investigators.
Agreement between clinician sonographer and expert sonographer for: n = 100
LV failure κ = 0.74 (0.59–0.89)
Hyperdynamic LV associated with vasoplegia κ = 0.61 (0.43–0.78)
A2C; apical 2 chambers, A4C; apical 4 chambers, A5C; apical 5 chambers, CI; confidence interval, ED; emergency department, EM; emergency medicine, EP; emergency physician, EPSS; E-point Septal Separation,; LR; likelihood ratio, LV; left ventricle, LVOT: left ventricle outflow tract, LVSF; left ventricle systolic function, PA; phased array transducer, POC; point-of-care, PGY; post-graduation year, POS; prospective observational study, PSLA; parasternal long axis, PSSA; parasternal short axis, PV; predictive value, ROS; Retrospective observational study, SBT; Simpson’s biplanar technique, SC: subcostal, SOB; Shortness of breath, STEMI; ST-segment elevation Myocardial infarction, TTE; transthoracic echocardiography, US; ultrasound, VTI; velocity time integral.

Meta-analysis

To determine the level of agreement between the clinician sonographer and expert sonographer for LVSF assessment a meta-analysis was performed using four effect measures of interest. Cohen/Conger’s Kappa statistics (simple and weighted) [34–36], sensitivity, specificity, positive and negative likelihood ratio with 95% confidence interval (CI) were independently pooled. Forest plots were used to summarise and compare the included studies. Heterogeneity was quantified using Higgin’s I2 statistics; any value of I2 >50% was indicative of significant heterogeneity. To account for heterogeneity random effect models were used to pool the study effect sizes. The presence of publication bias was assessed using Deeks’ funnel plot [37]. The meta-analysis was conducted on each method of assessment of LVSF when there was enough data provided (visual estimation, EPSS, VTI). Information obtained from the quality assessment tool was used for bias assessment and subgroup analysis. Analysis of subgroups was performed to explore the potential sources of heterogeneity in the agreement. The following subgroups were evaluated: level of experience of clinician sonographer, training on POC echocardiography for clinician sonographer prior study enrolment and year of publication (before and after 2010). Analyses were conducted using kappaetci, meta and Midas packages in Stata software (StataCorp, 2019, Stata Statistical Software: Release 16, StataCorp LLC, College Station, Texas, USA).

Results

A total of 1673 studies were identified and screened, and 25 full articles were eligible for review (Fig. 1). Thirteen studies that did not meet the inclusion criteria were excluded. Consequently, 12 studies [22–33] published between 2002 and 2020 (1229 scans on 1131 patients) were included in this systematic review. Eleven studies enrolled 159 clinician sonographers (88 emergency medicine residents/fellows, 70 emergency medicine attendings and one study enrolled one physician assistant [23]). One study did not specify the number of clinician sonographers [25]. An attempt was made to contact all authors to clarify aspects of the included studies; four replies were received [22,24,27,31].

F1
Fig. 1:
PRISMA flow diagram for the selection process for the systematic review studies. PRISMA, Preferred Reporting Items for Systematic reviews and Meta-Analysis statement.

Characteristics of the included studies

Overall, 12 studies were included in this review (Table 2). Eleven studies [22–30,32,33] were prospective observational cohort design, and one study [31] was a retrospective chart review. All studies were single centre, included only patients presenting to hospitals via the ED, and used floor-based ultrasound machines with a phased array transducer. In all studies, POC echocardiography was performed solely in the ED except one study which included POC echocardiography performed in ED, ICU and wards [28]. The inclusion criteria for patients varied considerably, two studies included only patients with hypotension [22,25], two recruited patients with dyspnoea [24,29], five studies recruited patients who required inpatient echocardiography for any reason [23,26–28,31], one study included any patient with suspected cardiac disease [30], one study included patient with acute circulatory failure [33] and one study included patients who required volume status or LVSF assessment due to hypotension or suspected heart failure [32].

Visual estimation of left ventricular systolic function

Visual estimation of volume changes between diastole and systole using ranked categories was the most commonly used method by clinician sonographers to assess LVSF (nine studies) [22,23,25,27–29,31–33]. Cardiac function was categorised as normal (ejection fraction ≥ 50%), reduced (ejection fraction 30–49%) or severely reduced (ejection fraction <30%) function in four studies [22,27,31,33], or as normal (ejection fraction >55%), reduced (ejection fraction 30–55%) or severely reduced (ejection fraction <30%) function in three studies [23,28,32]. One study did not use numerical cutoff values [25]. Two studies also included the category hyperdynamic function in addition to the three previously mentioned categories [25,33]. LVSF was ranked simply as normal or abnormal in two studies with cutoff ejection fraction value of 50% [31], or 55% [29]. Both the clinician sonographer and expert sonographer used visual estimation in three studies [22,27,33], in two studies clinician sonographers used visual estimation and the expert sonographer measured LVSF using Simpson’s Biplanar Technique (SBT) [29,31]. Three studies did not describe the expert sonographer method of LVSF estimation [23,25,32].

There was heterogeneity in the methods used to assess agreement between clinician sonographer and expert sonographer for visual estimation of LVSF (Table 2). Eight studies [22,23,25,27,28,31–33] reported a simple or weighted Cohan’s Kappa (0.46–0.79), one reported Pearson’s correlation [29] (for the two recruited emergency physicians values were 0.77 and 0.78). Five studies [22,23,27,28,31] reported raw/overall agreement, two reported agreement using Bland–Altman [22,27] and four calculated specificity and sensitivity for LVSF as normal or abnormal [28–31]. Overall, there was a high agreement in identifying LVSF as normal and with severe dysfunction. However, the agreement was moderate in identifying moderate dysfunction.

E-point septal separation

The second most frequently used method to assess LVSF was EPSS (three studies including 355 patients) [24,28,30]. No studies compared EPSS by both clinician sonographer and expert sonographer. Two studies compared EPSS by clinician sonographer with visual estimation by expert sonographer [24,30], one reported a Kappa (κ = 0.85) and the other reported a Spearman’s correlation (−0.84). One study [28] compared the EPSS by clinician sonographer to ejection fraction assessed using the Teichholz method by an expert sonographer, reporting with Pearson’s correlation of 0.73.

Velocity time integral

Cardiac index calculation using VTI and Left Ventricular Outflow Tract (LVOT) diameter was assessed in only one study [26], which reported the moderate agreement (κ = 0.40).

Image acquisition

There were some variations in the methods used to obtain images. In five studies [23,26,29–31], the clinician sonographer performed POC echocardiography prior to the expert sonographer while in two the expert sonographer scanned first [27,28]. In one study, the POC echocardiography was performed based on investigator availability [33]. In five studies [22,24–26,32], the expert sonographer reviewed video images taken by the clinician sonographer; in three studies [23,28,31], the cardiologist reviewed videos recorded by a sonographer and in three studies [27,29,30] expert sonographers (cardiologists) performed their own echocardiograms. Expert sonographer used the same ultrasound machines as clinician sonographer in four studies [26,27,29,33] and different machines in four [23,28,30,31]. The expert sonographer were emergency physicians in two studies [24,25], cardiologists/sonographers in 10 [22–24,26–32] (one study combined emergency physician and cardiologists as expert sonographer [24]) and intensivists expert in echocardiography in one [33]. In 11 studies [22,23,25–33], the second sonographer was blind to the findings of the first, while in one study, the expert sonographers were not blind to EPSS measures by clinician sonographers [24]. In four studies, there was a time difference of greater than 1 h between clinician sonographer and expert sonographer scans [23,28,29,31].

Quality assessment of studies

The QUADAS-2 quality assessment is depicted in Table 3. Studies were rated as low (), high () or uncertain (?) risk of bias. All studies scored at least one high-risk domain and seven studies scored two or more. Thus, the quality of the included studies was low, and all studies were assessed as high risk of bias. All studies excluded a proportion of their acquired POC echocardiography data, most commonly as a consequence of poor image quality and were consequently considered high risk (total enrolled sample 1309 patients and analysis sample 1131 patients). Eleven were rated as high risk of bias in patient selection due to convenience sampling or due to exclusion criteria, or both (body habitus or technical difficulties obtaining images) [22–30,32,33]. The protocol for the POC echocardiography performed by clinician sonographers varied widely between the studies with differing windows, machines, and transducer frequencies. Seven studies included low numbers of clinician sonographers selected from a larger clinician group with unclear reasons for clinician sonographer selection [22,23,26,28,29,32,33]. Five studies involved time intervals between POC echocardiography performed by clinician sonographers and the echocardiogram performed by expert sonographers, reflecting high risk of bias in flow and timing [28–31,33].

Table 3. - Quality assessment using QUADAS-2 of the included studies: ; high risk of bias, ; low risk of bias,?; unclear
Bias Applicability
Author Patient selection Index test Reference standard Flow and timing Patient selection Index test Reference standard
Moore et al. [22]
Randazzo et al. [26]
Secko et al. [27]
Weekes et al. [28] ? ? ?
Dinh et al. [29]
Bustam et al. [30]
McKaigney et al. [31]
Ünlüer et al. [32]
Farsi et al. [33] ?
Balderston et al. [23]
Monsomboon et al. [24]
Lafon et al. [25]
Percentage % Summary of QUADAS-2 results
Bias Applicability
Patient selection Index test Reference standard Flow and timing Patient selection Index test Reference standard
8.5% 33.5% 91.5% 50% 91.5% 91.5% 91.5%
91.5% 58% 0% 41.5% 8.5% 8.5% 0%
? 0% 8.5% 8.5% 8.5% 0% 0% 8.5%
QUADAS-2, Quality Assessment Tool for Diagnostic Accuracy Studies-2.

Meta-analysis

In seven studies (688 patients, 786 scans, >119 clinician sonographers; 60 attendings, and 59 emergency medicine residents/fellows) [22,23,25,27,29,31,32], clinician sonographers used the visual estimation method to assess LVSF and reported adequate data to conduct the meta-analyses. The primary data were extracted from the original articles for statistical analysis (Supplementary Table 5 in Supplemental Digital Content 1, https://links.lww.com/EJEM/A318). The remaining five studies [24,26,28,30,33] had inadequate data for meta-analysis and we were unable to obtain sufficient data from the authors. There were insufficient studies to conduct a meta-analysis for EPSS or VTI assessment of LVSF. Weighted kappa was calculated in five studies [22,23,25,27,32] where the LVSF was categorised as normal, reduced, or severely reduced. In one study [25] where LVSF ranked as hyperdynamic, we include this as normal function for analysis. In two studies [29,31] where LVSF was categorised as normal or abnormal, simple kappa was calculated from primary data. Then, an overall simple kappa was calculated for the seven studies [22,23,25,27,29,31,32] by combining the reduced and severe reduced function as abnormal. Combined data were then used to calculate sensitivity and specificity to identify normal or abnormal LVSF. A ‘positive’ finding is defined as abnormal LVSF and a ‘negative’ finding is defined as normal LVSF.

The meta-analysis of the seven studies [22,23,25,27,29,31,32] yielded simple Kappa, sensitivity, specificity, positive and negative likelihood ratio of 0.68 (95% CI, 0.57–0.79), 89% (95% CI, 80%–94%), 85% (95% CI, 80%–89%), 5.98 (95% CI, 4.13–8.68), and 0.13 (95% CI, 0.06–0.24), respectively. The weighted kappa for five studies [22,23,25,27,32] was 0.70 (95% CI, 0.61–0.80). The forest plots of the previous results are shown in Figs. 2 and 3, and in Supplementary Figs. 4 and 5 in the Supplemental Digital Content 1, https://links.lww.com/EJEM/A318.

F2
Fig. 2:
Forest plot of agreement of POC echocardiography by clinician sonographer for the assessment of LVSF as normal/abnormal using Simple kappa statistics as compared to expert sonographer. LVSF, left ventricular systolic function; POC, point-of-care.
F3
Fig. 3:
Forest plot of agreement of POC echocardiography by clinician sonographer for the assessment of LVSF as normal, reduced or severely reduced using weighted Kappa statistics as compared to expert sonographer. LVSF, left ventricular systolic function; POC, point-of-care.

The subgroup meta-analysis regarding the year of publication before or after 2010, yielded simple Kappa of κ = 0.70 (95% CI, 0.55–0.84) for the studies published after 2010 [25,27,29,31,32] and κ = 0.64 (95% CI, 0.46–0.84) for those published before 2010 [22,23]. There was no evidence of publication bias with significant asymmetry in Deeks’ funnel plot of the seven studies (P = 0.95) (Supplementary Fig. 6 in Supplemental Digital Content 1, https://links.lww.com/EJEM/A318).

Discussion

The results of this systematic review suggest that ED-based clinician sonographers (56% trainees, 44% attendings, with or without study-specific training) are able to interpret visual estimation of LVSF with a substantial agreement as compared to expert sonographer, with a pooled κ = 0.68 (seven studies) and κw = 0.70 (five studies). Visual estimation of LVSF is a rapidly performed, widely acceptable method and in experienced hands may be superior to formal measurements for assessment of LVSF [38–41], but requires considerable experience to perform accurately. The sensitivity (true positive rate for detection of abnormal LVSF), specificity (true negative rate for detection of normal LVSF), positive and negative likelihood ratios in seven studies were 89%, 85%, 5.98, and 0.13, respectively, suggesting that clinician sonographer can correctly identify normal or abnormal LVSF sufficiently for clinical practice when compared to expert sonographer’s interpreted echocardiography as the reference standard.

For every 100 patients classified as having normal LVSF by expert sonographer, and clinician sonographer identified 85 patients. For every 100 patients identified as having abnormal LVSF by the expert sonographer, the clinician sonographer correctly identified 89 patients. Clinician sonographer classified 11% wrongly as abnormal and 15% wrongly as normal. A narrative review of the twelve included studies suggests that there is a high agreement in identifying LVSF as normal as compared to severely reduced function. However, the agreement was moderate in identifying mildly/moderately reduced LVSF function.

Other methods to assess LVEF (not considered in this review) such as mitral annular plane systolic excursion and tissue doppler imaging may offer more precise assessment and have been reported as having good interobserver reliability between untrained sonographers and experts [42–44].

Measuring EPSS is a much simpler and quicker method to assess LVSF than the SBT technique and is easily taught to novice sonographers [45]. However multiple factors may cause an over or underestimate in the LVSF measurement, such as valvular disease or septal hypertrophy. In this systematic review, EPSS was used in three studies [24,28,30]. The level of agreement and correlation were variably expressed, and no meta-analysis was possible. These three individual studies reported a moderate to a high level of agreement between clinician sonographers and expert sonographer.

VTI assesses the volume of blood ejected via the aortic outflow tract and may be used to calculate the stroke volume, cardiac index and guide fluid therapy [46]. VTI depends on ejection fraction, LV filling pressures and vascular resistance, and so offers different data to clinicians as compared to ejection fraction and EPSS (and other assessments of LV function). One study included in this systematic review assessed cardiac index using VTI and LVOT diameter by clinician sonographer and expert sonographer (κ = 0.40) [26].

SBT method is widely used by cardiologists, but this technique is time-consuming to perform in the resuscitation phase of care and includes A4C and A2C views, with the latter not widely taught to clinician sonographers. None of the included studies used this method by clinician sonographer for LVSF assessment.

A 2019 narrative review [21] described the assessment of LVSF by emergency physicians and general practitioners with varying POC echocardiography experience. The authors reported an overall raw agreement of 84–93% (10 studies) [22–24,26–30,47,48], based on visual estimation, EPSS and VTI. This review had no a priori defined research question, clear protocol, critical appraisal, or meta-analysis of included studies. The authors included eight studies which are included in the systematic review reported here [22–24,26–30] and two studies that were excluded [47,48]. Shah et al. [47] enrolled adult and paediatric patients in ED and outpatient settings with POC echocardiography performed by internal medicine and social service doctors followed by expert review of the recorded images (cardiologist or emergency medicine ultrasound fellow). This study reported almost perfect agreement for LVSF (κ = 0.98). Dehbozorgi et al. [48] studied the diagnostic accuracy of thoracic and cardiac POCUS for acute heart failure as compared to clinietween clinician sonographers and expert sonographers. Therefore, these two articles did not meet the inclusion criteria for the work reported in this article.

Visual estimation of LVSF is well described in the cardiology literature [39,40,49,50]. However, there are few studies that investigate the interobserver agreement between cardiologists or between cardiologists and sonographers. One study including 136 patients with suspected left ventricular dysfunction explored agreement between echocardiograms performed by cardiology trainees using portable ultrasound machines and hospital-based echocardiograms performed by senior cardiac technicians using a floor-based ultrasound machine. The authors reported the excellent agreement (κ = 0.87) for the detection of LV systolic dysfunction [51]. Testuz et al. compared echocardiography performed by certified cardiologists using hand-held devices to departmental echocardiography interpreted by blinded cardiologists, again with excellent agreement for LVSF (κ = 0.89) [52].

Several studies have evaluated POC echocardiography for the assessment of LVSF in an ICU setting. Amiel et al. reported the substantial agreement (κ = 0.75) in assessing LVSF between two experienced intensivists trained in critical care echocardiography using hand-held POC echocardiography as compared to cardiology reported departmental echocardiography [53]. Biais et al. reported excellent agreement (κ = 0.87) in assessed LVSF between two different expert intensivist sonographers using hand-held POC echocardiography in ED and departmental echocardiography [54]. Two studies assessed the ability of trainees in internal medicine to assess LVSF. One study reported substantial agreement (κ = 0.77) while a second study reported moderate agreement (κ = 0.51) between POC echocardiography performed by internal medicine trainees and cardiology performed echocardiography [55,56]. An ICU-based pilot study reported that following 8 h of focused training to assess LVSF using POC echocardiography by noncardiologist trainees (two anaesthesia and two internal medicine) recorded substantial agreement (κ = 0.76) as compared to expert intensivist sonographers using floor-based devices [57].

There was significant heterogeneity identified among the included studies. The level of POC echocardiography experience for clinician sonographers ranged from no previous significant experience [27,32,33], <25 scans [24], 100 scans [28], completed an US workshop [23], ACEP certified [31], and not clearly described [22,25,26,29,30]. However, the level of experience of clinician sonographers did not predict the level of agreement with expert sonographers in this systematic review. Three studies enrolled only novice clinician sonographers with no significant previous experience in ultrasound and reported substantial agreement for visual estimation, κ = 0.61 to 0.79 [27,32,33]. In three studies [22,25,29], the clinician sonographers were all attending emergency physicians but the level of POC echocardiography experience was not clearly defined, and these studies reported widely different levels of agreement for visual estimation of LVSF. In two studies the clinician sonographers were a mixture of trainees and attendings [23,31]. In this systematic review, emergency physician attending did not have higher levels of agreement with expert sonographers than emergency medicine trainees.

There was also heterogeneity in additional training in POC echocardiography offered by the study teams prior to enrolment. Ten studies [22,23,25–30,32,33] offered study-specific training in POC echocardiography to clinician sonographer participants ranging from 10 min to 16 h (theoretical and hands-on training). In one study, the clinician sonographer did not receive any additional training with a calculated κ = 0.49; however, the clinician sonographers were credentialed to the standards defined by ACEP [31]. Implementing additional, targeted training in POC echocardiography is associated with an improvement in image acquisition and interpretation [58–61].

POC echocardiography technology has advanced considerably in the past two decades, with better quality machines at a reduced cost. Five studies that used visual estimation by clinician sonographers and published after 2010 [25,27,29,31,32] showed higher pooled agreement compared to those two studies before 2010 [22,23]: κ = 0.7 (95% CI, 0.55–0.84) vs. κ = 0.64 (95% CI, 0.46–0.84). This may reflect the improvement in technology and training for clinician sonographers [62].

Limitations

This systematic review has several limitations. All the data are from small, single centre observational studies. The study design, patient population, statistical analysis, clinician sonographer selection, POC echocardiography training and experience varied considerably. The risk of bias for included studies was high, mainly a consequence of convenience sampling, patient exclusion and selected clinician sonographers. However, this reflects the clinical use for POC echocardiography where images are not always attainable and not all clinicians are trained in ultrasound/POC echocardiography. ED performed POC echocardiography involves critically unwell patients who may be challenging to position optimally. Many emergency medicine training schemes do not include assessment of LVSF and in 10 studies [22,23,25–30,32,33] additional training was provided for clinician sonographers. This work is not generalisable to clinician sonographers without additional POC echocardiography training. Two studies [29,31] classified LVSF as normal or abnormal, a very blunt tool for clinical assessment. Classification of LVSF as normal, reduced, and severely reduced function may offer a more useful clinical approach. Only two studies assessed the interobserver agreement between the expert sonographer [22,24], which was similar to the agreement between clinician sonographer and expert sonographer. There were insufficient data to allow meta-analysis for EPSS and VTI. The reference standard in this systematic review was expert sonographer interpreted echocardiograms, which is not a gold standard for LVSF assessment. This limits the value of reported sensitivity and specificity. However, this is considered an acceptable reference standard for clinical practice. A multicentre observational study comparing clinician sonographer to expert sonographer, interobserver expert sonographer/interobserver clinician sonographer reliability and intra-observer reliability using multiple methods of LVSF assessment would provide valuable data on understanding the reliability, precision, and reproducibility of each technique.

Conclusion

Clinician sonographer (trainee or attending) had a substantial agreement with an expert sonographer for the POC echocardiography assessment of LVSF as ‘normal or abnormal’ and as ‘normal, reduced and severely reduced’ by visual estimation. Few studies assessed the agreement between clinician sonographer and expert sonographer for EPSS and VTI methods and meta-analysis was not possible. There is considerable heterogeneity among studies with limited generalisability. A standardised training curriculum for LVSF assessment is needed. The quality of the included studies was low. Further studies are warranted.

Acknowledgements

Open Access funding provided by the Qatar National Library.

Conflicts of interest

There are no conflicts of interest.

References

1. Labovitz AJ, Noble VE, Bierig M, Goldstein SA, Jones R, Kort S, et al. Focused cardiac ultrasound in the emergent setting: a consensus statement of the American Society of Echocardiography and American College of Emergency Physicians. J Am Soc Echocardiogr 2010; 23:1225–1230.
2. Breitkreutz R, Price S, Steiger HV, Seeger FH, Ilper H, Ackermann H, et al.; Emergency Ultrasound Working Group of the Johann Wolfgang Goethe-University Hospital, Frankfurt am Main. Focused echocardiographic evaluation in life support and peri-resuscitation of emergency patients: a prospective trial. Resuscitation 2010; 81:1527–1533.
3. Nowak RM, Sen A, Garcia AJ, Wilkie H, Yang JJ, Nowak MR, et al. The inability of emergency physicians to adequately clinically estimate the underlying hemodynamic profiles of acutely ill patients. Am J Emerg Med 2012; 30:954–960.
4. Hautz WE, Kämmer JE, Hautz SC, Sauter TC, Zwaan L, Exadaktylos AK, et al. Diagnostic error increases mortality and length of hospital stay in patients presenting through the emergency room. Scand J Trauma Resusc Emerg Med 2019; 27:54.
5. Ray P, Birolleau S, Lefort Y, Becquemin MH, Beigelman C, Isnard R, et al. Acute respiratory failure in the elderly: etiology, emergency diagnosis and prognosis. Crit Care 2006; 10:R82.
6. Atkinson PR, Milne J, Diegelmann L, Lamprecht H, Stander M, Lussier D, et al. Does point-of-care ultrasonography improve clinical outcomes in emergency department patients with undifferentiated hypotension? An International Randomized Controlled Trial From the SHoC-ED Investigators. Ann Emerg Med 2018; 72:478–489.
7. Levitt MA, Jan BA. The effect of real time 2-D-echocardiography on medical decision-making in the emergency department. J Emerg Med 2002; 22:229–233.
8. Mantuani D, Frazee BW, Fahimi J, Nagdev A. Point-of-care multi-organ ultrasound improves diagnostic accuracy in adults presenting to the emergency department with acute dyspnea. West J Emerg Med 2016; 17:46–53.
9. Volpicelli G, Lamorte A, Tullio M, Cardinale L, Giraudo M, Stefanone V, et al. Point-of-care multiorgan ultrasonography for the evaluation of undifferentiated hypotension in the emergency department. Intensive Care Med 2013; 39:1290–1298.
10. Sasmaz MI, Gungor F, Guven R, Akyol KC, Kozaci N, Kesapli M. Effect of focused bedside ultrasonography in hypotensive patients on the clinical decision of emergency physicians. Emerg Med Int 2017; 2017:6248687.
11. Russell FM, Ehrman RR, Cosby K, Ansari A, Tseeng S, Christain E, et al. Diagnosing acute heart failure in patients with undifferentiated dyspnea: a lung and cardiac ultrasound (LuCUS) protocol. Acad Emerg Med 2015; 22:182–191.
12. Jones AE, Tayal VS, Sullivan DM, Kline JA. Randomized, controlled trial of immediate versus delayed goal-directed ultrasound to identify the cause of nontraumatic hypotension in emergency department patients. Crit Care Med 2004; 32:1703–1708.
13. Anderson KL, Jenq KY, Fields JM, Panebianco NL, Dean AJ. Diagnosing heart failure among acutely dyspneic patients with cardiac, inferior vena cava, and lung ultrasonography. Am J Emerg Med 2013; 31:1208–1214.
14. Shokoohi H, Boniface KS, Pourmand A, Liu YT, Davison DL, Hawkins KD, et al. Bedside ultrasound reduces diagnostic uncertainty and guides resuscitation in patients with undifferentiated hypotension. Crit Care Med 2015; 43:2562–2569.
15. Sekiguchi H, Harada Y, Villarraga HR, Mankad SV, Gajic O. Focused cardiac ultrasound in the early resuscitation of severe sepsis and septic shock: a prospective pilot study. J Anesth 2017; 31:487–493.
16. Stickles SP, Carpenter CR, Gekle R, Kraus CK, Scoville C, Theodoro D, et al. The diagnostic accuracy of a point-of-care ultrasound protocol for shock etiology: a systematic review and meta-analysis. CJEM 2019; 21:406–417.
17. Ward RP, Mansour IN, Lemieux N, Gera N, Mehta R, Lang RM. Prospective evaluation of the clinical application of the American College of Cardiology Foundation/American Society of Echocardiography Appropriateness Criteria for transthoracic echocardiography. JACC Cardiovasc Imaging 2008; 1:663–671.
18. Spencer KT, Kimura BJ, Korcarz CE, Pellikka PA, Rahko PS, Siegel RJ. Focused cardiac ultrasound: recommendations from the American Society of Echocardiography. J Am Soc Echocardiogr 2013; 26:567–581.
19. Kendall JL, Bahner DP, Blaivas M, Budhram G, Dean AJ, Fox JC, et al. Emergency ultrasound imaging criteria compendium (policy statement). 2014. Ann Emerg Med 2016; 68:e11–e48.
20. Via G, Hussain A, Wells M, Reardon R, ElBarbary M, Noble VE, et al.; International Liaison Committee on Focused Cardiac UltraSound (ILC-FoCUS); International Conference on Focused Cardiac UltraSound (IC-FoCUS). International evidence-based recommendations for focused cardiac ultrasound. J Am Soc Echocardiogr 2014; 27:683.e1–683.e33.
21. Sorensen B, Hunskaar S. Point-of-care ultrasound in primary care: a systematic review of generalist performed point-of-care ultrasound in unselected populations. Ultrasound J 2019; 11:31.
22. Moore CL, Rose GA, Tayal VS, Sullivan DM, Arrowood JA, Kline JA. Determination of left ventricular function by emergency physician echocardiography of hypotensive patients. Acad Emerg Med 2002; 9:186–193.
23. Randazzo MR, Snoey ER, Levitt MA, Binder K. Accuracy of emergency physician assessment of left ventricular ejection fraction and central venous pressure using echocardiography. Acad Emerg Med 2003; 10:973–977.
24. Secko MA, Lazar JM, Salciccioli LA, Stone MB. Can junior emergency physicians use E-point septal separation to accurately estimate left ventricular function in acutely dyspneic patients? Acad Emerg Med 2011; 18:1223–1226.
25. Weekes AJ, Tassone HM, Babcock A, Quirke DP, Norton HJ, Jayarama K, et al. Comparison of serial qualitative and quantitative assessments of caval index and left ventricular systolic function during early fluid resuscitation of hypotensive emergency department patients. Acad Emerg Med 2011; 18:912–921.
26. Dinh VA, Ko HS, Rao R, Bansal RC, Smith DD, Kim TE, et al. Measuring cardiac index with a focused cardiac ultrasound examination in the ED. Am J Emerg Med 2012;30:1845–1851.
27. Bustam A, Noor Azhar M, Singh Veriah R, Arumugam K, Loch A. Performance of emergency physicians in point-of-care echocardiography following limited training. Emerg Med J 2014;31:369–373.
28. McKaigney CJ, Krantz MJ, La Rocque CL, Hurst ND, Buchanan MS, Kendall JL. E-point septal separation: a bedside tool for emergency physician assessment of left ventricular ejection fraction. Am J Emerg Med 2014;32:493–497.
29. Unlüer EE, Karagöz A, Akoğlu H, Bayata S. Visual estimation of bedside echocardiographic ejection fraction by emergency physicians. West J Emerg Med 2014;15:221–226.
30. Farsi D, Hajsadeghi S, Hajighanbari MJ, Mofidi M, Hafezimoghadam P, Rezai M, et al. Focused cardiac ultrasound (FOCUS) by emergency medicine residents in patients with suspected cardiovascular diseases. J Ultrasound 2017;20:133–138.
31. Balderston JR, Gertz ZM, Brooks S, Joyce JM, Evans DP. Diagnostic yield and accuracy of bedside echocardiography in the emergency department in hemodynamically stable patients. J Ultrasound Med 2019; 38:2845–2851.
32. Monsomboon A, Patarateeranon T, Tongyoo S, Surabenjawong U, Chaisirin W, Chakorn T, et al. Agreement between emergency physicians and a cardiologist on cardiac function evaluation after short training. Siriraj Medical Journal 2019; 71:253–260.
33. Lafon T, Appert A, Hadj M, Bigrat V, Legarcon V, Claveries P, et al. Comparative early hemodynamic profiles in patients presenting to the emergency department with septic and nonseptic acute circulatory failure using focused echocardiography. Shock 2020; 53:695–700.
34. Cohen J. A coefficient of agreement for nominal scales. Educational and psychological measurement 1960; 20:37–46.
35. Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968; 70:213–220.
36. Conger AJ. Integration and generalization of kappas for multiple raters. Psychol Bull 1980; 88:322.
37. Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol 2005; 58:882–893.
38. Shahgaldi K, Gudmundsson P, Manouras A, Brodin LA, Winter R. Visually estimated ejection fraction by two dimensional and triplane echocardiography is closely correlated with quantitative ejection fraction by real-time three dimensional echocardiography. Cardiovasc Ultrasound 2009; 7:41.
39. Rich S, Sheikh A, Gallastegui J, Kondos GT, Mason T, Lam W. Determination of left ventricular ejection fraction by visual estimation during real-time two-dimensional echocardiography. Am Heart J 1982; 104:603–606.
40. Amico AF, Lichtenberg GS, Reisner SA, Stone CK, Schwartz RG, Meltzer RS. Superiority of visual versus computerized echocardiographic estimation of radionuclide left ventricular ejection fraction. Am Heart J 1989; 118:1259–1265.
41. Mueller X, Stauffer JC, Jaussi A, Goy JJ, Kappenberger L. Subjective visual echocardiographic estimate of left ventricular ejection fraction as an alternative to conventional echocardiographic methods: comparison with contrast angiography. Clin Cardiol 1991; 14:898–902.
42. Matos J, Kronzon I, Panagopoulos G, Perk G. Mitral annular plane systolic excursion as a surrogate for left ventricular ejection fraction. J Am Soc Echocardiogr 2012; 25:969–974.
43. Park YS, Park JH, Ahn KT, Jang WI, Park HS, Kim JH, et al. Usefulness of mitral annular systolic velocity in the detection of left ventricular systolic dysfunction: comparison with three dimensional echocardiographic data. J Cardiovasc Ultrasound 2010; 18:1–5.
44. Adel W, Roushdy AM, Nabil M. Mitral annular plane systolic excursion-derived ejection fraction: a simple and valid tool in adult males with left ventricular systolic dysfunction. Echocardiography 2016; 33:179–184.
45. Gelabert C, Geckle R, Malhotra R, Haddad S, Ash A, Raio C, et al. 321 comparison of different methods of left ventricular ejection fraction in novice sonographers. Ann Emerg Med 2014; 64:S114.
46. Mielnicki W, Dyla A, Zawada T. Utility of transthoracic echocardiography (TTE) in assessing fluid responsiveness in critically ill patients - a challenge for the bedside sonographer. Med Ultrason 2016; 18:508–514.
47. Shah SP, Shah SP, Fils-Aime R, Desir W, Joasil J, Venesy DM, et al. Focused cardiopulmonary ultrasound for assessment of dyspnea in a resource-limited setting. Crit Ultrasound J 2016; 8:7.
48. Dehbozorgi A, Eslami Nejad S, Mousavi-Roknabadi RS, Sharifi M, Tafakori A, Jalli R. Lung and cardiac ultrasound (LuCUS) protocol in diagnosing acute heart failure in patients with acute dyspnea. Am J Emerg Med 2019;37:2055–2060.
49. Martin RP. Real time ultrasound quantification of ventricular function: has the eyeball been replaced or will the subjective become objective. J Am Coll Cardiol 1992;19:321–323.
50. Cheitlin MD, Armstrong WF, Aurigemma GP, Beller GA, Bierman FZ, Davis JL, et al.; ACC; AHA; ASE. ACC/AHA/ASE 2003 Guideline Update for the Clinical Application of Echocardiography: summary article. A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (ACC/AHA/ASE Committee to Update the 1997 Guidelines for the Clinical Application of Echocardiography). J Am Soc Echocardiogr 2003;16:1091–1110.
51. Jeyaseelan S, Goudie BM, Pringle SD, Donnan PT, Sullivan FM, Struthers AD. Agreement between community echocardiography and hospital echocardiography in patients suspected of having left ventricular systolic dysfunction. Postgrad Med J 2005;81:777–779.
52. Testuz A, Müller H, Keller PF, Meyer P, Stampfli T, Sekoranja L, et al. Diagnostic accuracy of pocket-size handheld echocardiographs used by cardiologists in the acute care setting. Eur Heart J Cardiovasc Imaging 2013;14:38–42.
53. Amiel JB, Grümann A, Lhéritier G, Clavel M, François B, Pichon N, et al. Assessment of left ventricular ejection fraction using an ultrasonic stethoscope in critically ill patients. Crit Care 2012;16:R29.
54. Biais M, Carrié C, Delaunay F, Morel N, Revel P, Janvier G. Evaluation of a new pocket echoscopic device for focused cardiac ultrasonography in an emergency setting. Crit Care 2012;16:R82.
55. Johnson BK, Tierney DM, Rosborough TK, Harris KM, Newell MC. Internal medicine point-of-care ultrasound assessment of left ventricular function correlates with formal echocardiography. J Clin Ultrasound 2016; 44:92–99.
56. Alexander JH, Peterson ED, Chen AY, Harding TM, Adams DB, Kisslo JA Jr. Feasibility of point-of-care echocardiography by internal medicine house staff. Am Heart J 2004; 147:476–481.
57. Vignon P, Dugard A, Abraham J, Belcour D, Gondran G, Pepino F, et al. Focused training for goal-oriented hand-held echocardiography performed by noncardiologist residents in the intensive care unit. Intensive Care Med 2007; 33:1795–1799.
58. Kerwin C, Tommaso L, Kulstad E. A brief training module improves recognition of echocardiographic wall-motion abnormalities by emergency medicine physicians. Emerg Med Int 2011; 2011:483242.
59. Jacob M, Shokoohi H, Moideen F, Pousson A, Boniface K. An echocardiography training program for improving the left ventricular function interpretation in emergency department; a brief report. Emerg (Tehran) 2017; 5:e70.
60. Frederiksen CA, Juhl-Olsen P, Nielsen DG, Eika B, Sloth E. Limited intervention improves technical skill in focus assessed transthoracic echocardiography among novice examiners. BMC Med Educ 2012; 12:65.
61. Jones AE, Tayal VS, Kline JA. Focused training of emergency medicine residents in goal-directed echocardiography: a prospective study. Acad Emerg Med 2003; 10:1054–1058.
62. Lee L, DeCara JM. Point-of-care ultrasound. Curr Cardiol Rep 2020; 22:149.
Keywords:

adult; cardiologists; echocardiography; emergency medicine; left ventricular dysfunction; meta-analysis; observer variation; point-of-care systems; sensitivity and specificity; systematic review

Supplemental Digital Content

Copyright © 2021 The Author(s). Published by Wolters Kluwer Health, Inc.