Share this article on:

A Comparison of Web-Based with Traditional Classroom-Based Training of Lung Ultrasound for the Exclusion of Pneumothorax

Edrich, Thomas MD; Stopfkuchen-Evans, Matthias MD; Scheiermann, Patrick MD, PhD; Heim, Markus MD; Chan, Wilma MD; Stone, Michael B. MD; Dankl, Daniel MD; Aichner, Jonathan; Hinzmann, Dominik MD; Song, Pingping MD; Szabo, Ashley L. MD; Frendl, Gyorgy MD, PhD; Vlassakov, Kamen MD; Varelmann, Dirk MD

doi: 10.1213/ANE.0000000000001383
Technology, Computing, and Simulation: Research Report

BACKGROUND: Lung ultrasound (LUS) is a well-established method that can exclude pneumothorax by demonstration of pleural sliding and the associated ultrasound artifacts. The positive diagnosis of pneumothorax is more difficult to obtain and relies on detection of the edge of a pneumothorax, called the “lung point.” Yet, anesthesiologists are not widely taught these techniques, even though their patients are susceptible to pneumothorax either through trauma or as a result of central line placement or regional anesthesia techniques performed near the thorax. In anticipation of an increased training demand for LUS, efficient and scalable teaching methods should be developed. In this study, we compared the improvement in LUS skills after either Web-based or classroom-based training. We hypothesized that Web-based training would not be inferior to “traditional” classroom-based training beyond a noninferiority limit of 10% and that both would be superior to no training. Furthermore, we hypothesized that this short training session would lead to LUS skills that are similar to those of ultrasound-trained emergency medicine (EM) physicians.

METHODS: After a pretest, anesthesiologists from 4 academic teaching hospitals were randomized to Web-based (group Web), classroom-based (group class), or no training (group control) and then completed a posttest. Groups Web and class returned for a retention test 4 weeks later. All 3 tests were similar, testing both practical and theoretical knowledge. EM physicians (group EM) performed the pretest only. Teaching for group class consisted of a standardized PowerPoint lecture conforming to the Consensus Conference on LUS followed by hands-on training. Group Web received a narrated video of the same PowerPoint presentation, followed by an online demonstration of LUS that also instructs the viewer to perform an LUS on himself using a clinically available ultrasound machine and submit smartphone snapshots of the resulting images as part of a portfolio system. Group Web received no other hands-on training.

RESULTS: Groups Web, class, control, and EM contained 59, 59, 20, and 42 subjects. After training, overall test results of groups Web and class improved by a mean of 42.9% (±18.1% SD) and 39.2% (±19.2% SD), whereas the score of group control did not improve significantly. The test improvement of group Web was not inferior to group class. The posttest scores of groups Web and class were not significantly different from group EM. In comparison with the posttests, the retention test scores did not change significantly in either group.

CONCLUSIONS: When training anesthesiologists to perform LUS for the exclusion of pneumothorax, we found that Web-based training was not inferior to traditional classroom-based training and was effective, leading to test scores that were similar to a group of clinicians experienced in LUS.

Supplemental Digital Content is available in the text.Published ahead of print May 6, 2016

From the *Department of Anesthesiology, Perioperative and Pain Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts; Department of Anesthesiology, University Hospital Campus Grosshadern, Ludwig-Maximilians-University, Munich, Germany; Department of Anesthesiology, University Hospital rechts der Isar, Technical University Munich, Munich, Germany; §Department of Emergency Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts; Department of Anesthesiology, Perioperative Medicine and General Intensive Care Medicine, Salzburg General Hospital, Paracelsus Medical University, Salzburg, Austria; and Ludwig-Maximilians-University Medical School, Munich, Germany.

Thomas Edrich, MD, is currently affiliated with the Department of Anesthesia and Surgical Intensive Care Medicine, Klinikum Landkreis Erding, Germany; and Paracelsus Medical University Salzburg, Salzburg, Austria.

Wilma Chan, MD, is currently affiliated with the Department of Emergency Medicine, Division of Emergency Ultrasound; Hospital of the University of Pennsylvania, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania.

Pingping Song, MD, is currently affiliated with the Department of Anesthesiology, Lahey Clinic, Burlington, Massachusetts.

Accepted for publication March 1, 2016.

Published ahead of print May 6, 2016

Funding: Internal.

The authors declare no conflicts of interest.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website.

This report was previously presented, in part, at the European Society of Intensive Care Medicine (ESICM) 2014 (only subset of data presented in abstract form).

Reprints will not be available from the authors.

Address correspondence to Thomas Edrich, MD, Department of Anesthesiology, Perioperative and Pain Medicine, Brigham and Women’s Hospital, Harvard Medical School, 75 Francis St., Boston, Massachusetts 02115. Address e-mail to thomasedrich@gmail.com.

Lung ultrasound (LUS) is a well-established method that can exclude pneumothorax with a sensitivity and specificity that is equivalent or superior to chest radiography.1 A 2011 international consensus conference proposed an ultrasound algorithm specifically designed to investigate pneumothorax.2 The presence of pleural sliding and the demonstration of the associated ultrasound artifacts (“sandy beach sign,” “B-lines”) are used to exclude a pneumothorax. The algorithm uses the detection of a “lung point” (this is the edge of the pneumothorax) for the positive diagnosis of pneumothorax. The authors of the consensus statement, however, make it clear that this sign is sometimes difficult to obtain and emphasize that, in the setting in which a pneumothorax is likely, clinical judgment must prevail when a lung point cannot be detected readily.

Although many anesthesiologists and intensivists are familiar with the use of ultrasonography to guide central venous catheter placement and for guidance of regional anesthesia techniques, some of which entail a risk of pneumothorax, the use of LUS for the detection of pneumothorax has not been widely taught. The utility of quickly detecting or excluding pneumothorax, however, has been reported both in the operating room and the intensive care unit.3–5 LUS is now an integral part of many ultrasound courses for intensivists, but these courses require significant scheduling effort and may not be feasible for many anesthesiologists. In anticipation of an increased training demand for LUS, efficient and scalable Web-based teaching methods should be developed.

We developed a Web-based training course that included self-directed practical exercises and compared this with a traditional classroom-based course with the same educational content. The primary goal of the 2 courses was to teach subjects how to exclude pneumothorax but also how to recognize lung point as a positive diagnostic sign of pneumothorax. Our primary hypothesis was that Web-based teaching would not be inferior to classroom-based instruction. Our secondary hypothesis was that the performance of the anesthesiologists after the Web-based training would be similar to the performance of a group of emergency medicine (EM) physicians who have had LUS training and used it routinely for clinical care.

Back to Top | Article Outline

METHODS

Approval was obtained from the IRBs of the 2 German, 1 Austrian, and 1 US institution involved in this prospective randomized interventional study. At both German centers, written informed consent was obtained from all subjects. This method of consent was approved by their respective IRB. At the Austrian and US center, the requirement for written informed consent was waived by their respective IRBs. At these 2 centers, consent was implied when subjects registered for the study. This method of implied consent was approved by their IRBs. Inclusion criteria included any resident, fellow, or staff physician in the departments of anesthesia or EM. Anesthesia physicians were recruited from all of the participating institutions, whereas the EM physicians were recruited only from the center in the United States. Anesthesia physicians who had received any formal training in LUS were excluded. All EM physicians had received some amount of formal training in LUS and indicated that they performed LUS regularly in their clinical practice. Members of group EM were asked to estimate the total number of LUS performed before the study.

Figure 1 summarizes the design of the study. Up to 10 subjects were recruited for each training session. After a pretest, consisting of a written and practical part, subjects were randomized in blocks of 7 to Web-based (group Web), classroom-based (group class), or no training (group control) by the use of a ratio of 3:3:1. For EM physicians (group EM), the study was terminated after the pretest without further intervention. Group class remained for traditional classroom teaching followed by a hands-on practice. Group Web received online access to 2 instructional videos and were sent home.

Figure 1

Figure 1

Classroom training for group class consisted of traditional lecture teaching lasting approximately 45 minutes administered by an anesthesiologist member of the study staff, who used a standardized PowerPoint lecture that conformed closely to the Consensus Conference on Lung Ultrasound.2,6 Immediately after the lecture, the subjects received approximately 20 minutes of hands-on training and performed LUS on a volunteer model using their clinical ultrasound machine under the supervision of the study staff. Given the planned class size of ≤5 subjects, this amounts to ≥4 minutes of direct hands-on training per subject.

Instead of classroom teaching, group Web received a narrated video of the same PowerPoint presentation lasting 25 minutes. This was followed by a 5-minute online demonstration video of LUS that also instructs the viewer to perform an LUS on himself or herself with a clinically available ultrasound machine and submit smartphone snapshots of the resulting images as part of a portfolio system. Images required were a B-mode image of the pleura with adjacent ribs, an M-mode image displaying the sandy-beach sign, and a B-mode image of A-lines (see links to both online videos in the footnotes).a,b Group Web received no other hands-on training. Subjects in group control were sent home after the pretest without any instruction and no access to the Web-based materials.

The class and control groups took the posttest on the day after the randomization. Likewise, the subjects in group Web were approached to complete the posttest within 24 hours of receipt of their smartphone images. Groups Web and class returned for a retention test 4 weeks later.

All tests (pretest, posttest, and retention test) had the same format, consisting of a combination of a written test and a practical test as described to follow:

All written tests consisted of a 10-question multiple-choice questionnaire with accompanying images (Supplemental Digital Content, http://links.lww.com/AA/B427) and 1 video clip.c These were designed by 2 of the authors and pertained to LUS for the detection of pneumothorax only. Pretest, posttest, and retention tests were created from the same original test by randomizing both the order of the questions and the order of the multiple-choice answers. Thus, although the questions cover the same material, the ability to memorize the answers is minimized.

In the practical test, the examiner prompted the subject to perform LUS on the examiner’s chest. Table 1 contains a summary of the manual and cognitive skills required in this test. For consistency, the examiner followed a precisely scripted order of questions as listed in the Supplemental Digital Content (http://links.lww.com/AA/B427). A checklist of items completed was documented, and all images requested were stored for later evaluation. Note that not all questions in the practical test were deemed to have equal clinical relevance and were therefore multiplied by an individual weighting factor (see last line of Practical Test in the Supplemental Digital Content, http://links.lww.com/AA/B427) for a maximal score of 10.5 points. In the practical test, 8 of 11 questions required recording of an image or video clip. All imaging was deidentified and was presented to 3 blinded reviewers in randomized fashion for separate evaluation after conclusion of the study. The agreement of the reviewers’ results was assessed with the Krippendorff test. All subsequent data analysis was performed with the median of the scores that the 3 reviewers had assigned to each question (majority rule).

Table 1

Table 1

The total score of each subject was calculated by adding the scores of the written and practical tests and was expressed in percent. Thus, a perfect performance in both the written test and the practical test yields a total of 20.5 points, which is 100%.

The power analysis involving the primary hypothesis was based on preliminary experience teaching and testing 7 inexperienced individuals before and after classroom teaching and finding that improvement of the combined written and practical testing was 26% with an SD of 20%. Based on the clinical judgment of the authors, a difference in improvement in excess of 10% was declared to be clinically meaningful; this was defined a priori as the noninferiority margin. When a 1-sided, 2-sample t test was used, a sample size of 50 anesthesiologists would be required for groups Web and class to detect an inferiority of group Web compared with group class beyond 10% with a power of 0.8. The enrollment target was increased by 20% to 120 in anticipation of possible exclusions.

All statistical analyses were performed using MATLAB (MathWorks, Natick, MA). A significance level of P value <0.05 was used for all comparisons. For data with normal distributions as verified by the Jarque-Bera test, comparisons were performed with 2-sample t tests or analysis of variance (ANOVA) with Bonferroni adjustment for the 10 multiple comparisons. Kruskal-Wallis ANOVA was used for nonparametric data, likewise with Bonferroni adjustment. Fisher exact test was used to compare gender. Klippendorff alpha coefficient was calculated to assess for interrater concordance.7

Back to Top | Article Outline

RESULTS

Enrollment of 138 anesthesiologists occurred from February until December 2014 from 4 large academic teaching hospitals, 1 in the United States (n = 59), 1 in Austria (n = 24) and 2 in Germany (n = 29 and 26). All EM physicians (n = 42) were enrolled at the US center. As shown in Figure 1, all anesthesiologists and 38 of 42 subjects in group EM completed the pretest. After randomization of the anesthesiologists into groups Web (n = 59), class (n = 59), and control (n = 20) and assignment of group EM (n = 38), 3 subjects were excluded because of unavailability of the subject for the posttest. Another 5 subjects did not complete the retention test as planned.

The median time delay between the pretest and posttests was significantly longer in group Web than that in groups class and control (9 vs 1 vs 1.5 days). The delay between the posttest and the retention test was not different between groups Web and class (median of 30 days in both).

The median ages and age ranges for groups Web, class, control, and EM were 32.5 (25–58), 35 (27–70), 34.5 (28–56), and 31 (26–44) years. Group EM was younger than group class; all other groups did not differ significantly in age by Kruskal-Wallis test. Median postgraduate-year training status (given with range) was 4 (1–25), 5 (1–30), 7 (1–25), and 4 (1–8) years for groups Web, class, control, and EM with group EM significantly less than groups class and control by Kruskal-Wallis test. Group EM had reported the total numbers of LUS performed before the study ranging from 15 to 112 (interquartile range, 25% to interquartile range, 75%) with a median of 50.

The stored images and clips from the practical tests were evaluated in a blinded fashion by the 3 reviewers. The interrater agreement was tested, and a Krippendorff alpha of 0.91 was found, which is categorized as “almost-perfect agreement.”7,8

Overall performance of each group in the pretest, posttest, and retention tests is compared in Figure 2. These data were not normally distributed; therefore, Kruskal-Wallis analysis was applied for this comparison.

Figure 2

Figure 2

Baseline performance of groups Web, class, and control did not differ significantly in the pretest. Posttest and retention test results of both groups were not significantly different from the baseline of group EM. Group control did not show improvement in the repeated test. Exact P values for these comparisons are given in the Supplemental Digital Content (http://links.lww.com/AA/B427).

After the teaching intervention, the improvement in written, practical, and overall test scores of groups Web and class was determined and compared with the change in score of group control that had received no teaching. Likewise, at a median of 30 days later, the retention of skills was documented in groups Web and class by the retention test. An ANOVA analysis was performed to compare the performance changes in the written, practical, and combined test scores as shown in Figure 3. The residuals for the ANOVA analyses of the written and the combined (written + practical) test scores were normally distributed as determined by the Jarque-Bera test (P = 0.50 and P = 0.23, respectively). The residuals for the practical test were not normally distributed (P = 0.01). Thus, this comparison was performed with the Kruskal-Wallis test.

Figure 3

Figure 3

Mean combined test scores (Fig. 3C) improved significantly in groups Web and class from the pretest to the posttest by 42.9% (±18.1% SD) and 39.2% (±19.2% SD), respectively. Group control showed a mean performance change of 1.2% (±13.9% SD) which was not significantly different from zero. In both groups Web and class, there was no significant deterioration of performance from the posttest to the retention tests. Both groups performed significantly better than group control as can be inferred from their nonoverlapping 95% confidence intervals (see the Supplemental Digital Content for details, http://links.lww.com/AA/B427).

Figure 4

Figure 4

By using a 2-sample t test to compare the overall performance of groups Web and classroom after the training intervention, we found an average difference of −3.7% with a 95% confidence interval of −10.6% to 3.2%. This result is presented in Figure 4 in relation to the a priori noninferiority margin of 10% in accordance with the Consolidated Standards of Reporting Trials (CONSORT) statement.9

Back to Top | Article Outline

DISCUSSION

This study was motivated by the desire to teach anesthesiologists the relatively simple skill of LUS for the exclusion of pneumothorax. Because anesthesiologists perform central line and nerve block placement near the thorax with an attendant risk of pneumothorax, this would be a useful skill and would allow for early exclusion of a pneumothorax and possibly reduce the reliance on chest radiographs in daily practice. Web-based teaching would allow for more efficient and scalable teaching, thus avoiding the burden of scheduling, cost, and time commitment inherent in classroom-style teaching.

The course designed for this study is based strictly on the recommendations of a widely accepted consensus conference.6 The techniques needed to exclude pneumothorax are a common part of well-known, classroom-based critical care ultrasound courses such as those provided by the Society of Critical Care Medicine or the American College of Chest Physicians. The duration of the classroom training in this study and the material covered are comparable to the amount of time devoted to the same subject in these courses.

Our Web-based training program contains elements similar to the “portfolio” system used by the American College of Chest Physicians: trainees in our Web-based course are required to practice ultrasound imaging on their own with equipment present in their own institutions and submit the images to demonstrate completion of their self-guided training.

The course was effective, as shown in Figures 2 and 3, where the performance of both groups Web and class improved significantly after training. This significant improvement occurred for both the written and the practical tests, as shown in Figure 3. Both modalities of training conferred knowledge that led to similar test performance as the group of EM physicians whom we considered to be a good “gold standard” because of their widespread use of point-of-care ultrasound (Fig. 2). Adequate retention of skills was demonstrated as well, with only nonsignificant decreases in test performance between the post and the retention tests as shown in Figures 2 and 3.

The improvement in the performance was attributable to the teaching intervention and not because of repeated testing. This was demonstrated in Figure 2: the pretest and posttest for group control was similar. Thus, the method of randomizing the questions and answer sequences to produce separate pretest, posttest, and retention tests from the same original test was effective.

A comparison of the learning effect in groups Web and class as shown in Figure 4 supports our primary hypothesis: the Web-based training program designed for this study was not inferior to the classroom-based training program. In fact, the average test improvement was numerically greater after Web-based training than after classroom-based training although this did not reach statistical significance.

In 2010, Platz et al.10 published a study similar to ours, demonstrating that a more complex set of ultrasound skills can be successfully taught to EM physicians with the use of Web-based teaching with minimal practical instruction. Although the subjects of their study had a heterogeneous training background and received a written test only without testing of practical manual skills, our study involved homogenous groups and explicit practical testing. Cuca et al.11 also demonstrated effective teaching of LUS using Web-based platform, albeit their subjects were medical students. They taught a wider spectrum of LUS, not just for pneumothorax detection, and performed only written testing.

The limitations of our study include the unequal recruitment of anesthesiologists among the centers. This reflected the differing sizes of the anesthesia departments. However, even when repeating the analysis with single centers, the noninferiority of Web training could be established in the US and the Austrian centers. In the 2 German centers, the groups did not differ significantly, but the confidence intervals extended beyond the noninferiority margin. In our study, all teaching materials and test questions were presented in English to avoid inconsistency. Although all subjects were fluent in English, this may have reduced the effectiveness of the training and increased the difficulty of the written test for non-native English speakers in Austria and Germany. The EM physicians were all recruited from the US center because their training is standardized. In the Austrian and in the 2 German centers, there were not sufficient numbers of physicians trained specifically in EM. Our study trained only anesthesiologists who were already familiar with the use of ultrasound technology as a result of their training with regional anesthesia and central catheter placement. The results of this study may not be applicable to other medical specialties. Because all examinations were performed on the examiners themselves, our study could not investigate the assessment of sonographic ability in differing body habitus. Also, the practical examination tested for the acquisition of signs that exclude pneumothorax, such as lung sliding. The detection of lung point that provides a positive diagnosis of pneumothorax was tested only in the written examination. As detailed above, the exclusion of pneumothorax is much easier than the positive diagnosis of pneumothorax. Thus, the strength in the described training programs lies in the exclusion of pneumothorax, not in the positive diagnosis of pneumothorax. Finally, our hypotheses were confirmed using theoretical and practical tests that we designed to reflect best clinical practice. However, caution must be exercised when inferring that mastery of these skills would improve patient care.

In conclusion, LUS for the purpose of pneumothorax exclusion can be taught to anesthesiologists effectively with the use of Web-based training. This efficient and cost-effective method should be considered when demand for training increases.

Back to Top | Article Outline

DISCLOSURES

Name: Thomas Edrich, MD.

Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.

Attestation: Thomas Edrich has seen the original study data, reviewed the analysis of the data, approved the final manuscript, and is the author responsible for archiving the study files.

Name: Matthias Stopfkuchen-Evans, MD.

Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.

Attestation: Matthias Stopfkuchen-Evans approved the final manuscript.

Name: Patrick Scheiermann, MD, PhD.

Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.

Attestation: Patrick Scheiermann has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

Name: Markus Heim, MD.

Contribution: This author helped design the study, conduct the study, and write the manuscript.

Attestation: Markus Heim approved the final manuscript.

Name: Wilma Chan, MD.

Contribution: This author helped conduct the study and write the manuscript.

Attestation: Wilma Chan approved the final manuscript.

Name: Michael B. Stone, MD.

Contribution: This author helped design the study, conduct the study, and write the manuscript.

Attestation: Michael B. Stone approved the final manuscript.

Name: Daniel Dankl, MD.

Contribution: This author helped conduct the study and write the manuscript.

Attestation: Daniel Dankl approved the final manuscript.

Name: Jonathan Aichner.

Contribution: This author helped conduct the study and write the manuscript.

Attestation: Jonathan Aichner approved the final manuscript.

Name: Dominik Hinzmann, MD.

Contribution: This author helped conduct the study and write the manuscript.

Attestation: Dominik Hinzmann approved the final manuscript.

Name: Pingping Song, MD.

Contribution: This author helped design the study, conduct the study, and write the manuscript.

Attestation: Pingping Song approved the final manuscript.

Name: Ashley L. Szabo, MD.

Contribution: This author helped conduct the study and write the manuscript.

Attestation: Ashley L. Szabo approved the final manuscript.

Name: Gyorgy Frendl, MD, PhD.

Contribution: This author helped design the study and write the manuscript.

Attestation: Gyorgy Frendl approved the final manuscript.

Name: Kamen Vlassakov, MD.

Contribution: This author helped design the study and write the manuscript.

Attestation: Kamen Vlassakov approved the final manuscript.

Name: Dirk Varelmann, MD.

Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.

Attestation: Dirk Varelmann has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

This manuscript was handled by: Maxime Cannesson, MD.

Back to Top | Article Outline

ACKNOWLEDGMENTS

We thank Chuan-Chin Huang, Sc.D., of the Surgical ICU Translational Research (STAR) Center at Brigham and Women’s Hospital for statistical support.

Back to Top | Article Outline

FOOTNOTES

a Teaching video: Lung ultrasound for pneumothorax detection. Available at: https://youtu.be/KbdQB0t303A. Accessed February 8, 2016.
Cited Here...

b Videotutorial: Lung ultrasound for pneumothorax detection. Available at: https://youtu.be/ysX9ilUXwGY. Accessed February 8, 2016.
Cited Here...

c Written test: video clip. Available at: https://youtu.be/Cn3F7dLF0Mw. Accessed September 27, 2015.
Cited Here...

Back to Top | Article Outline

REFERENCES

1. Wilkerson RG, Stone MBSensitivity of bedside ultrasound and supine anteroposterior chest radiographs for the identification of pneumothorax after blunt trauma.Acad Emerg Med20101711–7
2. Volpicelli GSonographic diagnosis of pneumothorax.Intensive Care Med201137224–32
3. Edrich T, Dunser MW, Lux M, Schaubmair H, Bacher B, Butturini E, Dinges C, Kesner KL, Davidson MJCASE 0-2015: intraoperative transthoracic cardiac and pulmonary ultrasonography.J Cardiothorac Vasc Anesth2015291702–11
4. Edrich T, Pojer C, Fritsch G, Hutter J, Hartigan PM, Stundner O, Gerner P, Berger MMUtility of intraoperative lung ultrasonography.A A Case Rep2015471–4
5. Edrich T, Song P, Dünser MW, Bacher B, Torgersen CA postoperative patient decompensates during transfer from an outside hospital.Chest2014145e14–6
6. Volpicelli G, Elbarbary M, Blaivas M, Lichtenstein DA, Mathis G, Kirkpatrick AW, Melniker L, Gargani L, Noble VE, Via G, Dean A, Tsung JW, Soldati G, Copetti R, Bouhemad B, Reissig A, Agricola E, Rouby JJ, Arbelot C, Liteplo A, Sargsyan A, Silva F, Hoppmann R, Breitkreutz R, Seibel A, Neri L, Storti E, Petrovic TInternational Liaison Committee on Lung Ultrasound (ILC-LUS) for International Consensus Conference on Lung Ultrasound (ICC-LUS)International evidence-based recommendations for point-of-care lung ultrasound.Intensive Care Med201238577–91
7. Hayes AF, Krippendorff KAnswering the call for a standard reliability measure for coding data.Commun Methods Meas2007177–89
8. Landis JR, Koch GGThe measurement of observer agreement for categorical data.Biometrics197733159–74
9. Piaggio G, Elbourne DR, Pocock SJ, Evans SJ, Altman DGCONSORT GroupReporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement.JAMA20123082594–604
10. Platz E, Goldflam K, Mennicke M, Parisini E, Christ M, Hohenstein CComparison of Web-versus classroom-based basic ultrasonographic and EFAST training in 2 European hospitals.Ann Emerg Med201056660–7
11. Cuca C, Scheiermann P, Hempel D, Via G, Seibel A, Barth M, Hirche TO, Walcher F, Breitkreutz RAssessment of a new e-learning system on thorax, trachea, and lung ultrasound.Emerg Med Int20132013145361

Supplemental Digital Content

Back to Top | Article Outline
© 2016 International Anesthesia Research Society