Sixty-three shoulders were scanned by both the specialist shoulder surgeon and the first-year orthopaedic resident. They agreed about the presence or absence of a full-thickness rotator cuff tear in 86% of the shoulders and showed a kappa value of 0.71 (standard error = 0.09), suggesting substantial agreement among the clinicians. The agreement among these clinicians and the radiology department was equally high, with the first-year orthopaedic resident agreeing with radiology about the findings in 85% of eighty-two shoulders that were scanned by both the resident and a radiologist (kappa value = 0.70, standard error = 0.08) and the specialist shoulder surgeon agreeing with radiology about the findings in 83% of seventy-eight shoulders that were scanned by both the specialist and a radiologist (kappa value = 0.67, standard error = 0.08).
Table III shows ultrasound sizing of correctly identified full-thickness rotator cuff tears from all four surgeons matched with intraoperative sizing for the two training periods. Ultrasound and intraoperative sizing of the full-thickness tears as small to medium or large to massive showed agreement of 84% (95% confidence interval [CI], 71% to 97%) for the initial training period of the first fifty scans and agreement of 95% (95% CI, 88% to 100%) for the later training period of the second fifty scans for the surgeons who completed 100 scans each.
For the 303 ultrasound scans performed by the surgeons in the study, there was the potential for 108 scans to identify a full-thickness tear. Eleven scans failed to identify a full-thickness tear; ten of these were reported as partial-thickness tears and only one was reported as an intact rotator cuff. All of the missed tears were small to medium sized. Fourteen scans incorrectly identified a full-thickness tear; nine of these had partial-thickness tears at arthroscopy.
The use of shoulder ultrasound by orthopaedic surgeons as an adjunct to clinical history and physical examination can provide important information at point of access for the patient and allows the provision of a so-called one-stop clinic, saving time and hospital visits, reducing demand on radiology departments, and potentially offering a cost saving.
This study aimed to bridge the gap between the uninitiated orthopaedic surgeon and those with many years of ultrasound experience in an effort to establish some evidence for a suitable minimum training period for individuals wishing to follow this independent learning method.
Current training guidelines in the use of musculoskeletal ultrasound for clinicians who are not radiologists require the learning clinician to be supervised directly by a qualified musculoskeletal ultrasonographer or radiologist for at least 150 to 300 scans17-19. The need for a supervising radiologist or sonographer to act as the gold standard within the confines of the radiology department is rather limiting. The substantial time investment required by the learning clinicians and supervising radiologists or sonographers to complete this training renders these protocols unfeasible for clinicians who are not radiologists.
Our learning method utilizes arthroscopy as a gold standard for the assessment of rotator cuff integrity. Combining this gold standard with same-day comparison of ultrasound and arthroscopic findings provides a learning method whereby surgeons can hone their ultrasound technique and image interpretation quickly and accurately by using a definitive outcome measure against which they can judge their findings.
The predictive values for ultrasound identification of a full-thickness tear attained by the surgeons using our learning method from their initial training period of their first fifty scans show a high level of proficiency even at this stage. The values attained during the later training period of the second fifty scans are comparable with published data for surgeons experienced in shoulder ultrasound and for musculoskeletal radiologists10,14-16. These data are detailed for comparison in Table IV.
Compared with the predictive values for the identification of full-thickness tears demonstrated by the local musculoskeletal radiology department (sensitivity of 93%, specificity of 85%, positive predictive value of 76%, negative predictive value of 96%, and accuracy of 87%), the results attained in the study show that the surgeons demonstrated a similar level of ability to identify full-thickness tears within the initial training period of their first fifty scans using the learning method.
The ultrasound measurement of tear size demonstrated by the surgeons was also comparable with the results seen from the local musculoskeletal radiology department. When categorizing the tears into clinically useful groups, as small to medium (≤3 cm) or large to massive (>3 cm), the agreement between preoperative ultrasound measurements and intraoperative sizing increased from 84% in the initial training period to 95% in the later training period. Due to the small sample size of tears identified, statistical analysis of the difference between these values was not valid. Results from the local musculoskeletal radiology department demonstrated agreement for the same measurements in 71% of the shoulders, which was again equaled by the surgeons within the initial training period. It is important to note, however, that preoperative radiology ultrasound scans were not performed on the day of surgery. The time delay between the radiology scan and the day of surgery may have allowed for tears to increase in size and impact the level of agreement between the radiology reports and intraoperative findings. This limitation does not detract from the high level of agreement between the surgeons’ ultrasound measurements and intraoperative findings.
Eleven scans failed to identify a full-thickness rotator cuff tear and fourteen scans mistakenly identified partially torn or intact tendons as containing a full-thickness tear. These mistakes may reflect the inexperience of the user, limitations of the imaging modality in terms of technological capability, or even patient variables such as anatomical differences and body habitus. Nonetheless, the rate and type of mistakes made are comparable with those noted in recent published results of ultrasound reporting compared with surgical findings by experienced users of shoulder ultrasound4-7,9,10,14-16.
Perhaps of most importance, the study showed a high negative predictive value for the detection of full-thickness rotator cuff tears for all of the participants. As a consequence, a negative scan performed in the acute or chronic setting can reassure the assessing surgeon that the presence of a full-thickness rotator cuff tear is highly unlikely and therefore the potential need for early repair is not indicated.
The learning protocol used in this study offers an independent method of learning for surgeons that may also be used for continued performance monitoring and self-assessment after completion of training to maintain a high standard of care. When using ultrasound in an outpatient setting, a surgeon can retain an image log and report findings as part of the medical record; predictive values can be reviewed for the patients who progress to surgery.
With any clinical investigation that relies on subjective evaluation and technical ability, there are limitations that may impact its evaluation. The study was limited by the nature of our patient group; arthroscopic diagnosis was the chosen outcome measure for comparison and, as such, the patients included in the study had already failed conservative therapy and may have had a higher prevalence of structural rotator cuff defects than a cross section of patients seen in a shoulder clinic. This may represent an example of spectrum bias as the prevalence of a condition within a study population may differ from the target population and affect the apparent predictive values obtained by a diagnostic test used in that population. Table IV details the prevalence of full-thickness tears in this study population and that of other recent publications, showing this study to have a comparable or lower prevalence of tears than the other investigations. This finding could lead to this study having a falsely lower sensitivity for the detection of full-thickness tears compared with the other publications.
During the patients’ clinic attendance, at least two months prior to surgery, the specialist shoulder surgeon in the study had previously performed a physical examination on the patients he scanned and, in some cases, would have seen a prior radiology report. These issues may have led to bias in his ultrasound reporting, although a substantial time had passed between prior clinical assessment and the day of ultrasound scanning for all shoulders, and we did not see a difference in performance between the specialist and the remainder of the group who had not examined the patients or had any prior knowledge of their history.
The ultrasound technique in this study was not a detailed musculoskeletal ultrasound scan, as may be performed by a radiologist, but aimed to establish the most important aspect of image analysis in rotator cuff pathology: the presence or absence of a full-thickness tear. Including this fast, affordable, and immediate imaging method as an adjunct to clinical evaluation does not eradicate the need for further musculoskeletal imaging but may help to screen patients effectively prior to more advanced imaging methods in some cases.
The study showed that surgeons could develop proficiency in ultrasound assessment of rotator cuff integrity in a relatively short period of time by using an independent training method and produce results that are comparable with published data for experienced musculoskeletal radiologists. It was not within the scope of this study, however, to establish a minimum level of proficiency required for surgeons to perform ultrasound as part of clinical practice or the number and frequency of ultrasound scans that need to be completed after the initial training period to maintain that level of proficiency.
In conclusion, the study demonstrates that orthopaedic surgeons with varying levels of clinical experience can become competent in shoulder ultrasound in a relatively short period of time by following an independent learning program. Importantly, this method of learning offers the opportunity for shoulder surgeons to use their everyday surgical practice to audit and improve their ultrasound assessment of the rotator cuff. Combined with the growing affordability and quality of portable ultrasound machines, this learning method facilitates independent training and ongoing validation for surgeon-performed ultrasound.
A figure showing the shoulder ultrasound training program for surgeons is available with the online version of this article as a data supplement at jbjs.org.
Investigation performed at the Oxford National Institute for Health Research Biomedical Research Unit, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
1. Ostör AJ Richards CA Prevost AT Speed CA Hazleman BL. Diagnosis and relation to general health of shoulder disorders presenting to primary care. Rheumatology (Oxford). 2005 Jun;44(6):800–5. Epub 2005 Mar 15.
2. Chard MD Hazleman R Hazleman BL King RH Reiss BB. Shoulder disorders in the elderly: a community survey. Arthritis Rheum. 1991 Jun;34(6):766–9.
3. Vecchio P Kavanagh R Hazleman BL King RH. Shoulder pain in a community-based rheumatology clinic. Br J Rheumatol. 1995 May;34(5):440–2.
4. Mack LA Matsen FA 3rd Kilcoyne RF Davies PK Sickler ME. US evaluation of the rotator cuff. Radiology. 1985 Oct;157(1):205–9.
5. Teefey SA Hasan SA Middleton WD Patel M Wright RW Yamaguchi K. Ultrasonography of the rotator cuff. A comparison of ultrasonographic and arthroscopic findings in one hundred consecutive cases. J Bone Joint Surg Am. 2000 Apr;82(4):498–504.
6. Brenneke SL Morgan CJ. Evaluation of ultrasonography as a diagnostic technique in the assessment of rotator cuff tendon tears. Am J Sports Med. 1992 May-Jun;20(3):287–9.
7. Wiener SN Seitz WH Jr. Sonography of the shoulder in patients with tears of the rotator cuff: accuracy and value for selecting surgical options. AJR Am J Roentgenol. 1993 Jan;160(1):103–7; discussion 109-10.
8. Hedtmann A Fett H. [Ultrasonography of the shoulder in subacromial syndromes with disorders and injuries of the rotator cuff]. Orthopade. 1995 Nov;24(6):498–508. German.
9. Ottenheijm RP Jansen MJ Staal JB van den Bruel A Weijers RE de Bie RA Dinant GJ. Accuracy of diagnostic ultrasound in patients with suspected subacromial disorders: a systematic review and meta-analysis. Arch Phys Med Rehabil. 2010 Oct;91(10):1616–25.
10. Teefey SA Rubin DA Middleton WD Hildebolt CF Leibold RA Yamaguchi K. Detection and quantification of rotator cuff tears. Comparison of ultrasonographic, magnetic resonance imaging, and arthroscopic findings in seventy-one consecutive cases. J Bone Joint Surg Am. 2004 Apr;86(4):708–16.
11. Bryant L Shnier R Bryant C Murrell GA. A comparison of clinical estimation, ultrasonography, magnetic resonance imaging, and arthroscopy in determining the size of rotator cuff tears. J Shoulder Elbow Surg. 2002 May-Jun;11(3):219–24.
12. de Jesus JO Parker L Frangos AJ Nazarian LN. Accuracy of MRI, MR arthrography, and ultrasound in the diagnosis of rotator cuff tears: a meta-analysis. AJR Am J Roentgenol. 2009 Jun;192(6):1701–7.
13. Fotiadou AN Vlychou M Papadopoulos P Karataglis DS Palladas P Fezoulidis IV. Ultrasonography of symptomatic rotator cuff tears compared with MR imaging and surgery. Eur J Radiol. 2008 Oct;68(1):174–9. Epub 2007 Dec 21.
14. Al-Shawi A Badge R Bunker T. The detection of full thickness rotator cuff tears using ultrasound. J Bone Joint Surg Br. 2008 Jul;90(7):889–92.
15. Ziegler DW. The use of in-office, orthopaedist-performed ultrasound of the shoulder to evaluate and manage rotator cuff disorders. J Shoulder Elbow Surg. 2004 May-Jun;13(3):291–7.
16. Jeyam M Funk L Harris J. Are shoulder surgeons any good at diagnosing rotator cuff tears using ultrasound?: A comparative analysis of surgeon vs radiologist. Int J Shoulder Surg. 2008 Jan;2(1):4–6.
17. American Institute of Ultrasound in Medicine. Official statement: training guidelines for the performance of musculoskeletal ultrasound examinations. 2009 Nov 14. http://www.aium.org/resources/statements.aspx
. Accessed 2012 Aug 15.
18. Board of the Faculty of Clinical Radiology. Ultrasound training recommendations for medical and surgical specialties. The Royal College of Radiologists; 2005. p 1–58.
19. Minimum training requirements for the practice of Medical Ultrasound in Europe. Ultraschall Med. 2010 Aug;31(4):426-7. Epub 2010 Aug 19.
20. Allen GM. Shoulder ultrasound imaging-integrating anatomy, biomechanics and disease processes. Eur J Radiol. 2008 Oct;68(1):137–46. Epub 2008 Apr 21.
21. Mack LA Nyberg DA Matsen FA 3rd. Sonographic evaluation of the rotator cuff. Radiol Clin North Am. 1988 Jan;26(1):161–77.
Disclosure: One or more of the authors received payments or services, either directly or indirectly (i.e., via his or her institution), from a third party in support of an aspect of this work. None of the authors, or their institution(s), have had any financial relationship, in the thirty-six months prior to submission of this work, with any entity in the biomedical arena that could be perceived to influence or have the potential to influence what is written in this work. Also, no author has had any other relationships, or has engaged in any other activities, that could be perceived to influence or have the potential to influence what is written in this work. The complete Disclosures of Potential Conflicts of Interest submitted by authors are always provided with the online version of the article.Copyright 2013 by The Journal of Bone and Joint Surgery, Incorporated