Armantrout, Elaine Ann PT, DSc, ECS; Hummel-Berry, Kathleen PT, PhD; Underwood, Frank PT, PhD, ECS; Nelson, Charlene PT, MA
Doctorate of Science in Clinical Electrophysiology, Rocky Mountain University of Health Professions, Provo, Utah.
Address correspondence to: Dr. Elaine Armantrout, E-mail: firstname.lastname@example.org.
Purpose: The purpose of this study was to determine compliance rates for following three electromyography (EMG) clinical practice guidelines and to determine the reliability of two performance measurement tools developed for this study.
Methods: This was a retrospective descriptive and correlation study of chart reviews of voluntary submissions of EMG reports performed by physical therapists (PTs). Chart reviewers included a PT, a neurologist, and a physiatrist. EMG reports leading to diagnosis of carpal tunnel syndrome (CTS) (n = 43), ulnar neuropathy at the elbow (n = 36), and cervical radiculopathy (n = 38) were collected. The performance measurement tools were analyzed for interrater reliability.
Results: The mean (median) score for CTS was 84% (89%), ulnar neuropathy at the elbow was 71% (72%), and cervical radiculopathy was 83% (85%). The reliability calculation (95% confidence interval) for the performance measurement tool for CTS was 0.74 (0.49–0.86), ulnar neuropathy at the elbow was 0.75 (0.56–0.86), and cervical radiculopathy was 0.57 (0.16–0.79). All three reliability calculations were statistically significant (P < 0.05).
Conclusions: The EMG reports performed by PTs had high compliance rates for following the three clinical practice guidelines. The performance measurement tools developed for CTS, ulnar neuropathy at the elbow, and cervical radiculopathy were moderately reliable. They may be useful for quality improvement projects in individual EMG laboratories.
The prerequisite elements to adequately perform clinical electromyography (EMG) include knowledge base areas, appropriate training, guidelines for practice, and clinical experiences (Table 1). Defining who is qualified to perform EMG studies depends on the acquisition of these prerequisite elements and adequacy of performance.
A key healthcare component that addresses issues of quality and adequacy of healthcare procedures (including EMG studies) is the use of established clinical practice guidelines.1,2 Clinical guidelines are used for education, reimbursement, quality improvement, and research purposes. Performance measurements translate the clinical guidelines into review criteria and tools. Outcome analysis demonstrates the relationship between the clinical guidelines and performance measurements. This analysis can determine the best patient care results for specific clinical conditions. The integration of clinical guidelines with performance measurements and outcome analysis can explain and reduce the variations in practice. This integration can also improve the quality, validate accountability, and lower the costs.
The American Association of Electrodiagnostic Medicine (AAEM), now known as the American Association of Neuromuscular and Electrodiagnostic Medicine, developed and published clinical practice guidelines called Practice Parameters for Carpal Tunnel Syndrome (CTS), ulnar neuropathy at the elbow, and cervical radiculopathy.3 A critical review of the literature by the Quality Assurance Committee of the AAEM determined the sensitivity and specificity for the diagnosis of these three upper extremity conditions.3 Use of EMG for the diagnosis of CTS has high sensitivity (49%–84%) and specificity (95%). Use of EMG for the diagnosis of ulnar neuropathy at the elbow has moderate sensitivity (37%–86%) and high specificity (95%). Use of EMG for the diagnosis of cervical radiculopathy has moderate sensitivity (50%–71%) and high specificity (65%–85%). These three clinical practice guidelines are reported to be reliable and valid.3
Physician compliance with these clinical practice guidelines ranges from 10% to 69% for CTS and from 66% to 69% for ulnar neuropathy at the elbow.4–6 There are no published reports on compliance with following the cervical radiculopathy guidelines. No published studies were found comparing EMG reports performed by physical therapists (PTs) with the AAEM practice parameters established for CTS, ulnar neuropathy at the elbow, or cervical radiculopathy.
The purpose of this study was to establish the compliance rates of EMG reports performed by PTs by comparing them with clinical practice guidelines developed by the AAEM for three upper extremity conditions. Secondary objectives were to determine the interrater reliability of the study's two performance measurement tools.
Two performance-measurement tools were developed for this study. Checklists were developed for each of the three AAEM practice parameters for CTS (Appendix A), ulnar neuropathy at the elbow (Appendix B), and cervical radiculopathy (Appendix C). The checklist items were weighted based on the strength of evidence reported in the practice parameters. Strength of evidence is based on the following six criteria: prospective study design, diagnosis of the condition is based on clinical criteria independent of the electrodiagnostic procedure under evaluation, the electrodiagnostic procedure is sufficiently described, the limb temperature is recorded, reference values are provided, and the criteria for abnormal findings are clearly stated. Items from published studies based on the literature review or studies that met all six criteria (class A) were awarded up to six maximum points. Items that met four or five of the six criteria (class B) were awarded up to four maximum points. Items meeting three or less of the six criteria (class C) were awarded up to two maximum points. The score for each of the checklists was calculated as the percentage of points acquired out of the maximum possible points. Expert opinion on how adequately the conclusions of the report reflected the EMG data was determined using a seven-point Likert rating scale (Appendix D).
The EMG reports were solicited from PT electromyographers from across the country over a three-month period from January to March 2006. A participating PT electromyographer may have submitted more than one EMG report for review, but each EMG report represented a different upper extremity condition. Exclusion criteria included other concomitant upper extremity peripheral nerve conditions, such as underlying nerve disease and/or other peripheral nerve pathology in addition to the single upper extremity condition. The EMG reports were devoid of any personal patient information as guided by the Health Information Portability and Accountability Act. The EMG reports masked the identity of the participating PT electromyographer before scoring.
The checklists, Likert rating opinion forms, and EMG reports were sent to three masked reviewers: a PT, a neurologist, and a physiatrist. These reviewers held board certification in EMG in their respective disciplines. They practiced in different sections of the country (East Coast, Midwest, and West Coast).
The Likert rating scores would provide an alternative account of the appropriateness and adequacy of the EMG reports. Interrater reliability of the checklists and Likert ratings were compared across the three reviewers for each of the three upper extremity conditions performed by calculation of intraclass correlation coefficients (ICC, 2.1). Validity was established by calculation of correlations (Pearson's product-moment or Spearman's r), which compared the checklist scores with the Likert rating scores.
The minimum required sample size was 23 EMG reports determined by an a priori calculation based on the checklist scores from a pilot study with an α level set at 0.05. The estimated sample sizes for the Likert ratings scores from the pilot study ranged from 61 to 4904, numbers deemed too large to attain for this study.
Approval for exempt review status was obtained. The checklists scores and Likert rating scores descriptive statistics including means, medians, SDs, and skewness along with interrater correlations using intraclass correlation coefficients were determined (Graduate Pack 11.0.4 for Macintosh OS X; SPSS Inc., Chicago, IL).
Forty-three carpal tunnel case reports, 36 ulnar neuropathy at the elbow case reports, and 38 cervical radiculopathy case reports were collected and sent to the three reviewers for scoring. This represented a 34% response rate. Not all respondents submitted case reports for each upper extremity condition. Of the 43 respondents, 40 held board certification in clinical electrophysiologic physical therapy from the American Board of Physical Therapy Specialties and four were females.
The checklist and Likert rating descriptive statistics were determined (Tables 2 and 3). The checklist and Likert rating correlations between the reviewers were calculated (Table 4). Box plot graphs describing the checklist and Likert rating scores were depicted (Figure 1). Most of the data sets (15 of 18) were skewed left, indicating that the data were not normally distributed. The checklist data sets for ulnar neuropathy at the elbow for the neurology and physiatry reviewers and cervical radiculopathy for the physiatry reviewer were normally distributed. Post hoc statistical power was calculated and was high (>0.94) for the checklist scores, but very low (<0.12) for the Likert rating scores. The overall checklist mean and median scores and Likert rating mean and median scores (averaged from all three raters) are listed in Table 5. Averaging all scores for all three upper extremity conditions resulted in an overall checklist mean (median) score of 79% (82%) and an overall Likert rating mean (median) score of 78% (83%).
Clinical practice guidelines are a critical component for assessing the quality of healthcare. Following clinical practice guidelines is useful when striving for best practice patterns by reducing variations in practice, lowering costs, and providing quality care. What should be the ideal goal for following clinical practice guidelines? No guideline would expect to have both 100% sensitivity and 100% specificity, so one would expect compliance rates to be <100%. Burgers et al7 in a study evaluating guideline characteristics suggest that the lower level of the highest compliance rate is 70%. The checklist mean scores ranging from 71% to 84% and median scores ranging from 72% to 89% were high for all three upper extremity conditions. Previously published studies describing physician compliance rates range from 10% to 69% and may be high in comparison with this study's outcomes. The previously published studies report frequency distributions as mean percentages. These previously published studies do not use weighted values for each item (based on the weight of the evidence in the literature). In other words, if each item is valued equally, then the reported compliance rates could be inflated when compared with this study. Weighting items in this study improved each item's meaningfulness by linking their values to the strength of evidence in the literature.
The checklist reliability calculations were moderately high for all three upper extremity conditions. This suggested that the checklist performance measurement tools developed for CTS, ulnar neuropathy at the elbow, and cervical radiculopathy were moderately reliable. The three checklists may be useful as quality assurance/quality improvement tools for an EMG laboratory.
Although the Likert rating mean scores were high for all three upper extremity conditions and very similar to the checklist scores (Table 5), this study determined that they were not reliable. The authors of the clinical practice guidelines do state that all three are reliable and valid based on a meta-analysis of previously published studies evaluated against stringent guideline development parameters.3 Why were the Likert rating scores unreliable? It may have been due to differences in the reviewers' provider credentials (PT, neurologist, and physiatrist) or to their regional differences (East Coast, Midwest, and West Coast). It is more likely that the reliability point estimates were low because of the lack of variability among the scores and an inadequate sample size (low statistical power). Most of the Likert rating scores were similarly high with very few low scores represented. Another possibility was that the three clinical practice guidelines were not valid in assessing interpretation. In other words, the clinical practice guidelines addressed how to perform an EMG report, but not how to interpret the report. An EMG report may well follow the guidelines, but its conclusion may be inadequate or incorrect. The reviewers' expert opinions of the EMG conclusions had high overall average scores, but they did not correlate. This issue may need to be considered by the clinical practice guideline authors. The question that the performance measurement tool attempted to address: “Is the conclusion supported by the data?” remains unanswered.
A limitation of this study was that the submission of the EMG reports was voluntary. A bias for submitting a best case type of report tailored to meet the clinical practice guidelines was possible.
This study suggested that PTs who perform EMG studies have high compliance rates for following established clinical practice guidelines. The checklist performance measurement tools developed for CTS, ulnar neuropathy at the elbow, and cervical radiculopathy were moderately reliable. They may be useful for quality improvement projects in individual EMG laboratories. Further study recommendations include evaluation of the effect of these clinical practice guidelines and their respective performance measurement tools on patient outcomes such as a definitive (final) diagnosis or treatment. It may be useful to compare compliance rates between different provider groups. Finally, the issue of including interpretation items into the three clinical practice guidelines needs to be addressed.
Appendix A: Checklist Carpal Tunnel Syndrome Cited Here...
TABLE. No Caption av...Image Tools
Appendix B: Checklist Ulnar Neuropathy at the Elbow Cited Here...
TABLE. No Caption av...Image Tools
Appendix C: Checklist Cervical Radiculopathy Cited Here...
TABLE. No Caption av...Image Tools
Appendix D: Likert Rating Scale Cited Here...
TABLE. No Caption av...Image Tools
1. American Medical Association. Attributes to Guide the Development and Evaluation of Practice Parameters/Guidelines. Chicago: American Medical Association; 1996.
2. Kaegli L. AMA clinical quality improvement forum ties it all together: from guidelines to measurement to analysis and back to guidelines. Jt Comm J Qual Improv. 1999;25:95–106.
3. American Association of Electrodiagnostic Medicine, American Academy of Neurology, American Academy of Physical Medicine and Rehabilitation. Guidelines in electrodiagnostic medicine. Muscle Nerve. 1999;22 (suppl 8).S139–S221.
4. Corwin HM, Kasdan ML. Electrodiagnostic reports of median neuropathy at the wrist. J Hand Surg [Am]. 1998;23:55–57.
5. Storm S, Beaver S, Giardino N, et al. Compliance with electromyography guidelines for patients undergoing carpal tunnel release. Arch Phys Med Rehabil. 2005;86:8–11.
6. Thibault MW, Robinson LR, Franklin G, et al. Use of AAEM guidelines in electrodiagnosis of ulnar neuropathy at the elbow. Am J Phys Med Rehabil. 2005;84:267–273.
7. Burgers JS, Grol RP, Zaat JO, et al. Characteristics of effective clinical guidelines for general practice. Br J Gen Pract. 2003;53:15–19.