Secondary Logo

Journal Logo

Improving the Quality of Evidence on Upper-Limb Prostheses Through the Standardization of Outcome Measurement

Resnik, Linda PT, PhD, FAPTA

Journal of Prosthetics and Orthotics: October 2017 - Volume 29 - Issue 4S - p P21–P24
doi: 10.1097/JPO.0000000000000160
Proceedings
Free

ABSTRACT This commentary discusses the challenges in conducting systematic reviews of studies comparing upper-limb prostheses given the wide variety of outcome measures used in scientific research. The paper calls for the use of a standardized brief core set of validated outcome measures. A culture change is needed to facilitate widespread adoption of a standardized core set in all routine clinical encounters and research studies. The result will be critically needed data to enable comparison of outcomes of different prosthesis types and components, and ultimately, stronger scientific evidence to assist in clinical and policy decision making.

LINDA RESNIK, PT, PhD, FAPTA, is affiliated with the Providence VA Medical Center, Providence, Rhode Island.

Disclosure: The author declares no conflict of interest.

Correspondence to: Linda Resnik, PT, PhD, FAPTA, Providence VA Medical Center, 830 Chalkstone Ave, Providence, RI 02908; email: Linda.Resnik@va.gov

In 2014, Carey, Lura, and Highsmith published a systematic review that examined differences between myoelectric and body-powered prostheses to inform evidence-based clinical practice regarding prescription of these devices and training of users.1 The evidence synthesis resulting from this review found mixed results and a paucity of data. Only 31 of 462 studies identified through rigorous searches met the inclusion criteria for the systematic review, and of these, nearly 60% were rated as having low methodological quality. Based on these articles, 11 evidence statements about the differences between myoelectric and body-powered devices were generated. However, 82% of these evidence statements were based on low-quality or insufficient evidence. The investigators also reported that very few outcome measures were used in multiple studies, making it particularly difficult to compare or pool results across studies.

Outcome assessments used in the included articles fell into 3 overall categories: surveys, laboratory and clinical assessments, and analysis of use and ability to perform activities of daily living (ADLs). The systematic review reported that investigators frequently developed their own outcome metrics or survey instrument and did not use standardized measures. The constructs addressed within the outcome assessments in the included studies were quite broad and ranged from timed manipulation and precision to kinematic measurements of joint angles to ADL independence and prosthesis rejection and patterns of wear.

The results of this systematic review underscore the need for more well-designed studies to address the comparative effectiveness of myoelectric and body-powered devices. The need for comparative effectiveness studies in upper-limb prosthetics is becoming increasingly more urgent, given the advances in the field and the proliferation of new devices, components, and technologies, many of which are associated with greater costs and require specialized training for clinicians and patients. Without a body of evidence quantifying the relative benefits of specific devices and technologies, the clinician has only expert opinion and experience, but insufficient research evidence, to inform decision making. Furthermore, it can be difficult, if not impossible, to justify the need for specific devices to third-party payers who may view the dearth of research evidence as a reason to restrict policy coverage.

The heterogeneity and breadth of outcome measures used in the articles included in this systematic review are not surprising. This phenomenon is consistent with findings from systematic reviews in fields as diverse as blood pressure monitoring and gynecological health.2,3 The challenge for any field is that a wide heterogeneity of outcome measures makes it difficult to conduct a systematic review across studies. It is also not surprising that Carey’s systematic review identified gaps in the literature. This is also common in well-conducted systematic reviews. The challenge for the field of amputation rehabilitation now, given the results of this systematic review, is to strategically address the identified knowledge gaps by charting a course that ultimately will lead to better-quality evidence. Thus, the purpose of this editorial is to make recommendations for standardization of outcome measures across the field of upper-limb prosthetics. This is needed to strengthen the quality of evidence and increase the potential usefulness of future research by collecting common data elements. This data collection will enable pooling of results across studies in future systematic reviews and maximize usefulness of research findings in development of clinical practice guidelines.

Standardization of outcomes measures for clinical trials has been proposed as a solution to the problems of inappropriate and non-uniform outcome selection.4 A variety of methods have been used to develop core outcome sets, and there is no consensus in the literature on which approach is the best.5 Nevertheless, it is widely recognized that the first steps in standardizing outcomes data collection for any field are identifying domains that are particularly relevant to the target patient population and then establishing priorities for the key domains that should be addressed. Efforts to identify core sets of content typically involve some type of consensus process, often involving the Delphi technique. Early examples of these types of national and international initiatives can be seen in the recommendations for a core set of measures for low back pain,6 the World Health Organization (WHO) recommendations for minimum data collection requirements for cancer trials trials,7 and the Outcome Measures in Rheumatology (OMERACT).8 Another more recent example is the efforts of the National Institutes of Health (NIH) to develop a brief multidimensional set of measures of neurological and behavioral function to assess cognition, emotional, motor, and sensory functions across study designs.9–13 These major initiatives have demonstrated that developing core sets through a consensus process is a time-consuming and effort-intensive process but has great potential for advancing scientific knowledge.

The field of upper-limb amputation rehabilitation would greatly benefit from the identification and adoption of a core set of measures for use in research and clinical care. Fortunately, the field has already made strides toward identification of appropriate measures through the work of the Upper Limb Prosthetic Outcome Measures Working Group (UPLOM)14,15 and recent systematic reviews.16–20 The UPLOM was an international group that proposed a toolkit of validated measures based on the conceptual framework of the International Classification of Functioning, Disability and Health (ICF).21 They suggested that the core set of measures would differ depending on the development stage of the prosthesis, with emphasis on physical function at the early stages of device development and emphasis on measurement of activities and participation when examining outcomes of clinical and home usage.15 The UPLOM recommendations addressed the following domains: functional assessment, hand function, goal setting, and satisfaction/quality of life. Their efforts resulted in a list of recommended measures that the authors reported had sufficient evidence in support of their measurement properties, as well as a list of measures that warranted further consideration. However, the recommendations fell short of recommending a mandatory core set to be used across research studies or across clinical care settings.17 Three other relatively recent systematic reviews of outcome measures linked the content of measures to the ICF framework16 and synthesized the available evidence on measurement properties.17,18 Collectively, these systematic reviews provide evidence to help select outcome measures with sound measurement properties, but none prescribe a minimum core data set and will not in and of themselves move the field forward toward standardization.

However, there are ongoing efforts, funded by the Bridging Advanced Developments for Exceptional Rehabilitation (BADER) consortium, to develop consensus on the content of a toolkit of measures for persons with upper-limb trauma and amputation. The toolkit project involves evidence synthesis from systematic reviews of measures,19,20 followed by a consensus conference of stakeholders from the US Department of Veterans Affairs (VA) and the US Department of Defense (DoD) to select measures for use across these systems of care.

Because patients with amputation often have disability caused by impairments in body structures and function with resulting activity limitations, one of the BADER-funded systematic reviews focused solely on outcome measures that assessed domains most commonly addressed in amputation rehabilitation: functions of movement and mobility, activities, self-care, and domestic life.19 I had the privilege of leading that systematic review that identified outcome measures and evaluated the focus, content, and measurement properties of each identified measure. Although the consensus-building process and testing of the preliminary toolkit is not yet complete, based on the results of Carey, Lura, and Highsmith’s systematic review, it is clear that the efforts to develop a core set must extend beyond the VA and DoD communities to reach wider and international research and clinical communities.

In my view, the field needs to reach agreement on a brief core set of measures that should be used in every clinical study of upper-limb amputation rehabilitation and collected in routine clinical encounters. We then need to identify supplementary measure sets for use in subpopulations. A core set of measures should address a very limited set of domains and include only those measures that have been validated in this patient population. Such a core set of measures must, in my opinion, be feasible to implement in routine clinical practice and research without excessive administrative burden in terms of time to administer or special training of the test administrator. Further, this core set of measures must include patient-centered outcomes, meaning outcomes that are meaningful to patients, families, and clinicians who work with them, and which are widely considered critical to the measurement of quality.22 Although such a minimal core data set would not include the full range of important metrics needed to address specific research questions, or clinical care needs, it would facilitate comparison of outcomes across studies and health care settings.

In my view, every clinician and every research study should collect several standardized outcome measures that assess impairments of physical function and activity limitation. I recommend this core set of measures (shown in Figure 1) including measures of grasp function and fine motor activity, ADL performance, self-reported disability, and quality of life. In addition, I recommend that a core set of measures for studies of prosthetic devices should evaluate device satisfaction and prosthesis use. Based on the results of our recent systematic reviews of measures and pending results of the BADER preliminary testing of the proposed toolbox, I hope that, in the near future, key stakeholders will adopt a brief set of standardized, validated measures. That said, I am also aware that, despite efforts to develop measures of activity performance,23–25 the field still lacks brief, concise validated measures of ADL performance for adults and more measure development and testing work is needed.

Figure 1

Figure 1

It is clear that the process of identifying and reaching consensus on supplemental measure sets for subpopulations of persons with upper-limb amputation will take longer and will no doubt require major investment to complete. In my view, supplementary sets of measures would be used to collect data on a broader range of important domains, and the supplemental set may vary for specific subsets of patients with upper-limb amputation, such as persons who are myoelectric device users only or persons with bilateral amputation. Furthermore, supplementary measure sets may need to be developed to assess outcomes of new technological innovations that are or will soon be tested such as sensory-enabled prostheses26,27 and osseointegrated prostheses.28 These supplementary measure sets will necessarily include “emerging measures” that are new and promising and assess some of the expected benefits of these technologies, but because of their newness and the small samples in which they will be used, they do not possess strong evidence in support of their measurement properties.

The upper-limb amputation field should move swiftly in adopting a standardized core set of measures for use in research and clinical practice. With the proliferation of electronic medical records and the opportunity to capture clinical data elements for use in quality improvement efforts and health services research, the availability and analysis of standardized outcome measures can be harnessed to improve quality of care. Importantly, we must recognize that identification and endorsement of core sets of measures alone will be insufficient to insure general adoption of these measures.

A culture change is needed to change practice and research so that core outcomes data collection is an integral aspect of our work. This culture change can be spurred by widespread efforts to disseminate information about core sets to the relevant clinical, research, and other stakeholder communities. Professional associations, consumer organizations, funding agencies, and leading journals can be extremely influential in reaching out to their constituents to strongly recommend or even require using the brief toolkit of outcome measures. With such a unified effort, future systematic reviews comparing myoelectric and body-powered prostheses may be able to pool evidence or even conduct meta-analyses of key outcomes. The benefits will be that patients, clinicians, and payers will have stronger scientific evidence to assist in decision making.

Back to Top | Article Outline

ACKNOWLEDGMENTS

This work was supported by VA RR&D A0264-S. The information in this manuscript does not necessary reflect the position or policy of the government; no official endorsement should be inferred. The views expressed herein are those of the authors and do not reflect the official policy or position of the US government.

Back to Top | Article Outline

REFERENCES

1. Hurst S. Better understanding of the patient’s autonomy [in French]. Rev Med Suisse 2013;9(377):594.
2. Uhlig K, Patel K, Ip S, et al. Self-measured blood pressure monitoring in the management of hypertension: a systematic review and meta-analysis. Ann Intern Med 2013;159(3):185–194.
3. Rahn DD, Abed H, Sung VW, et al. Systematic review highlights difficulty interpreting diverse clinical outcomes in abnormal uterine bleeding trials. J Clin Epidemiol 2011;64(3):293–300.
4. Clarke M. Standardising outcomes for clinical trials and systematic reviews. Trials 2007;8:39.
5. Williamson PR, Altman DG, Blazeby JM, et al. Developing core outcome sets for clinical trials: issues to consider. Trials 2012;13:132.
6. Deyo RA, Andersson G, Bombardier C, et al. Outcome measures for studying patients with low back pain. Spine (Phila Pa 1976) 1994;19(18 Suppl):2032S–2036S.
7. Miller AB, Hoogstraten B, Staquet M, Winkler A. Reporting results of cancer treatment. Cancer 1981;47(1):207–214.
8. OMERACT, Conference on Outcome Measures in Rheumatoid Arthritis Clinical Trials. Proceedings. Maastricht, The Netherlands, April 29-May 3, 1992. J Rheumatol 1993;20(3):527–591.
9. Baumrucker SJ, Adkins RW, Stolick M, et al. Surrogates with conflicting interests: who makes the decision?. Am J Hosp Palliat Care 2012;29(6):497–500.
10. Evans L, Ward NS. Prior experience is key for confidence in surrogate end-of-life decisions. Crit Care Med 2012;40(8):2494–2495.
11. Anghelescu DL, Faughnan LG, Oakes LL, et al. Parent-controlled PCA for pain management in pediatric oncology: is it safe?. J Pediatr Hematol Oncol 2012;34(6):416–420.
12. Majesko A, Hong SY, Weissfeld L, White DB. Identifying family members who may struggle in the role of surrogate decision maker. Crit Care Med 2012;40(8):2281–2286.
13. Mullock A. Deciding the fate of a minimally conscious patient: an unsatisfactory balancing act?: W v M and others [2011] EWHC 2443 (Fam). Med Law Rev 2012;20(3):460–469.
14. Hill W, Stavdahl O, Hermansson L, et al. Functional outcomes in the WHO-ICF model: establishment of the Upper Limb Prosthetic Outcome Measures Group. J Prosthet Orthot 2009;21(2):115–119.
15. Hill WK, Hermansson LN, Hubbard S, et al. Upper Limb Prosthetic Outcome Measures (UPLOM): a working group and their findings. J Prosthet Orthot 2009;21(4S):69–P82.
16. Lindner HY, Nätterlund BS, Hermansson LM. Upper limb prosthetic outcome measures: review and content comparison based on International Classification of Functioning, Disability and Health. Prosthet Orthot Int 2010;34(2):109–128.
17. Wright V. Prosthetic outcome measures for use with upper limb amputees: a systematic review of the peer-reviewed literature, 1970 to 2009. J Prosthet Orthot 2009;21(4S):3–63.
18. Heinemann AW, Connelly L, Ehrlich-Jones L, Fatone S. Outcome instruments for prosthetics: clinical applications. Phys Med Rehabil Clin N Am 2014;25(1):179–198.
19. Resnik L, Borgia M, Silver B, Cancio J. Systematic review of measures of impairment and activity limitation for persons with upper limb trauma and amputation. Arch Phys Med Rehabil 2017.
20. Resnik L, Borgia M, Silver B. Measuring community integration in persons with limb trauma and amputation: a systematic review. Arch Phys Med Rehabil 2017;98(3):561–580. e568.
21. World Health Organization. International Classification of Functioning, Disability, and Health. Geneva: World Health Organization; 2001.
22. Broderick JE, DeWitt EM, Rothrock N, et al. Advances in patient-reported outcomes: The NIH PROMIS(®) Measures. EGEMS (Wash DC) 2013;1(1):1015.
23. Resnik L, Adams L, Borgia M, et al. Development and evaluation of the activities measure for upper limb amputees. Arch Phys Med Rehabil 2013;94:488–494.
24. Resnik L, Borgia M, Acluche F. Brief activity performance measure for upper limb amputees: BAM-ULA. Prosthet Orthot Int 2017:309364616684196.
25. Resnik L, Borgia M, Acluche F. Timed activity performance in persons with upper limb amputation: a preliminary study. J Hand Ther 2017.
26. Hebert JS, Olson JL, Morhart MJ, et al. Novel targeted sensory reinnervation technique to restore functional hand sensation after transhumeral amputation. IEEE Trans Neural Syst Rehabil Eng 2014;22(4):765–773.
27. Tyler DJ. Neural interfaces for somatosensory feedback: bringing life to a prosthesis. Curr Opin Neurol 2015;28(6):574–581.
28. Jönsson S, Caine-Winterberger K, Brånemark R. Osseointegration amputation prostheses on the upper limbs: methods, prosthetics and rehabilitation. Prosthet Orthot Int 2011;35(2):190–200.
Keywords:

disability evaluation; upper limb; prostheses; outcome

© 2017 by the American Academy of Orthotists and Prosthetists.