The field of upper limb prosthetics has seen increased activity in recent years toward the collective goal of improved provision of artificial limbs for the user population. For example, there has been increased effort toward technological advances in prosthetic components,1–4 advances in prosthetic control strategies,5 and new surgical techniques for treating high-level limb deficiencies.6–8 A continued pressure on resources has also led to a greater need to justify the cost of providing these improved devices. As a result, there is a renewed interest in outcome measures to objectively measure performance and change in this population. Clinicians and researchers are interested in measuring progress as the user learns to use a prosthesis, also to gauge the effectiveness of a prosthesis or prosthetic component and to measure the impact of surgical interventions on the usage of prosthetic devices. Clinicians are also being asked to justify the costs of prescribing expensive advanced technology and to quantify the functional benefits to third party payers of prostheses. Therefore, it is important to be able to objectively measure differences in performance and the importance of particular features of a prosthesis to a prosthesis user.
It has become apparent that the different contributors to this process straddle numerous disciplines. Thus, discussions in multidisciplinary settings can result in confused interpretation. Although some terms are used in common, they are often used with very different implied meanings. For example, occupational therapists use terms like “function” and “activity” with very different intentions from what an engineer would infer from the same words. An occupational therapist sees function as the ability for a person to accomplish something in their daily lives. To prosthetists and engineers, function may refer to the technical performance of a device, (such as grip speed or force), whereas others may see function as the hand's ability to perform when being employed to undertake a particular task. Hence, no common terminology exists. When such terms are used in relation to outcomes, there is ample opportunity for misinterpretation of the meaning if the terminology is not clarified.
Literature reviews indicate that there are many assessment tools and measures that are intended to measure the performance of the upper limb.9–12 There are also tools designed to measure prosthetic control and prosthetic use. Some tools have been used for many years but may have uncertain psychometric properties. Other standardized measures are being used in parts or are modified to fit the needs of a particular center or culture, invalidating their design.9 Other tests were validated for use with a particular non–limb-deficient population and have been used untested in prosthetics. This implies that results from these tests cannot be compared and may be incomplete or invalid.
EVOLUTION OF AN APPROACH
It was this state of affairs that has led to a greater interest in the field for a means to address these problems. This article describes a process that aims to move toward a consensus to use outcome measures in a standardized fashion within the professions. It involved professionals and users connected in the design, supply, fitting, and training of the use of prosthetic arms. Over the past few years, through a series of meetings and discussions, these groups have been moving in this direction. An early step in this direction was the Myoelectric Control Symposium (MEC) in 2005.
The MEC, held triannually at the Institute of Biomechanical Engineering, University of New Brunswick in Fredericton, Canada, traditionally provides a 2-day preconference education session for clinicians involved in upper limb prosthetics. Topics have generally focused on aspects of basic skills and knowledge. In the years leading up to MEC 2005, it became apparent that this session needed to be more about sharing information and resources than learning basic skills. So in 2005, a 2-day workshop was organized by one of the authors (WH). Approximately 75 delegates, including occupational therapists, prosthetists, physicians, engineers, and other researchers from around the world attended the workshop. The aim of the workshop was to provide an in-depth look at some of the key functional assessments being used in upper limb prosthetics at that time, followed by discussion among experienced clinicians and researchers about whether current outcome measures fully met the needs of the upper limb deficient population.
A variety of different tools were discussed, including individualized goal-setting measures, various functional assessments, both observational and self-rated as well as quality of life measures The ensuing discussions revealed a general feeling that there were many more options for assessment of the pediatric population than the few available for the adult population. An overview and summary of the workshop and conclusions can be found in Wright.13
From the discussions at MEC, it was apparent that the objective measure of prosthetic outcomes was not only of interest to clinical therapists, but also it was equally important to researchers and engineers involved in the design of new prosthetic components. It was also noted that there were potentially many cultural differences and that several of the assessments being used had a cultural bias or language barrier, particularly those that are task based, such as the UNB Test of Prosthetic Function.14 This led to further discussions among a small group of interested clinicians and researchers from MEC, which led to the establishment of a workshop held in Trondheim, Norway in March 2007. The aim of the workshop was to begin a process toward greater standardization. It was sponsored by the Research Council of Norway (grant number 174940/D15), the Norwegian University of Science and Technology (NTNU), and Foreign Affairs and International Trade, Canada, and jointly organized by NTNU and Institute of Biomechanical Engineering. The organizers were among the authors (WH, PK, and ØS).
The Trondheim workshop brought together a small group of international researchers with experience in the field of upper limb prosthetic research and fitting. It aimed to illuminate key issues from many perspectives to be addressed in order to facilitate a more consistent means of making assessment of outcomes from prosthetics research and fitting. In planning the workshop, input was requested from the subdisciplines of:
- User training and outcome assessment (Occupational Therapists)
- Biomedical Engineering/Technical Development
- Prosthesis users
By gathering information from these perspectives, it was hoped to identify the important facets of prosthetic use that should be measured. The intention was to establish some consensus among the participants on whether there were appropriate assessment tools that were being used universally in clinical settings and to establish whether these assessments were appropriate for use in research settings. Prosthetic users attended the workshop to allow the group to learn which facets of prosthetic use were important to an end user and if the profession was indeed attempting to measure them. The final participant list consisted of:
- Five occupational therapists (two from North America, two from Sweden and one from Norway)
- Two prosthetists (one from North America, one from Sweden)
- Two engineers/researchers (one from Norway, one from Canada)
- Two prosthesis users from Norway, one a congenital amputee and one traumatic (both transradial).
The initial outline for the workshop consisted of the following questions:
- What are we currently using as assessment tools?
- What are the needs for assessment from the various stakeholders (users, clinicians, technicians, industry, researchers, etc.)?
- What is hindering progress in the area of effective outcome assessment (technological limitations, organizational or commercial limitations, lack of appropriate assessment tools, etc.)?
- Can we identify specific short-term and long-term goals that will contribute to more effective and realistic outcome assessment?
LIFE CYCLE OF PROSTHESIS DEVELOPMENT
The ideas that emerged from the workshop provided impetus and a clearer direction for the next stages. This insight was that, essentially, there is a life cycle of any device as it moves from the research arena to the field, through the clinic and into the home. At each point there needs to be assessment of the device to ascertain whether it fulfills the goals set when the requirements were identified and the initial design was carried out, or when a prosthesis is used in the field that the user is trained appropriately. This information is fed back up the chain so that any deviation from the desired goals can be corrected or remedied, (Figure 1, black lines).
The process falls into four major areas:
- Research—where the ideas for new devices and techniques are considered and created.
- Development—where the ideas are matched to the demands of production and field application.
- Clinical use—the customized fitting of the device to the person and training of the device's use.
- Home—the daily use of the device by the target population.
The important insight to note from this process is that at each phase, the information needed to be fed back to the preceding phases. The information to be captured is specific to the area within which it is being tested. For example, during the research and development phases there needs to be a method of capturing the important technical information about how a component works. Thus, with the creation of a new powered hand, it is important to measure prehension speed, grip force, etc. At the same time, the design of that hand needs input from prosthetists with regards to how the hand mechanism will fit within a complete prosthesis and how it is to be controlled. Feedback is also required from therapists with regards to how it works within real-life settings and tasks. Most importantly, input is required from prosthesis users with regards to how the hand functions in their own life situations, and how appealing it is to them. Evaluation tools are needed to be able to cover each of these areas. Moreover, tools developed for one domain may not provide meaningful insight in another domain as it has no provenance there.
In 2002, the World Health Organization established the International Classification of Functioning, Disability and Health (ICF) with regards to development of healthcare products or services15 (Figure 2). The ICF model is intended to serve as a common language across health disciplines and has been recommended as an organizing framework for outcome measure selection and goal setting.
According to the definitions of the ICF domains,15 prostheses are regarded as assistive devices and, hence, belong to the “Environmental factors” domain of the ICF model. If it is interpreted this way, it limits the model significantly and ignores the experience of many prosthesis users. On the basis of our experience, for the prosthesis user the “assistive device” is an extension of the body and a replacement of “body function/body structure.” We suggest therefore that the model should be interpreted as follows:
The aim of the ICF process means that it is reasonable to suggest that the domains, defined in the ICF model, clearly relate to the provision of prosthetic devices and treatment/care of upper limb amputees, and the areas of assessment easily fit within this framework: e.g., the domain of Body Structures and Functions relates to device performance and can capture measures such as speed, grip force, and range of motion. The Activity domain refers to the carrying out of tasks, which relates to the assessment of function with a prosthesis. The Participation domain refers to involvement in real life situations, which is reflected in assessment of the impact of the prosthesis function from a user's perspective in his or her own life situations.
The domains reflect the aspects of the provision and the mode of treatment/care required. Once expressed this way, it becomes easier to see that each domain has its own set of particular requirements and so the process of assessing progress or function within each domain has to be different. It also becomes more apparent that previous attempts to measure function without such insight were less likely to prove effective, as the measurement might not be focused on the right domain, or using the right means to measure it.
There are three main types of assessment tools in use currently: timed measures, observational measures, and self-rated or proxy-rated measures. Put in this context it is more obvious that it is not likely that it would be possible to cover all the domains within the ICF model using only one assessment tool or one type of measure.
Naturally, each of these methods has their own biases and shortcomings. However, the most important conclusion to draw from this is that for a complete analysis, knowledge of each domain is necessary. One method can only serve some of the range. What is needed to unify them is not one test to cover all domains, but a single approach, with different tests to elucidate the information (Figure 1, light gray).
A second insight is that, through the development life cycle of a prosthesis, different stakeholders have different levels of importance to the process and different assessment techniques dominate/operate. One way to express the relative importance is shown in Figure 3.
Thus, after the development of a prosthesis: initially, the engineer wishes to know if a prosthetic hand design can open wide enough to admit objects or close fast enough to be practical. She or he would then use basic Functional/technical tests, such as simple measurements based on motion tracking. Once the device is more advanced, the assessment moves on to Activity-based measures Can the device pick up household objects? Can it hold on to them and move them about? This information is important to the engineer, but now the input from the clinical team becomes important as their insight into its long-term use becomes relevant. Early fittings in the clinic also need observational-based or self-rated measures of prostheses use as the clinicians need to compare the device to others, and to monitor progress in using a specific device.
Finally, the device moves on to the home/community, and the activities may revolve around tasks specific to the needs of that user. At this stage, the outcome measure may tell the clinician about the functional capabilities of the device or of the person's ability and something of the person's motivations. When the therapist wants to know more about how the user feels about their device and how it integrates into their daily lives, then the information will more likely to be obtained through a questionnaire. Hence, it can be seen that at each stage a different tool is used to obtain the information, and that each is important and provides a different insight. Some techniques overlap into different domains, but only with multiple assessments will the full picture be clear. From this, it can be seen that different existing tools cover different areas of the continuum.
An additional factor to consider when choosing a method is the design of such outcome measures. The design of a measure tends to control how well it is received by practitioners, and if it is to be used generally. To be used, the tool must therefore be user friendly, i.e., easy to administer, inexpensive and the results must be easily understandable and interpretable. Another aspect, equally important, of the test is its psychometric properties, i.e., the validity and reliability of the test. This ensures that the test is measuring what it is intended for, that the measures are repeatable, and do not depend on who conducts the test or when or where it is performed.
At the close of the Trondheim workshop, it was becoming clear that because a single standard measure was not possible, what was required instead was a common, unified approach to outcome measures. Thus, if a set of tools was developed that could cover all the domains, it was likely that one test might cover two overlapping areas (such as the development and the clinical application sphere), but clearly it was unlikely that a single tool could cover the use within the home and during the research phase.
This work was presented to a larger group of professionals, including occupational therapists, engineers, physicians, and researchers at a special interest meeting during the 12th World Congress of the International Society of Prosthetics and Orthotics in Vancouver in August 2007. The aim of the presentation was to garner interest and comments from a wider group of interested professionals. What emerged from the meeting was a clear interest in the field. Subsequently, a broader working group, representing adult, pediatric, and research and development needs, known as the Upper Limb Prosthetic Outcome Measures (ULPOM) Group, was formed to take the ideas forward. It is moderated by one of the authors (SS). Its intention is to enable a “toolkit” of validated procedures for the entire development cycle to be developed and promoted within the appropriate professions.16,17 This toolkit is not aimed at prescribing a single method or a single assessment tool for each domain, rather it is a collection of all the tests developed that can be used in upper limb prosthetics.
The initial aim of the ULPOM group process is to identify all the possible assessments currently available through the literature and then to critically assess whether they are appropriate for the ULPOM application. Thirty-five assessments covering areas of hand function, activity measures, goal-setting, quality of life, and client satisfaction have so far been identified.
Depending on the results of the analyses, the group will determine one of three different recommendations for each test:
- Accept—the test has psychometric merit and is clinically useful for upper limb prosthesis users.
- Consider—with modifications or psychometric validation testing the instrument has potential to be useful for upper limb prosthesis users.
- Reject—the test does not have psychometric merit and/or is not clinically useful for the target group.
After this analysis, it will be possible to determine whether an adequate toolbox exists, or if there are gaps within the range that need addressing within the range of tests, or in the case that the tests have been validated for other groups how much effort would be required to extend its remit to prosthetics. Finally, this information will be placed within a document, which will be distributed widely and aimed to help create a consensus.
This process has begun and initial results were reported at the recent MEC symposium in August 2008. A combination of breakout groups and plenary sessions has attempted to disseminate the ideas and encourage comments and suggestions from fellow professionals. After the publication of this article, similar comments are encouraged. An interest group has been created for discussion purposes relating to outcome measures for upper limb prosthetics. To join this interest group, please email: email@example.com.
It is possible that an approach toward the problem of a consistent means of analysis of upper limb prosthesis use and application can be developed. The ULPOM group has begun to work toward this goal through an analysis of the existing literature and through promotion of the ideas to encourage a consensus within the profession. The initial analysis work has commenced and a draft version of the recommendations is expected to be available for comment within the next 18 months.
The authors thank those who have contributed their time and insights into the process described here, especially the members of the Trondheim workshop: Tordis Magne, Kerstin Caine- Winterberger, Stewe JÖnsson, Kathy Stubblefield, Laura Miller, Sigbjørn Rønning, Stian Salomonsen and the other members of the ULPOM working group: Kristin Gulick, Virginia Wright and MacJulian Lang.
1. Newman P. Multi-articulating hands and fingers—technical strategies for improved function and myoelectric uptake, (Abstract). In: MEC '08 Measuring Success in Upper Limb Prosthetics
, August 13–15, 2008. Fredericton, New Brunswick, Canada; 2008:183.
2. Uellendahl JE, Uellendahl N. Experience fitting partial hand prosthesis using prodigits.
In: MEC '08 Measuring Success in Upper Limb Prosthetics
, August 13–15, 2008. Fredericton, New Brunswick, Canada; 2008:71–73.
3. Puchhammer G. Michelangelo 03—A versatile hand prosthesis, featuring superb controllability and sophisticated bio mimicry (Abstract). In: MEC '08 Measuring Success in Upper Limb Prosthetics
, August 13–15, 2008. Fredericton, New Brunswick, Canada; 2008:162.
4. Schulz S, Pylatiuk C, Kargov A, et al. Design and Preliminary Experience with Fludihand MK III. In: MEC '08 Measuring Success in Upper Limb Prosthetics
, August 13–15, 2008. Fredericton, New Brunswick, Canada; 2008:197–199.
5. Zhou P, Lowery MM, Englehart K, et al. Decoding a new biological neural-machine interface for superior artificial limb control. J Neurophysiol
6. Kerstin Hagberg, Eva Häggström, Stewe Jönsson, et al. Osseoperception and Osseointegrated Prosthetic Limbs Psychoprosthetics
. London: Springer; 2008:131–140.
7. Kuiken T, Miller L, Lipschulz R, et al. Targeted reinnervation for enhanced prosthetic arm function in a woman with a proximal amputation: a case study. The Lancet
8. Dietl H. Prostheses control based on TMR a case study. In: MEC '08 Measuring Success in Upper Limb Prosthetics
, August 13–15, 2008. Fredericton, New Brunswick, Canada; 2008:166–170.
9. Light CM, Chappell PH, Kyberd PJ, et al.Critical review of functionality assessment
in natural and prosthetic hands
. Br J Occu Ther
, January, 62(1), pp 7–12, 1999.
10. Bazzini G, Orlandini D, Moscato D, et al. I Test Di Funzionalità Della Mano In Amputati Di Aart Superiore Con Protesi (Translated Hand Functionality Tests of upper limb amputees who wear a prosthesis). Giornale Italiano Di Medician Del Lavoro Ed Ergonomia
11. Buffart LM, Roebroeck ME, Peschbatenburg JMFB, et al. Assessment
of arm/hand functioning in children with a congenital transverse or longitudinal reduction deficiency of the upper limb. Disabil Rehabil
12. Metcalf C, Adams J, Burridge J, et al. A review of clinical upper limb assessments within the framework of the WHO ICF. Musculoskeletal Care
13. Wright V. Measurement of functional outcomes with individuals who use upper extremity prosthetic devices: current and future directions. J Prosth Ortho
14. Sanderson ER, Scott RN. UNB Test of Prosthetic Function: A Test for Bilateral Amputees (Test Manual)
. Fredericton: University of New Brunswick; 1985.
15. World Health Organization. Towards a Common Language for Functionary, Disability and Health: ICF Beginner's Guide (WHO/EIP/GPE/CAS/01.3). Geneva: WHO; 2002.
16. Demers L, Desrosiers J, Ska B, et al. Assembling a toolkit to measure geriatric rehabilitation outcomes. Am J Phys Med Rehabil
17. Child Health Care Quality Toolbox. Fact Sheet. AHRQ Publication No. 01-0025
. Rockville, MD: Agency for Healthcare Research and Quality; 2004. Available at: http://www.ahrq.gov/news/chtoolfact.htm