Discussing the purpose and focus of pattern recognition training necessarily requires a brief discussion of existing myoelectric control paradigms. After introducing the current state of the art, pattern recognition will be introduced as an alternative prosthesis control method. Subject training concepts will then be proposed that stem from the development of pattern recognition algorithms. The objective of this article is to provide a conceptual understanding of pattern recognition–based prosthesis control with a focus on training techniques that may help improve a subject’s ability to use pattern recognition successfully. When pattern recognition–based prostheses become commercially available, an amputee and clinician can use the proposed training paradigm to better prepare the amputee for a pattern recognition–based prosthesis.
Myoelectric prostheses are controlled through the acquisition and processing of the electrical signal that generates muscle contractions. This signal is known as the myoelectric or electromyographic (EMG) signal and may be recorded intramuscularly or from the skin surface. Prosthesis control using the EMG signal is typically accomplished through surface recording from electrodes placed on the skin covering target muscles or muscle groups. Common electrode sites for upper limb amputees are the flexor and extensor muscles of the forearm, the biceps and triceps of the upper arm, and muscles of the chest and back depending on the level of amputation.1 Electromyographic signals are recorded from one or two electrode sites on the residual limb to control a single degree of freedom (DOF) in the prosthesis. The detected EMG signals from these sites are typically compared against preset thresholds, and a prosthesis action is performed as determined by the current prosthesis mode. This control scheme is typically referred to as direct control and requires direct placement of electrodes on the target muscles.2,3 The prosthesis may have several modes that give the user the ability to open or close the whole hand, extend or retract the index finger, or rotate the wrist among other possible movements, but only one such movement type can be achieved at a time based on the current mode of the prosthesis.1
A more complex control scheme known as proportional control operates in a similar fashion but adds the ability of the user to control the speed of the prosthesis movement as a function of EMG signal amplitude or muscle contraction strength.1–3 Proportional control considers the difference between the myoelectric signal amplitude and a preset threshold to both initiate prosthesis movement in the desired direction and to control the speed of the prosthesis movement. A shortcoming of both direct and proportional control is the requirement to switch prosthesis modes whenever a new type of movement is desired.1,3 Various mode switching schemes exist, but the typical requirement is to perform some combination of co-contractions, extended contractions, or repeated pulse contractions that will register with the prosthesis controller as a mode switching command.3 This requirement is often difficult, unintuitive, and unreliable for many prosthesis users.4
As available myoelectric prostheses provide increasingly more useful and complex functionality, a natural and intuitive pattern recognition–based method of prosthesis control will likely become even more desirable. Using a pattern recognition control scheme will present different challenges, but it does provide two distinct advantages over the direct and proportional control schemes. It effectively eliminates the requirement for the user to make an unrelated movement sequence to access different prosthesis modes, and it allows the user to make natural, intuitive movements to control corresponding prosthesis movements.1,2 A transradial amputee would no longer use the flexor and extensor muscles in the same manner for multiple movements as may be required in conventional methods. With pattern recognition, the amputee controls various DOFs with the appropriate musculature. The unique EMG patterns associated with various muscle movements allow for a larger of number of DOFs to remain active. Later discussion will clarify how a multiple-DOF prosthesis will still be controlled sequentially; simultaneous control of wrist and hand is not currently possible in a broad sense for surface EMG pattern recognition.
Pattern recognition–based control functions in a fundamentally different way from direct or proportional control. Instead of relying on two optimal electrode sites to control a single DOF, pattern recognition uses many electrodes to control several DOFs and provide better resolution of more nuanced or complex movements. In pattern recognition research, it is common to use between 4 and 12 electrodes uniformly spaced circumferentially around a transradial amputee’s forearm.5–7 Different numbers of electrodes and different electrode configurations may be required based on a higher level of amputation and whether the subject has undergone targeted muscle reinnervation surgery and requires more precise electrode placement.8 The significant increase in electrodes over conventional setups produces a more complete picture of overall muscle activity. There are many pattern recognition techniques used successfully in this field, but at the core, they all seek to observe distinguishable characteristics of each movement trained during a previously recorded training session.9–11 Pattern recognition techniques use these training data to provide a baseline so that future muscle activity can be classified in real time as one of the trained movements most likely to be the user’s intention.
Pattern recognition control has a lofty goal: enable the amputee to perform a large number of prosthesis movements simply by attempting to reproduce or mimic intuitive movements using residual muscle signals on the amputated limb. There are numerous potential reasons why this may prove difficult for each individual amputee. Each amputee will have unique circumstances surrounding the limb loss and potentially some associated comorbidities related to nervous or vascular damage present in the residual limb. This uniqueness among amputees results in high variance among a group of amputees when looking at residual muscle strength and coordination based on the residual muscle mass and length of the residual limb. There may also be differences between amputees based on how much time has elapsed since the amputation and which types of prostheses have been used in the meantime. Improved muscle control and phantom limb awareness are both linked to myoelectric prosthesis use, which will likely result in an initial advantage for a previous myoelectric user.12 An individual with a more recent amputation may have the best chance of feeling or remembering certain movements. Other amputees may have difficulty visualizing how to attempt a requested muscle movement that has not been performed or even considered in many years or even decades. Because of many such differences, it is fair to say that each individual will approach pattern recognition with varying degrees of initial success. Furthermore, it is expected that the number of movements that can be controlled and the quality of control attained will vary from individual to individual. Initial success may be an indicator of long-term potential, but only after proper training can informed expectations about a long-term outcome be made.
Pattern recognition training will be presented in three phases, with in-depth discussion of their respective purposes and goals:
- 1) Initial demonstration and conceptual instruction;
- 2) In-clinic testing and initial training; and
- 3) At-home training.
INITIAL DEMONSTRATION AND CONCEPTUAL INSTRUCTION
Knowing that people learn in many different ways, it is beneficial if the instructor teaching these concepts (prosthetist, occupational therapist, or other trained staff) has already learned the skill of pattern recognition control. The instructor can then relate his/her personal experience learning this task while demonstrating the desired outcome with a virtual prosthesis on a computer screen during a subject’s first appointment. The attractiveness of the end result will likely gain immediate interest and create some initial motivation.
This initial demonstration exposes the amputee to a computer-based training system with which EMG signals are recorded, amplified, and processed by a real-time signal processing and pattern recognition algorithm that animates a virtual prosthesis on a computer screen (Figure 1). In our pattern recognition training system, the user follows visual cues presented on a computer screen and executes the same movement with his/her phantom limb. Subjects who have no perception of a phantom limb try to mimic the muscle activity of their intact limb. Some subjects may be under the impression that they must simply think about making the movement, but it is important to explain that only by attempting to make the movement with their phantom limbs do they actually generate the necessary muscle activity in the residual limb. The recorded data are used to train a movement classification algorithm. After the training phase is complete, the user now controls a virtual prosthesis visible on a computer screen (Figure 1). This virtual prosthesis can be programmed to reflect any natural configuration of the hand and wrist. The virtual prosthesis is animated to approximate the rate of movement observed in commercially available prostheses.
The term training is used frequently to describe two separate tasks discussed in this article. The computer-based training system described here provides a platform for the amputee to learn pattern recognition skills while exploring different prosthesis options or while awaiting a final prosthesis fitting. This form of skill training is synonymous with “learning” and is the focus of this discussion. A second type of training is algorithm training, by which a movement classifier is taught what to look for in future EMG data. This training is equivalent to carrying out mathematical analysis of EMG patterns and programming or calibrating a myoelectric prosthesis. Efficient, user-friendly prosthesis calibration is a separate research focus and is not the purpose of the pattern recognition training system described in this article. Programming a prosthesis controller has been achieved without a computer in a process called prosthesis-guided training.13 It is possible to record EMG signals and train the pattern recognition algorithm with components embedded entirely within the prosthesis, thus eliminating an external connection to a computer. Although this approach provides a substantial benefit for on-the-go prosthesis calibration, it will be most effective when the user has sufficient initial training in basic pattern recognition control.
Intuitive understanding of the pattern recognition approach cannot easily be drawn from seeing the EMG recordings, so providing a conceptual understanding of pattern recognition will help give the subject a better grasp of the learning task and explain why certain techniques or approaches might improve performance. The literature on EMG pattern recognition is full of equation-rich models and methods, but it has been our experience that only occasionally will a subject express interest in a more mathematical description of pattern recognition.7–12 A high level of mathematical understanding is not required for the amputee, prosthetist, or occupational therapist to effectively train basic pattern recognition control. To briefly summarize, the first step is to collect the myoelectric signals from many electrode sites and then calculate various measures or features of each channel in a process called feature extraction. Commonly used features include the mean absolute value of a signal as well as the signal’s waveform length over a short time window.5,9 Next, we use the extracted feature data to either create a new classifier or evaluate new data using an existing classifier. The final step varies greatly among different pattern recognition methods. Pattern recognition algorithms are designed to solve the mathematical problem of classification. In classification problems, we use example data from various categories to classify new data as a member of one of the known categories. It can be difficult to explain or sketch an example of how pattern recognition classifies a large number of unique muscle patterns (Figure 2). Many subjects are not interested in the mathematical basis of pattern recognition, so it helps to use a simple analogy to explain the pattern recognition concept and the focus of pattern recognition training.
To provide an example explanation of how pattern recognition works, let us consider how we identify a friend in a crowd. We look for a person of the right height, weight, hairstyle, facial features, clothing, and so on. All of these contribute to the pattern we envision that best describes our friend. Because we know what to look for, a drastic hairstyle or hair color change might evoke a response such as “I almost didn’t recognize you.” We can overcome slight unexpected variations, but if the changes we observe are too severe, the person may no longer fit the expected pattern and might go unnoticed. We do the same with voice recognition on the telephone and song recognition on the radio. Now, let us extend this idea to the problem of recognizing patterns in muscle activity.
Hand and wrist motions are generated by the combined inputs of more than 20 muscles in the forearm. Muscles can be described as anterior, posterior, deep, or superficial, and they can be functionally grouped into a flexor/pronator group and an extensor/supinator group.14 When we place an array of electrodes around the arm capable of recording the EMG activity of different muscle groups and subgroups during specific movements, we attempt to construct a comprehensive picture of what the muscle movement looks like. The large number of electrodes provides the type of valuable information that we need to pick the right muscle movement out of a large group of candidate movements. In conventional two-site control, we get two pieces of information to decide which movement to choose. This forces us to keep the number of options small, perhaps limiting control of the prosthesis to a single DOF, that is, hand open and hand close. In contrast, the extra information obtained by additional electrodes allows a prosthesis to have more active DOFs without sacrificing the ability to reliably control each one.
MUSCLE PATTERN CONSISTENCY AND DISTINGUISHABILITY
After explaining the concept of pattern recognition to a new subject, we introduce the dual purpose of pattern recognition training: consistent and distinguishable muscle patterns. Consistency and distinguishability are the two key words to repeat throughout training because these are the underpinnings of successful pattern recognition. Practice and repetition are necessary to build consistency in any motor movement. Consider playing the piano at a very high level. A concert pianist performs with great consistency that can come only from much practice. Muscles are trained to repeat the same motions again and again. Just like quality piano playing, pattern recognition relies on each muscle to do its part as practiced during training to yield a consistent result. This encompasses the first goal of pattern recognition training: developing consistent muscle patterns that an algorithm has seen before. Consistency alone can produce significant improvement in virtual prosthesis control (Figure 3).
The second part of pattern recognition training is conducted as needed and seeks to make each movement more distinguishable. To continue the piano analogy, a well-trained pianist can make slight modifications to change the rhythm of a song. Adding syncopation or other rhythmic differences may allow the listener to classify the same song as a completely different style. Just like a pianist who develops unique muscle pattern variations to produce different styles, it is necessary to ensure that each trained movement is unique as viewed by the classification algorithm. Because it is unrealistic to think that a subject can make each movement exactly the same way each time, we need to incorporate enough uniqueness into our trained movements that some trial-to-trial variability is still tolerated. For some subjects, this becomes the most important step in gaining reliable control of a pattern recognition–based device (Figure 4).
MAKING MORE DISTINGUISHABLE MOVEMENTS
A prosthesis movement in the pattern recognition setting is a more difficult task than a normal hand movement. Consider the many variables associated with the opening of a hand from a closed-fist position. As the hand fully opens, the fingers may be relaxed, strained, grouped, or spread apart. The hand can be opened quickly or slowly, as a whole unit or one digit at a time. The end result can always earn the same label of “open hand,” but it was achieved in many different ways. Pattern recognition as approached in this research is focused on end-state recognition. It is likely that the transition period from one movement to another can be confused more easily than the end state of the movement. Pattern recognition cannot accurately decipher the many variations or transition states we are capable of introducing into a single movement. Only the perceived end state drives the pattern recognition algorithm, and it will be a combination of the prosthesis’ mechanical properties and embedded programming that determine the manner in which the amputee actually completes the prosthesis movement. Many prosthesis users may find that even their best efforts to make consistent, repetitive movements cannot completely distinguish some movements from others. In the virtual prosthesis environment pictured in this article (Figure 1), a confusing muscle pattern input results in a fidgety hand that seems to twitch or oscillate between movements despite the subject’s effort to maintain a constant hand or wrist position. This behavior should be expected when a subject first learns pattern recognition control, but if it persists and is associated with only a few desired movements, this is an indication that one or more movements should be modified to make it more distinguishable (Figure 5). Movement confusion can also result from the manner in which the training data were obtained, specifically with regard to the relative significance of movement onset and movement offset periods in the complete movement cue period.
Before considering how we might attempt to make a particular movement more distinguishable, it is important to consider how a complex hand and wrist movement will be completed with pattern recognition control. In normal activities of daily living, we routinely couple hand movements with wrist movements. When a person rotates the wrist to have the palm facing down, a naturally coupled movement is to open the hand at the same time. We are typically reaching for something to pick up or perhaps setting our hands on a keyboard. When we rotate our palms up, we are likely to open our hands to place something in them. These types of complex movements present a challenge for pattern recognition control as current methods require that coordinated movements be conducted sequentially. Hand actions must be interpreted independently from wrist actions. A coupled hand and wrist movement will likely confuse the pattern recognition controller regarding our true intentions (Figure 6). The user likely trains hand opening and wrist rotations as separate movements, so to accomplish a more complex action such as reaching for an object, the prosthesis user must first open the hand and then rotate the wrist, or vice versa.
The goal here is to train and perform hand and wrist movements using only the required muscles. Using an able limb to mirror the phantom hand position will help initially for two reasons. First, it provides the trainer a visual representation of what the unilateral amputee is attempting. Second, it forces the trainee to consider the entire phantom hand position, which increases the amputee’s awareness of phantom hand and wrist position. This practice may expose possible limitations in phantom hand range of motion and feeling, and this knowledge is essential when discussing potential ways to modify movements for greater distinguishability.
After attempting to isolate the desired movement from any extraneous movements, the next key task is to figure out what to do with the nonessential hand and wrist components. Some individuals may find that their greatest performance improvements come from determining some optimal action to perform with these nonessential components.
The ease with which a particular grasp like the fine pinch can be achieved in the virtual environment varies from person to person. When a person does have difficulty producing and maintaining a particular grasp, we try slight variations of the original movement to find a more distinguishable muscle pattern. For one subject, extending and grouping the three remaining fingers changed the pinch movement from the least accurate to one of the most accurate movements that the subject could perform without affecting the accuracy of other movements (Figure 7). It was actually the position of the three remaining fingers that offered such a unique muscle pattern that it eliminated previous confusion.
INTACT LIMB TRAINING
Up to this point, the discussion and associated pictures have reflected the lessons learned by predominantly able-bodied subjects. These are experiences and realizations that prosthetists or occupational therapists will learn best through their own personal experience. Much of this knowledge is directly applicable to amputee training. The subtle differences that make muscle patterns reliable or unreliable are most easily observed when an individual can visually compare an intact limb to the decoded movements depicted by the virtual prosthesis. A unilateral amputee should initially spend some time learning these concepts with his/her intact limb. A bilateral upper limb amputee can gain this conceptual understanding through observation of another person’s initial training.
For able-bodied subjects, we can presume some element of mirrored symmetry between muscles on both arms. This allows the learning accomplished in training with one arm to be transferred to future attempts with the other arm. Although some practice may still be required to match the final skill level achieved with the first limb, a subject can use all of the conceptual learning and movement refining completed up to that point to begin at a much higher level than when the subject was completely naive. Despite this benefit, amputee training with the intact limb should be used only as a small component of the conceptual learning phase for amputees. Excessive training with the intact limb could potentially have negative effects because the symmetry that likely exists in able-bodied subjects may not be extended to all amputees, and the information gained with the intact limb may be misleading or impossible to use with the residual limb. The demonstration and conceptual instruction are necessarily accomplished first and may likely consume a considerable amount of time, but this phase is crucial to making the subsequent training productive.
IN-CLINIC TESTING AND INITIAL TRAINING
After completing a demonstration and some conceptual instruction, an initial testing session should occur in the clinic environment. Such testing should be carried out by a trained staff member who can ensure proper placement of electrodes and efficient interaction with the training program software. The goal of this initial testing is to establish a starting point for the at-home training. It is important to find a reasonable set of movements that can be achieved with modest accuracy. In our experience, trying too many movements in the beginning yields very poor results and has a negative effect on a subject’s motivation. The benefits of keeping the initial movement set small are 2-fold: early success will maintain the motivation required to train, and good performance for some movements will not be eclipsed by struggles with other movements.
We typically start with the simplest useful movement set: hand at rest, an open hand with fingers extended, and a hand closed in a fist. If the person has reasonable success, we add forearm pronation and supination. At this point, the movement set controlled by pattern recognition already includes all the movements available to many prosthesis users, perhaps even more if the user does not have a wrist unit as part of his/her prosthesis. More importantly, the user is controlling them in a simpler and more efficient way. If the subject continues to be successful with larger movement sets, we continue adding desirable movements until there is some difficulty achieving all the desired movements. Once this initial threshold is determined, a realistic plan can be developed to reinforce existing movement control while progressively adding more movements. The threshold for acceptable control will be unique to each individual, so the subject and clinician should agree upon the desirable set of movements to be used initially for functional training after a prosthesis is fitted.
During the initial testing with the amputee, it is important for the clinic staff member to watch the amputee complete the movements with the intact limb. It is critical to observe how the individual is completing the movements as assessed by response time to the movement cue, transition speed to a new movement, and apparent consistency of hand and wrist positions. Although these metrics are more easily observed and any errors are corrected with the intact limb during initial testing, they can sometimes be observed with the residual limb during later training and testing. Mirrored movements can be made later to show what the amputee is attempting with his/her phantom limb. When the user has visual feedback from a real-time decoding virtual prosthesis, there is one common error many users make in response to a misclassification. As also observed by Simon et al.,13 many subjects will initially try to repeat the same movement as originally performed but with more force. This is the equivalent of raising one’s voice to be better understood, but unfortunately, pattern recognition does not work that way. When the stronger contraction fails to produce the desired result, the user may try something completely different from what was performed during training. These natural responses will likely be observed and corrected when the subject is conducting training with the intact limb, but if not, these are still important points to address. Increased force or making a drastic modification will likely generate EMG signals that resemble a different movement rather than clarify the desired movement.
Once the subject has completed an initial supervised testing session, a trained staff member can oversee the first pattern recognition training with the subject. The staff member ensures that the key principles of consistency and distinguishability are present in each desired movement and familiarizes the subject with the various components of the training system. Before an individual is ready for at-home training, he/she should be comfortable running the software program as well as placing electrodes or donning any equipment required as part of the training system. Detailed written instructions will facilitate transferring this knowledge to the person’s home, but supervised practice setting up and running the training system will help mitigate any lack of computer skills on the user’s part. The initial testing and training phase concludes when the clinician has found a starting point for at-home training and the amputee is comfortable working with the training system software and equipment.
Before a training methodology to be used at home is proposed, it is important to first discuss why this phase is so important to overall pattern recognition control success. Typical occupational therapy techniques may be much less effective for pattern recognition users because they do not provide detailed feedback to the individual. Only with the visual biofeedback coming from a virtual prosthesis can the user see that the practiced movements are actually eliciting the desired response from the virtual prosthesis. Simply practicing movements without feedback may improve the strength and endurance of an individual’s muscles, but it will not necessarily reinforce the specificity and muscle memory we seek to achieve for pattern recognition control. In addition, conventional therapy practices without feedback allow the possibility that the user will repeatedly practice movements that will need to eventually be modified for greater distinguishability, making that transition more difficult than if the movements were modified before extensive training.
At-home training can be initiated during the prosthesis fitting process while the user waits for a final prosthesis. Skills can be learned, sharpened, and maintained while waiting for a device. This is probably best accomplished by maintaining portable training systems in the occupational therapy clinic to be loaned to amputees as needed for at-home training. A sound business model for the procurement and ownership of portable training systems by the clinic or the individual amputee will be required for the successful implementation of pattern recognition–based prostheses. If conducted in a clinic, the desired training paradigm could require daily clinic visits for 2 weeks or more to accomplish training goals. Movement repetition alone can yield the positive effects of movement consistency over the course of the first two or three daily training sessions. Movement distinguishability efforts may require many additional sessions for some subjects to fully grasp the pattern recognition concept of distinguishability. At that point, subjects may begin suggesting logical adjustments to the way they train and perform various movements. Only the amputee can ensure that these adjustments remain intuitive and repeatable, both of which are critical to effective pattern recognition control.
We continue the discussion of the importance of at-home training with a variety of physiological and motor learning implications. The amputee who is taking time off from work or driving from out of town to conduct training at a clinic is likely motivated to maximize the time spent doing the training and reduce the overall number of trips required. Unfortunately for this person, motor learning is best accomplished in smaller sessions spread over a few days.15 In fact, the most effective motor learning is conducted with consideration given to sleep cycles.16 Training sessions should be separated by a night’s sleep because it is believed that the brain consolidates the new skills learned each day while sleeping at night. Some motor task studies report improved performance when subjects are tested the day after initial training as compared with testing later on the same day.17 At-home training facilitates this optimal form of motor learning whereby an amputee can conduct small, manageable amounts of training on a daily basis.
Another well-investigated learning concept is the role of the trainee in deciding the goals and format of the training program.18 The trainee may likely have good ideas for movement modifications or things to focus on during current and future sessions. The amputee is the only person involved who can describe which muscle movements or phantom limb movements can be visualized or controlled. A motivated trainee will determine the movements he/she really wants to be able to make and should not be forced to adhere to an arbitrarily decided movement set defined by a clinician. When the trainee is responsible for the pace of the training program, the immediately successful trainee can accelerate the program to avoid boredom, and the struggling trainee can slow it down to lessen the frustration. There are numerous ways the clinician can stay involved in the process: help troubleshoot computer or hardware issues, brainstorm new ideas for movement modifications or movement sets to try, or just offer encouragement. This interaction can be accomplished through webcam discussions and remote desktop viewing software to work with remote clients or research participants.
A final physiological benefit of at-home training compared with clinic visits pertains to the variable nature of phantom limb pain and the effects of any regularly used medication. One of our research participants initially described his phantom hand as being in a cramped fist position that varies from day to day. Furthermore, his ability to move the hand out of this position changes daily and sometimes throughout the day. There are certain times of day when an amputee may feel the most control of his/her phantom limb and other times when he/she feels no control over it. In these cases, it is highly likely that such an amputee might be prescribed a combination of pain management and muscle relaxing medication. Phantom limb pain and the independent and combined effects of various medications may result in a highly unpredictable training schedule that is better accomplished at home when possible instead of during clinic visits scheduled in advance.
From a purely financial perspective, the time and costs associated with daily clinic visits would likely eliminate a daily training regimen as a viable option for many people. Conducting an identical pattern recognition training program at home allows amputees to follow an aggressive training schedule without commuting to a clinic, taking time off from work, or worrying about insurance coverage for repeated clinic visits. Amputees can instead accomplish this training in their free time in the comfort of their homes, thus reducing the impact on the clinic staff and the overall cost of the training.
A TRAINING MODEL AND MANAGING EXPECTATIONS
Daily exposure to virtual prosthesis control is the key to pattern recognition training as a motor learning task. This training should be incremental or progressive in nature. Such a training protocol might start with basic pattern generation for two or three unique and useful movements such as a completely relaxed hand at rest, an open hand with fingers extended, and a hand closed in a fist. Then progressively add more movements until a subject can reliably control all of the desired movements. With reliable movement control, the subject can experiment with control schemes that operate multiple DOFs. There are many ways to design a multiple-DOF prosthesis controller, and a user may have a particular preference after testing various options. This task is secondary to establishing reliable control of individual movements. Finally, a comprehensive training plan would conclude with virtual task completion that requires frequently performed sequences of movements that will assist the prosthesis user in performing activities of daily living.
It is desirable to start this training sequence as early as possible after amputation to prevent loss of muscle coordination and unfavorable cortical reorganization. Combating muscle atrophy is an important task in this phase, and pattern recognition training can be used to preserve both muscle strength and functional muscle coordination. The “Golden Window” for initial prosthesis fitting is said to be 30 days to minimize the amputee’s opportunity to become unilaterally dominant.19 It should be expected that training before prosthesis fitting will be even more critical for pattern recognition–based prosthesis acceptance than for a body-powered or traditional myoelectric device. Using a training system to prepare a recovering amputee for an advanced prosthetic device is possible while still waiting for the residual limb to fully recover from surgery.
Daily use of a pattern recognition–based prosthesis will require the user to train a movement classifier, explore its effectiveness, and then either retrain the device or accept the results. These steps appear in the 1-hr training session format we use with study participants (Figure 8). A subject repeatedly provides training data to the movement classification algorithm and over time learns how best to train the system. We save and track the results of each evaluation to quantify any improvement in performance over multiple sessions. Because desired movements may change from person to person, the user alone decides how long to continue training to add more movements or obtain better overall control.
MEASURING IMPROVEMENT IN VIRTUAL PROSTHESIS CONTROL
Success can be measured in many ways, but in recent years, the Motion Test and Target Achievement Control Test have shifted the focus of pattern recognition work toward more functional metrics of prosthesis control.8,20 We adopt three variations of these published metrics that we feel best assess a person’s ability to control a prosthesis: 1) movement completion percentage tracks how frequently the subject can manipulate the virtual prosthesis to achieve the position presented by the movement cue; 2) movement completion time describes the speed and efficiency with which the user completes the movement and is negatively affected when the user’s inputs are temporarily confused with other movements during the movement transition stage; 3) finally, postselection classification accuracy measures the ability of the user to maintain a consistent muscle pattern. This metric considers the portion of each movement cue after both the subject’s reaction time and the initial transition time into the movement. It disregards all movement classification decisions preceding the first instance of the correct movement being detected.
For a basic movement like a fine pinch, these metrics describe whether the fine pinch is ultimately achieved by the user, how long it took to achieve the fine pinch, and how stably the user can maintain the fine pinch by continuing to provide a fine pinch muscle pattern. For more complicated real or virtual tasks, simply recording the success or failure and time required can suffice. With these easily quantifiable metrics describing a subject’s performance, both the subject and the clinician can track progress and make informed decisions about modifying the current training goals.
Defining realistic expectations that can be broadly applied to the amputee population is difficult. Proper pattern recognition training will likely improve an individual’s performance on virtual prosthesis control tasks, but the rate and magnitude of improvement are difficult to predict for several reasons. In the small amputee population that has interacted with our pattern recognition training system, we find that regular myoelectric prosthesis users who have been forced to maintain some degree of strength and muscle tone in their residual limbs have described better control and awareness of their phantom limbs. These individuals also demonstrated better initial control of the virtual prosthesis. This strength, coordination, and phantom hand position awareness can be reacquired for a predominantly body-powered prosthesis user, but the process may take considerable time. The cortical reorganization that gradually reduced or eliminated neural representations of portions of the amputated limb may take lengthy training to reverse. It is impractical to discuss the fine differences in thumb position for various grasps if the amputee cannot sense the position of his/her phantom thumb. Some of our research participants reported improved awareness and control of their phantom limb position over the course of training. One wrist disarticulation amputee, who exclusively used a body-powered prosthesis or no prosthesis, began the study with a phantom hand that had been stuck in a cramped fist position for 10 years. Over the course of 2 weeks, the phantom hand gradually relaxed and began to open. Both phantom limb pain reduction and favorable cortical reorganization are linked to myoelectric prosthesis use and virtual or augmented reality training.21,22 Our own experience with subjects interacting with this pattern recognition training system supports this notion. Perhaps most important are the observed improvements of each amputee who used a pattern recognition training system with respect to our three measures of virtual prosthesis control (Figure 9).
Although there remain many problems to solve in the clinical implementation of pattern recognition prostheses, we find that the problem of learning pattern recognition control is quite tractable. When an amputee can see and understand the significance of creating consistent and distinguishable movements through interaction with a visual biofeedback training system, several barriers to effective pattern recognition control are removed. The amputee learns the key principles of pattern recognition control during demonstration and conceptual instruction from a trained clinician. The amputee then conducts initial supervised training with a trained clinician to personally explore the subtleties of pattern recognition control. This training directly translates conceptual understanding into something the amputee can feel. This combination of understanding and feeling makes it possible for the amputee to practice making consistent, distinguishable muscle patterns while training at home. Individuals who are motivated to practice and learn this skill can improve their ability to intuitively control the increasing functionality available in the newest prostheses. The methods and results discussed in this article suggest that pattern recognition–based prosthesis control is a skill many amputees can learn, and for amputees to successfully adopt a pattern recognition–based prosthesis, focused pattern recognition training will be a vital part of the transition. The development of effective pattern recognition training methods will be a source of active research for many years to come.
1. Scheme E, Englehart K. Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. J Rehabil Res Dev
2011; 48: 643–660.
2. Parker P, Englehart K, Hudgins B. Myoelectric signal processing for control of powered limb prostheses. J Electromyogr Kines
2006; 16: 541–548.
3. Williams TW. Control of powered upper extremity prostheses. In: Atkins DJ, Meier RH, eds. Functional Restoration of Adults and Children With Upper Extremity Amputation
. New York, NY: Demos Medical; 2004: 207–224.
4. Atkins DJ, Heard DCY, Donovan WH. Epidemiologic overview of individuals with upper-limb loss and their reported research priorities. J Prosthet Orthot
1996; 8: 2–11.
5. Scheme EJ, Englehart KB, Hudgins BS. Selective classification for improved robustness of myoelectric control under nonideal conditions. IEEE Trans Biomed Eng
2011; 58: 1698–1705.
6. Sensinger JW, Lock BA, Kuiken TA. Adaptive pattern recognition of myoelectric signals: exploration of conceptual framework and practical algorithms. IEEE Trans Neural Syst Rehabil Eng
2009; 17: 270–278.
7. Hargrove LJ, Li G, Englehart KB, Hudgins BS. Principal components analysis preprocessing for improved classification accuracies in pattern-recognition-based myoelectric control. IEEE Trans Biomed Eng
2009; 56: 1407–1414.
8. Kuiken TA, Li G, Lock BA, et al.. Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms. JAMA
2009; 301: 619–628.
9. Englehart K, Hudgins B, Chan ADC. Continuous multifunction myoelectric control using pattern recognition. Technol Disabil
. 2003; 15: 95–103.
10. Chan ADC, Englehart KB. Continuous myoelectric control for powered prostheses using hidden Markov models. IEEE Trans Biomed Eng
2005; 52: 121–124.
11. Chu J, Moon I, Mun M. A real-time EMG pattern recognition system based on linear-nonlinear feature projection for a multifunction myoelectric hand. IEEE Trans Biomed Eng
2006; 53: 2232–2239.
12. Flor H, Birbaumer N. Phantom limb pain: cortical plasticity and novel therapeutic approaches. Curr Opin Anaesthesiol
2000; 13: 561–564.
13. Simon AM, Lock BA, Stubblefield KA. Patient training for functional use of pattern recognition-controlled prostheses. J Prosthet Orthot
2012; 24: 56–64.
14. Boles CA, Kannam S, Cardwell AB. The forearm: anatomy of muscle compartments and nerves. Am J Roentgenol
2000; 174: 151–159.
15. Walker MP, Brakefield T, Seidman J, et al.. Sleep and the time course of motor skill learning. Learn Memory
2003; 10: 275–284.
16. Walker MP, Brakefield T, Morgan A, et al.. Practice with sleep makes perfect: sleep-dependent motor skill learning. Neuron
2002; 35 (1): 205–211.
17. Fischer S, Hallschmid M, Elsner AL, Born J. Sleep forms memory for finger skills. Proc Natl Acad Sci U S A
. 2002; 99: 11987–11991.
18. McNevin NH, Wulf G, Carlson C. Effects of attentional focus, self-control, and dyad training on motor learning
: Implications for physical rehabilitation. J Phys Ther
2000; 80: 373–385.
19. Malone JM, Fleming LL, Roberson J. Immediate, early, and late postsurgical management of upper-limb amputation. J Rehabil Res Dev
1984; 21: 33–41.
20. Simon AM, Hargrove LJ, Lock BA, Kuiken TA. Target achievement control test: evaluating real-time myoelectric pattern-recognition control of multifunctional upper-limb prostheses. J Rehabil Res Dev
2011; 48: 619–628.
21. Flor H, Nikolajsen L, Jensen TS. Phantom limb pain: a case of maladaptive CNS plasticity? Nat Rev Neurosci
2006; 7: 873–881.
22. Flor H. Cortical reorganisation and chronic pain: Implications for rehabilitation. J Rehabil Med
2003; 35: 66–72.