Secondary Logo

Journal Logo

ORIGINAL RESEARCH ARTICLES

Mechanisms of Sporadic Control Failure Related to the Skin-Electrode Interface in Myoelectric Hand Prostheses

Stavdahl, Øyvind PhD; Mugaas, Terje MSc; Ottermo, Maria Vatshaug PhD; Magne, Tordis MSc; Kyberd, Peter PhD

Author Information
Journal of Prosthetics and Orthotics: January 2020 - Volume 32 - Issue 1 - p 38-51
doi: 10.1097/JPO.0000000000000296
  • Free

Abstract

The fact that prostheses controlled via the surface electromyogram (SEMG) occasionally fail to operate according to the user's intentions has been reported as one of the most highly desired factors to be improved,1 and the problem is well known among clinicians (Schonhowd T, personal communication, August 28, 2018). Several factors may cause such sporadic malfunction events, including electrical interference and skin impedance changes due to perspiration. The main focus of this article is concerned with another important class of disturbances, which are mechanical in nature: variations in skin-electrode contact forces or changes in relative positions between the residual limb and the electrodes. Such disturbances cause the SEMG to exhibit atypical behavior that is misinterpreted by the prosthesis control system.2 These problems are known to most users, and the low-level physical mechanisms at play have been described in some detail,2,3 but little hard evidence has been published on the actual mechanisms at work on a macro level inside a prosthesis socket during such malfunctions. The aim of this study was to reveal some of this evidence. We expect this knowledge to have practical relevance for improved socket and electrode designs, SEMG processing techniques, and user training, because problems that are properly understood can be more easily and effectively mitigated through targeted engineering efforts.

Despite efforts in improving prosthesis design, prostheses based on extraskeletal suspension and SEMG electrodes are still widely used. A recent study by Widehammar et al.,4 including subjects with this type of prosthesis, investigated the user's experience of environmental influence on prosthesis use. They found that although both daily and nondaily users were satisfied with the functionality of their prosthesis, hardly any of the participants trusted it completely. One of the reasons for this mistrust is that a certain amount of relative motion is inevitable between prosthesis and residual limb, which will alter the electromyograms (EMGs) and cause possible malfunctions. All external forces are transferred through the prosthesis socket to the residual limb, causing the socket, and thus the electrodes, to be pressed harder against the residuum, pulled away, displaced sideways, rotated, or any combination of these. Due to tissue compliance, changes in contact force will always be accompanied by some relative displacement, and vice versa. SEMG disturbances caused by such mechanical factors are thus collectively referred to as motion artifacts.2,3

The properties of the observed SEMG may also be altered if the limb is operated in working positions that are different from the ones used during system training. This phenomenon, referred to as “the limb position effect,” has been observed both in healthy5 subjects and in subjects with transradial amputation.6 The result is the same as for motion artifacts in that the prosthesis control system fails to correctly identify the user's intent. Previous studies indicate that this effect can be mitigated by appropriate system training strategies6,7 and/or by introducing new sensor modalities such as accelerometry.8 In the aforementioned studies, the limb position effect was studied as a static phenomenon, but dynamic motions and training have also been examined.9,10 These studies showed that dynamic motions yield higher classification accuracy than static motions. Even the simplest commercial two-electrode controllers are in effect two-input pattern recognition systems. Thus, they can arguably be expected to be susceptible to the limb position effect, although to a lesser degree than multichannel systems. It is therefore considered to be of equal relevance for the present study as motion artifacts.

Artifact attenuation in SEMG signals has been researched extensively for numerous applications. In prosthetics, one example is the attenuation of electrocardiogram (ECG) artifacts, which is of particular importance when using SEMG sites on or near the torso.11 When it comes to removing mechanically induced artifacts in prosthesis applications, the literature is considerably scarcer. Lovely et al.12 pointed out the problem and suggested an implantable myoelectric sensor as part of the solution. Significant research efforts have been put into implantable myoelectric sensors (see, for example, De Luca et al.13 and Widehammar et al.14), and a study by Kristjansson et al.15 indicates that implanted myoelectric sensors do provide reliable and as improved functional benefits for the user. Among the benefits of this technology are improved signal-to-noise ratio, no electrode lift-off, the ability to more selectively record the activity of deeper musculature, and partial elimination of the limb position effect.16 On the downside, implanted devices imply challenges related to power supply, infection, and encapsulation, some of which are gradually being solved through technological progress. Other disadvantages, like the need for surgery during installation and extraction, are more fundamental and will likely keep some subjects from adopting this technology in foreseeable future.

In commercial prosthesis control systems, movement artifacts are usually attenuated by high-pass filtering the raw SEMG signal with a cutoff frequency of approximately 20 Hz. This filter is said to remove most of the transient noise induced by normal upper-limb movements, though the exact appropriate cutoff frequency is subject to debate.3 Such filtering is not believed by the authors to have any relevance to the static position effect.

Lovely2 gives a concise record of physical phenomena associated with movement artifacts. Most of these are unpredictable and may cause significant disturbance even at minute electrode displacements. Some disturbances are fundamentally nonlinear, for example, changes in the effective signal gain, and thus cannot be removed through linear filtering. An extreme form of this occurs during electrode lift-off, in which case one or all of the electrode terminals completely lose contact with the skin (Figure 1). Electrode lift-off can be detected indirectly based on, for example, resistance measurements17 or reflected near-infrared light.18 Although lift-offs, as well as other phenomena related to movement artifacts, are well known and easily observable in laboratory settings, the research literature hardly reports any evidence as to what exactly causes the prosthesis to sporadically malfunction in practical use.

F1
Figure 1:
Illustration of partial (A) and full (B) electrode lift-off. The figure depicts a typical active SEMG electrode, similar to the one used in the multimodal myoelectric unit (MMU), as viewed parallel to the skin.

These phenomena might be illuminated by explicitly measuring the skin-electrode contact forces during operation of the prosthesis. In-socket force or pressure measurements have been demonstrated repeatedly, using mechanical, hydraulic, and pneumatic devices, as well as ones based on Hall effect sensors; see Kenney et al.19 for a brief review and Radmand et al.20 and Connan et al.21 for recent developments. All these studies have aimed at using information related to force or muscle bulge explicitly for control purposes. For the purpose of this study, which is to explore the mechanisms related to sporadic myoelectric control failure, we have developed an augmented SEMG device with built-in contact force measurements, that is, a multimodal myoelectric unit (MMU).22 The MMU has been applied to a group of experienced prosthesis users for realistic measurements. In Comert and Hyttinen,23 force sensors were used to measure pressure between a dry electrode and the skin in a laboratory setting, and it was shown that motion artifact and the impedance relationship to the applied motion depend on the applied force. To our best knowledge, this work still represents the first reported measurements of the actual contact force between SEMG electrodes and skin surface while worn by prosthesis users to perform everyday tasks. Although the force information is still applicable as additional control information, it is hypothesized that the force information will allow us to identify the causes of malfunction, at least qualitatively, and thus provide guidance in the quest for improved prosthesis function. To illuminate the possibilities and limitations of the additional information provided by the added sensors, a significant amount of space is given to the description of the MMU device itself.

The study is limited to transradial amputations, one particular brand of SEMG electrodes and terminal devices (TDs), and Münster/Northwestern type socket designs,24,25 but the results are believed to have validity beyond this category.

METHODS

The experimental protocol was approved by the Regional Committees for Medical and Health Research Items–REC Central. All subjects signed a written informed consent document before participation.

NOMENCLATURE

Abbreviations used in this article are listed in Table 1.

T1
Table 1:
Abbreviations used in this article

SUBJECTS

A total of 15 subjects were included in the study. For one subject, only age, sex, TD type, and amputation level were recorded. Thus, statistics related to other metrics are based on the remaining 14 subjects.

One subject was female and 14 were male. Ages ranged from 19 to 69 years (mean, 47 years; standard deviation, 16 years). The most recently amputated subject had been using a prosthesis for 3 years, whereas the most experienced subject had 62 years of experience (mean, 26 years; standard deviation, 17 years). All subjects had transradial amputations, with amputation levels distributed as follows: proximal forearm, seven; mid-forearm, four; distal forearm, three; and one not recorded. Four subjects were users of electric split-hooks, 10 used electric hands, and one used both types of TDs. For one subject, the cause of amputation was not recorded; six subjects had a congenital absence, and seven had lost their limb due to trauma. The participant data are summarized in Table 2.

T2
Table 2:
Participant data

THE MMU

Each unit comprised a differential 13E200 electrode (Otto Bock HealthCare GmbH), which has a built-in preamplifier and produces an output that is roughly proportional to the amplitude of the SEMG. The electrode placed on the medial (flexor) controls closing of the hand, whereas the one placed on the lateral (extensor) controls opening of the hand. Four FS1500 force sensors (Honeywell Sensing and Control), each connected to a separate INA122UA instrumentation amplifier (Burr Brown Corp.), were employed for contact force measurements. The electrode was mechanically coupled to the force sensors with a layer of elastic foam rubber, sandwiched between two semirigid plastic sheets, and all parts were eventually stacked within a plastic housing (Figure 2).

F2
Figure 2:
The MMU. A, 1, SEMG electrode; 2, foam spring; 3, force sensor board. B, Fully assembled MMU.

The foam rubber acts as a spring that allows the electrode an excursion of up to 3 mm when exposed to contact forces, similar to when the electrode is mounted the traditional way. The purpose of the plastic sheets is to distribute the spring force over the back surface of the SEMG electrode and the four force sensors. The electrode's suspension tabs, which can be seen as black cylindrical projections at each end of the electrode unit, protrude through slots in the MMU housing to allow the necessary 3 mm of play. Each MMU was calibrated to yield a reading of 0% from all force sensors when no external force was present, and 100% when the SEMG electrode was maximally depressed (i.e., metal electrodes flush with edge of housing). Table 3 summarizes the MMU's main characteristics.

T3
Table 3:
MMU technical specifications

Although a single force sensor might enable detection of such states as total electrode lift-off (Figure 1B) or excessive contact force, with separate sensors in each corner of the device, we achieve a joystick effect through which we can detect both magnitude and direction of the contact force. Furthermore, this configuration facilitates the detection of partial lift-off (Figure 1A), which may cause the electrode output to behave unpredictably and thus preclude any successful control of the prosthesis.

EXPERIMENTAL PROTOCOL

The participants were first asked to perform a series of three standardized activities: pigeon-hole, tray, and hand behind back.

The pigeon-hole test is inspired by a procedure originally used for assessment of the Belgrade hand in the 1970s.26 In the original setup, the user's ability to grasp and let go of different objects at four different height levels was tested. In the present study, we used only two objects, namely, a light hollow cylinder (approximately 90 g) and a small suitcase (approximately 1.3 kg). The suitcase can be seen in Figure 3. During the experiment, the user would stand in front of a rack (regular, adjustable bookshelves as shown in Figure 3) with 3 × 3 compartments (“pigeon holes”) with the task of lifting the object from one compartment at knee height to another compartment at shoulder height and then back. We only looked at diagonal movements, so as to provoke situations where the prosthesis was in an extreme position. The test was repeated 3 times for each side (starting at bottom right or left) and for each object. Each single grasp-move-release sequence was counted as one “run” in the analysis.

F3
Figure 3:
Synchronization between video footage and signals. Countersunk screws were inserted from the inside of the socket to engage with threads in the MMU casing or external metal nuts.

The tray test was included to provide information about how the prosthesis behaves when the subject is moving. One problem reported by the users was unpredictable prosthesis behavior when carrying things like a tray from one place to another.27 Sudden stops and turns and walking on stairs were also reported to be problematic. In this test, the participant was asked to carry a tray (290 g) with an object (a wooden brick with a weight of 350 g) on top with his/her prosthesis, while holding another object in the other hand to increase the cognitive load. The tray was made of an apparently fragile material so as to implicitly emphasize the importance of not dropping it. The objects were carried first up and then down a six-step flight of stairs, with sudden turns on the upper and lower landing. This was repeated three times.

The “hand behind back” activity looked at the performance of the prosthesis in an extreme position. The participant was asked to move the arm behind his/her back and then to the front three times while holding a light cylinder (approximately 90 g). At each end position, the participant was asked to operate (i.e., open and close) the device. This was repeated three times.

Finally, the participant was asked to identify other situations where (s)he had experienced sporadic control failure of the prosthesis and to move his/her prosthesis accordingly, in order to provoke a similar control failure. This was also repeated three times. Every test took approximately 1.5 hours, including short breaks between tasks.

Sporadic control failure events were categorized as failure to open (FO), failure to close (FC), involuntary open (IO), or involuntary close (IC). The first two of these event categories correspond to the prosthesis failing to open or close, respectively, at a time when the user solicited such action. The latter two represent movements of the prosthesis at a time when the user did not intend to trigger any movement. When a particular category of control failure was observed repeatedly within a time frame of approximately 3 seconds and within the same overall movement or posture, this was counted as a single occurrence. The suitable extent of this frame was established through preliminary experimentation. Control failures were detected based on observations by the researcher providing instructions to the user, and orally confirmed by the user upon request. The user would sometimes also report perceived failures that were not apparent to the observer.

EXPERIMENTAL SETUP

Two MMUs, a lateral measuring muscle activity related to opening of the hand and a medial for measuring activity related to closing, were mounted in the socket of a transradial prosthesis with the MMU housing leveled with the inside of the socket (Figure 4). To copy the conditions in the user's ordinary prosthesis as closely as possible, the experimental prosthesis was built out of the test socket used for manufacturing of each subject's ordinary prosthesis. The experimental prosthesis thus had a basic inner socket geometry identical to that of the subject's ordinary prosthesis. The socket was split along its ventral side, and a threaded adjustment devise was attached across the split to narrow the split and thus obtain an even snugger fit. The socket also exhibited the original markings for the position of the electrodes, which allowed the MMUs to be mounted in exactly the same positions as those of the user's ordinary prosthesis. No outer socket was used; the test socket was extended to the correct length by means of a PVC tube (Ø = 40 mm), which was glued to the distal end of the socket. The total weight of the experimental prosthesis closely resembled that of each subject's ordinary device.

F4
Figure 4:
Test socket with both MMUs mounted. The figure shows the overlay sample counter added to the video image (red outline). “Normal” indicates that the signal was interpreted as being in a normal state, that is, within typical boundaries for the SEMG.

A wrist adapter for the TD was mounted in the distal opening of the tube. All input signals were fed to an NI USB-6211 analog input/output module (National Instruments Corp.), which was connected to a laptop computer via a 5-m USB cable extension. The input/output module was placed in a small pouch that was attached to the user's clothes near the waist. The computer software was implemented in LabView (National Instruments Corp.) and configured to sample all MMU signals at 100 Hz and display them on the computer screen in real time. To make the prosthesis behave in its normal manner, the signals from the electrodes were relayed back to a pair of analog output channels that were connected to the TD's electrode input terminals. The signal amplification was adjusted in software to yield similar sensitivity to that of the user's ordinary prosthesis. The computer was set up to log all input and output signals to a hard drive, along with video footage recorded during the signal acquisition. The video allowed us to thoroughly study significant events offline and establish exactly what happened in every situation. Synchronization was done by adding an overlay sample counter to the video image, as shown in Figure 3. “Normal” indicates that the signal was interpreted as being in a normal state, that is, within typical boundaries for the SEMG.

DATA ANALYSIS

The force, SEMG, and video data were scrutinized to establish the cause of each control failure event. Particular attention was paid to signal patterns that appeared to recur across different events within the same category. With this in mind, each recorded scenario was labeled based on similarities in SEMG and force signal patterns immediately before or at the time of the control failure and assigned to one of the categories of total lift-off (TLO), partial lift-off (PLO), low force (LoF), or unidentified (UI).

TLO occurs when all electrode terminals lose physical contact with the residual limb. Similarly, we define PLO as the lifting of at least one but not all electrode terminal(s) from the skin surface. These are well-known failure modes in myoelectric prostheses, although hardly documented. The immediate cause of electrode lift-off is that the tissue is moved away from the electrode site, or vice versa, in excess of what the elasticity of the electrode suspension, or that of the soft tissue, can compensate for. This situation is usually secondary to external forces displacing the socket with respect to the residual, or changes in residual geometry due to joint or muscle tissue movement. A TLO was expected to produce contact force signals identical to zero until electrodes and tissue reconnected. Correspondingly, during a PLO, we expected to see two of the force measurements, corresponding to the one electrode terminal being lifted, to attain a constant value of or in the vicinity of zero.

During LoF situations, the contact forces become unusually low, as directly observable in the force measurement signals. This state can be thought of as a potential precursor to a lift-off event. In this perspective, LoF, PLO, and TLO may be thought of as representing the same fundamental problem at different levels of severity. Situations that were labeled “unidentified” represent instances of sporadic control failure in which the contact force data reveal no apparent mechanical reason for the failure.

It was detected that both MMUs had been significantly out of calibration during at least parts of the experiment, most likely because nonideal mounting surfaces caused deformations of the MMU housing. Assuming that the foam rubber exhibits linear spring characteristics (i.e., obeys Hooke's law), such deformations will cause a mere offset in the force measurements. Therefore, all force signals were offset-adjusted as follows before subsequent analysis:

  • In recordings that exhibited at least one period of lift-off, the mode (i.e., the most frequently occurring sample value) of the signal during the lift-off period was subtracted from the raw signal.
  • In signals without evidence of lift-off, the lowest sample value of the entire signal was subtracted from the raw data.

In the latter case, it is unlikely that the adjusted signals attained the correct numerical values unless the offset-adjusted signal spanned the entire interval from 0% to 100% of force. Consequently, these data cannot be considered quantitatively meaningful. They do, however, maintain their qualitative information.

RESULTS

GENERAL OBSERVATIONS

Figure 5 shows typical sensor readings from a pigeon-hole run. The figure clearly shows the user opening and closing the hand by activating extensor and flexor muscles, respectively, as reflected in the SEMG amplitude signals. The gross collective behavior of the contact force signals suggest that, after having closed the hand around an object, the object was lifted so that the gravitational force of the object was transferred through the socket/residual interface, causing the soft tissue to be squeezed against the electrodes and thus a general increase in the contact forces. The elevated contact force is quite variable during the moving of the object, especially on the lateral side. After the TD has been opened to release the object and then closed again, all forces stabilize at a low level.

F5
Figure 5:
Typical sensor readings for a pigeon-hole activity without reported control failures. The two left panes show the four force sensor readings and the sensed SEMG signal, respectively, from the medial (flexor) MMU, whereas the panes to the right show the same information for the lateral (extensor) MMU. Recorded events include the following (time references are approximate): t = 14 seconds: opening the TD; t = 15 seconds: closing the TD around an object; 15 seconds < t < 17 seconds: moving the object to another shelf; t = 17 seconds: opening the TD to release the object; t = 18 seconds: closing the TD. See Table 1 for a list of abbreviations used in the figure.

It is interesting to observe the variable degree of correlation between SEMG amplitude and associated contact forces. Sometimes, these signals are highly correlated, like during the first close event at t = 15 seconds, where the medial SEMG and all eight force signals exhibit a synchronous peak. However, at other times, these signals seem to be significantly less correlated, for example, during the second close event at t = 18 seconds. This indicates that the force signals contain information about not only muscle contraction, but also other effects, most likely disturbances.

SPORADIC CONTROL FAILURE

Eight of the 15 subjects reported occurrences of sporadic control failure during the experiment. The tray test induced no observable control failure; for the other three activities, the occurrences are summarized in Table 4. The columns indicate the count of control failure occurrences (mean number of occurrences per subject ± standard deviation), the number of subjects with control failures, and the percentage of runs during which control failures occurred.

T4
Table 4:
Number of control failure observations during different activities

All three subjects who reported control failure during the hand behind back activity also reported control failure during pigeon-hole. One subject only reported control failures during the other activity. The number of recorded control failure events in each event category is summarized in the rightmost column of Table 5. We note that the IO category accounts for virtually half of all the recorded events, whereas no IC events occurred. FO and FC occurrence rates were comparable. The bottom row of the table summarizes the number of events assigned to each label, whereas the body of the table shows the number of recorded events for each combination of event category and label. TLO and PLO collectively accounted for 64.3% of the recorded failure events, whereas TLO, PLO, and LoF together represented 73.8% of all events.

T5
Table 5:
Number of sporadic control failures by event category and label

INVOLUNTARY OPENING

Figure 6 shows representative signal readings from the MMU during an IO/TLO event, captured during the activity hand behind back. The following inferences can be made on the basis of these graphs:

F6
Figure 6:
Example of a typical MMU read-out during total lift-off. The following observations were noted: t = 20 seconds: successful closing; 21 seconds < t < 25 seconds: hand behind back, failure to open (FO); t = 25 seconds: hand moved towards front, involuntary opening (IO). See Table 1 for a list of abbreviations used in the figure.
  • o The FO during the interval t = 21 seconds to t = 25 seconds is caused by total electrode lift-off on the lateral side, as indicated by the corresponding zero valued force signals.
  • o The IO at t = 25 seconds is caused by spikes in the electrode output signals. In the force graphs, one can see that these spikes coincide with the SEMG electrode's reconnection with the residual limb after the preceding period of lift-off; this reconnection is indicated by significant increase in at least two of the force signals from the lateral MMU starting at t = 25 seconds.

This implies that it is not the lift-off itself, but rather the touchdown (reconnection of the electrode with the skin), that causes the involuntary movement. This particular chain of events was observed in conjunction with five of the nine IO events related to TLO or PLO. Another four cases of IO/TLO happened without any visible evidence of reconnection.

The single IO event classified as LoF was similar to the latter four, except for a minor disturbance in the force. The last nine cases of IO were classified as UI. Four of these exhibited an increase in SEMG level coincident with rising contact force levels, whereas in the last five there was no apparent correspondence between force and SEMG levels.

FAILURE TO OPEN

Of the 11 recorded FO events, nine were noted as being related to TLO. In some cases, as exemplified in Figure 6, the force levels stayed at zero for an extended period. In other cases, the electrodes occasionally reconnected with the skin as shown for the lateral MMU in Figure 7. The video recording of this particular experiment contains evidence that at least once during the time frame 27 seconds < t < 32 seconds the prosthesis motor was indeed activated, but without yielding the intended result. Thus, this might be an instance of involuntary closing that passed undetected because the hand was already closed.

F7
Figure 7:
Failure to open (FO) with TLO and occasional touchdown at very low force levels. The following observations were noted: t = 26 seconds: hand in front of body, successful opening; t = 27 seconds: successful closing; t = 28 seconds, t = 30 seconds, and t = 32 seconds: hand behind back, failure to open (FO); t = 34.5 seconds: hand in front of body, successful opening. See Table 1 for a list of abbreviations used in the figure.

Two FO events happened during LoF conditions, as illustrated in Figure 8. We see from the figure that the lateral MMU electrode is more or less in a TLO condition, but there is a finite, measurable contact force during the failing attempts to open the TD.

F8
Figure 8:
Failure to open (FO) under low force conditions (LoF) during a pigeon-hole activity. The following observations were noted: the amputated arm was stretched forwards, upwards, and laterally at t = 272 seconds, and stayed in this posture until t = 280 seconds; 274 seconds < t < 277 seconds: failure to open (FO). See Table 1 for a list of abbreviations used in the figure.

The last of the FO events, which is shown in Figure 9, was marked UI. Neither the LEMG graph nor the video footage from this experimental run suggests specific points in time where the user tried to issue an “open” command. Both electrodes appeared to have skin contact during most of the failure period, but as virtually no LEMG activity was recorded, the force variations cannot be correlated with any discrete opening attempts.

F9
Figure 9:
Failure to open (FO) with unidentified cause (UI) during a pigeon-hole activity. The user repeatedly but unsuccessfully tried to open the terminal device during the entire interval depicted in the figure, until eventually succeeding at t = 55.5 seconds. See Table 1 for a list of abbreviations used in the figure.

FAILURE TO CLOSE

Five instances of FC during TLO conditions were recorded. These were similar to the FO/TLO events, except that the lift-off occurred at the medial electrode site. Likewise, a single case of FC under LoF conditions was observed, qualitatively resembling FO/LoF but at the opposite site.

The two FC/PLO events, however, were qualitatively different from all other events. The MMU data from one of these are presented in Figure 10. Given that these are examples of FC, our attention is drawn to the graphs from the medial MMU since “close” commands are communicated via the medial electrode site. However, the forces on the medial side clearly exhibit nonzero values and the EMG level saturates, which suggests that the TD should indeed receive a valid “close” command. Interestingly, the MMU on the opposite side experiences PLO during the entire FC event, whereas the associated EMG attains moderate to high levels.

F10
Figure 10:
Failure to close (FC) accompanied by partial lift-off during a pigeon-hole activity. At t = 57 seconds, the TD is quickly and successfully opened and then closed; t = 59 seconds: successful opening; t = 61 seconds: partial lift-off (PLO) occurs at the lateral electrode site (signals LPP and PLA); 62 seconds < t < 66 seconds: the user repeatedly but unsuccessfully tries to close the terminal device. See Table 1 for a list of abbreviations used in the figure.

Finally, a single instance of FC with unidentified cause was recorded. The signal recordings from this event are depicted in Figure 11. The force graphs suggest that there was no lift-off, but one notes that the “open” signal at t = 53 seconds, which can be seen as a raised level of the SEMG from the lateral MMU, is kept at a significant level even during the attempted closing from t = 54 seconds to t = 56 seconds.

F11
Figure 11:
Failure to close (FC) with unidentified cause (UI) during a pigeon-hole activity. At t = 53 seconds, the TD is successfully opened; 54 seconds < t < 56 seconds: failure to close, followed by successful closing at the end of this interval; t = 59 seconds: successful opening. See Table 1 for a list of abbreviations used in the figure.

DISCUSSION

The primary findings of this study are 2-fold. First, although control failures related to varying contact between electrodes and residual limb have generally been attributed to lift-off, there is evidence that it is rather the touchdown (reconnection) that is causing the errors observed in this study. This is an important distinction, as it may call for a different set of countermeasures. Second, the presence of contact force sensors provide additional information that can be utilized for mitigating said control problems, as well as for general control inputs. The low general correlation between SEMG and force signals as well as the close causal connection between limb position and external load on the one side and pressures and forces inside the prosthesis socket on the other suggest that the information conveyed through the force measurements may be useful for mitigating the infamous limb position effect.5,8 These results and the extent of their validity are discussed further in the following sections.

LIMITATIONS OF THIS STUDY

The present results should be interpreted with caution due to the inevitable limitations of the study. These limitations relate to the following issues: all experiments were based on users with transradial amputation and variations of Münster/Northwestern type sockets. Amputation level is believed to influence the severity of sporadic control failures, in that a short residual implies higher local contact force variations and thus increased likelihood of failure. Similarly, different socket designs will influence the way in which the residual limb is displaced and deformed inside the socket during use. Radically different suspension techniques based on, for example, osseointegration28 or soft roll-on liners29 will obviously behave very differently with respect to these phenomena.

Different SEMG electrode designs will respond differently, both mechanically and electrically, when exposed to mechanical perturbations. Furthermore, the motor consequences of SEMG artifacts are determined by the control scheme implemented by the system, and as such one system may behave correctly in a situation where another fails.

The activities performed by subjects during this experiment resemble activities of daily living, but they were selected explicitly to create conditions under which some users experience sporadic control failure.

All subjects were relatively experienced prosthesis users with well-fitted sockets, hence represent a subset of the user population at large.

For these reasons, the quantitative results in general have a limited applicability to routine prosthesis use or other prosthesis designs. The qualitative aspects, however, are believed to have great generality in that they exemplify phenomena that are likely to occur in a wide range of user and equipment categories.

GENERAL OBSERVATIONS

As expected, we observed a certain degree of correlation between SEMG amplitude and the electrode/skin contact forces. However, this correlation appeared to be highly variable, which suggests that the force sensors indeed capture additional information that is not contained in, or easily extractable from, the SEMG signals. Contact force measurements might provide information related to both user intent and other relevant phenomena like the position effect and movement artifacts. Future research should therefore assess the added modalities' applicability as general inputs to modern control schemes based on pattern recognition and sensor fusion.

THE MMU

The present version of the MMU exhibited several weaknesses. As described in the Data Analysis section, once the device was mounted in a prosthesis socket, it was essentially out of calibration. If this was actually caused by deformation of the housing, a more rigid housing material would reduce the problem. To the extent that this deformation was constant during each experiment, an in-socket recalibration should be added to the protocol.

Two additional factors limit the device's fidelity. First, minute force changes might be masked by friction between the SEMG electrode and the housing. The second factor is that the foam rubber spring has a limited dynamic response due to the air that needs to pass into or out of its pores during expansion or compression. This response can be seen in Figure 6 as a brief undershoot of the LDA and LDP signals at t = 21 seconds. Although the latter effect might in principle be eliminated by temporal inverse filtering, both these limitations should be addressed during future redesign.

CONTROL FAILURES

The recorded control failure events were unevenly distributed over the four categories. The most frequently recorded event, involuntary opening, may be the most serious one, because unsolicited opening of the TD while handling an object may cause loss of grip and thus injury or material damage. This suggests that the IO failure mode should be given priority in the efforts to improve the reliability of the prosthesis control system.

IO events related to electrode lift-off seem to be equally often caused by the lift-off itself as by the subsequent touchdown. Some of these situations might include numerous rapid lift-off/reconnect events, akin to “contact bounce.” This might not be observable in the MMU output force data due to the limitations discussed previously. Thus, when the force exhibits a sudden change, we cannot tell if the SEMG responds to a lift-off, a reconnection, or both.

The scenario of Figure 9 was labeled UI. In this case, there is no evidence of lift-off in any of the force signals. The true offsets of these graphs are therefore unknown, and the existence of this single FO/UI event is not given any emphasis in this study.

It should be mentioned that in one of the subjects, the extensor MMU indicated constant or barely changing contact forces at a medium level during the entire experiment. This may have been caused by a mechanical failure in the MMU itself, most likely that the SEMG electrode had fastened somewhat in the middle of its excursion range. The corresponding data should therefore not be regarded as quantitatively representative in any way. Qualitatively, however, we believe that the control failures observed in this subject represent the same typical scenarios as those found in the rest of the study group, because a partially depressed electrode with a barely flexible suspension in fact resembles the reality of some users' actual prostheses. These data have therefore been included in the analysis.

The study group size and prevalence of sporadic control failure in the present study do not allow for a stringent statistical analysis, but our observations seem to support the following statements about the mechanisms of failure and possible solutions.

As expected, electrode lift-off is associated with the loss of control and with involuntary prosthesis movements. However, it seemed to be the event of touchdown, rather than the loss of connection, that induced unsolicited movements. This failure mode may be alleviated by disabling electrode output during lift-off and only re-enabling it once proper reconnection has been established. PLO, which was only observed a few times during this study, is known to often cause saturated electrode outputs due to the heavily unbalanced input impedances it represents. The suggested temporary disabling of electrode outputs will prevent even this failure mode from causing involuntary movement (although acceptance by users of the tradeoff between a temporarily inactive system and one that opens inadvertently needs to be investigated).

As many as 26.2% of all the recorded failure events were categorized as UI, most of which were involuntary openings. In four of these cases, we observed an increase in contact force coincident with increased SEMG output, which might be attributed to movement artifacts or a form of position effect, for example, increased electrode sensitivity as the electrode terminals are pressed against and encompassed by soft tissue, thereby reducing resistance and increasing capacitive coupling between electrodes and muscle. Another hypothesis, which we believe is correct in at least some of the observed cases based on the observed EMG activity, is that the user inadvertently performs actual, centrally controlled muscle contractions. These contractions can be thought of as remnants of motor programs once used for controlling the intact limb, for example, for active joint stabilization or gravity compensation during certain movements or in certain positions, but that are detrimental to the prosthesis control task at hand. Clearly, further research should be carried out to confirm or disprove this hypothesis. If it is correct, these contractions can be considered an aspect of a “phantom limb position effect,” which points to the value of introducing pattern recognition methods and additional sensor modalities like accelerometers even in single-function contemporary prostheses. Alternatively, targeted user training with biofeedback from the detected SEMG might be appropriate to un-train these contractions.

Not a single involuntary closing event was recorded. These events are inherently harder to observe—once the TD holds an object, further closing of the hand is virtually unperceivable unless if the object is soft. Thus, no conclusions can be made with respect to the prevalence of IC events in our experiment, but as mentioned in conjunction with Figure 6, certain observations suggested that IC events were in fact occurring. A future experimental setup should include a deformable object, as this could increase the possibility of observing IC events.

Lift-off–related events including the “lift-off precursor” state LoF accounted for a majority of the recorded control failures. This suggests that electrode lift-off should be given attention as a possible point of improvement. Such improvements might include redesigned electrode suspension that offers more compliance, so that the system can tolerate more tissue movement while still maintaining good electrode/skin contact. Snugger sockets contribute similarly, although a tradeoff must be made with respect to comfort. Although such mechanical improvements may have an immediate effect, the authors believe that a more thorough understanding of the influence of tissue movement on SEMG signals may pave the way for more fundamental improvements in the future, especially in advanced multifunction systems. Achieving such understanding requires further research that should be based on multimodal sensors with hi-fidelity raw EMG and force signals, and perhaps even explicit measurements of sideways skin displacement using, for example, optical mouse technology, to allow examination of more subtle connections between these quantities. The influence of the general shape and condition of the residual limb, the control strategy employed, and other demographic parameters on the resulting control dependability should also be considered.

Implanted electrode devices13,14 certainly bear the potential for reducing or eliminating many of the failure modes covered in this article, specifically those associated with electrode lift-off and touchdown. However, other problems like the limb position effect and mere mismatch between the muscle contraction patterns produced by the user and the ones needed for activating the desired prosthesis function are likely to call for algorithmic improvements and possibly added sensor modalities. Considering also that many users will probably want to avoid the surgery associated with implanted electrodes, it seems more than likely that surface electrodes will remain a major contender for the foreseeable future. An interesting development over the last decade has been the use of surface electrodes together with soft liners that in some embodiments potentially eliminate electrode lift-off (e.g., see Reissman et al.30 for one of the latest developments), but in general, this technology introduces other challenges and has yet to be broadly adopted by users. For the vast number who still relies on more traditional SEMG electrodes, the insights presented in this article point to possible improvements that might contribute to reducing sporadic control failures and thus improve on the general reliability of the prosthesis. Our addition of force or pressure sensors to the electrode assembly is an example of sensor fusion, a generic technique that has been applied with great success in experimental settings.8 As sensors of all kinds become ever smaller and less expensive, we believe that future prosthesis control systems will include multiple sensors specifically chosen for their sensitivity to various disturbances, allowing these to be effectively attenuated.

CONCLUSIONS

The changes in forces perpendicular to the skin and shear forces between EMG electrodes and residual limb were recorded, whereas 15 prosthesis wearers used their hand prostheses to perform four everyday tasks identified as prone to sporadic failures of the control of the hand. Different modes of failure were observed: FO, FC, and IO. Based on the force signals, the cause of each failure was classified as TLO, PLO, LoF, or UI. Involuntary opening of the hand was identified as the most common failure, which is also the most undesirable. The failures seem to be associated not only with the electrode moving away from the skin, but to an even larger extent with the electrode reconnecting with the skin. The force sensors allowed many of these events to be identified, and this suggests that the output of the electrode amplifier could possibly be disabled when the controller identifies an interfering event to avoid the failure.

Although 26.2% of the events did not have an easily identified cause based on electrode lift-off or touchdown, we did observe contractions that might be remnants of the natural control of the once unamputated arm but that are detrimental to the prosthesis control. It may be possible to unlearn these contractions or to resolve the problem through more advanced control algorithms as a “phantom limb position effect” through more advanced control algorithms.

ACKNOWLEDGMENTS

K. Tomm Kristensen of Norsk Teknisk Ortopedi AS, Ottestad, Norway, and Hans Petter Aursand of Orthopedic-technical Department, St. Olav's Hospital, University Hospital of Trondheim, Norway, and their respective technical staff are acknowledged for their invaluable contributions to the recruitment of relevant subjects, production and adaptation of prosthesis sockets, and for innumerable helpful hints during the course of the work.

REFERENCES

1. Kyberd PJ, Wartenberg C, Sandsjö L, et al. Survey of upper-extremity prosthesis users in Sweden and the United Kingdom. J Prosthet Orthot 2007;19(2):55–62.
2. Lovely D. Chapter 3 Signals and signal processing for myoelectric control. In: Muzumdar A, ed. Powered Upper-Limb Prostheses. Control, Implementation and Clinical Application. New York: Springer; 2004:35–53.
3. De Luca CJ, Gilmore LD, Kuznetsov M, Roy SH. Filtering the surface EMG signal: movement artifact and baseline noise contamination. J Biomech 2010;43(8):1573–1579.
4. Widehammar C, Pettersson I, Janeslatt G, Hermansson L. The influence of environment: experiences of users of myoelectric arm prosthesis—a qualitative study. Prosthet Orthot Int 2017;42(1):28–36.
5. Scheme E, Fougner A, Stavdahl O, et al. Examining the adverse effects of limb position on pattern recognition based myoelectric control. Paper presented at: Annual International Conference of the IEEE Engineering in Medicine and Biology 2010; Buenos Aires, Argentina.
6. Liang C, Yanjuan G, Guanglin L. Effect of upper-limb positions on motion pattern recognition using electromyography. Paper presented at: 4th International Congress on Image and Signal Processing (CISP) 2011; Shanghai, China.
7. Hargrove L, Englehart K, Hudgins B. A training strategy to reduce classification degradation due to electrode displacements in pattern recognition based myoelectric control. Biomed Signal Process Control 2008;3(2):175–180.
8. Fougner A, Scheme E, Chan AD, et al. Resolving the limb position effect in myoelectric pattern recognition. IEEE Trans Neural Syst Rehabil Eng 2011;19(6):644–651.
9. Radmand A, Scheme E, Englehart K. On the suitability of integrating accelerometry data with electromyography signals for resolving the effect of changes in limb position during dynamic limb movement. J Prosthet Orthot 2014;26(4):185–193.
10. Shin S, Tafreshi R, Langari R. Myoelectric pattern recognition using dynamic motions with limb position changes. Paper presented at: 2016 American Control Conference (ACC) 2016; Boston, MA.
11. Zhou P, Lock B, Kuiken TA. Real time ECG artifact removal for myoelectric prosthesis control. Physiol Meas 2007;28(4):397–413.
12. Lovely DF, Hudgins BS, Scott RN. Implantable myoelectric control system with sensory feedback. Med Biol Eng Comput 1985;23(1):87–89.
13. Merrill DR, Lockhart J, Troyk PR, et al. Development of an implantable myoelectric sensor for advanced prosthesis control. Artif Organs 2011;35(3):249–252.
14. Ortiz-Catalan M, Branemark R, Hakansson B, Delbeke J. On the viability of implantable electrodes for the natural control of artificial limbs: review and discussion. Biomed Eng Online 2012;11:33.
15. Kristjansson K, Sigurdardottir JS, Sverrisson AÖ, et al. Prosthetic control by lower limb amputees using implantable myoelectric sensors. Paper presented at: International Conference on NeuroRehabilitation (ICNR2016) 2016; Segovia, Spain.
16. Chi A, Smith S, Womack I, Armiger R. The evolution of man and machine—a review of current surgical techniques and cutting technologies after upper extremity amputation. Current Trauma Reports 2018.
17. Wilson AW, Losier YG, Parker PA, Lovely DF. A bus-based smart myoelectric electrode/amplifier—system requirements. IEEE Transactions on Instrumentation and Measurement 2011;60(10):3290–3299.
18. Herrmann S, Attenberger A, Buchenrieder K. Prostheses control with combined near-infrared and myoelectric signals. In: Moreno-Díaz R, Pichler F, Quesada-Arencibia A, eds. Computer Aided Systems Theory – EUROCAST 2011. Berlin, Germany: Springer; 2012:601–608.
19. Kenney LP, Lisitsa I, Bowker P, et al. Dimensional change in muscle as a control signal for powered upper limb prostheses: a pilot study. Med Eng Phys 1999;21(8):589–597.
20. Radmand A, Scheme E, Englehart K. High-density force myography: a possible alternative for upper-limb prosthetic control. J Rehabil Res Dev 2016;53(4):443–456.
21. Connan M, Ruiz Ramirez E, Vodermayer B, Castellini C. Assessment of a wearable force- and electromyography device and comparison of the related signals for myocontrol. Front Neurorobot 2016;10:17.
22. Stavdahl Ø, Kyberd PJ, Magne T, et al. Multimodal input device with SEMG and contact force sensors. MEC'11: Myoelectric Controls Symposium 2011; Fredericton, Canada.
23. Comert A, Hyttinen J. A motion artifact generation and assessment system for the rapid testing of surface biopotential electrodes. Physiol Meas 2015;36(1):1–25.
24. Hepp O, Kuhn GG. Upper extremity prostheses. Paper presented at: Second International Prosthetics Course 1959; Copenhagen, Denmark.
25. Lake C. The evolution of upper limb prosthetic socket design. J Prosthet Orthot 2008;20(3):85–92.
26. Kay HW. An evaluation plan for the Beograd hand. Advances in External Control of Human Extremities 1969; Dubrovnik, Yugoslavia.
27. Wininger M, Kim NH, Craelius W. Pressure signature of forearm as predictor of grip force. J Rehabil Res Dev 2008;45(6):883–892.
28. Jonsson S, Caine-Winterberger K, Branemark R. Osseointegration amputation prostheses on the upper limbs: methods, prosthetics and rehabilitation. Prosthet Orthot Int 2011;35(2):190–200.
29. Daly W. Clinical application of roll-on sleeves for myoelectrically controlled transradial and transhumeral prostheses. J Prosthet Orthot 2000;12(3):88–91.
30. Reissman T, Halsne E, Lipschutz R, et al. A novel gel liner system with embedded electrodes for use with upper limb myoelectric prostheses. PloS One 2018;13(6).
Keywords:

prosthetic hand; electromyography; control failure; contact force measurement; biomedical transducers; multimodal sensors

Copyright © 2019 American Academy of Orthotists and Prosthetists