Chaubey, Pravin MS; Rosenbaum-Chou, Teri PhD; Daly, Wayne CPO, LPO, FAAOP; Boone, David PhD, CP, MPH
Tactile sensory feedback, or haptics, is a fundamental element of life. When prosthetic limbs are provided for persons with amputations, this key function is largely missing. The residual limb within a prosthetic socket experiences indirect proprioception, but the sensations are highly attenuated by the structure and materials of the prosthetic device. Reduced haptic feedback results in reduced gripping accuracy for upper-limb amputees using commercially available electrical-powered prosthetic arms. Further improvement to dexterity of prostheses requires development of haptic methods to enable a finer sense of touch and grasp control lost to amputation.
To improve grasp control, researchers have investigated the use of direct force feedback for closed-loop prosthetic control. Meek et al.1 studied a force feedback system that applied a force to the user’s skin in proportion to the grip force at the prosthetic hand. In addition, they used vibration to the user’s skin in proportion to the vibration at the prosthetic hand.1 Patterson and Katz2 used a pressure cuff system that applied pressure to the arm in proportion to pressure on the prosthetic hand. These studies consistently found that the addition of any supplemental haptic feedback could improve prosthetic users’ control of their grasping force. Since the direct force or pressure feedback systems have tended to be large and cumbersome with demanding power consumption,1 vibrotactile feedback may provide a better alternative for clinical use as they are small, unobtrusive, and require less power.3 Therefore, if vibrotactile feedback also improves grip force control, it could more quickly and cost effectively be transitioned into a commercial product.
Previously, vibration-producing tactors have been shown to improve object manipulation in unimpaired persons4–6 and prosthetic hand grasping force accuracy in unimpaired persons with previous experience in using vibrotactile feedback.3 After an extensive literature review, no study was found that evaluated the ability of vibrotactile feedback to improve grip force accuracy in an upper-limb amputee population.
Sensation from vibration-producing tactors can be varied by changing the amplitude, frequency (dependent on speed of vibration motors), or pulse rate (dependent on rate that vibration motor is turned on and off). To increase the amplitude of the vibrotactile sensation, more than one tactor has been stacked coaxially to generate constructive vibrations.7 Results demonstrated an amplitude discrimination of 75% to near 100% when assessing one to three tactors. Although these results are promising, stacking tactors coaxially would make concealing these tactors in the prosthesis more challenging.
Frequency of the vibration can be altered by changing the voltage driving the motors. Previously, the discrimination between 156 and 122 Hz was 92% and 156 and 140 Hz was 86%.7 Another study found that participants were able to discriminate 6 levels of vibration frequency ranging from 60 to 360 Hz.8 However, accuracy was not reported. A combination of frequency and amplitude resulted in subjects’ ability to discriminate six different stimulation patterns with an accuracy of 78%.7 However, one drawback of frequency is its sensitivity to motor restraint.9 When a constant electrical signal is applied to a vibration motor, a motor that is mounted loosely to a person’s arm will vibrate at a lower frequency than would a motor that is firmly restrained.9 Therefore, a change in arm position could change the motor restraint and, thus, frequency of the motor’s vibration.
Varying the pulse rate of the vibration is another vibration characteristic that can also be used to better differentiate sensations. Pulse rates are created by starting and stopping the vibration of the tactor at specific rates. Pulsing has the advantage of being independent of motor restraint. Previously, pulse rates were used to improve prosthetic hand grasping force accuracy in unimpaired persons with previous experience in using vibrotactile feedback.3 However, the ability to discriminate between pulse rates, optimal location for pulse rate detection, optimal pulse signal, and pulsing desensitization time were not reported.
The research objectives of our study were as follows:
1. Optimize tactor effectiveness by determining locations on the skin for effectiveness and comfort during stimulation, optimal feedback signal (among square, sinusoidal, and saw tooth), and skin desensitization time.
2. Assess the effectiveness of vibratory feedback on cognitive loading interpretations. This involved measuring the ability of persons with transradial amputation to generate myoelectric signals controlling the gripping force while interpreting visual feedback, vibratory feedback, and no feedback.
3. Assess the perceptions of persons with amputations regarding the haptic feedback experience.
DEVELOPMENT OF HARDWARE TEST PLATFORM
A hardware platform was constructed that allowed subjects to follow a predetermined protocol for grasping objects with and without haptic feedback and for collecting data. A pressure sensor (Flexiforce® Tekscan, Inc., South Boston, MA) on the target object conveyed pressure data to the controller unit, which generated appropriate signals for a feedback device based on a predetermined mapping algorithm (Figure 1). The bolt and nut at the center of the object were used to provide a consistent location and surface area for the application of pressure to the sensor. C2 tactors® (Engineering Acoustics, Inc, Casselberry, FL) were chosen as the mode of vibratory feedback for this study because they have demonstrated success previously.3,4 The tactors were placed on the upper arm of the residual limb because it was uncertain if the tactors would fit comfortably within the socket of all subjects without modification to the socket. Vibration was modulated by pulse rate and waveform. Duty cycle for the pulse rate was 50%. A carrier frequency of 250 Hz was held constant because frequencies in the range of 220 to 250 Hz have been shown to strongly stimulate Pacinian corpuscle skin mechanoreceptors.10
The pressure sensors were put through a sequence of loading/unloading cycles for calculating no-load offsets and calibration curves. The pressure sensor used for subject testing was within Tekscan’s product specifications for repeatability, hysteresis, drift, and linearity. The main controller box was connected to the prosthesis through wires. The mapping algorithm was implemented using Labview® (National Instruments, Austin, TX). The mapping algorithm changed the pulse rate linearly from 0 to 8 Hz with grip force. Each subject’s maximum grip force resulted in an 8-Hz pulse. A custom signal conditioning circuit was built to condition the pressure sensor signal and to provide variable gains. Software development consisted of four separate programs, one for each research aim. The feedback signals were synthesized using standard sinusoidal frequency modulation technique.
The process for initializing grip with the prosthetic hand and receiving haptic feedback based on the grip force involved the following steps. First, subjects generated myoelectric signals to drive prosthetic hand actuators. The pressure sensor on the target object measured the grip force. Next, the sensor signals were processed using a data acquisition device (NI-DAQ, National Instruments, Austin, TX) to generate appropriate signals for the tactors. When the subject grasped the target object, the vibration pulse rate of the tactors increased with increased gripping force. The graphical interface plotted and recorded input and output signals (Figure 2).
A total of seven transradial subjects were recruited for this study. The upper-limb research subjects were aged between 21 and 71 years and were at least 3 years post amputation. All subjects were using myoelectric prostheses. Research protocols were approved by the relevant institutional review board and all subjects consented.
OPTIMIZING TACTOR EFFECTIVENESS
TACTOR LOCATION AND PULSE RATE DISCRIMINATION
Twelve tactor locations on the upper arm of the residual limb were selected for testing. Three tactors were placed on each side of the arm (medial, lateral, anterior, and posterior). On each side, a tactor was placed proximal, middle, and distal with equal distance between them using the proportion of each subject’s upper-arm length (Figure 3). Regions with scar tissue were avoided. Electromyographic (EMG) electrodes were placed over wrist flexor and extensor muscles. The tactor was placed 50 mm away from the electrodes to limit interference with the EMG signal. The choice of 50 mm was based on empirical tests during preliminary work. Because an amputee’s ability to relate haptic feedback to pressure at the fingertip depends on the resolution of the tactor signal and skin sensitivity, each location was tested for sensitivity and comfort. The tactor was vibrated with a modulated pulse rate between 1 and 8 Hz. Audible feedback indicated the start and end of each trial. The software incrementally increased the pulse rate by 1 Hz every 3 to 5 seconds, so the subject did not get accustomed to regular patterns. The pulse rates were not randomized because randomization would change the difficulty of the pulse rate discrimination. For example, it would be easier to discriminate 2- from 8-Hz pulse rate than to discriminate 7 from 8 Hz. Therefore, the testing order for each research subject was the same to keep the discrimination difficulty of the pulse rates the same. With the discrimination difficulty of pulse rates controlled, the sensitivity of different tactor sites could be compared. Each tactor site received one pulse discrimination trial (1–8 Hz).
After the start command, subjects were asked to convey every time they felt a change in pulse rate until the end of trial. The sum of the number of times the subject correctly detected a frequency change was recorded for each location. Because the purpose of the test was to characterize skin response, the closer the score was to 7 (perfect score), the better the resolution (or accuracy) for detecting pulse rate changes at that location. At the end of each test, subjects were asked to rate the comfort at that location (1 being least comfortable and 5 being most comfortable). To normalize results across patients, they were asked to judge it holistically and not for any one particular pulse rate change. In case of a tie when two locations received the same resolution score and comfort score, false-positives (subject stated a change but a change did not occur) were used as a tiebreaker to determine which location could more accurately detect changes in vibration pulse rate.
DETERMINING THE OPTIMAL VIBRATORY FEEDBACK WAVEFORM
After determining the optimal location for tactor placement, tests were performed to determine the feedback waveform that yielded the best resolution and comfort levels. The three waveforms tested were square, sawtooth, and sinusoidal. Only one trial of each waveform was performed to minimize subject burden. Other, more complex stimulation patterns may be beneficial but were beyond the scope of the current work. The order of the waveforms tested was randomized. Data relating to resolution and comfort levels of each waveform were collected using the same protocol for the tactor location testing. For this test, instead of the location changing after reaching 8 Hz, the waveform changed, and the test was repeated in the same location after about a minute break.
DETERMINING TIME TO SKIN DESENSITIZATION
It is known that thresholds for sensory perceptions are not consistent over time. If a sensation persists for a long time, the nervous system gradually adapts to it. The hypothesis was that continuous vibration over time would reduce the person’s ability to detect changes in vibration pulse rate. It was therefore necessary to know the time the skin takes to become desensitized to vibration. Using the subject’s optimal location for tactor placement and the subject’s optimal feedback waveform, each subject repeated the resolution test until deterioration was seen in his/her ability to distinguish incremental pulse rate changes. The time taken by subjects for their resolution to start deteriorating was recorded. A 1- to 2-Hz deterioration in pulse rate discrimination indicated start of skin desensitization. When this occurred, the trial was stopped and duration of the trial was recorded in seconds.
VIBRATORY FEEDBACK ON COGNITIVE LOADING INTERPRETATIONS
A rubber and acrylic block mounted with a pressure sensor (Figure 1) was used as the target object to be grasped by the subjects. It allowed sufficient resistance for pressure sensors to measure different levels of distinguishable grasping force. Also, because of its gradual deforming nature, subjects could relate the deformation of the object to grasping force. The target object was set up in a way that the object could be grasped without arm movement. This was done to minimize variability and to focus only on the effectiveness of vibrotactile feedback during grasping. The optimal tactor location and feedback waveform previously determined were used in all cognitive loading tests.
While the subjects squeezed the target object with their prosthetic hand, the grasping force was recorded using a customized graphical user interface in Labview (National Instruments, Austin, TX) (Figure 1). At the start of the session, subjects were asked to squeeze the target object as hard as they could to determine their maximum grasping force, and subsequently, three target force levels: 40%, 60%, and 80% of maximum grip force were generated. The maximum grip force test was repeated before each set to recalibrate the three target force levels since muscle fatigue could occur over time.
The trials were divided into five sets (Table 1). During the grasping tests, research subjects were asked to reach target force levels within 5 seconds to help prevent muscle fatigue. Within each set, the 40%, 60%, and 80% of maximum grip force targets were each repeated three times for a total of nine trials per set.
The mean proportional error in achieving the target force level was calculated by taking the difference of the target force value and mean force reading from last 3 seconds of each trial. The difference was normalized with respect to the target force value. All sets were repeated on a different day to determine retest variability.
SUBJECT PERCEPTIONS REGARDING HAPTIC FEEDBACK EXPERIENCE
Perception plays an important role in prosthetic usage. After completion of the tests, subjects were asked questions relating to their overall perception of the vibrotactile haptic feedback. They were asked to grade between 1 (poor) and 5 (excellent) for each question shown below:
1) Rate your level of comfort on the skin with vibrotactile feedback.
2) Rate your overall feeling that you achieved the correct grip pressure with vibrotactile stimulation alone (without relying on visual feedback).
3)Rate your ability to operate a myoelectric or body-powered hand and react to the vibratory sensation at the same time.
4) What level of discomfort did you experience when using the system (1, a lot of discomfort, to and 5, no discomfort)?
5) Do you feel that a portable haptic system in a prosthetic arm would be useful for everyday grasping activities?
Each question was given equal weight because each amputee has his/her own unique set of priorities. Each of the questions was viewed as equally important.
Objective 1: Descriptive statistics were used to describe the optimal location for tactor placement on the upper arm. The scores from the sinusoidal and square waveform were compared with a paired t-test (Stata/IC version 10.1). The sawtooth waveform was not the best waveform for any subject and was therefore not statistically analyzed.
Objective 2: To account for the lack of independence introduced by using repeated measurements on the same study subjects, a mixed effects linear regression model was used. Subjects were included as a random effect to allow inferences to the population of subjects with each subject having his/her own intercept. The remaining factors were treated as fixed effects to make inferences to the specific values of the factors: sets (1, 4, 5) and target force (light, medium, strong). The factor trial (1, 2, or 3) was treated as random effect, being nested within subject, for sets that did not provide feedback (set 1). For sets that did provide feedback (sets 4 and 5), the factor trial was first included as a fixed effect to assess a sequential learning effect and was then included as a random effect, simply nested within subject, if the learning effect was not present or inconsequential.
To calculate the correct standard deviation, we used the linearized standard error, also called the Huber/White/robust sandwich variance estimator. The standard error from this approach was used to calculate standard deviation with the following equation: standard deviation = standard error × square root of sample size. In addition, the p values from multiple comparisons were adjusted using the Tukey-Ciminera-Heyse procedure because it maintains an α of 0.05. A paired t-test was used to determine if there were statistical significant changes between the first and second days of testing. The difference between the target force and the actual grasping force indicated how well subjects used vibratory haptic feedback to improve their controls.
Objective 3: The subject perception score from each question was averaged and reported. All average scores of 3 or less was considered an area that needs improvement before acceptance of a vibratory haptic device.
The results showed that the biceps region was the most preferred in terms of resolution and user preference for placement of a vibratory feedback device (Figure 4). When testing on each subject’s most accurate location, subjects were able to detect changes between 2 and 3 Hz, 3 and 4 Hz, 4 and 5 Hz, 5 and 6 Hz, and 6 and 7 Hz with an average of 86% accuracy (range, 57% to 100%). Accuracy was an average of 71% for detecting a 1 to 2 Hz change, and 57% for detecting a 7 to 8 Hz change. Overall accuracy in detecting 1-Hz changes between 1 and 8 Hz was 80%. Among the three waveforms tested, the most effective was the sinusoidal waveform (p = 0.047). The average time taken for the subject’s skin to become desensitized was 66 seconds, with a range of 34–102 seconds.
Results from cognitive loading effect showed an improvement in grasping force due to haptic feedback at 60% of the maximum grasping force for set 4 (p = 0.036) and set 5 (p = 0.026) (Figure 5). Even though statistical improvements were not found at the 80% force level, the percentage error while using haptic feedback improved from day 1 to 2 at the 80% force level (p = 0.007, Figure 6). This indicated that more practice in using vibratory haptic feedback may further reduce gripping errors.
With regard to user perception, subjects gave highest scores to the level of comfort using the vibratory haptic feedback followed by usefulness for everyday grasping tasks (Figure 7). They felt comfortable in simultaneously using the myoelectric controller and reacting to the haptic feedback at the same time.
However, further improvement was desired in being able to achieve the correct grip pressure when using vibrotactile feedback. Subjects suggested that three discrete vibration pulse rates relating to separate force levels may be more informative than using continuous vibration pulse rate changes in proportion to the forces as in this study. In addition, they suggested both an increase in vibration intensity as well as an increase in vibration pulse rate with increased grip force.
Although external vibrotactile stimulation had been previously shown to be both promising and practical,3,5,7,8 effective methods for applying vibrotactile stimulation in a prosthetic arm application needed further elucidation. Therefore, four fundamental issues relating to external vibrotactile stimulation by varying pulse rate were investigated: optimal tactor location on the upper arm, feedback signal type, skin desensitization, and the ability to assist in controlling grasping force.
The optimal tactor locations on the upper arm were most often on the proximal anterior region for the subjects. Because the vibration used a carrier frequency of 250 Hz that was pulsed between 1 and 8 Hz, both Pacinian corpuscles and Merkel disk receptors were likely activated in the subjects’ tissue.11 After an extensive literature review, no study was found that performed vibration sensitivity tests in the locations evaluated in this study. However, because Pacinian corpuscles and Merkel disk receptors also facilitate tactile sensation,11 studies evaluating tactile sensitivity may be applicable to vibration sensitivity. Previous studies have found that sensitivity to tactile discrimination is greater in the shoulder region than in the upper-arm region when using the two-point discrimination test.12 Therefore, the results in this study are in agreement with studies on tactile discrimination because locations closer to the shoulder were found to be more sensitive than distal locations. Because studies comparing the sensitivity of the anterior region of the upper arm to the medial, lateral, and posterior regions was not found, future studies should verify whether the anterior portion of the upper arm is indeed more sensitive to vibration or tactile discrimination relative to medial, lateral, and posterior regions.
The pulse rate detection accuracy of 80% for this study’s seven conditions was similar to the frequency/amplitude detection accuracy of 78% for six conditions.7 This indicates that pulse rate modulation can be a viable option for conveying vibratory feedback when trying to avoid the effects of tactor motor restraint on vibration frequency9 and obtrusive tactor sizes, as when stacking multiple tactors coaxially to activate different vibration amplitudes.7
When the sinusoidal waveform was used, subjects were better able to detect changes in vibration. Sinusoidal wave patterns often occur in nature, including ocean waves, sound, and light waves. Therefore, people may be more accustomed to detect it. However, more research is needed to understand the perception of vibration waveforms on upper arms because this study was limited to seven subjects.
Subjects, on average, became desensitized to the change in pulse rate after 66 seconds. Because skin sensitivity varies with each patient, further studies with a larger patient population would be useful. Future applications for modulated vibratory feedback for upper-limb prosthetic control should consider pausing stimulation after about 30 seconds to allow the user to regain sensitivity to the stimulation.
While using the optimal tactor placement and waveform for the subject, the gripping accuracy tests were performed. Only the 60% of maximum force target showed an improvement with vibratory feedback. Chatterjee et al.3 also found an improvement in grasping force accuracy with vibratory feedback when targeting 60% of the subject’s maximum gripping force but not at 40% or 80%. However, because the study of Chatterjee et al.3 did not report the pulse rate range used to modulate the vibratory waveform, it is difficult to know if their subjects were experiencing similar vibratory feedback at the 60% of maximum force level. In the current study, all subjects were trying to target approximately 3.3 Hz for 40% of maximum force, 4.9 Hz for 60% maximum force, and 6.5 Hz for 80% of their maximum force. The study results indicated that 4.9-Hz modulation was the easiest for subjects to recognize and target.
There were several study design limitations. First, the pulse rate discrimination study involved increasing the pulse rate by 1 Hz instead of increasing and decreasing by different pulse rate increments. By both increasing and decreasing the pulse rate by different amounts, the direction of the pulse rate change and minimum detectable pulse rate change could have been evaluated. However, this would have dramatically increased the number of tests and therefore increased subject burden and the potential for skin desensitization to interfere with results. However, now that this study has determined that lower pulse rates are easier to distinguish and that a greater than 1-Hz change is needed for greater discrimination, future studies can explore the minimum detectable change in pulse rate with a fewer number of testing conditions.
The second study design limitation is that this study tested grip accuracy rather than observing successful manipulation of heavy, brittle objects. This is because it would have required a greater engineering effort to add pressure sensors to brittle objects or to the prosthetic hand for this type of study. Therefore, this study added a pressure sensor to a durable object and measured grip accuracy. This proof-of-concept study provides justification to pursue the greater engineering effort of adding pressure sensors to prosthetic hands. The next step would be to add pressure sensors to prosthetic hands and perform real-world gripping tasks with and without vibratory feedback.
There were several study limitations that may have prevented vibrotactile feedback from helping subjects achieve more accurate grasps at the other target forces. First, in an effort to minimize subject burden, subjects started the grasp accuracy testing without previous training with the vibrotactile system. Set 3, where subjects could see how much grip force they were applying from the computer bar graph while also receiving vibrotactile feedback, was the only opportunity subjects had to relate the vibration stimulation to their known grip force. At the 80% of maximum grip force level, subjects did get better at using vibrotactile feedback on the second day of testing; however, this improvement did not exceed gripping accuracy of the control sets. Because past studies have shown that training can enhance the effective use of vibrotactile feedback in achieving accurate grip forces,3,13 future studies should give subjects more time to practice before measuring grip force accuracy.
Another limitation was that two of the subjects had myoelectric controls that were not working optimally. There was either a long delay between muscle activation and movement of the myoelectric hand or no response from the myoelectric hand. When it was obvious that the hand was not responsive, a trial was repeated. However, there may have been some trials where minor delays occurred between activation and hand movement. Because all uncharacteristic delays may not have been obvious to the investigators, some trials may have been recorded that should have been repeated. However, inaccuracies due to faulty myoelectric controls should have affected each set similarly. Although only two subjects had clearly faulty myoelectic controls, nearly all of the subjects learned for the first time that there was a lag between their applied myosignal and grip force that was apparent only when using haptic feedback (visual bar graphs or vibrotactile). This lag in response time seemed to contribute to undershooting and overshooting the target force, resulting in oscillations near the target force. Therefore, a positive outcome to this limitation was that vibrotactile feedback in a prosthesis may also help users identify degradation of their myoelectric controls so that corrective action can be considered. Indeed, a previous study demonstrated that vibrotactile feedback improved object manipulation when myoelectric hand controls displayed random lags in response time (feed-forward uncertainty).6 Further work in the field needs to incorporate improved coupling between the myoelectric controls and actuation of the prosthetic hand.
Although care was taken to minimize muscle fatigue, this still may have been a factor in the last sets with vibrotactile feedback. Subjects commented that the 80% of maximum force level was difficult to achieve and maintain for the 5-second trial duration. However, muscle fatigue should have been less of an issue in this study than in a previous study that used 10-second trial intervals.3 In Chatterjee et al.,3 improved grip accuracy was demonstrated only in able-bodied experienced users of vibrotactile feedback. The current study demonstrated similar results but in upper-arm prosthetic users who did not have previous experience with vibrotactile feedback. This indicated that reduced trial time and optimizing tactor location and vibration waveforms may have resulted in improved gripping accuracy with vibrotactile feedback for prosthetic users inexperienced with vibrotactile feedback.
Although the results of this study are promising for developing vibrotactile feedback for upper-limb prostheses, further refinement of the vibrotactile feedback needs to occur so that users can interpret the feedback more accurately. In addition, the tests performed in this study were done in laboratory environment. To evaluate effectiveness, real-world human subject trials that involve daily living activities need to be conducted.
Participants found vibratory feedback to be useful in controlling grip force at the level of 60% of their maximum grip force. A level of 60% is equivalent to 14–18 lb of grip force and would be needed to pick up a wine bottle or some of the thicker plastic water bottles. More training with vibratory feedback would help an amputee’s ability to improve overall control as demonstrated at 80% of the maximum grip force level. Further improvements to the interpretation of vibratory feedback may occur if 1) amplitude and/or carrier frequency along with pulse rate of the vibratory feedback simultaneously increased with increasing force and 2) vibration sensation increased in steps with distinct force levels such as light, medium, and strong grip force rather than a unique vibration sensation with all unique forces. Vibratory haptic feedback using pulse rate modulation seems to be a promising technique in controlling grip force of upper-arm prostheses.
This work would not have been possible without valuable input from the subjects who participated in the study.
© 2014 by the American Academy of Orthotists and Prosthetists.