Feedback Simulation for Acupressure Training and Skill Assessment

Abstract

Introduction 

Previous acupressure studies have yielded varying results. This could be due to differences in the amount of pressure applied to the acupressure point (acupoint) by study personnel within a study as well as between studies. Standardizing the level of pressure applied at an acupoint could improve clinical care and future research.

Methods 

As part of an ongoing randomized clinical trial of postoperative acupressure, five trainees were asked to perform 2 minutes of acupressure and light touch sessions on a simulator. The applied weight was recorded every minute. Individual skill assessment was performed using cumulative sum analysis. Six pretraining and 20 posttraining measurements in each acupressure and light touch group were compared with an expert's simulation values.

Results 

Before training (baseline), there was significant difference in applied weight (grams) between the expert [5705 (636)] and five trainees [2998 (798), P = 0.004]. Four of the five trainees crossed the lower decision limit assessing proficiency in the acupressure group, and all five trainees were successful in the light touch group. The trainees' average number of measurements needed to cross the lower decision limit (H₀), that is, defining that an individual failure rate does not statistically differ from the acceptable failure rate, was 21.3 measurements for acupressure. After this feedback simulation, trainees' scores showed no significant difference (P > 0.05) when assessed against the expert.

Conclusions 

Feedback simulation for acupressure training and skill assessment, evaluated by cumulative sum analysis, may help in improving the standardization of acupressure therapy performed during clinical practice or research.

Application of pressure to specific external sites on the body (acupressure) has been practiced for hundreds of years, particularly in the Western Pacific Region, as a component of traditional medicine.¹

Similar to acupuncture,^2–4 acupressure therapy has several physiological effects.^5–7 Applying pressure on defined cutaneous acupoints¹ can lead to improvements in symptoms such as postoperative nausea and vomiting,⁸ pain,⁹ and sleep disturbances¹⁰ with few potential adverse effects. Studies on acupressure in the clinical setting have shown varied results.⁸ This heterogeneity may be due to differences in pressures applied to the acupoints across these studies. Standardizing the level of pressure applied to an acupoint could improve teaching and research of acupressure therapy.

Simulation-based training seems particularly suited to train for such technical skill as acupressure therapy. Convergent evidence shows that simulation-based medical education can improve healthcare based on learner competences.¹¹ Considering pressure application, technologically enhanced simulation may be efficient in improving this technical skill and translate to patient care.¹² Simulation based on pressure monitoring using a weighing scale has been reported to be an efficient learning setting for a cricoid pressure maneuver¹³ and could be of interest for acupressure therapy. Cumulative sum (CUSUM) analysis is a novel method of assessing individual skill performance in behavioral tasks.¹⁴ By graphically plotting individual results, it allows for detection of fulfillment of or deviation from an established acceptable standard. Here, we describe a feedback simulator-based training program, including CUSUM analysis, for individuals being trained to perform acupressure, as part of an ongoing randomized clinical trial of postoperative acupressure.

METHODS

Participants

With approval from the Stony Brook University Institutional Review Board (study protocol #867005-3 approved on March 9, 2016), we trained study personnel to apply acupressure as part of a randomized controlled trial of acupressure to improve patient satisfaction and quality of recovery in hospitalized patients (randomized controlled trial of Acupressure to imProve Patient satisfaction and quality of RecOVEry in hospitalized patient “the APPROVE study”). Six research team members participated in this training plan; one member was a certified expert in acupuncture and acupressure (XG) and five members were being trained to perform the study intervention. Our acupressure expert is a board-certified anesthesiologist who is also licensed in acupuncture and acupressure. He has been a licensed practitioner of acupuncture and acupressure for the past 10 years. He started acupuncture and acupressure consultation at the university hospital in 2006 and opened its dedicated acupuncture and acupressure clinic in 2012. In addition, he has been teaching acupuncture and acupressure since 2006. The five learners are clinical researchers, four of them with no previous background in acupressure therapy and one who is a licensed massage therapist. The protocol was designed to be consistent with the Revised Standards for Reporting Interventions in Clinical Trials of Acupuncutre (STRICTA): Extending the CONSORT Statement.¹⁵

Simulator Settings

The acupressure simulator consisted of a platform scale (Oxo, Chambersburg, Pa; range, 0–22 lb) draped with a skin-colored wound dressing with an acupoint marked on the wound dressing surface. The use of a weighing scale for training standardized force application has been previously validated.^13,16–18 The scale was placed on a table edge and tarred before each session (Fig. 1) to allow for a thumb angle similar to the clinical setting for hand and forearm acupoints. The digital screen of the weighing scale was masked from the expert and the trainees except during self-regulated practice.

FIGURE 1: Acupressure simulator setting.

Measurements

The expert and five trainees were asked to perform two levels of pressure. The first pressure level was “true” acupressure, defined as 2 minutes of steady, manual pressure applied on the marked location (Fig. 1). This pressure was used in the active intervention arm of the clinical trial. The second pressure level was defined as 2 minutes of steady, manual light touch applied on the simulator marked location. This pressure was used in the sham (control) arm of the clinical trial. At least 1 minute of rest was allowed between each intervention. During every 2-minute intervention, the participant was blinded from the weighing scale's digital screen, and the weight measured from the scale was recorded every minute (ie, 2 measurements per intervention were taken).

First, the expert was asked to perform ten 2-minute acupressure interventions and ten 2-minute light touch interventions, and the measured weight was recorded after each minute (ie, 2 times per intervention). These measurements allowed us to calculate the reference value success range.

Secondly, the five trainees were asked to follow the following training plan (Fig. 2):

FIGURE 2: Graphical abstract of individual training plan.

1. Baseline (pretraining): Each participant performed the 2-minute acupressure interventions 3 times and the 2-minute light touch interventions 3 times. The measured weight was recorded after each minute (ie, 2 times per intervention).
2. Self-regulated practice: After baseline assessments were performed, the trainees were informed of the reference values (expert's weight measurements) and encouraged to partake in 15 minutes of self-regulated learning on the scale with full access to view the digital screen.
3. Frequently repeated feedback sessions: Each participant then had to perform the 2-minute acupressure intervention 10 times and the 2 minutes light touch interventions 10 times. The measured weight was recorded after each minute (ie, 2 times per intervention). After each measurement recording (every minute), feedback about the measured weight was provided to the trainee (frequently repeated feedback). After these 20 interventions were completed, if the trainee's results did not fulfill the statistical criteria for skill proficiency, they were asked to perform additional 2-minute intervention in blocks of 5 times, until they fulfilled the efficiency criteria (see data analysis).

Statistics

All analyses were performed by the trial's statistician using SAS 9.3, Cary, NC.

Pretraining Monitoring

Trainee baseline measurements were compared with the expert's simulation values using a mixed linear model, with the group (trainee or expert) as a fixed effect and subject ID as a random effect.

Cumulative Sum Analysis for Individual Skill Assessment

The CUSUM curve is a sequential analysis technique, originally developed as a way to measure quality control over time.¹⁴ This method of analysis has been applied to measure the learning process and subsequently evaluate performance of a clinical skill over time. In the most basic sense, each subject performs a skill numerous times, and each performance is assessed as either a success or failure. Once a certain number of the subject's attempts are considered “successful” (relative to the number of failures), the subject can be deemed “proficient” in performing the skill.

The CUSUM analysis technique requires a set of predefined factors to determine both success and proficiency, including an acceptable failure rate and unacceptable failure rate, as well as level of acceptable type 1 (α) and type 2 (β) errors (Table 1). Additional CUSUM plotting parameters can be calculated from these predefined factors. When a subject succeeds at performing the skill, success is indicated on the CUSUM plot as a downward deviation toward the acceptable failure rate benchmark (H₀). This numerical CUSUM success value (S), that is, the amount that is subtracted from the subject's CUSUM total, is derived from the acceptable and unacceptable failure rates. On the other hand, when a subject fails at performing the skill, failure is indicated as an upward deviation toward the unacceptable failure rate benchmark (H₁). The CUSUM numerical value added to the subject's total is defined as 1 − S. Once the subject's CUSUM value crosses the acceptable failure rate, the subject's performance is considered to be no different from an acceptable failure rate and thus presumed to be performing the skill proficiently. Conversely, once the subject's CUSUM plotting crosses the unacceptable failure rate H₁, the subject's performance is considered to be different from acceptable failure rate. Within these two limits, no conclusion can be made regarding individual performance difference from acceptable failure rate.

TABLE 1: Cumulative Summary Calculated Fields

For our training program, the most stringent acceptable and unacceptable failure rates were determined by the expert's simulation performance. As a result, we chose an acceptable failure rate (P₀) of 15% and an unacceptable failure rate (P₁) of 30% (CUSUM characteristic calculations are reported in the Table 1). As per previous study recommendations,¹⁹ α and β levels were both set to 0.1. Using these metrics, the a priori number of initial consecutive successful measurements needed to cross H₀ was 12.

The reference values success range was predetermined as (2) SDs on either side of the expert's mean values for both acupressure interventions and light touch interventions, respectively. Each trainee simulation measurement (2 per simulation intervention) was then classified as either a success or failure and plotted in the CUSUM plot.

Postproficiency Analysis

To test whether “postproficiency” measurements were different from the expert's values, measurements for each trainee below the H₀ line were compared with the expert's simulation values using a mixed linear model, with group (trainee or expert) as a fixed effect, and subject ID as a random effect.

RESULTS

Acupressure

There was a significant difference between the expert's simulation scores and trainees' pretraining scores [expert mean (SD) = 5705 (636) g, trainee mean (SD) = 2998 (798) g, P = 0.004] (Table 2). The expert's success range for acupressure was 4433 to 6977 g. Cumulative sum charts for the acupressure simulation are presented in Figure 3. Four of the five trainees had to perform at least one additional block of five procedures to cross the lower decision limit (Table 3). The trainees' average number of measurements needed to cross the lower decision limit (H₀), that is, defining that an individual failure rate does not statistically differ from the acceptable failure rate, was 21.3 measurements. One trainee did not cross the H₀ threshold and did not perform acupressure on patients in the randomized clinical trial. Of the other four trainees, after crossing the threshold, each trainee had between five and ten proficient measurements [total number of proficient measurement = 29, mean (SD) weight of proficient measurements = 5254 (699) g]. Trainee proficient scores showed no significant difference when assessed against the expert's scores (P = 0.10) (Fig. 5).

TABLE 2: Trainee and Expert Comparisons for Acupressure and Light Touch Baseline (Pretraining) Values

FIGURE 3: Cumulative summary plot for acupressure simulation exercises.

TABLE 3: Summary Results for Acupressure and Light Touch Simulations

FIGURE 5: Expert and trainee baseline and postproficient measurement comparison. A and C, Mixed linear models showed a significant difference between the trainee group's baseline values and the expert's values for both acupressure (P = 0.002) and light touch (P = 0.02). B and D, However, after trainees performed well enough to cross the H ₀ line, postproficient measurements were not significantly different from the expert's values for acupressure (P = 0.10) and light touch (P = 0.12).

Light Touch

There was a significant difference between the expert's simulation scores and trainee pretraining scores [expert mean (SD) = 42.9 (18.5) g, trainee mean (SD) = 373.0 (384.0) g, P = 0.02] (Table 2). The expert's success range for light touch was 5.9 to 79.9 g. Cumulative sum charts for the light touch simulation are presented in Figure 4. One trainee needed an additional block of five procedures to cross the lower decision limit (Table 3).The trainees' average number of measurements needed to cross the lower decision limit (H₀), that is, defining that an individual failure rate does not statistically differ from the acceptable failure rate, was 14.8 measurements. After crossing the threshold, each trainee had between five and nine proficient measurements [total number of proficient measurements = 41, mean (SD) weight for proficient measurements = 32.76 (19.36) g]. Trainee proficient scores showed no significant difference when assessed against the expert's scores (P = 0.12) (Fig. 5).

FIGURE 4: Cumulative summary plot for light touch simulation exercises.

DISCUSSION

The results of this analysis suggest that frequently repeated feedback simulation with CUSUM analysis can be useful to train individuals to standardize the amount of force applied when performing acupressure. Our low fidelity simulation setting, based on the cricoid pressure training studies,¹³ is a novel application in the field of acupressure therapy. It also emphasizes how simulation could improve healthcare trials methodology by standardizing the studied intervention. The CUSUM approach can usefully be combined with the simulation setting to assess learner proficiency in such technical skills and is a promising statistical tool for the field.

To our knowledge, no previous study has focused on interindividual variation in applied pressure when performing acupressure. This variation could contribute to the heterogeneity of results observed in clinical trials of acupressure. For example, in the Cochrane systematic review and meta-analyses including PC6 acupoint stimulation⁸ on nausea, the I2 of the analysis was 89%, which is indicative of high heterogeneity. One limitation of standardizing the force applied during acupressure is the interindividual variability in consideration to patient pressure sensitivity. However, to improve quality of care and research, we believe that variability of the provided force should be minimized.

This simulation-based training model using frequently repeated feedback seems suited to improve individual skill in standardizing the applied force for acupressure interventions. A meta-analysis by Cook et al¹² showed that technology-enhanced simulation training for healthcare professionals is efficient in improving knowledge, skills, and behaviors. Our proposed training setting may be efficient in improving a behavior skill because it encompasses several recommended learning strategies²⁰ including providing feedback, practicing repeatedly, providing multiple learning methods (ie, self-regulated and instructor-regulated), maintaining a controlled environment, individualizing learning, and defining outcomes. However, this simulation setting was only designed to train for pressure application, not acupoint location identification. The location of the acupoints on the APPROVE study patients were marked according to the World Health Organization definitions¹ using a sharpie pen before randomization to improve the standardization of this study procedure.

The CUSUM analysis is a statistical method based on the outcome of a procedural or behavioral process. It graphically represents the cumulative trend of an individual's performance for a process. By crossing the a priori defined lower decision limit H₀, the CUSUM graph statistically assesses that the individual failure rate does not statistically differ from the a priori defined failure rate.¹⁴ Considering the interindividual variation involved in fulfilling technical or nontechnical skills proficiency,^19,21 the CUSUM approach offers an interesting way to assist in decision making for training purposes.

This report of our training program has several limitations. First, the expert reference standard's acceptable threshold was based on one expert's practice. However, we did not aim to assess a universal optimal acupressure force, but rather to propose a way to standardize a chosen force application. This standardization may allow for external reproduction of our trial's results. In addition, our target force was not considered a mandatory force, but rather the target force to be applied during our acupressure intervention unless this level of force was painful in which case the highest force tolerated was used. Another limitation could be the low number of trainees. However, taking into consideration the CUSUM approach, each trainee was considered as their own control, and increasing the number of trainees would not have increased the power of the individual calculations. We did not directly measure pressure but rather weight. However, on the basis of the available literature, for example, cricoid pressure simulation,^16–18 we made a first-degree approximation that under constant gravity and similar finger surface area between team members, weight measurements offer an acceptable surrogate (1000 g corresponding to 9.8 N¹⁷). Another limitation of our study is that we did not assess retention of the training. Participants were given feedback from previous attempts during the simulation session, which is different from the clinical setting. This could result in a difference between these observed results and clinical practice. We still recommend external validation of our results to confirm the mean required number of attempts to cross the lower decision limit to help design future such training plans.

CONCLUSIONS

Frequently repeated feedback simulation for acupressure training and skill assessment evaluated by CUSUM analysis may help in improving the standardization of acupressure therapy performed in clinical practice or research. This educational setting could be also applied to other force-based technical skills such as cricoid pressure during induction of general anesthesia.