Total knee arthroplasty (TKA) is effective for relieving pain and improving function in patients with end-stage arthritis of the knee.1–4 However, a complete recovery may not be achieved, with up to 20% of patients with a certain degree of dissatisfaction after TKA5,6; pain, stiffness, and weakness are among the leading causes of this dissatisfaction.7 Weakness, which is often already present before surgery, still remains after surgery for long time; differences have been shown between subjects after TKA surgery and healthy adults of the same age.8 These following functional deficits may have major consequences for the patients from a clinical standpoint: reduced walking speed and balance, difficulties with stairs, and increased risk of falls.9
Neuromuscular electrical stimulation (NMES) has been proposed as an adjunct to traditional rehabilitation programs for patients after TKA, especially for those with deficit of the voluntary activation of the muscle and postoperative weakness.10,11 The contractions induced by electrical stimulation can lead to efficient muscle training with a greater and more selective recruitment of type II muscle fibers than voluntary contractions.12,13 In addition, the inputs generated by NMES may facilitate plastic changes in the networks of the sensitive-motor neurons at the level of the central nervous system, thus leading to strengthening of the signals and pathways dedicated to the muscle control and strength.14,15
The primary purpose of this meta-analysis was to analyze the randomized controlled trials reported in literature that compared the results of a traditional rehabilitation program after TKA with a rehabilitation program implemented with the NMES.
MATERIALS AND METHODS
Literature Search Strategies
The methodology of this study complies with preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines of 200916 (Appendix A).
The studies included in the meta-analysis were required to meet the following inclusion criteria: (a) studies evaluating the efficacy of NMES in patients undergoing unilateral TKA; (b) patients: subjects of both sexes at least 18 yrs; (c) intervention: traditional rehabilitation associated with postoperatory NMES (for a minimum duration of 4 wks) in comparison with traditional rehabilitation alone; and (d) outcomes and study design: studies with clinical outcome clearly defined “a priori,” controlled randomized studies and presence of a control group. “English language” and “the last 16 yrs” have been established as limits for the research.
A standardized data extraction was used for data collection and Excel program was used for archiving and analysis. The characteristics of the selected studies have been extracted and are the following: publication date, demographic characteristics of participants (mean age, percentage of females, mean body mass index), studies that provide an intervention on patient, sample sizes (number of subjects in the intervention and control groups), the study design, the type of action taken (commencement, duration, and frequency of treatment; intensity, frequency, duty cycle, and pulse duration; electrode size), outcomes used by individual studies, and the period of follow-up.
Two independent reviewers performed the whole process; discrepancies were resolved by the intervention of a third reviewer. Authors were contacted to obtain missing data.
Analyses of Potential Bias
According to the Cochrane Handbook for Systematic Reviews of Interventions, the overall risk of bias was determined considering to the following domains: generate random sequence, concealment of allocation, blind setting of participants and staff, blind assessment of the results, incomplete data, and studies conducted by device companies.
The synthesis of the quantitative data was performed using electronic tables in Microsoft Excel specially designed for creating meta-analysis. The tables were made following the guidelines provided by the Creative Commons Attribution License and published on Bio Med Central Research Notes.17
When data on at least three studies were available, a meta-analysis was carried out. Heterogeneity across studies was quantified using I2 statistics and tested with Q Cochran's statistics. Because of a substantial heterogeneity, the pooled mean difference was estimated along with the corresponding 95% confidence intervals, using the random-effect model weighting scheme. To assess publication bias, funnel plot techniques, Begg's rank test, and Egger's regression test were intended to be used, as appropriate given the known limitations of these methods. Statistical analyses were carried out with R, Version 3.2.5.
This study conforms to all PRISMA guidelines and reports the required information accordingly (see Checklist, Supplemental Digital Content, https://links.lww.com/PHM/A514).
The PRISMA flow diagram reports the assessed items and the reasons for exclusions (Fig. 1): six randomized controlled trials reflected the inclusion criteria and were included in the systematic review,18–23 whereas four randomized controlled trials have been included in the meta-analysis. From the starting 36 studies identified through the search in the databases, six studies have been excluded after duplicates removed and abstract/title evaluation because the main topic was not in line with inclusion criteria. The study of Stevens-Lapsley et al. (2012)24 was excluded because it was a substudy of the work done by the same team.21 Similarly, the study of Laufer et al. (2010)25 was excluded because it formed a substudy of the work done by Petterson et al.22 Three case reports10,11,26 and a case series27 have been excluded for the design of the study. A clinical trial28 has been excluded because it evaluated the use of NMES only before surgery, whereas another study was excluded because it compared the outcomes between multipath NMES and conventional NMES.29 From the remaining 22 studies, sixteen studies have been excluded after full-text evaluation.
Characteristics of the Studies
Characteristics of Participants
The size of the studies included ranged from 30 to 200 participants involving a total of 496 patients. Approximately half of the patients followed a standard rehabilitation protocol (control group) and the other part followed a rehabilitation program plus a session of NMES (NMES group). A number of 250 patients were randomized in NMES group, whereas 246 were randomized in the control group; follow-up periods ranged between 4 and 52 wks. Overall, the median age was 67.2 yrs, with 69.9% of women and a mean body mass index of 29.4. In the trials included in the meta-analysis, patient selection was controlled with particular attention to the criteria listed previously, including only relatively healthy subjects who had knee replacement surgery for unilateral arthritis of the knee. In these studies, patients who had significant pain in other joints of the lower limbs, other pathologies (uncontrolled diabetes, etc.), or cardiovascular diseases have been excluded. An exception to the criteria mentioned previously was the study of Levine et al.20 where patients with knee arthritis and bilateral pain were included. In this study, the patients with co-morbidities such as epilepsy, lower limb ischemia, or decreased cognitive function were not included. Table 1 reports the demographic of the participants and the characteristics of the studies.
Characteristics of the Procedures
The characteristics of the NMES among the various studies were similar, although some variations of parameters were present. The frequencies of stimulation in all studies were sufficient to induce a tetanic contraction of the muscle, with variations between 40 and 100 beats per second (hertz). The duration of the impulse applied to the muscle stretches between 250 and 400 μs. The waveform current was asymmetrical biphasic, asymmetrical sinusoidal, biphasic, or biphasic symmetrical synchronized one. All studies used the maximum intensity tolerated by the patient. On the contrary, the duty cycle (ratio between the time when the machine dispenses the impulse and time of rest) showed considerable variations between different studies. In most studies, both groups (control group and NMES group) followed a rehabilitation protocol that included strength exercises, flexibility, and functional type workouts. The NMES groups performed in addition NMES sessions as shown in the Table 1.
Description of Bias
Table 2 summarizes the quality and the risk of bias of all individual randomized controlled trials included in the meta-analysis. The overall risk of bias is high for the nature of this kind of study, because most of the included randomized controlled trials were at high risk or uncertain risk of bias in these domains (participants and evaluators blinding). Nevertheless, the quality of the included studies was acceptable, in particular if considering that all studies were randomized, that the quality of the data was good and complete, that this was not a drug therapies study, and that is this kind of studies where blindness can be complicated. The study of Petterson et al.22 had the lowest risk of bias, and it was a single blind study, which responded positively to all the questions examined, with the exception of concealment of allocation, which was not mentioned. On the contrary, the study that proved a major risk of bias was that of Stevens-Lapsley et al.21; in this study, the evaluators were aware of the type of treatment received by participants, and plus, this was the only study that received economic support.
In the meta-analysis, publication bias was not assessed because there were inadequate numbers of included trials to properly assess a funnel plot or more advanced regression-based assessments. Figures 2 and 3 report the forest plots and Table 3 reports the data of the Mental Component Score and of the Physical Component Score of the 36-Item Short Form Health Survey at 12 wks, which are the only items where comparable data were available in at least four studies. For both meta-analyses, heterogeneity is significant (Q statistics P values = 0.04, with total heterogeneity equal to 60% for both analyses, suggesting substantial heterogeneity).30
The absence of a defined protocol involved that, among the various studies, many different outcomes were used to evaluate the efficacy of NMES. Table 4 reports all the outcomes described in the studies, divided into two groups. The first group of outcomes evaluated the physical performance of patients during the rehabilitation phase; aim of these outcomes was to describe appropriately the physical adaptation of the patients to the prosthesis and how this affects the everyday performances such as walking, getting up, sitting, and climbing stairs. The second group included outcomes whose primary purpose was to provide an estimate of the overall functionality of the knee from the patient perspective. These are well-known questionnaires about social and emotional functions, pain and disability, and how these affect the subject's life.
Analysis of the Results
In the first study of Avramidis et al.,18 the participants, randomized in two groups (NMES and control), were assessed before surgery and at 6 and 12 wks after surgery. The NMES group presented the longest distances covered in the 3-min walking test than the control group with a statistically significant difference, both at 6 wks (P = 0.0002) and at 12 wks (P < 0.0001). On the contrary, the differences measured between the two groups regarding the Physiological Cost Index and Hospital for Special Surgery Knee Score were not statistically significant (P > 0.2).
Petterson et al.22 evaluated the participants preoperatively and then at 3 and 12 mos after surgery and did not show significant differences (P > 0.08) for all examined outcomes between the two types of interventions (physical exercises or physical exercises associated with NMES). These two groups of participants, however, showed better results than a third control group of 41 patients who followed a standard rehabilitation protocol (“standard of care”). The outcomes assessed at 12 mos after the intervention showed a statistically significant difference (all P < 0.01).
Stevens-Lapsley et al.21 showed a statistically significant improvement (P < 0.05) after 3.5 wks in favor of the NMES group regarding the following outcomes: strength of the hamstrings and of the quadriceps femoris, functional performance 6-min walking test, stair-climbing test, test up and go, and range of movement for active extension. These differences were still present, although reduced after 52 wks.
In the study of Avramidis et al.,19 the Knee Society Score showed statistically significant differences in the functional and total score at 6 and 12 wks after surgery. The other used outcomes (Oxford 12-item knee score and 3-min walking test) also emphasized that the two groups had statistically significant differences (with better results for the NMES group) in the early postoperative period, which gradually declined over time.
Levine et al. (2013)20 compared the two groups using the following six different outcomes: degree of extension, the degree of flexion, Knee Society Score (functional and related to pain), Western Ontario and McMaster Universities Arthritis Index, and Timed Up and Go test. The parameters were examined before surgery, at 6 wks, and then at 6 mos. The study showed that the NMES group was not less than the control group for any considered outcomes, at both intervals.
Demircioglu et al. (2015)23 showed a significantly better visual analog scale scores at the first and third months after TKA in the NMES group than in the control group (P = 0.0), although knee flexion and extension ranges were significantly better only at the first-month follow-up visit (P = 0.05). The comparisons of Western Ontario and McMaster Universities Arthritis Index results at month 1 revealed that pain, stiffness, and total scores of the NMES group were significantly better than those of control group, and significantly better physical function and 36-Item Short Form Health Survey subscales, except mental health, were found for the NMES group at the first month of follow-up.
The origin of the weakness of the quadriceps muscle after TKA is multifactorial; it can be related to specific causes such as the trauma of the surgery and factors associated with the pre-existing knee arthritis, but also, it can be associated with general factors such as obesity, morbidities, or age. The mechanism that underlies the weakness can be of muscular nature (e.g., atrophy, decrease in the number, and size of muscle fibers) or related to neuronal causes (e.g., reduced voluntary muscle activation). The contribution of the neuronal type seems to be prevalent respect to the muscular one, especially in the first postoperative weeks after TKA; on the contrary, after approximately 4–5 mos, the muscular component takes over.9 Because weakness can be a severe impairment for the recover after TKA, NMES has been proposed in addition to standard rehabilitation programs.
This study included six randomized controlled trials performed from 2003 to 2015, which together collected data on 496 patients undergoing TKA. All studies evidenced a deficit of strength and functionality in the first period after surgery, compared with data collected before the surgery—this was an effect of surgery itself. These deficits receded during the follow-up, also in response to the rehabilitative treatment, and after only 6 mos, the results were better than before surgery. All patients had excellent functional outcomes and physical performances, thus confirming the good results achieved at present with TKA and the subsequent rehabilitation.
Concerning on the use of NMES after TKA, the results of a recent systematic review31 and a critical review32 were similar to ours. Volpato et al.31 was inconclusive about NMES efficacy and suggested further evidence to support or deny its use after TKA; the data suggested that the postoperative treatment can improve the femoral quadriceps function, but concerns remained because of the low-quality evidence of the studies. However, their data analysis showed no inferiority of the rehabilitative treatment with NMES compared with physical therapy alone. Kittelson et al.32 underlined how the examined studies differed substantially in methodology for timing, duration, treatment volume, and intensity of NMES interventions. Their purpose was to synthesize the current state of evidence for NMES in post-TKA rehabilitation, and they concluded that high-intensity NMES performed regularly during the immediate postoperative phase helped attenuate dramatic losses in quadriceps strength after TKA, thereby resulting in overall improvements in strength and function.
In this review, it must be underlined that the absence of a standardized common rehabilitative protocol in the six studies analyzed prevented a uniformity of outcomes. Related to this heterogeneity, outcomes were then divided into two categories: those based on tests that evaluated physical performance and those that evaluated the functionality of the knee arthroplasty and the quality of life. This approach allowed to gather and analyze the homogeneous data properly and to extract the data that best reflected the real situation. On the other side, persons who during the rehabilitation period benefited from the NMES in addition to normal physical therapy got higher scores compared with those who received physical therapy alone (in particular, the tests that best expressed this difference were the test up and go, stair climbing test, and 6-min walking test).
One of the findings of this meta-analysis was that the differences highlighted by one or several outcomes were strong in the first few weeks/months after surgery and then gradually waned with passing of the time. It has been hypothesized that the long-term advantages are not significant because the patients who did not use the NMES recover muscular strength with the standard rehabilitation and the normal use of the knee, thus reaching the level of the patients who underwent NMES treatment after a certain period. Nevertheless, NMES is useful to gain muscular strength immediately after surgery. This situation was related to the fact that in the early postoperative period, there is an important deficit of muscle activation of neuronal type (or central). Recently, it has been demonstrated that there is a relationship between the use of NMES and changes in central nervous system; particularly, NMES activates regions of the cortical sensorimotor network (primary sensorimotor cortex, premotor cortex, supplementary motor area, and secondary somatosensory area).33 It leads to a neuroplastic reorganization of the cortex through the activation of peripheral sensory neural axons that send afferent signals from the stimulated muscle to the central nervous system, improving the voluntary activation.34,35 Moreover, it has been demonstrated that NMES current intensities above motor threshold increase corticospinal excitability, whereas NMES current intensities at sensory threshold decrease corticospinal excitability.35,36 For this reason, a treatment with a device that allows muscle activation, such as NMES, was essential to prevent muscle atrophy and dysfunction. The involvement of the nervous system during NMES has been studied also by Maffiuletti13 and Golaszewski et al.,14 who refer to the activity of the cortical areas; nevertheless, we could not find any specific reference about NMES after TKA and its relationship with central nervous system.
Another evidence of the study concerns the optimal time for initiation of NMES. Petterson et al.22 did not show any statistically significant difference between the NMES group and the control group, but this probably was a consequence of the fact that patients have begun treatment 4 wks after surgery, whereas in the other five studies, the NMES started after 24–48 hrs. Adding treatments with NMES to conventional rehabilitation is therefore to be considered; the more effective the more the shortfall of the muscular voluntary activation is pronounced, just like happened exactly during the early postoperative period.
A difference between the studies regarded the timing and the regularity of the application of the NMES. In the study of Petterson et al.,22 NMES was applied to the patients 2–3 times per wk for approximately 15 mins each time, whereas in the studies of Avramidis et al.18,19 and Stevens-Lapsley et al.,21 patients were instructed to apply by themselves. These different approaches led to widely differing treatment volumes between the studies. Clearly, the intensive treatment on a daily basis was more effective than the short biweekly sessions.
The intensity of the electrical stimulation also played a pivotal role in the effectiveness of the treatment. All studies used the maximum intensity tolerated by the patient; nevertheless, it has been noted that in certain cases, the NMES may activate the nociceptive receptors, causing discomfort that can limit the effectiveness of the treatment. Probably, the use of electrodes with a large surface of skin contact can be a solution to this problem, because of the possibility to reduce the current density (ratio of current and contact surface).
Most studies reported that the NMES allows a better functional recovery after TKA surgery, particularly for individuals with a pronounced deficit of muscular activation of neuronal type; nevertheless, we could not find a statistical significant difference after the meta-analysis. In addition, no adverse effects have been correlated with the treatment with NMES, which has proven to be safe and free from risks to the health of the patient. A cutaneous reaction due to the adhesive of the electrodes, with consequent discomfort reported by some individuals, has been the only adverse effects reported.19 The application of NMES did not affect the integrity and the stability of the arthroplasty. It is true that the differences tend to diminish after a certain time, but first is the control group that reaches the values of the NMES group and not this one, which loses the “advantage.” Second and the most important, the faster is the recovery, the better are the results in the long time for the patients. The compliance to treatment was remarkable because the treatment was very simple.
The major limitation identified in this meta-analysis was linked to reduced sample size of the studies (excluded the study of Petterson et al.22). Another limitation was represented by the heterogeneity of rehabilitation protocols, in particular with regard to the setting of the NMES, but especially for the timing of commencement, duration, and frequency of treatment. In addition, for the studies of Avramidis et al.18,19 and for study of Stevens-Lapsley et al.,21 a blind assessment of the outcomes was not possible because of the reduced funds, and therefore, they incurred into a detection bias.
Neuromuscular electrical stimulation after TKA is easy to apply and can be done daily by instructed persons, allowing a continuous and durable muscular training. The use of NMES is safe and does not present any risk of injury secondary to treatment or implant damage. Particularly in the immediate period after TKA, a faster functional recovery has been reported (statistically not significant), both from the point of view of the physical performance and of the joint function.
1. Bistolfi A, Lee GC, Deledda D, et al.: NexGen® LPS mobile bearing total knee arthroplasty: 10-year results. Knee Surg Sports Traumatol Arthrosc
2. Bistolfi A, Massazza G, Rosso F, et al.: Cemented fixed-bearing PFC total knee arthroplasty: survival and failure analysis at 12–17 years. J Orthop Traumatol
3. Hawker G, Wright J, Coyte P, et al.: Health-related quality of life after knee replacement. J Bone Joint Surg Am
4. Bistolfi A, Massazza G, Lee GC, et al.: Comparison of fixed and mobile-bearing total knee arthroplasty at a mean follow-up of 116 months. J Bone Joint Surg Am
5. Bistolfi A, Bettoni E, Aprato A, et al.: The presence and influence of mild depressive symptoms on post-operative pain perception following primary total knee arthroplasty. Knee Surg Sports Traumatol Arthrosc
6. Bourne RB, Chesworth BM, Davis A, et al.: Patient satisfaction after total knee arthroplasty: who is satisfied and who is not? Clin Orthop Relat Res
7. Escobar A, Quintana JM, Bilbao A, et al.: Responsiveness and clinically important differences for the WOMAC and SF-36 after total knee replacement. Osteoarthritis Cartilage
8. Bade MJ, Kohrt WM, Stevens-Lapsley JE: Outcomes before and after Total Knee Arthroplasty compared to healthy adults. J Orthop Sports Phys Ther
9. Ravi B, Nan Z, Schwartz AJ, et al.: Fall risk score at the time of discharge predicts readmission following total joint arthroplasty. J Arthroplasty
10. Mintken PE, Carpenter KJ, Eckhoff D, et al.: Early neuromuscular electrical stimulation to optimize quadriceps muscle function following total knee arthroplasty: a case report. J Orthop Sports Phys Ther
11. Petterson S, Snyder-Mackler L: The use of neuromuscular electrical stimulation to improve activation deficits in a patient with chronic quadriceps strength impairments following total knee arthroplasty. J Orthop Sports Phys Ther
12. Gregory CM, Bickel CS: Recruitment patterns in human skeletal muscle during electrical stimulation. Phys Ther
13. Maffiuletti NA: Physiological and methodological considerations for the use of neuromuscular electrical stimulation. Eur J Appl Physiol
14. Golaszewski S, Kremer C, Wagner M, et al.: Functional magnetic resonance imaging of the human motor cortex before and after whole-hand afferent electrical stimulation. Scand J Rehabil Med
15. Stackhouse SK, Binder-Macleod SA, Stackhouse CA, et al.: Neuromuscular electrical stimulation versus volitional isometric strength training in children with spastic Diplegic cerebral palsy: a preliminary study. Neurorehabil Neural Repair
16. Moher D, Shamseer L, Clarke M, et al.: Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev
17. Neyeloff JL, Fuchs SC, Moreira LB: Meta-analyses and forest plots using a Microsoft excel spreadsheet: step-by-step guide focusing on descriptive data analysis. BMC Res Notes
18. Avramidis K, Strike PW, Taylor PN, et al.: Effectiveness of electric stimulation of the vastus medialis muscle in the rehabilitation of patients after total knee arthroplasty. Arch Phys Med Rehabil
19. Avramidis K, Karachalios T, Popotonasios K, et al.: Does electric stimulation of the vastus medialis muscle influence rehabilitation after total knee replacement? Orthopedics
20. Levine M, McElroy K, Stakich V, et al.: Comparing conventional physical therapy
rehabilitation with neuromuscular electrical stimulation after TKA. Orthopedics
21. Stevens-Lapsley JE, Balter JE, Wolfe P, et al.: Early neuromuscular electrical stimulation to improve quadriceps muscle strength after total knee arthroplasty: a randomized controlled trial. Phys Ther
22. Petterson SC, Mizner RL, Stevens JE, et al.: Improved function from progressive strengthening interventions after total knee arthroplasty: a randomized clinical trial with an imbedded prospective cohort. Arthritis Rheum
23. Demircioglu DT, Paker N, Erbil E, et al.: The effect of neuromuscular electrical stimulation on functional status and quality of life after knee arthroplasty: a randomized controlled study. J Phys Ther Sci
24. Stevens-Lapsley JE, Balter JE, Wolfe P, et al.: Relationship between intensity of quadriceps muscle neuromuscular electrical stimulation and strength recovery after total knee arthroplasty. Phys Ther
25. Laufer Y, Snyder-Mackler L: Response of male and female subjects after total knee arthroplasty to repeated neuromuscular electrical stimulation of the quadriceps femoris muscle. Am J Phys Med Rehabil
26. Lewek M, Stevens J, Snyder-Mackler L: The use of electrical stimulation to increase quadriceps femoris muscle force in an elderly patient following a total knee arthroplasty. Phys Ther
27. Stevens JE, Mizner RL, Snyder-Mackler L: Neuromuscular electrical stimulation for quadriceps muscle strengthening after bilateral total knee arthroplasty: a case series. J Orthop Sports Phys Ther
28. Walls RJ, McHugh G, O'Gorman DJ, et al.: Effects of preoperative neuromuscular electrical stimulation on quadriceps strength and functional recovery in total knee arthroplasty. A pilot study. BMC Musculoskelet Disord
29. Morf C, Wellauer V, Casartelli NC, et al.: Acute effects of multipath electrical stimulation in patients with total knee arthroplasty. Arch Phys Med Rehabil
30. Higgins JP, Thompson SG, Deeks JJ, et al.: Measuring inconsistency in meta-analyses. BMJ
31. Volpato HB, Szego P, Lenza M, et al.: Femoral quadriceps neuromuscular electrical stimulation after total knee arthroplasty: a systematic review. Einstein (Sao Paulo)
32. Kittelson AL, Stackhouse SK, Stevens-Lapsley JE: Neuromuscular electrical stimulation after total joint arthroplasty: a critical review of recent controlled studies. Eur J Phys Rehabil Med
33. Smith GV, Alon G, Roys SR, et al.: Functional MRI determination of a dose-response relationship to lower extremity neuromuscular electrical stimulation in healthy subjects. Exp Brain Res
34. Hortobágyi T, Maffiuletti NA: Neural adaptations to electrical stimulation strength training. Eur J Appl Physiol
35. Chipchase LS, Schabrun SM, Hodges PW: Peripheral electrical stimulation to induce cortical plasticity: a systematic review of stimulus parameters. Clin Neurophysiol
36. Chipchase LS, Schabrun SM, Hodges PW: Corticospinal excitability is dependent on the parameters of peripheral electric stimulation: a preliminary study. Arch Phys Med Rehabil
APPENDIX A: Literature Search Strategies
Electronic databases: PubMed, Cochrane Library, PEDro
#2. Electrical stimulation
#3. Total knee arthroplasty OR total knee replacement
#4. #1 AND #2 AND #3
Limits: 2000/10/01 to 2016/10/01, Humans, English,
Meta-analysis/randomized controlled trial/controlled clinical trial/systematic reviews
PICOS approach (Patients, Intervention, Comparator, Outcomes, Study design)16