Clinical trials (CTs) are generally considered the ultimate proving ground for treatment strategies; this recognition extends across all of health care. However, where CTs are ubiquitous in the many subspecialties of medicine, there are as yet few CTs in the allied health fields.1 And yet, the disciplines of allied health are perhaps the most fertile ground for CTs. In this article, we give motivation for CT adoption in prosthetics and orthotics (P&O) and closely related fields and give impetus through overview of the process CT design.
Without meaning to preempt questions of science and medicine, we can strengthen the case for CTs in P&O, physical therapy and occupational therapy, athletic training and exercise physiology, dietetics and nutrition, and others, by pointing to pragmatics: A major barrier to funding of CTs is feasibility of recruitment.2–5 One particularly potent reason why patients are willing to participate in research is their own personal interest in contributing to a public good.6,7 As of today, there is perhaps more opportunity for patients in the care of allied health professionals to have impact versus any other field of medicine.
To wit, we note the disproportion in issuance of clinical practice guidelines in conventional medical disciplines and the allied health fields: Professional associations in medicine regularly issue and update guidelines for clinical practice; the guidelines for practice in the allied health fields are sparse at best.8,9 The medical guidelines come about through years of refined study through CTs. Ironically, one natural consequence of the proliferation of CTs in medicine is that where guidelines exist, the balance of opinion regarding competing treatments (“equipoise”) disappears; as a result, the opportunities for subsequent CTs in medicine are becoming ever more granular and subject to the law of diminishing returns: Once the broad questions are answered, CTs begin to ask more detailed and specific questions related to the minutiae. By contrast, the allied health fields, practitioners still have much latitude to ask “the big questions,” and thus patients have substantial opportunity to contribute to the research that will shape clinical practice in the future.
For that matter, in P&O in particular,10–12 but also in other fields of allied health,13–15 there is a groundswell of interest in engagement, innovation, and synergistic practice; these are the key ingredients to successful CT design and operation. By assembling a critical mass (and diversity) of role players, investigators in the allied health professions can carry forward into an intensive effort of broadly impactful clinical science inquiry. Summarily, more than elsewhere in medicine today─and more than ever before in our own field─the patients and practitioners in allied health disciplines are in an unparalleled position to work together in shaping policy by way of CTs.
The critical first step toward launching a CT is to identify what is the question that needs to be answered.16,17 Within P&O, this has historically been accomplished through collaborative enterprise via the State of the Science (SOS) conferences, sponsored by the American Academy of Orthotists & Prosthetists, with support from the US Department of Education and Centers for Disease Control. These conference reports and the perennial State of the Science omnibus integrate the input of field leaders, subject matter experts, and stakeholders representing the payer, the patient population, and provider partners in the chain of care. Because the SOS reports reflect the perspectives of a diverse cadre of invested parties, they are key driver documents to CT origination. In other fields, the mechanism for identifying target topics varies in intensity and form; the American Physical Therapy Association House of Delegates' Strategic Plan, for example, blends its research priorities with a broader vision for the practice. But each field has─or should have─its own consensus documents where the community can find the roadmap for the research landscape.
Once priority topics have been identified within the field, it is up to the community to self-assemble into working groups for the design and execution of CTs. Although this activity can be incited by various agencies, for example, calls to action from the professional associations, seed funding from the private sector, and circumstantial incentive due to payer policy changes, the work itself is inherently community driven and must be instigated by intrepid investigators. This typically happens organically: when two or three colleagues start the conversation and invite others to join in, letters of intent are written, and grant support can be obtained.
In most circumstances, CTs are going to be conceived of as multicenter trials. The advantages to multicenter trials are numerous: inclusion of a patient population that is diverse by geography and socioeconomics and engaging a team of practitioners that train and practice with different styles improves study generalizability. Multicenter trials allow for efficient operation by distribution of labor; operating a trial at multiple sites allows for the sharing of wisdom between site investigators; and the burden of recruitment is shared so that there is less risk of saturation of any one market.
The study team should comprise an appropriate mix of personnel, including a lead or principal investigator (PI) or co-PIs, site investigator(s), as well as persons with specialty in data collection and analysis, statistics and study design, and study operations to include meeting planning, bursary, and quality control/oversight. Depending on study objectives, it may be appropriate to hire a health economist, adverse event specialist, medical billing specialist, or subject matter expert, for example, geriatrician or device engineer. Particularly for the purposes of establishing study need and feasibility, it may be appealing to engage with an epidemiologist. Once the trial is collecting data, it will be important to appoint a data safety monitoring board or, equivalently, a data monitoring committee. This board typically comprises field experts with no study affiliation and at least one independent statistician, charged with periodic review of blinded and unblinded study data to ensure ongoing adherence to best practices and, as applicable, predetermined boundaries for study stoppage due to safety and efficacy.
Team members can be extant staff, fee-basis contractors, or newly added through conventional hire. And the time commitment, day-to-day activities, and team makeup will change throughout the various stages of the study: Consultation with subject matter experts is critically important in the planning stage; study monitoring is not relevant until study launch; and data archival and results publication occur only after study close.
There are a number of parameters to consider when designing a CT, and they are all interdependent: changing one parameter will have consequences on other parameters. It can be challenging to determine where to start with setting the parameters; typically, the study's planning team identifies the subset of parameters that are either 1) most topically critical or 2) most implicated in study efficiency and discusses the parameter set as a system of moving parts, rather than one parameter in isolation. Here, we give an overview to universally relevant parameters and provide guidance for prioritization where applicable.
Typically, this is the first parameter to be identified and is usually determined in the earliest stages of study planning. Choice of outcome measure is critically influential on study impact: It is important that the outcome measure reflect the prioritized outcomes in clinical practice, so that clinicians reading the final published report will find the study relevant and informative. Whenever feasible, the outcome measure should also be robust, scrutable, and uncomplicated. In medicine, it is typical to select all-cause mortality as an outcome; investigators planning a CT in the allied health arena are more likely to use a functional outcome measure or observable performance measure Here, caution is advised: Subjective, clinician-administered tests are notoriously fickle,18–20 and even widely used objective performance measures can be capricious and yield counterintuitive results.21–23 It is helpful, but not essential, for the outcome measure to have been used in previous studies; previous reporting in other studies helps to establish expectations for the study in planning and contextualizing its results upon completion.
Once the primary outcome measure is selected, it will be important to define the follow-up period. Whether to follow for days or years depends on the time course of treatment, the nature of the question, and the resources for participant tracking. Some studies are well suited to a short-term follow-up, for example, infection rates after surgery or tolerability of a new therapy, whereas some studies require extended follow-up, for example, long-term outcomes after intensive rehabilitation. Furthermore, it must be determined whether the study will follow an “intention to treat” (ITT) design, where outcomes are measured regardless of crossover, or a “per-protocol” (PP) design.24–26 If PP, a policy will need to be adopted for how to handle the data collected from crossover patients, that is, whether to censor the data, withdraw the participant from the study altogether, or otherwise.
In the same conversation as selection of outcome measure is the hypothesized effect size. What is the expected baseline value for this study population? What is the expected change by study's end for those participants in study arm A versus those in study arm B? The chosen effect size has a direct impact on the study: Small effect sizes place less “burden of proof” on the investigational therapy and therefore are more likely to yield a positive result. At the same time, smaller effect sizes present a higher challenge, statistically speaking: All other things being equal, a small hypothesized effect size requires a larger sample size, which is likely to incur a direct additional cost onto the study. Moreover, a study reporting a small effect may have limited impact. It is the responsibility of the study's planning team to be thoughtful and deliberative in selecting the effect size that balances study impact versus efficiency.
In the CTs setting, sample size is determined before consent is obtained from the first patient. One of the most important principles of CTs is that the parameters are defined in advance of study launch, with consensus thresholds for error tolerances and distributional properties of the outcome measure. There are five major determinants of study sample size: effect size, variability of effect, the amount of α (type I) error and β (type II) error deemed acceptable, and whether the test will be one sided or two sided.27 Type I error equates to the risk of erroneously concluding the existence of a treatment difference; type II error equates to the risk of erroneously concluding no difference. Defining α and β values is at the discretion of the planning committee but─as with all other parameters─is determined through both exhaustive review of the relevant literature and deliberation among the planning committee. Almost without exception, α is set to .05, and β is typically .1 or .227.
Although the main determinants of study sample size are as above, there are a number of other parameters that must be considered when calculating sample size in a CT. First is loss to follow-up: Almost every outcome requires some compliance from some actor not directly in the employ of the study, that is, the study participant, his/her family, the payer, or agent. Mortality outcomes are particularly attractive because the death registries are, on balance, thorough and reasonably timely; few participants will elude a well-operated vital status database. At the other extreme, study outcomes that require face-to-face contact with the participant, particularly those that require completion of an intensive examination, and especially those outcomes that are collected after a long follow-up period or exposure to a treatment with low tolerance, will inevitably incur some loss to follow-up. Conventionally, the number of participants anticipated to be thus lost are added directly onto the study sample size, so that the true number of outcomes recorded at study's end reflects the projected sample size required of the prespecified α and β. In addition, it is important to consider “drop-in” and “dropout,” that is, participants assigned to one treatment that cross over into the other treatment. In both ITT and PP designs, the study sample size calculation will likely be impacted. Lastly, we mention the need to consider rates of patient accrual and crossover. Because many outcomes are time dependent, it is important to declare assumptions regarding whether patient accrual into the study will be uniform or changing over time: It may be that many patients are “waiting in the lobby” to join the study, or it may be that it would take many months for word to spread about the study; if recruitment into the study is nonuniform, then this will change the average follow-up. In the same way, if participants are more likely to cross over or become lost early versus late in follow-up, this will also impact the study sample size calculation. For extended discussion on the nuances of study size estimation in a CT, we refer to highly referenced source materials.28–30
Whereas the purpose of the study is to collect data, it is imperative that the data collection tools be thoughtfully designed and completed ahead of study launch. What data will be collected, by whom, and where will it be stored? How will data be captured to permit analysis: Will paper-based forms be transcribed by a human or by character-recognition packages; will tablet-based data collection be used instead? Depending on the study objectives, there may be quite a lot of data to collect related to both baseline and follow-up measures. Most studies identify key secondary or tertiary measures: typically, these include outcomes deemed important but ultimately ill-suited as primary outcome and exploratory measures that would benefit from further development or which may provide additional insight into the participants' response to treatment. We harken to the old adage, “every piece of data costs something,” usually money, by way of man-hours incurred in transcription, proof-checking, validation, quality control, and analysis. Whereas federally funded CTs must register with clinicaltrials.gov pursuant to eventually repositing some data there, data curation and archiving will have to be an ongoing process.
A major point of deliberation in the design of a CT will be “who is the target patient population?” Defining proper inclusion/exclusion criteria is critically important, both for its implications for study interpretation and its potential to impact study feasibility. Many laboratory studies include a statement of justification to the effect of inclusion criteria being intentionally broad and exclusion criteria few in number and narrow in scope. While there are inherent advantages to broad inclusion criteria in terms of study recruitment, and generalizability is typically viewed favorably, heterogeneity can undermine the study's power by introducing too much variability in the outcome. If there are published guidelines related to the study treatment, for example, clinical practice guidelines for treatment candidacy or published criterion for Centers for Medicare and Medicaid Services reimbursement, then these typically make very good starting points for defining study inclusion/exclusion, with the proviso that additional criteria should be added for ability to provide informed consent/assent and ability to complete study follow-up, as appropriate.
There are many other parameters that hold relevance to the design and operation of a CT, but scope and space limitations prevent thorough discussion here. We mention a few: Patient compensation should strike the balance between properly incentivizing patient participation and coercion; an informed consent questionnaire can often provide affirmation that the patient understands the study before participating (or open a dialogue for extended discussion)31; and whether a consent or release of information can be ethically authorized by a legal representative. For multisite studies, it is important to determine the number and location of sites: more sites improves the breadth of the trial but increases the burden of study management. Whether and how to establish (and enforce) site recruitment targets should be established pro forma.
The ubiquity of CTs is surely contributory to the circumstance of medical practice as the paragon of health care provision. But practitioners of the allied health fields, including prosthetists and orthotists, have equal opportunity to engage in this level of evidence-based practice. Indeed, because of the highly networked and innovation-driven nature of the field, our highly motivated patient population, and our integration with multiple medical subspecialties and other allied health disciplines, P&O practitioners may have the greatest mobility into the CT arena. The Big Questions are ready to be asked and deserve to be answered.
1. Burns PB, Rohrich RJ, Chung KC. The levels of evidence and their role in evidence-based medicine. Plast Reconstr Surg
2. Gorelick PB, Harris Y, Burnett B, Bonecutter FJ. The recruitment triangle: reasons why African Americans enroll, refuse to enroll, or voluntarily withdraw from a clinical trial: an interim report from the African-American Antiplatelet Stroke Prevention Study (AAASPS). J Natl Med Assoc
3. Hunninghake DB, Darby CA, Probstfield JL. Recruitment experience in clinical trials
: Literature summary and annotated bibliography. Control Clin Trials
1987;8(suppl 4):6S–30S. doi:10.1016/0197-2456(87)90004-3.
4. Lovato LC, Hill K, Hertert S, et al. Recruitment for controlled clinical trials
: literature summary and annotated bibliography. Control Clin Trials
5. Townsley CA, Selby R, Siu LL. Systematic review of barriers to the recruitment of older patients with cancer onto clinical trials
. J Clin Oncol
6. Schaefer GO, Emanuel EJ, Wertheimer A. The obligation to participate in biomedical research. JAMA
7. Trauth JM, Musa D, Siminoff L, et al. Public attitudes regarding willingness to participate in medical research studies. J Health Soc Policy
8. Lin RS, Stevens PM, Wininger M, Castiglione CL. Orthotic management of deformational plagiocephaly: consensus clinical standards of care. Cleft Palate Craniofac J
9. Hakkennes S, Dodd K. Guideline implementation in allied health professions: a systematic review of the literature. Qual Saf Health Care
10. Agarwal A. Essentials of Prosthetics & Orthotics: With MCQs & Disability Assessment Guidelines
. Jaypee Brothers: New Delhi, India; 2013.
11. Highsmith MJ. Editorial: clinical innovations in prosthetics
. Technol Innov
12. van Twillert S, Geertzen J, Hemminga T, et al. Reconsidering evidence-based practice in prosthetic rehabilitation: a shared enterprise. Prosthet Orthot Int
13. Cameron D, Cockburn L, Nixon S, et al. Global partnerships for international fieldwork in occupational therapy: reflection and innovation. Occup Ther Int
14. Zidarov D, Thomas A, Poissant L. Knowledge translation in physical therapy: from theory to practice. Disabil Rehabil
15. Janssen J, Hale L, Mirfin-Veitch B, Harland T. Building the research capacity of clinical physical therapists using a participatory action research approach. Phys Ther
16. Piantadosi S. Clinical Trials: A Methodologic Perspective
. Somerset, NJ: Wiley; 2013 http://public.eblib.com/choice/PublicFullRecord.aspx?p=4037039. Accessed September 30, 2016.
17. Pocock SJ. Clinical Trials: A Practical Approach
. Chichester, England: John Wiley & Sons Ltd;1983; http://onlinelibrary.wiley.com/book/10.1002/9781118793916. Accessed September 30, 2016.
18. Malhotra S, Pandyan AD, Day CR, et al. Spasticity, an impairment that is poorly defined and poorly measured. Clin Rehabil
19. Fleuren JF, Voerman GE, Erren-Wolters CV, et al. Stop using the Ashworth Scale for the assessment of spasticity. J Neurol Neurosurg Psychiatry
20. Pomeroy VM, Dean D, Sykes L, et al. The unreliability of clinical measures of muscle tone: implications for stroke therapy. Age Ageing
21. Wininger M. A generalized strategy for measuring performance in the velocity-position phase plane. J Mot Behav
22. Wininger M. Common roadblocks for biomaterials metrologists. J Funct Biomater
23. Wininger M, Kim NH, Craelius W. Spatial resolution of spontaneous accelerations in reaching tasks. J Biomech
24. Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ
25. Sedgwick P. Per protocol analysis. BMJ
26. Montori VM, Guyatt GH. Intention-to-treat principle. CMAJ
27. Lilford RJ, Johnson N. The alpha and beta errors in randomized trials. N Engl J Med
28. Shih JH. Sample size calculation for complex clinical trials
with survival endpoints. Control Clin Trials
29. Lakatos E. Sample sizes based on the log-rank statistic in complex clinical trials
30. Lakatos E. Sample size determination in clinical trials
with time-dependent rates of losses and noncompliance. Control Clin Trials
31. Guarino P, Lamping DL, Elbourne D, et al. A brief measure of perceived understanding of informed consent in a clinical trial was validated. J Clin Epidemiol