Fiss, Alyssa LaForme PT, PhD, PCS; McCoy, Sarah Westcott PT, PhD; Bartlett, Doreen J. PT, PhD; Chiarello, Lisa Ann PT, PhD, PCS; Palisano, Robert J. PT, PhD; Stoskopf, Barbara RN, MHSc; Jeffries, Lynn PT, PhD, PCS; Yocum, Allison PT, DSc, PCS; Wood, Audrey PT, MS
Multisite research is important for improving the evidence base in physical therapy.1 Multisite research allows for collaboration of researchers and expert clinicians with varied knowledge, interests, and expertise and completion of studies that are not feasible to conduct at a single site. Multisite studies are needed to meet sample size requirements for research on populations with lower incidences2 and to improve generalization of findings. Another advantage is the potential to foster collaboration between practitioners and researchers and, as a result, enhance translation of findings into practice.
Despite the benefits, multisite studies are complex and require considerable resources, infrastructure, and coordination. Previous articles have described the challenges with multisite research including site coordination,2 consistency of data collection,2,3 selection of team members and team leadership,4 consensus on research procedures,4 and decisions on authorship.3,4 The time and effort required to receive institutional review board (IRB) approval for all sites is a concern that has been expressed by several investigative teams.5–9
The purpose of this article is to describe key considerations for planning and implementing multisite research based on our experiences and reflections in conducting a large, international, multisite study. The successes and challenges we have encountered are presented and recommendations provided. Our aim is to assist researchers and practitioners in planning, implementing, and completing multisite studies that evolve from practice and advance knowledge pertinent to pediatric physical therapy. We also hope that this article is useful to practitioners when reading research to appreciate the context in which data are gathered.
OUR CONTEXT: THE MOVE & PLAY STUDY
We are in the final year of a multisite study funded jointly by the Canadian Institutes of Health Research and the National Institute of Disability and Rehabilitation Research, Department of Education, United States, titled: “Understanding the Determinants of Basic Motor Abilities, Self-care and Play in Young Children With Cerebral Palsy.” Our short title “Move & PLAY” stands for “MOVEment and Participation in Life Activities for Young children.” Using a prospective cohort design and consistent with an interactionist perspective guided by the World Health Organization's International Classification of Functioning, Disability and Health (ICF),10,11 the objectives are to test a model of child, family, and service determinants that together explain change in gross motor abilities, self-care, and play of young children with cerebral palsy (CP) across 1 year. The model proposes that for children with CP, impairments in body structure and function, change in basic motor abilities (activity), and engagement in self-care and play (participation) are modified by personal factors (things that are associated with people but not their health condition) and physical, social, and attitudinal aspects of the environment.12 We believe that the results of this study will contribute to more effective and efficient delivery of rehabilitation services for preschool children with CP by identifying determinants that are amenable to change. In addition, determinants that are not amenable to change will assist with establishment of realistic and attainable goals and with prognostic discussions with children and their families.
Our target sample was 963 young children with CP (18 months to 5 years of age) and their parents from 3 regions across Canada and 4 regions in the United States. Details of the data collection sites (including associated investigators, coordinators, site liaisons, assessors, interviews, and students) are outlined in Table 1. The final enrollment included 430 children and their parents.
Outcomes include basic motor abilities (activity) and engagement in self-care and play (participation). Children were assessed by physical therapists and occupational therapists who achieved reliability on criterion tests of at least 80% item agreement for measures of motor function, play, and primary and secondary impairments. This standard for reliability testing is consistent with previous work conducted through CanChild Centre for Childhood Disability Research.13–15 Data were also collected from parents on family function, child adaptive behavior and health conditions, and community and rehabilitation services.16 Data analysis will include confirmatory factor analysis and evaluation of how well the data fit with our proposed model.12 Our study timeline is outlined in Table 2.
RECOMMENDATIONS FOR MULTISITE RESEARCH
Create a Collegial Research Team
Creating an effective and efficient team requires careful consideration and understanding of personal commitment, skills, and expertise. All members of the research team should have commitment, effective communication, and a voice in decision making. Previous researchers have commented that the selection of the research team and leadership are initial challenges faced in multisite research.4 In developing a research team, researchers can consider the following key characteristics of effective service provider teams: (1) shared goal; (2) commitment to establish a relationship; (3) demonstration of honesty, trust, and respect; (4) system for interaction and communication; (5) collaboration in decision making; and (6) development and implementation of an action plan to ensure work is completed.17,18 We believe that creating a collegial team is essential for the success of the project as a whole.
Our research team consists of 8 investigators, 3 project coordinators, 2 parent consultants, and 1 professional consultant. Many of the investigators had previously worked together on projects and had conducive working relationships. Key considerations in forming the team were that members previously enjoyed working together, had a level of respect and trust for each other, and willingness to compromise when decisions are made. We cannot emphasize enough the importance of forming a team that will support each other especially when challenges arise. After a more intensive implementation phase, team members now participate in monthly conference calls with designated follow-up and timelines. Each team member's opinions and ideas are respected and valued, and consensus of the team is used for decision making. Team members take advantage of the opportunity for in-person meetings at national conferences for additional study coordination and social outings (which we value!) to strengthen the team relationships.
Create a “Community of Practice” for Recruitment and Retention of Data Collectors
Creating a “community of practice”19 may assist study participants and data collectors in developing a sense of responsibility and participation in the study as a whole. In a research context, communities of practice are collaborative partnerships that link researchers together with practitioners and families to design, implement, and disseminate research activities. A community of practice provides opportunities for mentoring, maximizes resources, and helps ensure that procedures are acceptable to families and practitioners. Practitioners and families provide invaluable guidance regarding the research approach, measures, and dissemination of products in formats that are meaningful and useful to a wider audience.20,21 Communities of practice may help to bridge the research to practice gap.21,22
Data for the Move & PLAY study were collected and coordinated by 90 pediatric therapists who completed assessments in home and clinic settings, phone interviews, and coordinated recruitment at various sites. Table 1 outlines study personnel. We encouraged participation of therapists and parent consultants in various aspects of the study. Therapists were engaged in the selection of indicators leading to choices of measures we are currently using.23 Pilot testing was completed with therapists, parents, and children before submitting the grants for funding to ascertain the feasibility and acceptability of data collection methods. The parent consultants were involved in the development of study materials and provided feedback on study procedures via teleconferences and e-mails with investigators. All members were invited to participate in formation of substudies and are currently being invited to participate in dissemination activities.
Information has been offered to assessors via regular educational newsletters, assessor teleconferences, online surveys, a listserve, e-mails, and a study Web site: http://www.canchild.ca/en/ourresearch/moveplay.asp. Assessor newsletters have contained updates on the study progress, reminders and clarifications on assessment guidelines, and resources for evidence-based practice. Assessor teleconferences, online surveys, and the listserve allow the assessors to provide feedback to the investigators on the research process, successes and challenges with assessments, and considerations for data analysis that the investigators might have otherwise missed. Online surveys were completed after each assessment phase, to gather data about therapist perspectives on the process and measures. The use of a listserve for assessor networking was not as successful as we had anticipated as therapists were often overextended and did not engage in the listserve process. Research teams may also consider the use of a study Web site to post updates and materials for assessors to access at their convenience. Study investigators have been available by phone or e-mail and were able to attend assessment sessions with the therapist assessors to answer any questions or provide feedback on assessment procedures. An important aspect of the study coordinators' role was to maintain frequent e-mail and phone contact with local site liaison persons and study assessors, for the purpose of trouble-shooting issues of concern regarding recruitment and retention of families, and to ensure compliance with the study protocol. This close contact also built and strengthened constructive and collaborative relationships with local sites. Frequency of communication with assessing therapists was determined by the needs of individual assessors, with some assessors requesting more frequent contact. As we move into entry and analysis of study data, we are sharing preliminary descriptive results and engaging data collectors and parent consultants in assisting with interpretation.
When forming the research team, it is important to emphasize to physical therapists the time commitment of the study and ask them to consider any circumstances that might preclude participation to the conclusion of the study. It is also helpful to identify therapists who are likely to “champion” the study and facilitate solutions to potential barriers at their sites. Despite our best efforts, challenges were experienced in maintaining active engagement of therapists over the 2-year data collection phase. Therapists often noted difficulties in balancing participation in the assessments with clinical workloads and other professional and personal responsibilities. We recommend considering a limited number of key assessing therapists who demonstrate high engagement in the research project. This may increase the investment of therapists and lead to decreased challenges with monitoring and motivating therapists to stay engaged in the study. In our experience, when local sites (eg, children's rehabilitation centers) provided support and cooperation to the study by allowing assessments to take place within the context of the physical therapists' normal workday, the problem of maintaining therapist engagement was virtually eliminated. In these cases, remuneration for study visits was paid directly to the institution. Where this arrangement existed, therapists' motivation and commitment to the study remained high. For community-based therapists, the research team can establish contracts for a specified number of hours to ensure that they have the time to balance research activities within their regular workloads. In both situations within Move & PLAY, the therapist or the center was paid in commensurate with the prevailing rates. It is also important for researchers to remain flexible and responsive to modifying nonessential study aspects and to try various methods to support assessors. As an example for Move & PLAY, we assisted some assessors by scheduling appointments for them when they requested or by allowing them to divide a data collection session into 2 visits when needed.
Create Methods for Retention of Participants
Involvement of the consumer at every step of the research from design, data collection, analysis, and dissemination ensures that study materials are more easily understood and interpreted by the consumer and allows for a more accurate representation of the consumers' insights.21,24 To create a positive research experience for study participants, efforts should be taken to minimize the burden of research. A collaborative relationship with the designated liaison at each site is important and may improve the retention of participants.
To ensure that our research approach is family-centered, 2 parent consultants participated in the study development and continue to consult as the research proceeds. To decrease the burden of data collection, families were consulted for their availability for assessment sessions, and data could be collected in the home. Positive communication with families was provided through informative, 3- to 4-page, family newsletters sent after each study assessment. The newsletters contained updates on the study progress and tips on child development activities from the study researchers. In the newsletters, families were encouraged to provide anonymous feedback on their experiences and to contact a study investigator with any concerns or questions. Multiple means of contact with study coordinators were provided in the newsletters. We will continue to engage our parent consultants as we move into the analysis and dissemination phases of our work.
We believe in the idea that “retention starts at recruitment.” Systems for tracking study participants should be started at the initiation of the study planning, and methods of maintaining contact with families over time should be determined on the basis of the needs and differences that may be present for each individual site. In the Move & PLAY study, families were encouraged to share as much contact information as they were comfortable sharing, within the context of human subjects review board guidelines at each site. A concerted effort was made to obtain multiple modes of contact information including home, cell, and work phone numbers, e-mail addresses, mailing address, and permanent contact information from a friend or relative who may be able to reach the family if contact information changed. Families were asked to identify their preferred mode of communicating with study personnel. This was extremely important when families moved or children “aged out” of their current therapy setting and then “disappeared.” Having multiple modes of communication with families helped the research team stay in contact with the families throughout their year-long study participation, minimizing attrition. In some areas, therapists who had provided services to a child were able to help locate and contact families who had moved throughout the study process. Investigators in future projects may want to explore the incorporation of current technology (Web sites, social networking sites, or listserves) into methods for communicating with families. The use of technology should be based on the preferences of study participants and regional differences should be considered.
Creativity and flexibility are needed to maintain family participation. During our study, 4 families moved to regions of the US outside of our study regions. In 3 cases, assessment of the child and family at their new residence was arranged. In 1 case, the therapist was able to schedule the visit while traveling to a professional conference. Another therapist was able to travel to the family on an inexpensive flight, and a third family was assessed by a therapist who was willing to travel to the family's new residence after a personal trip. For 1 family who moved from Canada to India, arrangements were attempted to collect parent-report data after the family had moved, even though physical assessment of the child was not possible; however, this did not work out. By maintaining flexibility and creative scheduling, we have been successful in retention of participants.
Mentor the Next Generation of Researchers
We believe that the inclusion of young researchers and students provides them with a valuable experience and facilitates timely and financially fiscal research. New researchers have the benefit of collaborating with more experienced researchers, gaining practical experience with research methods, gaining firsthand experience in problem solving, and participating in larger funded projects that may not be otherwise possible. These collaborations lead to improved skill attainment of the novice researcher, and enhanced research capacity building, which has been identified as a need in the allied health care professions.21
Two physical therapists who recently completed their PhDs are investigators in the Move & PLAY study. These 2 investigators have broadened their experiences by taking a lead on 2 separate substudy grants and preparation of various dissemination materials, while receiving feedback from other team members and increasing their confidence and research skills. These researchers have also been provided the opportunity to be mentored by more experienced colleagues to refine techniques for preparing poster presentations, national conference presentations, and manuscripts for publication in professional practice journals, thus increasing their confidence and research productivity.
We had a total of 26 graduate students participating in the study. Table 1 outlines the students participating in this project. One 4th-year undergraduate student conducted an honor's thesis associated with our program of research. Students have been responsible for activities such as preparation of training materials, development of criterion testing and monitoring of completion of criterion testing, participant recruitment, data collection, data entry, preliminary data analysis, updating of background literature reviews, and preparation of a poster template; they are also invited to participate in team teleconferences. They have participated in data collection sessions either as an assessing therapist or by assisting the assessing therapist. Students have also contributed by completing substudies of the larger project, including the reliability and validity of measures developed specifically for this study. The study is also providing 1 PhD student with an opportunity to pursue a related question for her dissertation research. In general, the PhD students were involved in the main Move & PLAY study activities, participating in the study as part of their paid graduate assistantships and at times, receiving course credit for completing activities as part of a research practicum. DPT students participated in subprojects of the larger study, such as reliability of measures, receiving course credits toward the requirements for their professional programs. Although involving students in multisite research does require organization, we did not find more difficulties in mentoring students in a multisite project than in smaller projects. Within Move & PLAY, we found it beneficial for PhD candidates to mentor DPT students, providing the PhD students with the opportunity to learn and practice valuable teaching methods.
Create and Continually Update a Broad Disseminate Plan
We believe that it is essential to start a plan to disseminate the findings of the study early and to consider additional substudies. Girot and colleagues4 stated that one of their greatest challenges in multisite research was determining adequate dissemination and addressing intellectual property issues. By discussing and planning dissemination early in the study design process, this challenge can be significantly lessened. Dissemination plans may change and evolve as the study progresses, so continued conversations regarding dissemination should occur. Written guidelines to establish authorship on all study materials are recommended to alleviate misunderstandings among team members.3
The National Institute of Disability and Rehabilitation Research recommends developing a logic model to guide dissemination products based on the anticipated outcomes and the effect of those outcomes on providers and families. In accordance with these federal guidelines, a comprehensive dissemination plan for practitioners and consumers was created as part of the grant proposal. We had early discussion of a dissemination plan for papers and other knowledge translation products for practitioners and consumers. We chose to adapt and blend the authorship guidelines outlined by Russell and Bourbonniere25 on the CanChild Web site: http://canchild.icreate3.esolutionsgroup.ca/en/canchildresources/resources/authorshipguidelines.pdf and those used in a previous multisite study of activity and participation of children with CP involving 2 of the investigators. (See Appendix 1 Supplemental Authorship Guidelines File available at http://links.lww.com/PPT/A16). Key points for authorship include contribution to the study conception, design, methodological support, analysis or interpretation of data, contribution to the writing, reviewing and final approval of manuscripts, and the willingness to take public responsibility for the content of the paper. Written guidelines for authorship were created and agreed on by all study team members before beginning production of any dissemination materials.
Dissemination plans are frequently reviewed to ensure that opportunities are provided for all study collaborators to participate and all team members have a record of who is responsible for particular materials. Prioritizing the dissemination plan across the breadth of dissemination opportunities has been completed collaboratively, based on the aims of the study, the main goals of the research team, and an individual team member's capacity and desire to complete particular products. Revisions to the dissemination plan were also deliberated after initial data analysis to determine how best to present descriptive information and measurement development papers. In addition to written resources for families and practitioners, our plan includes reports to State and Provincial Interagency Coordinating Councils and Technical Assistance Programs, publications for policy implications, and curriculum recommendations for professional education programs. Multiple poster and platform presentations, as well as conference educational sessions, have been given at local, national, and international conferences. Investigators are encouraged to identify outcomes and outputs that are useful to providers and families, building these products into the comprehensive grant plan.
Multisite research also provides avenues for conducting additional studies. We have added 4 substudies to the original project and remain open to additional substudies suggested by team members, as long as relationships with children and families, assessors, and interviewers are not compromised.
Dissemination efforts are very time consuming. Although it is important to build in time and resources for dissemination during the time a study is funded, in reality, complete dissemination often extends past the end of funding. If additional or subsequent knowledge translation would be beneficial, researchers are encouraged to explore smaller sources of funding for products that target families and service providers. For the Move & PLAY study, we have applied for and received the Canadian Institutes of Health Research for an end-of-study Knowledge Translation Grant, primarily to support our main project coordinator and parent consultants to assist with the so-called “nonpeer review” products that we are planning for families and service providers. (See Appendix 2 Supplemental Dissemination Plan Template available at http://links.lww.com/PPT/A17)
OUR CHALLENGES AND RECOMMENDATIONS FOR MULTISITE RESEARCH
Defining a Scope of Work Commensurate With Funding
We recommend advocating and applying for 5-year granting cycles in grant submissions for large multisite studies when possible. However, many agencies do not fund 5-year grant cycles, especially for first-time applicants. For larger projects, shorter granting cycles may not provide sufficient time for preliminary study activities, such as IRB applications, preparation of training and testing materials, assessor training, and participant recruitment; similarly, the time for later activities involving dissemination can be compromised. For example, Cadell and colleagues5 reported that they had planned to begin participant enrollment around 6 months into a 3-year grant cycle, but with extensive IRB delays, their enrollment did not begin until well into the 2nd year. It is important for researchers to ensure an optimal match between the scope of the study and the funding period of granting agencies, considering all of the preliminary time requirements. If the time and resources are available, researchers may prepare preparatory materials in advance, such as IRB templates or training and testing materials. Pilot testing may provide an opportunity to begin organizing and creating study materials that would benefit subsequent larger projects. However, we recognize that early preparation of materials is not always feasible with grant funding not guaranteed and with some IRBs discouraging submissions for large studies prior to confirmation of project funding.
Our 3-year granting cycle did not allow sufficient time to obtain IRB approvals, prepare all study materials, instruct therapists in data collection, conduct criterion testing, recruit the full sample of participants, and reserve funded time for dissemination. Initial delays required us to close participant recruitment before obtaining the desired sample to comply with funding cycle deadlines. Delays also required us to ask and obtain an extension to complete our longitudinal data collection, final data analysis, and desired study dissemination. A 5-year grant would have provided us with the time and resources to complete all of these activities.
IRB Approval: Process for Coordination of Multisite Approval
We recommend advocating for collaboration among University and clinical site IRBs, or an integrated IRB process. Previous studies described the IRB process as difficult and time-consuming in multisite research because of the need for multiple IRB submissions.5–9 Institutional review boards were initially designed to provide oversight to local single-site studies and often do not have procedures to deal with multicenter studies.26 For example, even if a study has previous approval from another institution's IRB, subsequent institutions will often still require their own full review of the project.5 This can lead to long delays in data collection that need to be anticipated. In the United Kingdom, the ethics review burden for health research is managed through the “Integrated Research Application System,” using 1 standard application form, submitted online with reviews conducted weekly.27 This eliminates the need for multiple IRB submissions. This model of ethics review has been advocated in the United States and Canada for multisite studies but is currently not available.5,28 Researchers should discuss with their institution's IRB the possibility of reciprocal acceptance of IRB approval from other institutions involved in multisite projects and should develop materials that can be shared among sites.
This study required 23 IRB applications with several amendments as well as yearly renewals. This represents submissions to both University and clinical site IRBs. The process was time-consuming and required extensive resources to manage all required documents and submissions. We did develop general templates for study materials and information; however, many of the IRBs had different procedures or varied requests for study materials, which required that multiple versions of documents be drafted to meet individual IRB requirements. Small variations in methods for recruitment, consent documents, and follow-up contact of participants had to be incorporated at each location to meet site-specific IRB requirements. It is critical to budget for sufficient administrative support to complete this requirement.
Planning Efficient Preparation of Therapists for Data Collection
We recommend integrating training workshops and criterion testing for data collectors to facilitate the start of data collection, even if this adds to the time of initial training when everyone is face-to-face. This will decrease the amount of time and effort required to ensure that all data collectors complete the required reliability testing.
Our training workshops were limited to 1 day to minimize expenses and to respect therapists' other time commitments. Therapists were then asked to complete reliability criterion testing online on their own after the workshop. Therapists took several months to 1 year from the time of the workshops to complete the criterion testing. This caused a prolonged reliability-testing period that delayed the start of data collection. Therapist feedback indicated that they would have preferred to complete the criterion testing during the workshop, so as to have dedicated time to complete the training. We recommend a longer, comprehensive training program to complete therapist training and criterion testing in an efficient manner. Longer training sessions may also provide an opportunity to increase therapist momentum and excitement for the study as they are integrating into the team process.
Difficulty With Recruitment and Retention of Participants
Designate, invest in, and pay “champions” at each study site who understand and support the study and will foster enthusiasm for recruitment and retention of participants. Initially identifying effective “champions” is important. It is beneficial if these “champions” are therapists or administrators in a position of access to large numbers of potential participants. A study of knowledge brokers recommended using desirable characteristics for educationally influential physicians to determine which persons may make effective liaisons for study information.29 These characteristics include being familiar with the target population (within Move & PLAY—young children with CP and their families), being engaged on a regular basis with these children and families, having strong interpersonal skills, being respected by their peers, and being enthusiastic about their role in the project (approaching children and families to discuss their potential participation) and the project as a whole.30,31 Whenever possible, we advocate for direct communication with children and families by a representative of the study. This ensures that children and families are presented with the opportunity to participate and to make their own decisions.
In the Move & PLAY study, we achieved approximately 45% of the target sample size, with 91% retention at the conclusion of data collection. Informal discussions with other research teams suggest that initial estimates of sample size more often than not prove unattainable for reasons that could not have been anticipated. The young age of our study participants (18 months to 5 years of age) and the uncertainty of a diagnosis of CP for many young children made recruitment particularly challenging. Large and complex urban rehabilitation centers or hospitals, which serve vast geographic areas, were the most challenging to engage in recruitment, because of the many “layers” present between the researchers and the therapists who have direct contact with potential participants. In addition, these types of centers often had other studies competing for participants, as well as therapists' time. A central registry of children with a diagnosis of CP was not available in any of our study regions. Central registries are being used to collect clinical information about children with CP and to identify families interested in participating in research in countries throughout Western Europe and Australia.
During study development, contacts were made at each site to help determine an anticipated recruitment goal; however, during recruitment the anticipated numbers of children did not materialize for a range of reasons, including participation of families in other studies, dual diagnoses of autism, therapists believing that families were too busy to participate, and therapists reporting fewer children with CP on their caseloads. Whereas we believe that the number of sites involved in our project was sufficient, identifying families who were interested in participating in the study became a challenge. We were not certain of the success of providing information about the study to family indirectly though community agencies and providers approved by the IRB. We found that relying on therapists was often a less successful recruitment method, because of therapists' competing needs and lack of time to commit to recruitment. Recruitment was most successful in 2 scenarios: when families could be contacted directly through hospital mailings with a return letter to indicate their interest in participating in the study, and, when “champions” were available, particularly when working in large hospitals or rehabilitation centers with access to patient databases for study mailings, or direct contact with eligible families. These champions helped significantly increase participant enrollment in their regions.
Participant retention often required multiple attempts to reconnect with families. Despite best efforts by study personnel, we have had 41 losses: 35 families have withdrawn from the study and there have been 6 deaths prior to completion of data collection. Families have been withdrawn from the study because they had been lost to follow-up, found the demands of participating difficult with their numerous family responsibilities, received a new diagnosis making them ineligible to participate, or the participating child passed away.
Challenges in Administrative Management
An important study aspect is to develop consistent communication with budget officers and designated personnel for tracking budgets at study sites. It would be beneficial to ensure funding to pay for the required time to complete grant management.
Project administration, especially financial administration, is a component of research on which investigators may receive little instruction or guidance, yet to is essential to the day-to-day operations of a research study. Postaward management requires time and expertise to ensure that personnel and resources are available throughout the implementation of the study. We have been challenged to obtain and coordinate multiple financial agreements with participating institutions and monitor budgets from 2 funding sources. The universities where investigators are employed have different procedures and forms. The multitude of financial agreements between universities and the community therapists who are collecting data have required additional time and effort. We recommend that the principal investigator and project manager meet with their university sponsored projects' officer and accounts payable representative to review the regulations of the funding agency, preparation and submission of annual grant reports, establishment of independent contracts, management of subcontracts, and procedures for submitting and obtaining purchasing approvals and processing invoices. Investigators are encouraged to advocate for an effective university infrastructure that provides explicit policies, procedures, and support for grant management. Although it depends on the specifics of each study, investigators should budget for weekly time to review and process expenses and monthly time to review and balance the budget to ensure appropriate cash flow.
Creating and sharing study databases for data analysis has been complicated because of the large scope of our study. Our databases are large and have required extensive cleaning and checking to maintain accuracy. Our error rate to date has been 0.1% across the various measures. Because of the large size of the databases, it has often been difficult to quickly exchange information across sites to allow data analysis to occur at multiple sites. Because of IRB requirements, the sharing of data also became a challenge. Some of the sites allow us to send de-identified data via e-mail, while other IRBs do not allow electronic sharing of data. This required additional time and financial resources to transfer files to CDs and courier them to a central location for data entry. De-identified data have had to be verified multiple times by study identification number to ensure accuracy of data entry across the longitudinal study.
Multisite studies involving collaboration of researchers and practitioners have the potential to increase evidence for pediatric physical therapy by enabling research with a large sample from a wide geographical area. These larger, more diverse samples may improve generalization of findings and knowledge translation. We have presented the successes and challenges from our current project with the hope that our experience and recommendations will assist researchers and practitioners in the development and implementation of successful collaborative research. Specifically, we found that our greatest successes were in developing relationships among the research team and with study participants, both families and assessing therapists. We were also successful in developing a strong model for research capacity building for the next generation of researchers and in organizing a comprehensive dissemination plan. Our recommendations to groups interested in future multisite research projects include allowing adequate time and resources for study planning, IRB submissions, and training of data collectors. It is also important to devote adequate resources and personnel to ongoing study tasks including management of IRB issues, budgets, study materials, and study databases. Comprehensive planning for and investment in key personnel (who we referred to as “champions”) for participant recruitment and retention of study participants is also strongly recommended. Multisite research can be successful with forward planning, collaboration, and perseverance.
We thank Beth Tieman, PT, PhD, for creating the short form of our project name, Move & PLAY, and for her contributions to the development of this study.
1. Winstein C, Pate P, Ge T, et al. The Physical Therapy Clinical Research Network (PTClinResNet): methods, efficacy, and benefits of a rehabilitation research network. Am J Phys Med Rehabil. 2008; 87: 937–950.
2. Currow DC, Agar M, Tieman J, Abernethy AP. Letter to the editor: multi-site research allows adequately powered palliative care trials; web-based data management makes it achievable today. Palliat Med. 2008; 22: 91–92.
3. Weinberger M, Oddone EZ, Henderson WG, et al. Multisite randomized controlled trials in health services research: scientific challenges and operational issues. Med Care. 2001; 39(6):627–634.
4. Girot EA, Goodman B, Ross K, Latter S, Jackson D. Focus. Recognizing the challenges of collaborative, multi-site research. NT Res. 2004; 9(6):411–420.
5. Cadell S, Ho G, Jacques L, Wilson K, Davies B, Steele R. Considerations for ethics in multisite research in paediatric palliative care. Palliat Med. 2009; 23: 274–175.
6. Silverman H, Hull SC, Sugarman J. Variability among institutional review boards’ decisions within the context of a multicenter trial. Crit Care Med. 2001; 29: 235–241.
7. Dziak K, Anderson R, Sevick MA, Weisman CS, Levine DW, Scholle SH. Variations among institutional review board reviews in a multisite health services research study. Health Serv Res. 2005; 40: 279–290.
8. McWilliams R, Hoover-Fong J, Hamosh A, Beck S, Beaty T, Cutting G. Problematic variation in local institutional review of a multicenter genetic epidemiology study. JAMA. 2003; 290: 360–366.
9. Vick CC, Finan K, Kiefe C, Neumayer L, Hawn MT. Variation in institutional review processes for a multisite observational study. Am J Surg. 2005; 190: 805–809.
10. Bartlett DJ, Macnab J, Macarthur C et al. Advancing rehabilitation research: an interactionist perspective to guide question and design. Dis Rehabil 2006; 28: 1169–1176.
11. World Health Organization. The International Classification of Functioning, Disability and Health. Geneva: World Health Organization; 2001.
12. Bartlett DJ, Palisano R. A multivariate model of determinants of motor change for children with cerebral palsy. Phys Ther. 2000; 80: 598–614.
13. Rosenbaum PL, Walter SD, Hanna SE, et al. Prognosis for gross motor function in cerebral palsy: creation of motor development curves. JAMA 2002; 288: 1357–1363.
14. Hanna SE, Bartlett DJ, Rivard LM, Russell DJ. Reference curves for the Gross Motor Function Measure (GMFM-66): percentiles for clinical description and tracking over time among children with cerebral palsy. Phys Ther 2008; 88: 596–607
15. Hanna SE, Bartlett DJ, Rivard LM, et al. Stability and decline in gross motor function among children and youth with cerebral palsy aged 2 to 21 years. Dev Med Child Neurol 2009, 51:295–302.
16. Bartlett DJ, Chiarello LA, McCoy SW, et al. The measurement model of the Move & PLAY study: The protocol of a study using comprehensive outcomes research in rehabilitation. Phys Ther. 2010; 90(11):1–13.
17. Briggs MH. Building Early Intervention Teams: Working Together for Children and Families. Gaithersburg, MD: Aspen Publishers; 1997.
18. Rapport MJK, McWilliam RA, Smith BJ. Practices across disciplines in early intervention: the research base. Infants Young Childr, 17(1):32–44, 2004.
19. Wenger E. Communities of practice: the social fabric of a learning organization. Healthc Forum J. 1996; 39: 20–26.
20. Meyers AR, Andresen EM. Enabling our instruments: accommodation, universal design, and access to participation in research. Arch Phys Med Rehabil. 2000; 81:S5–S9.
21. Bartlett DJ, Lucy S. A comprehensive approach to outcomes research in rehabilitation. Physiother Can. 2004;56 237-247.
22. Turnbull AP, Friesen BJ, Ramirez C. Participatory action research as a model of conducting family research. J Assoc Persons Sev Handicap. 1998; 23(3):178–188.
23. Bartlett DJ, Palisano RJ. Physical therapists’ perceptions of factors influencing the acquisition of motor abilities of children with cerebral palsy: implications for clinical reasoning. Phys Ther. 2002; 82(3):237–248.
24. Cooke J. A framework to evaluate research capacity building in healthcare. BMC Fam Pract. 2005; 6:44
26. Nowak KS, Bankert E, Nelson RM. Reforming the oversight of multi-site clinical research: a review of two possible solutions. Account Res. 2006; 13(1):11–24.
28. Ethics IAPoR. The Tri-council Policy Statement: Ethical Conduct for Research Involving Humans. 2nd draft edition. Ottawa, Ontario, Canada: Government of Canada; 2008.
29. Canadian Institutes of Health Research. Moving motor growth research into clinical practice: Do knowledge brokers make a difference?(MOP-75901). $280,600 awarded to Russell D, Rosenbaum P, Bartlett D, Hanna S, Walter S, Roxborough L, Darrah J, in April 2006 for a period of 3 years.
30. Hiss RG, MAcDonald R, Davis WR. Identification of physician educational influentials in small community hospitals. Resid Med Educ. 1978; 17: 283–288.
31. Kauffman DM, Hodder I, Ryan K. A study of the educationally influential physician. J Contin Educ Health Prof 1999; 19: 152–162.
Canada; cohort study; cooperative behavior; international cooperation; Internet; institutional review boards; families; multicenter studies as topic; patient dropouts; patient selection; personnel selection; physical therapy (specialty/organization & administration); research/organization and administration; research subjects; United States
© 2010 Lippincott Williams & Wilkins, Inc.