Share this article on:

CE: Defining and Understanding Pilot and Other Feasibility Studies

Morris, Nancy S. PhD, ANP-BC; Rosenbloom, Deborah A. PhD, RN, AGACNP-BC

AJN The American Journal of Nursing: March 2017 - Volume 117 - Issue 3 - p 38–45
doi: 10.1097/01.NAJ.0000513261.75366.37
Feature Articles

Nurses are becoming increasingly involved in conducting clinical research in which feasibility studies are often the first steps. Understanding why and how these studies are conducted may encourage clinical nurses to engage with researchers and take advantage of opportunities to participate in advancing nursing science. This article provides an overview of feasibility studies, including pilot studies, and explains the type of preliminary data they seek to provide in order to make larger, future studies more efficient and successful. By way of example, the authors discuss a feasibility study they conducted that illustrates the key components and necessary steps involved in such work.

This article provides an overview of feasibility studies, including pilot studies, and explains the type of preliminary data they seek to provide in order to make larger, future studies more efficient and successful. As an example, the authors discuss a feasibility study they conducted.

Nancy S. Morris is an associate professor in the Graduate School of Nursing at the University of Massachusetts Medical School in Worcester, and Deborah A. Rosenbloom is an assistant professor in the College of Nursing at the University of Massachusetts in Amherst. Contact author: Nancy S. Morris, nancy.morris@umassmed.edu. The authors and planners have disclosed no potential conflicts of interest, financial or otherwise.

Figure.

Figure.

How often have you thought to yourself, “There must be a better way to do this”? Some of what we do as nurses is grounded in strong evidence and yields positive outcomes, but we often encounter problems in care delivery and nursing processes that appear to have no clear solutions. If we can envision a novel approach, it's important to conduct research to determine its effectiveness. A well-designed research study can be expensive, however, typically requiring funding from an organization or foundation. To obtain funding, it's usually necessary to present data demonstrating that studying the approach is feasible and to make a strong case that the proposed study will answer the questions of interest.

Feasibility studies can provide such data. They can demonstrate that a research design is achievable and that recruitment for an anticipated larger study is possible. They can validate study procedures or identify procedures in need of revision. All of this information is helpful in convincing a funding agency that a study is worthy of support.

In this article, we will explain the various types of feasibility studies, as well as the terms currently used to describe them. We will encourage you to consider their research value and discuss how you might conduct such work yourself or support this type of inquiry. To illustrate the necessary components of such research, we will describe a feasibility study we conducted in preparation for an anticipated future study exploring the relationship between RNs and family caregivers of hospitalized older adults.

Back to Top | Article Outline

DEFINING OUR TERMS

Many authors use the terms “pilot work,” “pilot trials,” “pilot studies,” and “feasibility studies” interchangeably. While all of these terms are used to describe preliminary research conducted before a larger study, there is a growing consensus among researchers that we should recognize distinctions among them and be more consistent in our usage, because the way we define our terms determines the necessary components of our studies.1, 2 Eldridge and colleagues propose a conceptual framework in which “pilot studies are a subset of feasibility studies, rather than… being mutually exclusive.”2 While recognizing that there is variation in current research literature, for the sake of consistency this article will use the following definitions proposed by Eldridge and colleagues2:

  • feasibility study–research conducted to determine whether something can or should be done and, if so, how
  • randomized pilot study–a small-scale feasibility study, conducted with randomization of participants, that evaluates the practicability of carrying out all or part of an intervention and other processes to be undertaken in a larger, future study, and may or may not include alternative approaches
  • nonrandomized pilot study–a small-scale feasibility study, conducted without randomization of participants, that evaluates the practicability of carrying out all or part of an intervention—and, possibly, other processes—to be undertaken in a larger, future study
  • feasibility study that is not a pilot study–a feasibility study that does not incorporate the intervention or other processes to be undertaken in a future trial, but which may address the development of interventions or processes

We maintain that use of these definitions and support of the reporting guidelines recommended in the 2016 extension of the Consolidated Standards of Reporting Trials (CONSORT) 2010 statement1 promotes transparent, standardized reporting, thereby facilitating interpretation and comparison of studies.

Since pilot studies include all or part of an intervention, as well as possibly other processes to be undertaken in a future trial, the steps in a pilot study are similar to those taken in larger research studies3:

  • Plan for the protection of human subjects.
  • Describe the setting and location of data collection.
  • Explain the eligibility criteria for participants, in accordance with those to be used in the anticipated larger study.
  • Provide intervention details for each group.
  • Clarify sample size.
  • Report the point estimate of effects and corresponding confidence intervals to illustrate precision of data.
Back to Top | Article Outline

THE VALUE OF PILOT AND OTHER FEASIBILITY STUDIES

All feasibility studies, including pilot studies, are conducted to determine how well study components work together.4 Researchers use the results to demonstrate that the design they've proposed for a larger study is realistic—that the study procedures can be carried out and incorporated in a larger follow-up study. Although some outcome data are obtained, the number of participants in most feasibility studies is too small to allow any conclusions to be drawn about the efficacy of the results. However, good work in these areas can help researchers obtain funding for larger studies, especially when feasibility has been demonstrated and quantified.5 From subsequent larger randomized studies, researchers may be able to draw conclusions that can be generalized.

Bednash and colleagues call attention to the need for nursing to generate new knowledge, translate findings, and join with the larger health care community to promote evidence-based care.6 As health care delivery and financing continue to change, with an increasing emphasis on patient-centered care and self-management, it has become increasingly important for nurses to innovate patient care through participation in research.

Nurses often conceptualize solutions to problems, but it's challenging to go from a great idea to a full-scale clinical trial that establishes efficacy or effectiveness. A feasibility study may be the first step in moving an idea into the research arena, whether it's a pilot study or not.

Back to Top | Article Outline

GETTING STARTED IN FEASIBILITY RESEARCH

Box.

Box.

All feasibility studies, including pilot studies, start with a problem or a question (see A Step-by-Step Guide to Feasibility Research). Think about the challenges you've faced while providing nursing care—for example, trying to protect healthy skin from wound drainage or enhancing sleep on a unit with a high noise level. Some of these problems may suggest a question that can be answered with research. After identifying a problem or question, the next step is to review the literature to see what is and is not already known on the topic—to identify gaps in our knowledge.

Table 1

Table 1

Feasibility studies may focus on any aspect of research—the processes involved, necessary resources, study management, or a scientific question (see Table 1 3, 5, 7). Your study should concentrate on obtaining the information that will be most useful in supporting your proposal for a larger study.

Study processes are often the subjects of feasibility research. If you plan to recruit study participants by posting flyers or mailing invitations, for example, you may want to conduct a feasibility study to test whether these strategies are successful so you can revise your recruitment efforts, if necessary, for the larger study. If planning a trial that will involve participant randomization, trying out your randomization strategy with a smaller group can help to determine its efficacy. You can also assess retention in the various groups studied—are people in both the control and intervention groups willing to complete all aspects of the study? Studying retention before carrying out a large trial allows you to make changes that may increase the likelihood of high retention in all groups.

Sometimes you need to know if your instruments are appropriate for a targeted study population. An instrument or survey that has been tested and validated for use with adolescents, for example, may not be valid in adults over age 70. It's helpful to test all instruments to be used on the target population. You want to know if they can understand the directions, interpret the questions as intended, read the font size, and complete the instrument within the allotted time.

Identify any logistical issues or resource limitations. If several steps are involved in data collection, it's helpful to check the sequencing and timing with a small group of participants—and make any necessary adjustments—prior to a larger study. This also allows you to determine whether research assistants are sufficiently skilled to carry out procedures and follow the protocol without deviation. If you find inconsistency in protocol delivery, the protocol can be revised so as to avoid such problems in the larger study.

It may be appropriate to conduct a feasibility study before planning a pilot (intervention) study. The key areas of focus for such studies are as follows8:

  • acceptability of the intervention to those both delivering and receiving it
  • demand for the intervention
  • likelihood of its successful implementation
  • practicability, given available resources
  • adaptability of the intervention to different circumstances or populations
  • integration of the intervention within an existing system
  • potential for expansion, given the cost of the intervention and any related disruption to organizational culture, structure, or goals
  • promise of efficacy
Back to Top | Article Outline

INITIAL STEPS IN PLANNING A FEASIBILITY STUDY

To demonstrate how nurses might initiate this type of research, we'll describe the feasibility study we conducted in order to obtain data to support our plans for a grant proposal for a larger, future research study.

First, we identified a problem and a question—that it's difficult for nurses to detect subtle functional and mental changes in hospitalized older adults if they are unfamiliar with the patients’ baseline status. Our idea was that family caregivers may help nurses by sharing their observations and personal knowledge of the hospitalized family members. Our clinical experience suggested that family caregivers and nurses would benefit from an educational program that encouraged communication and active participation of family members in the care of hospitalized older adults. Before we could design such a program, however, we needed to understand the relationship between family caregivers and nurses.

Targeted health education programs must consider the learner's health literacy, which, according to Berkman and colleagues, is the “degree to which individuals can obtain, process, understand, and communicate about health-related information needed to make informed health decisions.”9 Limited health literacy was reported in up to 55% of 503 caregivers in one study, highlighting the importance of tailoring educational approaches to this group.10 One component of the larger research study we anticipated conducting is health literacy assessment of hospitalized older adults’ family caregivers. For this reason, we questioned the acceptability and feasibility of collecting such data from this group within the hospital environment.

Our literature review confirmed that family-centered care is beneficial to patients, families, and health care practitioners,11 but revealed little about best methods for engaging family members in the provision of care. The Institute for Patient- and Family-Centered Care and the American Hospital Association suggest that, in partnering with patients and families, health care providers gain “the benefit of their help and insights to better plan and deliver care.”11 Specifically, family-centered care is associated with the following outcomes12:

  • significantly better mood and engagement of patients
  • more patient–staff interactions that meet emotional and psychological needs
  • greater family caregiver satisfaction

Ten years ago, Corlett and Twycross suggested that ineffective communication and issues of power and control may prevent the ideal exchange of information between families and nurses.13 There were clearly gaps in our knowledge. More research is needed to understand the best ways for nurses to communicate and partner with families regarding the provision of care in the hospital.

We refined our general question and formulated the following specific research questions, based on our literature review:

  • How do RNs view the quality of their communication with family caregivers of hospitalized older adults, as measured by the Relational Communication Scale (RCS)?14
  • How do family caregivers view the quality of their communication with RNs caring for their hospitalized older family members, as measured by the RCS?14
  • How do RNs view the quality of their partnership with family caregivers of hospitalized older adults, as measured by the 10-item communication subscale of the Partnership Questionnaire (PFB)?15, 16
  • How do family caregivers view the quality of their partnership with RNs caring for their hospitalized older family members as measured by the communication subscale of the PFB?15, 16
  • Do family caregivers find it acceptable and feasible to complete a health literacy assessment tool, such as the Newest Vital Sign (NVS),17 while visiting their hospitalized older family members?

Our literature review indicated that we didn't have enough information to make a strong case for conducting a full-scale research study. To move forward, we needed to conduct a feasibility study through which we could obtain data further supporting work in this area.

We decided to conduct a descriptive, cross-sectional feasibility study focused on obtaining data related to process (would family caregivers be willing and able to complete the NVS health literacy assessment while visiting hospitalized family members?) and science (what preliminary data can we gather about the RN–family caregiver relationship and the quality of communication between the two groups using the RCS and the communication subscale of the PFB?). Our hope was that findings from this feasibility study would provide support for future research into nurse–family caregiver partnerships specifically related to the care of hospitalized older adults. Since our goal was to recruit family caregivers of patients ages 65 or older, we chose to collect data on the medical unit of an academic medical center that provides care to many older adults who have family caregivers.

We began by contacting the nurse manager to explain our goal and to ask if she would be interested in having us conduct our study on her unit. She indicated interest and confirmed that there would be no competing demands or major changes occurring on the unit during the period of data collection, which was important because such disruptions could influence both our recruitment efforts and our findings.

The nurse manager suggested that we attend a staff meeting to explain the proposed study to the nursing staff. Through that meeting, we were able to

  • enlist staff support before we moved forward with data collection.
  • elicit nurse input on timing.
  • identify potential RN participants.
  • set expectations.

The meeting also demonstrated our respect for the work of the unit staff. We asked the staff to suggest times of day they thought our data collection would be least disruptive to workflow and most conducive to family caregiver and RN recruitment. At their recommendation, we asked the unit's nurse educator to identify potential family caregivers of patients ages 65 or older. Our frequent communication with the unit nurse educator and nurse manager throughout the period of data collection kept them involved and aware of our progress as we recruited participants. It also provided them with access to the research team in the event that any issues arose from our presence on the unit.

Back to Top | Article Outline

FEASIBILITY RESEARCH IN ACTION

Because our study would involve human subjects, after enlisting the support of the unit's nurse manager and confirming our study setting, we submitted our plan for review and approval to the institutional review board affiliated with the medical center. We obtained informed consent from all participants.

Sample size. For small-scale feasibility studies, Hertzog suggests that the aims of the study should determine sample size: “For assessing clarity of instructions or item wording, acceptability of formatting, or ease of administration, a sample of 10 or even fewer may suffice. However, if the aims… are to estimate internal consistency or test–retest reliability or to assess item performance to evaluate or revise an instrument, such a small sample may be inadequate.”18 Given that our study aims were to investigate aspects of study processes such as recruitment strategy, acceptability of the instruments, and length of time required to complete the instruments, we considered that a sample of 10 would be adequate. Billingham and colleagues conclude their review of sample sizes for pilot and other feasibility studies with the recommendation that such studies don't necessarily require a sample size calculation, but researchers should be able to justify their chosen sample size.19

Our convenience sample included 15 dyads, each of which consisted of the family caregiver of a hospitalized older adult who was receiving treatment at UMass Memorial Health Care, a tertiary academic medical center in Worcester, Massachusetts, and the RN caring for that older adult. One RN participated twice as two of her patients had family caregivers enrolled in the study.

Family caregivers were eligible to participate if they were

  • the spouse, relative, or domestic partner of a hospitalized adult; ages 65 or older; and spent a minimum of 30 minutes per day at the hospital with that hospitalized adult.
  • familiar with the older adult's routines and usual state of health.
  • able to read, write, and communicate in English.

Data collection and instrumentation. Based on our research questions, we used the following three scales in our study:

  • the RCS to assess development of an interpersonal relationship between the family caregiver and the RN14
  • the 10-item communication subscale of the PFB to measure perceived quality of the partnership between the RN and the family caregiver15, 16
  • the NVS to assess health literacy of the family caregivers17

All three scales have been found to have good psychometric properties.

Our research team collected data on different days of the week between 10 AM and 6 pm. When a research team member was available to collect data, the nurse educator on the unit alerted the researcher to potential participants—family caregivers who were currently visiting patients ages 65 or older. The researcher then spoke with the caregiver to explain the study, determine eligibility, invite participation, and obtain informed consent from those who were interested in participating. The caregiver completed the PFB communication subscale, the RCS, and a brief demographic form. The researcher administered the NVS assessment tool orally. After the family caregiver completed all instruments, the RN for the caregiver's family member was invited to participate in the study; if interested, the RN provided informed consent and completed the PFB communication subscale, the RCS, and a brief demographic form.

Our study used descriptive statistics for the analysis and reporting of results, as most feasibility studies do.20 Our sample can be described as follows:

  • All caregivers (15) and RNs (14) enrolled in the study were female.
  • Family caregivers could be classified into one of four relationship categories—spouses (six), daughters (six), sisters (two), domestic partner (one).
  • All caregivers had known the patient for more than 10 years.
  • Five caregivers (33%) had received formal caregiver training.
  • All RNs were experienced, with only five (36%) having been an RN for fewer than five years.
  • Eight RNs (57%) had a bachelor of science in nursing.

The RNs reported interacting with the caregivers anywhere from 10 minutes to more than two hours. More family caregivers than RNs (53% versus 7%) perceived the other as “accessible” and their communication as “clear.” When asked to describe their partnership as excellent, very good, fair, distant, or poor, one family caregiver and one RN rated it as “restricted.” Most family caregivers (73%) and RNs (58%) rated the relationship as “very good.” Because our sample size was small, these results suggest only that nurses and caregivers may perceive their relationship differently and that more research is needed before any meaningful conclusions can be drawn. We did, however, establish that the RCS and the PFB communication subscale were easy to administer in a hospital environment.

All the family caregivers completed the NVS. Of the 15 caregivers, 12 (80%) demonstrated adequate health literacy. Only three caregivers (20%) exhibited a high likelihood of limited health literacy. Two caregivers expressed worry about “getting things wrong.” Although our caregiver sample was too small to allow us to draw meaningful conclusions about caregiver health literacy, our experience suggested that caregivers are willing and able to complete the NVS while in the hospital visiting their family member.

Feasibility study results focus on the value of outcomes for future studies rather than on the specific findings. In our feasibility study, we identified discrepancies in the way family caregivers and RNs perceive their relationship. When asked to describe the relationship as excellent, very good, fair, distant, or poor, 42% of the RNs and 27% of the caregivers described it as either “distant” or “fair.” Since ineffective communication between family caregivers and nurses hinders patient care, this finding suggests a need for further study to explore the

  • type, duration, and frequency of interactions between caregivers and RNs.
  • expectations caregivers and RNs bring to their relationship.
  • communication skills and style of caregivers.
  • communication skills and style of RNs.

Although we assessed each group's perception of the other's “accessibility,” we did not specifically assess the ease with which caregivers and RNs communicated, which may be different conceptually than accessibility. This difference will be important to consider in the design of future studies.

Although all caregivers in our study completed the health literacy screen, our use of a convenience sample is a source of potential bias. Those caregivers with limited health literacy may have chosen not to participate in the study. To avoid this potential bias, future studies should consider using a sampling strategy more likely to engage caregivers at all health literacy levels. Researchers might plan to recruit from public as well as private hospitals in both urban and rural areas, for example, and to ensure that all recruitment materials are written clearly and at a level accessible to those with low health literacy.

Feasibility studies provide information needed to determine whether future study in an area would be valuable and successful. The findings may convince you not to carry out a larger study, or may cause you to modify your design and protocols. Because pilot studies incorporate interventional aspects of anticipated larger studies, they can identify potential unanticipated harm to participants as well as the usefulness of surrogate end points in larger, future studies.21 Our feasibility study provided us with preliminary data supporting the need to conduct additional research into the relationship between RNs and family caregivers with a particular focus on communication and partnerships.

It's important to disseminate the results of feasibility studies. A recent extension to the CONSORT statement specific to randomized pilot and other feasibility studies conducted in advance of randomized controlled trials provides guidance on the essential elements to share.1 These guidelines emphasize that the primary aim of such studies is to assess the feasibility, rather than the efficacy or effectiveness, of some aspect of the anticipated randomized controlled study.

Back to Top | Article Outline

GETTING INVOLVED IN RESEARCH

Conducting feasibility studies is one way to get involved in research, but there are others. You can ask to join a research team or support the efforts of investigators conducting research where you work. Welcoming nurse researchers to your work site and sharing your thoughts and ideas is one way to advance nursing science. If a researcher wants to recruit patients or family members from your unit, you may be able to spark interest among colleagues and support recruitment efforts. You might participate in disseminating the results through a presentation or a publication. Seek mentoring from researchers and ask to be a part of their research team. By embracing research in the practice setting, nurses can contribute to a culture of inquiry focused on better understanding human responses to actual and potential health problems.

Back to Top | Article Outline

REFERENCES

1. Eldridge SM, et al CONSORT 2010 statement: extension to randomised pilot and feasibility trials BMJ 2016 355 i5239
2. Eldridge SM, et al Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework PLoS One 2016 11 3 e0150205
3. Thabane L, et al A tutorial on pilot studies: the what, why and how BMC Med Res Methodol 2010 10 1
4. Arain M, et al What is a pilot or feasibility study? A review of current practice and editorial policy BMC Med Res Methodol 2010 10 67
5. Leon AC, et al The role and interpretation of pilot studies in clinical research J Psychiatr Res 2011 45 5 626–9
6. Bednash GP, et al Building the research enterprise in the academic environment. Quality advancement in nursing education/Avancées en formation infirmière 2014 1 1
7. van Teijlingen E, Hundley V The importance of pilot studies Nurs Stand 2002 16 40 33–6
8. Bowen DJ, et al How we design feasibility studies Am J Prev Med 2009 36 5 452–7
9. Berkman ND, et al Health literacy: what is it? J Health Commun 2010 15 Suppl 2 9–19
10. Morrison AK, et al Low caregiver health literacy is associated with higher pediatric emergency department use and nonurgent visits Acad Pediatr 2014 14 3 309–14
11. American Hospital Association. Strategies for leadership: patient- and family-centered care. n.d. http://www.aha.org/advocacy-issues/quality/strategies-patientcentered.shtml.
12. Goldberg SE, et al Care in specialist medical and mental health unit compared with standard care for older people with cognitive impairment admitted to general hospital: randomised controlled trial (NIHR TEAM trial) BMJ 2013 347 f4132
13. Corlett J, Twycross A Negotiation of parental roles within family-centred care: a review of the research J Clin Nurs 2006 15 10 1308–16
14. Burgoon JK, Hale JL Validation and measurement of the fundamental themes of relational communication Commun Monogr 1987 54 1 19–41
15. Halweg K, et al Zur erfassung der ehequalität: ein vergleich der “Dyadic Adjustment Scale” (DAS) und des “Partnerschaftsfragebogens” (PFB). / Assessment of marital quality: a comparison of the Dyadic Adjustment Scale (DAS) and the Partnership Questionnaire (PFB) Diagnostica; Informationsorgan über psychologische Tests und Untersuchungsmethoden 1992 38 4 312–27
16. Zimmermann T, et al Fear of progression in partners of chronically ill patients Behav Med 2011 37 3 95–104
17. Weiss BD, et al Quick assessment of literacy in primary care: the newest vital sign Ann Fam Med 2005 3 6 514–22
18. Hertzog MA Considerations in determining sample size for pilot studies Res Nurs Health 2008 31 2 180–91
19. Billingham SA, et al An audit of sample sizes for pilot and feasibility trials being undertaken in the United Kingdom registered in the United Kingdom Clinical Research Network database BMC Med Res Methodol 2013 13 104
20. Lancaster GA, et al Design and analysis of pilot studies: recommendations for good practice J Eval Clin Pract 2004 10 2 307–12
21. Loscalzo J Pilot trials in clinical research: of what value are they? Circulation 2009 119 13 1694–6

For 164 additional continuing nursing education activities about research, go to www.nursingcenter.com/ce.

Keywords:

culture of inquiry; feasibility study; nursing research; pilot study; pilot trial; pilot work; preliminary study

Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved.