Internal validity of a randomized clinical trial of a nursing intervention is dependent on intervention fidelity. Although several methods have been developed, evaluating audio or audiovisual tapes for prescribed and proscribed interventionist behaviors is considered the gold standard test of treatment fidelity. This approach requires development of a psychometrically sound instrument to meaningfully categorize and quantify interventionist behaviors.
To outline critical steps necessary to develop a treatment fidelity instrument.
A comprehensive literature review was conducted to determine procedures used by other researchers. The literature review produced five quantitative studies of treatment fidelity, all in the field of psychotherapy, and two replication studies. A synthesis of methodologies across studies combined with researchers' experiences resulted in identification of the steps necessary to develop a treatment fidelity measure.
Seven sequential steps were identified as essential to the development of a valid and reliable measure of treatment fidelity. These steps include (a) identification of the essential elements of the experimental and control treatment modalities; (b) construction of scale items; (c) development of item scaling; (d) identification of the units for coding; (e) item testing and revision; (f) specification of rater qualifications and development of rater training program; and (g) development and completion of pilot testing to test psychometric properties. Development of the Possibilities Project Psychotherapy Coding Questionnaire is described as an illustration of the seven-step process.
The results show the essential steps that are unique to the development of treatment fidelity measures and show the feasibility of using these steps to construct a psychometrically sound treatment-specific fidelity measure.
Karen Farchaus Stein, PhD, RN, FAAN, is Associate Professor; and Judy T. Sargent, MS, RN, CS, is Doctoral Student and Research Assistant, School of Nursing, University of Michigan, Ann Arbor.
Nicholas Rafaels, MS, is Programmer/Data Analyst, Division of Allergy and Clinical Immunology, Department of Medicine, The Johns Hopkins University, Baltimore, Maryland.
Accepted for publication October 4, 2006.
Karen Stein was supported by National Institutes of Health, National Institute of Nursing Research (NINR) Grants R01 05277-01, 1 R55 NR 05277-01, and Judy Sargent was supported by an NIH Institutional NRSA predoctoral traineeship during the preparation of this article. Thank you for the contributions of Nora Arato, Lucy Miller, Adam Lewis, and Amelia Deschamps, members of the Possibilities Project Research Team, in the preparation of this manuscript.
Corresponding author: Karen Farchaus Stein, PhD, RN, FAAN, School of Nursing, University of Michigan, 400 North Ingalls, Ann Arbor, MI 48109 (e-mail: firstname.lastname@example.org).