Peno-Green, Laura MD; Verrill, David MS; Vitcenda, Mark MS; MacIntyre, Neil MD; Graham, Helen PhD, RN
The assessment of patient and program outcomes is an essential element in the practice of pulmonary rehabilitation (PR). Measuring patient outcomes can assist the healthcare provider in identifying therapeutic interventions that have the largest clinical impact for an individual patient and thus can enhance patient counseling. This information can be applied to the postrehabilitation activity prescription to promote individualized recommendations. The conclusions derived from measuring program outcomes can prompt performance improvement (PI) projects directed toward optimizing care delivery. Outcome results can also potentially improve reimbursement rates because healthcare administrators and third-party payers often expect this information. Each of these points underscores the importance of tracking and analyzing the effectiveness of PR through a system of structured outcome assessments.
Several major organizations have endorsed the clinical implementation of outcomes in PR. The American Association of Cardiovascular and Pulmonary Rehabilitation (AACVPR) first advocated measuring outcomes in 1995.1 These concepts were reinforced in the 2006 publication AACVPR Guidelines for Pulmonary Rehabilitation Programs.2 The American Thoracic Society (ATS) and European Respiratory Society also published practice guidelines for PR in 2006,3 recommending the integration into clinical practice of patient outcomes such as the assessment of patient symptoms, exercise capacity, and health-related quality of life (HRQL).
When tracking outcomes, it is preferable to use tools that have been shown to be reproducible, responsive, and valid.1 In doing so, the risk of conclusion error is reduced. The objective of this statement is to discuss several outcome measures that are identified within the AACVPR PR Outcome Matrix. The outcome measurements presented should be useful in monitoring patient progress and program performance.
THE PR OUTCOMES MATRIX
The AACVPR developed the PR Outcome Matrix (Table 1) in 2006 with the purpose of grouping outcome measurements with certain similarities to assist programs with the selection of assessment tools that would measure the effectiveness of comprehensive PR programs. The matrix consists of 4 domains or measurement categories. Patient outcomes are identified within the clinical, behavioral, and health domains and program outcomes within the service domain. Measurements are further subdivided into the core components of care, which are the interventional targets used in the practice of PR. The matrix is subject to changes in terminology and clinical care standards; thus, through a formal process, the AACVPR provides periodic revisions.4
The AACVPR encourages program staff to perform a multidimensional outcome assessment, selecting more than 1 outcome measurement from each domain to maintain program excellence. The basis for this recommendation is the understanding that patient outcomes are best achieved “not by substitution of 1 measurement or assessment strategy for another, but by measuring them in concert.”5(p776) Selecting tools to evaluate multiple program components within each domain will help assess the program impact on patient adherence to interventional therapies.
Collecting information required for a multidimensional outcome assessment is achievable even within the time constraints of a busy practice. The traditional approach to PR outcome assessment involves the collection of information that defines patient parameters at baseline in order that an individualized prerehabilitation exercise prescription is developed as well as the identification of patient goals and educational objectives. This assessment also requires documentation of patient clinical status at discharge. Thus, to move beyond the traditional approach to a more comprehensive outcome date collection strategy, there is need to gather information objectively through standardized assessment. This approach enables program staff to collect and compare data from multiple clinical, behavioral, and health variables from baseline to various timelines including program completion.
A sample patient outcome assessment form is presented in Table 2. Using a tabular format, such a form can be useful in providing an organized process for documenting progress toward patient and program goals, as well as a means to easily communicate these findings to referring providers.
The Clinical Outcome Domain
Measurements within the clinical domain contain both physiological and psychological patient outcome variables. Physiological measurements are directed toward symptoms, exercise capacity, and performance of activities. Psychological measurements include those mental processes of mood, such as depression, as well as cognition, memory, and orientation. The clinical outcome domain also incorporates assessments of nutrition and tobacco use, which, although reflect behavior as well, are components of patient assessment dealing with objective clinical data.
Breathlessness and leg fatigue during exercise are common symptoms encountered in patients entering a PR program. Identification and evaluation of these symptoms are important as both have been shown to be independent predictors of functional and health status.6 The Borg Rating of Perceived Exertion7 or the visual analog scale8 can be used to assess severity of these symptoms during exercise training or exercise testing.9
An assessment of cough and sputum production should be included as part of the symptom evaluation as suggested by the ATS, although the utility of available validated cough-specific tools during PR is unknown.3 In selected individuals, however, a symptom-specific questionnaire such as the cough-specific quality-of-life questionnaire10 and Leicester Cough Questionnaire11 may be useful.
Exercise capacity assessment
Exercise capacity is commonly measured by evaluations of endurance. Examples of commonly used exercise tools include the 6-minute walk test (6MWT), the shuttle test, and cardiopulmonary exercise testing. The 6MWT is standardized, practical, and frequently used by PR programs. The 6MWT protocol is described in the ATS guidelines.9 Although not included in the current version of the PR matrix, the shuttle test can be used as an alternative to the 6MWT.12 The shuttle test is an externally paced incremental exercise test that measures distance walked up and down a 10-m corridor marked by cones during which the pace is set by audio tones on a cassette. Cardiopulmonary exercise testing measures physiological responses during exercise that are not provided with either the 6MWT or the shuttle test. However, this test is costly and may not be readily available to many programs. When performing serial exercise assessment, oxygen administration and timing of bronchodilators should be taken into account because they can affect exercise performance.13–17
Physical activity assessment
A distinction is made by the ATS between exercise capacity and physical activity because exercise capacity does not necessarily correspond with performance of daily activities.3 Patient self-reporting and direct staff observation of activities of daily living are 2 methods for evaluating physical activity.3 However, these methods can be difficult to standardize and time-consuming. For this reason, the use of activity assessment questionnaires is an alternative method. The University of California—San Diego Shortness of Breath Questionnaire18 and the Modified Medical Research Council (MMRC) scale19 are 2 validated and available questionnaires. An advantage of the University of California—San Diego Shortness of Breath Questionnaire is that perceived dyspnea of different activities is quantified. The advantage of the MMRC scale is that duplication of staff efforts is reduced if the BODE index20 is also used as an outcome measurement.
The BODE index (Table 3) is a 10-point, multidimensional scale that provides a comprehensive evaluation of patients with chronic obstructive pulmonary disease (COPD).20 Although originally designed as a predictor of mortality, more recently it has been shown to be a useful tool when assessing the effectiveness of interventions in COPD, such as PR. Components of this index include body mass index (B), degree of airflow obstruction (O), dyspnea (D), and exercise capacity (E). Degree of obstruction is measured by forced expiratory volume in 1 second (FEV1), degree of dyspnea is defined by the MMRC score, and exercise capacity is assessed by the 6MWT. The BODE index is considered a clinical outcome domain measurement because it incorporates an assessment of dyspnea and exercise capacity.
Two studies have evaluated the BODE index specifically within the context of PR. One study demonstrated that the PR training effect can be objectively measured using this index.21 The index has also been found to be helpful in predicting the response to inpatient PR in COPD patients with respiratory failure.22 Thus, the BODE index can be used as a patient outcome tool provided a pre- and postrehabilitation comparison is made. The added advantage of using and reporting the BODE index is the prediction of mortality,20 which can prove valuable to PR staff and referring providers.
Muscular strength and flexibility assessment
An assessment of upper and lower body muscular strength as well as flexibility can be used as an adjunct to the exercise capacity assessment. The 1 repetition maximum test can be used and has been shown to be safe in PR patients.23 This test identifies the heaviest weight that can be lifted only once. An additional test is the handgrip test, which involves a single-hand compression of a grip dynamometer to estimate overall upper body strength.24 A consideration when performing strength testing is that measurements are dependent on effort, motivation, and practice.25 Strength testing for patients with primary pulmonary vascular disorders is not recommended because the potential of the Valsalva effect elevates the risk for serious adverse effects.2 For flexibility testing, joint range of motion can be measured with goniometric devices, which consist of a 180° protractor.
Depression and anxiety have been identified in patients with chronic respiratory disorders, but the prevalences are variable.25 Depression can influence HRQL26 and physical activity status,27 emphasizing the importance of screening for these symptoms. Depression has been shown to improve with participation in PR even without specific psychosocial interventions,28,29 which further justifies the inclusion of depression assessment among patient outcome measurements. Both symptom screening and outcome tracking may be accomplished with the same questionnaire. A common depression questionnaire used in clinical practice is the Beck Depression Inventory–II,30 which requires only 5 to 10 minutes to complete and score. A frequently used alternative is the Center for Epidemiology Studies–Depression Mode Scale.31
A comprehensive PR program should include a nutrition assessment and education.2 For many patients with chronic respiratory illnesses, dyspnea and increased metabolic demands can interfere with proper consumption of a balanced diet and maintenance of normal weight. In addition, a low body mass index is linked to higher mortality in patients with severe COPD.32 In contrast, the prevalence of metabolic syndrome in patients with COPD is also of concern for long-term health.33
Using an outcome assessment strategy will assist programs in determining whether nutrition education has been effective. Common measurements used for the nutritional assessment include patient weight, body mass index, abdominal circumference, and skinfolds. Although useful, these measurements may not change significantly during the time span of PR. For this reason, nutritional questionnaires outlined in the behavioral domain section should complement these clinical assessments.
A core component of care for PR is tobacco cessation. Although there is controversy whether to admit someone into a PR program who is actively using tobacco products, tobacco use should be assessed in all PR patients. Objective measurements of tobacco use are serum levels of cotinine or carboxyhemoglobin. The advantage of these measurements is the confirmation of tobacco use or abstinence so that a determination can be made whether the tobacco cessation aspect of the PR program has been successful. The stages of change related to tobacco use are outlined in the behavioral domain and can serve as an adjunct tool to these serum measurements.
The Behavioral Outcome Domain
Outcome measurements within the behavioral domain evaluate the psychological processes involving patient conduct and self-efficacy. Outcome tools used for this domain have been recently described.34 There are several additional tools available for measuring patient knowledge, nutritional habits, and tobacco use.
Patient knowledge assessment
Patient education should include information on self-management as well as prevention and treatment of exacerbations.25 A basic understanding of disease processes should also be included. A patient knowledge assessment can be used to evaluate the effectiveness of educational components and to provide patient feedback to reinforce important concepts. A sample test is provided within the AACVPR Guidelines for Pulmonary Rehabilitation Programs.2
Questionnaires are useful in evaluating dietary patterns. Tools include the Northwest Fat Questionnaire,35 which measures fat intake, the Diet Habit Survey,36 which measures fat and carbohydrate intake, and the Block Food Questionnaire,37 which provides fat, carbohydrate, protein, vitamin, and mineral assessments. Although these tools have been validated and shown to be useful in many patient populations, they apparently have not been studied specifically in PR patients.
Determination of tobacco status and readiness for change are central to the tobacco assessment because they provide a specific target for education. The readiness-to-change model for tobacco cessation is a continuum and divided into 5 stages: precontemplation, contemplation, preparation, action, and maintenance.38 Although validated questionnaires are not available, simple questions can be developed by the program to standardize its investigation. Example questions can include “Are you currently smoking?” if so, “How much do you smoke and are you ready to stop?” and if a ex-smoker, “How long ago did you quit?” It is recommended that questions be succinct and directed toward a third-grade level of education so that they can be understood by most patients. Assigning a numeric value to each stage will enable tabular recording for pre- and postrehabilitation.
The Health Outcome Domain
Health outcome domain measurements offer a “snapshot” of patient health status. These measures attempt to capture the total impact of medical and behavioral therapy on patient perceived health state and clinical status. Measures defined in this domain include HRQL, disease-related morbidity and mortality, and healthcare utilization.
Health-related quality of life
An important component of the health domain is HRQL, which is an expression of physical and emotional well-being. Multidimensional questionnaires are most useful when assessing HRQL. HRQL tools used can be divided into 2 categories: generic or disease-targeted. Generic questionnaires have a broad application to many health and disease states and can be applied to any population.39 Normative data for these tools are often available for cross-population comparisons. Disease-targeted questionnaires measure the health impact of a particular disease. This type of questionnaire is characterized by an enhanced sensitivity in detecting abnormalities specific to the disease group being evaluated.
Several generic and disease-targeted questionnaires have been shown to be useful in patients with chronic lung disease. Common generic questionnaires include the Short Form Health Status Survey40 and the Nottingham Health Profile.41 The Short Form Health Status Survey contains 36 items measuring physical, psychological, and social functions. The Nottingham Health Profile measures energy, pain, emotion, sleep, mobility, social isolation, and activities of daily living. Examples of disease-targeted questionnaires applicable to pulmonary disease include the Chronic Respiratory Disease Questionnaire42 and the St. George's Respiratory Questionnaire.43 The Chronic Respiratory Disease Questionnaire measures dyspnea, fatigue, emotional function, and self-efficacy. The St. George's Respiratory Questionnaire measures symptoms, activity, and the impact of disease. A more detailed discussion of these questionnaires is available.44
Several considerations must be made when selecting and administering these multidimensional questionnaires. Questionnaire selection should be determined on the basis of program resources, such as expense and administration time. Disease status should also be taken into account with serial testing since disease exacerbation can negatively impact HRQL for patients with COPD.45
Morbidity and mortality assessment
The morbidity and mortality subcategory within the health outcome domain includes any untoward events that occur during PR. For the purposes of this document, an untoward event is defined as a physiological complication that occurs while the patient is participating in a medically supervised PR session that necessitates immediate cessation of exercise and/or immediate medical assistance requiring physician or physician-extender intervention. It is incumbent upon programs to monitor and record untoward events that occur during exercise sessions in order that programs make necessary adjustment in policies and procedures to improve patient safety and decrease the likelihood of untoward events.
Healthcare utilization is one of the most important outcomes for third-party payers, regulators, and administrators. This assessment includes hospital admissions, emergency department visits, medical care, and unplanned physician appointments. These data must be carefully scrutinized largely because of potential limitations in patient recollection of events. Staff of PR programs may also check hospital, emergency department, or clinic records for this information. To determine the utilization of healthcare resources and its impact, individual and group data must be collected for a staff-determined period of time prior to PR and following PR participation to make comparisons.
The Service Outcome Domain
Service Domain measurements provide information regarding overall program delivery and effectiveness of specific PR protocols. Common measurements include enrollment rates, completion rates, dropout rates, referral rates, and measures of patient satisfaction. The objective of these measurements is to gain insight into program structure, program management, and service delivery.
Aggregate data derived from patient outcome measurements are used to evaluate the effectiveness of specific program protocols. For example, the difference between the pre- and postrehabilitation 6MWT can be calculated for each patient. The aggregate value for all patients can then be divided by the number of patients completing the endurance component of the program to generate the average change. Thus, an inference as to the effectiveness of the endurance training component of the program can be made. If effectiveness is not as expected, a PI project can be developed for optimizing the intervention. The effectiveness of the PI project can be determined by comparing values from one time period to another.
Pulmonary rehabilitation is designed and implemented to provide patient improvements in certain parameters, but importantly, improvements should be of sufficient magnitude to be clinically significant. The minimal clinically important difference (MCID) is defined as the smallest change in a parameter that results in a patient's perception of benefit or that would cause a change in patient management.46 For example, the MCID for the 6MWT is 54 m in that it has been shown that a distance of at least 54 m is needed for a patient to discern a perceptible improvement in functional status.47 Studies of outcome tools will often report the MCID. When available, clinicians need to understand this value and how it relates not only to individual patient response but also to overall patient outcomes.
Outcome assessment complements patient care by measuring and analyzing data that define the patient response to therapeutic interventions. Findings can guide program staff with patient counseling and the postrehabilitation exercise prescription and enhance the postrehabilitation continuum of care. The PR Outcome Matrix is useful in assisting program staff with the selection of appropriate outcome measurement tools. It is recommended that the type and quantity of outcome measurements derived from the matrix conform to a multidimensional assessment that can enhance the understanding of overall patient response to PR interventions and program efficacy.
The authors thank the AACVPR Outcomes Committee members for their valuable input, particularly Denise Albert, Teresa Corbisero, Val Kramer, Manoj Mithal, and Joy Reardon. The authors are also grateful to Linda Venus and Jean Steelman for their assistance with the preparation of the manuscript.
1. Pashkow P, Ades PA, Emery CF, et al. Outcome measurements in cardiac and pulmonary rehabilitation by the AACVPR Outcomes Committee. J Cardiopulm Rehabil
2. American Association of Cardiovascular and Pulmonary Rehabilitation. Guidelines for Pulmonary Rehabilitation Programs.
3rd ed. Chicago, IL: Human Kinetics; 2004.
3. Nici L, Donner C, Wouters E, et al. American Thoracic Society/European Respiratory Society statement on pulmonary rehabilitation. Am J Respir Crit Care Med
4. American Association of Cardiovascular and Pulmonary Rehabilitation. Outcome tool resource guide, 2002. http://www.aacvpr.org/publications
(AACVPR members only). Accessed January 2009.
5. Ware JE. Conceptualizing and measuring generic health outcomes. Cancer
6. Katsura H, Yamada K, Wakabayashi R, Kida K. The impact of dyspnoea and leg fatigue during exercise on health-related quality of life in patients with COPD. Respirology
7. Borg GA. Psychophysical bases of perceived exertion. Med Sci Sports Exerc
8. Wewers ME, Lowe NK. A critical review of visual analogue scales in measurement of clinical phenomena. Res Nurs Health
9. American Thoracic Society. ATS statement: guidelines for the six-minute walk test. Am J Respir Crit Care Med
10. French CT, Irwin RS, Fletcher KE, Adams TM. Evaluation of a cough-specific quality-of-life questionnaire. Chest
11. Birring SS, Prudon B, Carr AJ, et al. Development of a symptom specific health status measure for patients with chronic cough: Leicester Cough Questionnaire (LCQ). Thorax
12. Singh SJ, Morgan MD, Scott S, et al. Development of a shuttle walking test of disability in patients with chronic airways obstruction. Thorax
13. Leggett RJ, Flenley DC. Portable oxygen and exercise tolerance in patients with chronic hypoxic cor pulmonale. BMJ
14. Leach RM, Davidson AC, Chinn S, et al. Portable liquid oxygen and exercise ability in severe respiratory disability. Thorax
15. Casaburi R, Kukafka D, Cooper CB, et al. Improvement in exercise tolerance with combination of tiotropium and pulmonary rehabilitation in patients with COPD. Chest
16. Hay JG, Stone P, Carter J, et al. Bronchodilator reversibility, exercise performance and breathlessness in stable chronic obstructive pulmonary disease. Eur Respir J
17. Grove A, Lipworth BJ, Reid P, et al. Effects of regular salmeterol on lung function and exercise capacity in patients with COPD. Thorax
18. Eakin EG, Resnikoff PM, Prewitt LM, et al. Validation of a new dyspnea measure: the UCSD Shortness of Breath Questionnaire. University of California, San Diego. Chest
19. Mahler DA, Wells CK. Evaluation of clinical methods for rating dyspnea. Chest
20. Celli BR, Cote CG, Marin JM. The body mass index, airflow obstruction, dyspnea, and exercise capacity index in chronic obstructive pulmonary disease. N Engl J Med
21. Cote CG, Celli BR. Pulmonary rehabilitation and the BODE index in COPD. Eur Respir J
22. Biscione GL, Mugnaini L, Pasqua F, et al. BODE index and pulmonary rehabilitation in chronic respiratory failure. Eur Respir J.
23. Kaelin ME, Swank AM, Adams KJ, et al. Cardiopulmonary responses, muscle soreness, and injury during the one repetition maximum assessment in pulmonary rehabilitation patients. J Cardiopulm Rehabil
24. Pierson LM, Miller LE, Pierson ME, et al. Validity of hand-held dynamometry for strength assessment in cardiac rehabilitation. J Cardiopulm Rehabil
25. Ries AL, Bauldoff GS, Carlin BW, et al. Pulmonary rehabilitation—Joint ACCP/AACVPR evidenced-based clinical practice guidelines. Chest
26. Yohannes AM, Rooni J, Waters K, et al. Quality of life in elderly patients with COPD: measurements and predictive factors. Respir Med
27. Graydon JE, Ross E. Influence of symptoms, lung function, mood, and social support on level of functioning of patients with COPD. Res Nurs Health
28. Emery CF, Leatherman NE, Burker J, MacIntyre NR. Psychological outcomes of a pulmonary rehabilitation program. Chest
29. Guell R, Resqueti V, Sangenis M, et al. Impact of pulmonary rehabilitation on psychosocial morbidity in patients with severe COPD. Chest
30. Beck AT, Steer RA, Brown GK. Beck Depression Inventory–II Manual
. San Antonio, TX: The Psychological Corporation; 1996.
31. Radloff L. A self-report depression scale for research in the general population. Appl Psychol Meas
32. Landbo C, Prescott E, Lange P, et al. Prognostic value of nutritional status in chronic obstructive pulmonary disease. Am J Respir Crit Care Med
33. Marquis K, Maltais F, Duguay V, et al. The metabolic syndrome in patients with chronic obstructive pulmonary disease. J Cardiopulm Rehabil
34. Verrill D, Graham H, Vitcenda M, Peno-Green L, Kramer V, Corbisiero T. Measuring behavioral outcomes in cardiopulmonary rehabilitation: an AACVPR statement. J Cardiopulm Rehabil
35. Retzlaff BM, Dowdy AA, Walden CE, et al. The Northwest Lipid Research Clinic Fat Intake Scale: validation and utilization. Am J Public Health
36. Conner SL, Gustafson JR, Sexton G, et al. The Diet Habit Survey: a new method of dietary assessment that relates to plasma cholesterol changes. J Am Diet Assoc
37. Block G, Hartman AM, Naughton D. A reduced dietary questionnaire: development and validation. Epidemiology
38. Prochaska JO, Goldstein MG. Process of smoking cessation. Implications for clinicians. Clin Chest Med
39. Tulsky DS, Chiaravalloti ND. Managing quality of life in rehabilitation medicine. In: Physical Medicine and Rehabilitation
: Principles and Practice.
4th ed. Philadelphia, PA: Lippincott Williams & Wilkins; 2005:1194–1195.
40. Boueri FMV, Bucher-Bartelson BL, Glenn KA, Make BJ. Quality of life measured with a generic instrument (Short Form–36) improves following pulmonary rehabilitation in patients with COPD. Chest
41. Smith KW, Avis NE. Measuring quality of life. Principles Pract
42. Guyatt GH, Berman LB, Townsend M, et al. A measure of quality of life for clinical trials in chronic lung disease. Thorax
43. Jones PW, Quirk FH, Baveystock CM, Littlejohns P, et al. A self-completed measure of health status for chronic airflow limitation: the St. George's Respiratory Questionnaire. Am Rev Respir Dis
44. Kaplan RM, Ries AL. Quality of life as an outcome measure in pulmonary diseases. J Cardiopul Rehabil.
45. Aaron SD, Vandemheen KL, Clinch JJ, et al. Measurement of short-term changes in dyspnea and disease-specific quality of life following an acute COPD exacerbation. Chest
46. Hajiro T, Nishimura K. Minimal clinically significant difference in health status: the thorny path of health status measures. Eur Respir J
47. McGavin CR, Artvinli M, Naoe H, et al. Dyspnea, disability, and distance walked: comparison of estimates of exercise performance in respiratory disease. BMJ
© 2009 Lippincott Williams & Wilkins, Inc.