Quantifying and improving the quality of healthcare are a priority for healthcare organizations. Although outcomes measurement for accountability often is mandated by external organizations, such as the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) and the Centers for Medicare/Medicaid Services (CMS), the primary function of such measurement is to serve as an internal force for evaluating, understanding, and improving the quality of patient care. 1
In 1992, the American Association of Cardiovascular and Pulmonary Rehabilitation (AACVPR) established an Outcomes Committee in response to a need to document the benefits from rehabilitation efforts directed at patients with cardiovascular and pulmonary disease. The committee’s primary purposes were to identify priority areas for outcomes evaluation and to provide suggestions for methods and measures. The AACVPR first published outcomes recommendations for cardiac and pulmonary rehabilitation programs in 1995, 2 emphasizing the importance of a comprehensive approach to outcome measurements within routine clinical practice. Specific recommendations included the importance of identifying measures that reflect the results of care in three domains: the behavioral, clinical, and health domains. Program staff were advised to track outcomes consistently within their clinical practice, with the minimum expectation of including at least one behavioral, one clinical, and one health domain measure in their outcomes evaluation.
The purpose of the current statement is to update the previous recommendations for outcomes evaluation in cardiac rehabilitation/secondary prevention programs. This statement complements the most recently published Guidelines for Cardiac Rehabilitation (CR) and Secondary Prevention Programs, 3 and is in accordance with the AACVPR and American Heart Association collaborative paper on the Core Components of Cardiac Rehabilitation/Secondary Prevention. 4 A similar outcomes statement for pulmonary rehabilitation will be available after the publication of the Third Edition of the Guidelines for Pulmonary Rehabilitation.
The objectives of this document are to review the purposes of outcomes evaluation and to provide practical information that may help in the implementation of an outcomes-directed approach within routine patient care and program management functions. Sample forms are provided with recommendations for an outcomes-directed approach to care at two levels: (1) assessment of patients at program entry to help guide treatment plans and repetition of the measures at defined intervals (eg, at completion of 12 weeks of CR programming) to evaluate patient outcomes, and (2) evaluation of program effectiveness on the basis of aggregate patient and service-related outcomes data. The importance of connecting outcomes evaluation with quality improvement processes is reinforced throughout this document. A brief discussion of future directions for CR in outcomes evaluation concludes this report.
Table 1 describes important activities that have occurred since the first AACVPR outcomes statement, 2 which influenced the development of the current recommendations. It is important to recognize that recommendations for outcomes evaluation must be dynamic to reflect new findings in outcomes research related to CR and secondary prevention. The current recommendations have been updated and coincide with revised evidence-based guidelines for the treatment of coronary artery disease and the associated risk factors. 5–10
Other research 11,12 has provided information on the effectiveness of behavioral and psychosocial interventions that is specific to CR settings, which can help guide treatment strategies. Psychosocial and health status measurement tools have been tested with CR patient populations, 13,14 and the information gained can help to refine decision making in the choice of the most appropriate tools for individual programs and their patient population.
Furthermore, as clinical programs continue to publish their own outcomes data, 15,16 these observations can help to identify issues that may be applicable for quality improvement initiatives. The dissemination of “real-world” CR experiences, including the reporting of program observational data and lessons learned from quality improvement projects, can help guide future and more effective approaches to CR/secondary prevention care. The combination of outcomes research and well-documented practical experiences in the CR setting can contribute to the advancement of outcomes evaluation and, ultimately, to an improvement in patient outcomes and program effectiveness.
To accomplish this goal, the outcomes evaluation process must be meaningful to practitioners at the patient level and must add value to the overall effectiveness of the program. 17 Outcomes evaluation needs to be a proactive program expectation, and must not be driven primarily by reactive requirements from external forces such as program certification applications, administrative reports, or other mandated projects. 18 However, it is recognized that most programs have limited personnel and resources, particularly as they may relate to outcomes data management (eg, standardized data collection, completion of forms, computer data entry, data analyses, and report generation). The development of user-friendly forms and patient assessment protocols that serve dual purposes (medical record documentation and outcomes evaluation) continues to be a work-in-progress for most programs.
No single form, assessment protocol, or questionnaire (eg, tool, instrument) will fit the needs of all programs. Although programs offer similar services, there is wide variability among programs related to patient populations, service delivery methods, and available resources for outcomes evaluation. Therefore, this statement does not direct program staff to use a specific questionnaire, but rather provides a structural framework for outcomes measurement that will guide programs in the development of standardized assessment protocols that fit their specific needs. If this is accomplished, programs will be able to fulfill at least the minimum expectations of outcomes evaluation as described in the recently published clinical guidelines. 3
A team effort with input from those providing direct patient care, the program medical director, and the program manager is needed in the development of a functional outcomes evaluation for a specific program. The ability to enter and manage data efficiently from the point of patient care to the generation of program summary reports is important for the integration of outcomes evaluations into routine patient care and program management processes. A computer and a database that meets the needs of the program are wise investments for accomplishing this goal. However, if databases are allowed to become unfocused or too large, data management becomes burdensome and detracts from the intended purpose of quality improvement.
CORE COMPONENTS OF CARE AND THE OUTCOME MATRIX
Cardiac rehabilitation/secondary prevention programs are advised to provide specific core components of care to optimize cardiovascular risk reduction, reduce disability, and promote healthy behaviors, including long-term adherence to these behaviors. 3,4 Information regarding evaluation, intervention, and expected outcomes in each of the core components is described in detail elsewhere. 4Table 2 provides data from the core components that outlines a battery of outcome measures recommended for CR/secondary prevention programs. At the individual patient level, these measures are necessary for assessing the patient’s baseline status and identifying priority secondary prevention needs. Reevaluation of the same measures at the end of the program documents progress toward the secondary prevention therapy goals.
Cardiac rehabilitation programs need to establish a standardized method of data collection and maintain effective communication with other healthcare providers who also provide care for the referred patient. Secondary prevention goals are not isolated to the CR setting, and collaboration with other healthcare providers is essential in the effort to identify the specific needs of the patient and guide him or her to goal achievement. For example, lipid assessment is standard care in secondary prevention, 5 but duplicate testing is unnecessary and not cost effective. A program-specific protocol defining acceptable periods and methods for documenting laboratory data from referring physicians or the acute care setting permits the capture of important information without needless additional testing. It also promotes a collaborative relationship with other care providers. The establishment of consistent data collection methods within the program is not only necessary at the patient level, but also imperative in the interpretation of aggregate patient data for evaluating program effectiveness.
The structural framework suggested for outcomes measurement is the Outcomes Matrix, as illustrated in Table 3. The Matrix was developed by the AACVPR Outcome Committee 19 to help link outcomes evaluation to the core components of care. The framework provides a menu of outcome measures within the behavioral, clinical, and health domains and is in accordance with the core components of CR/secondary prevention programs. 4 It also introduces the service domain, which includes other program measures described in more detail later.
Although the factors to measure are well defined, the measures to use may not be as clear-cut. The program staff members need the flexibility to choose measures that are meaningful for their patient population, providing results that can be interpreted and used by the staff. Decisions about what measures to use are influenced by the level of resources available in individual programs and the practicality of implementing the selected measures within routine patient care and program functions.
Although improved health and health-related quality of life are the ultimate goals of care, 20 it is the accumulation of positive changes that leads to the more global measures of health. For example, when CR/secondary prevention interventions are implemented, changes in behavioral domain measures may be observed before changes in clinical domain measures, and changes in clinical domain measures may be observed before changes in health domain measures. Although changes in outcomes are much more complex than the following simplified illustration suggests, it may help engender understanding concerning the expected sequence of change as a direct result of the care provided in CR (Figure 1).
Patient outcome measures need to include at least one outcome measure for each core component of care that can be integrated consistently into preprogram and follow-up patient assessments (Table 3). The battery of measures needs to include an assortment of patient outcome measures (for the behavioral, clinical, health domains) and a service domain measure that will provide sufficient data for evaluating the effectiveness of the program in delivering each component of care, as well as overall program effectiveness. These data will contribute important information for developing quality improvement initiatives. The following discussion describes some specific examples of measures for each domain.
Behavioral domain measures reflect the patient’s ability to make the recommended lifestyle changes that potentially lead to goal achievement in the clinical and health domains. These measures often are neglected or underused as important tools for guiding individual treatment plans and evaluating the effectiveness of the education and behavioral interventions. Behavioral domain measures are most commonly collected through administration of validated self-report questionnaires and by observation of patient behaviors during program participation. Suggested behavioral domain measures include physical activity (eg, minutes of physical activity or energy expenditure per week), diet habit scores, smoking (eg, current smoking status and number of cigarettes or amount of tobacco used per day), and demonstration of skills in stress management techniques. Other important behavioral domain measures include adherence to medications and other recommended secondary prevention strategies.
Clinical domain measures are necessary for assessing the patient’s status at program entry to determine the priority strategies in the treatment plan. The measures are repeated at scheduled intervals (eg, at program exit) to evaluate the patient’s progress and guide the development of long-term goals in the discharge plan. Clinical domain measures probably are the most familiar to program staff because they are routinely collected when patients are enrolled in CR. Ideally, medical record documentation should include at least one measure that provides an assessment of the patient in each core component of care: exercise training-functional capacity (maximal or submaximal), lipid management (lipid profile), hypertension management (blood pressure), diabetes management (patients with diabetes, glycosated hemoglobin (HgA1C) or fasting blood glucose level), weight management (body mass index), and psychosocial management (eg, depression screen and other psychosocial measures). If a clinical domain measure representing a specific component of care for the majority of patients is not feasible, then a behavioral domain measure needs to represent that specific core component of care. For example, if a patient is prescribed a limited number of sessions or a brief period in CR, repetition of laboratory tests for assessing lipid management at completion of the supervised sessions may not be warranted or meaningful at that time. However, a dietary habit measure would provide some information about the patient’s progress in making the desired changes known to improve lipid management.
Health domain measures, or global outcomes, include morbidity, mortality, health status, and health-related quality of life. Although morbidity and mortality are valid health domain measures, interpretation of the data at the program level usually is not meaningful or practical because of small sample sizes, the brief interval between measures, and other influences beyond the program’s scope. Outcomes research has pointed to the importance of the patient’s viewpoint on the goals of medical care, so “patient-centered outcomes” are increasingly viewed as the focus of medical interventions in which patient values shape the goals of medical care. 21 A measure of patient-perceived health status or health-related quality of life usually is the most feasible measure for inclusion in CR settings. There are generic and disease-specific measures of health status and health-related quality of life. Whereas generic measures provide general information about patients’ perceived health-related quality of life, specific measures may be more responsive to treatment impact than generic measures. 14
Finally, service domain measures represent an area of outcomes measurement that although often tracked by programs, does not typically fit within the behavioral, clinical, or health outcome domain measures. Service domain measures are those that the program staff can use to evaluate a number of indices regarding the effectiveness of the program’s structure and methods for delivering services. Patient satisfaction is a specific example of a service-related measure that is becoming increasingly more important as a strategic measure in healthcare organizations. 22 Other measures in the service domain include utilization rates (referral, enrollment, and completion rates), costs of providing care, and satisfaction with the various interventions provided.
In summary, a minimum expectation in outcome evaluation is selection of an outcome measures battery that can be administered to the majority of patients and includes a measure for each component of care (behavioral or clinical domain, ideally both), a measure of health status or health-related quality of life, and a measure in the service domain (patient satisfaction, utilization rates, cost data, and the like). The information gained in routinely reviewing the summarized reports should be used to answer the question, “How effective is our treatment overall and in each component of care?”
Patient assessment usually is the most familiar step in outcome evaluation because it is a part of routine clinical care. However, the documentation of patient assessment may not reflect an outcomes-directed approach. Ideally, as each individual patient is enrolled for therapy, standardized baseline assessments are performed, documented, and used to guide the development of individualized goals and a treatment plan. The treatment plan needs to reflect the patient’s unique status relative to the established treatment goals with a specific plan of action to help guide the patient in achieving these short- and long-term health goals. An ideal patient assessment protocol involves collecting and entering the patient data into a database during the initial intake process. This not only reduces staff time by eliminating duplicate documentation (writing data on the form and then reentering data into the database), but also provides an opportunity for immediate generation of a patient report. This patient report can be used for patient and physician feedback to facilitate goal setting and treatment plans, and for medical record documentation. At follow-up assessment, the patient is reevaluated with the measures used at baseline to determine the progress that was made toward the goals. The patient’s outcomes evaluation is used to help guide discharge plans for achieving long-term secondary prevention and health goals. A sample patient assessment and evaluation report that reflects an outcomes-directed approach is illustrated in Figure 2.
Implementation of treatment plans that target individualized patient goals is a minimal expectation in the CR/Secondary Prevention Guidelines. 3 Even if no computer or database is available within a program, paper-and-pen documentation of the patient assessment is necessary to reflect a comprehensive and outcome-directed approach to care as part of the medical record. This patient-level process, the essence of outcomes evaluation, needs to be integrated and documented within routine clinical practice.
Evaluating program effectiveness in improving patient outcomes is an important program function. It provides information that helps programs evaluate how effective the treatment activities were for a group of patients (aggregate patient data) in reaching goals for secondary prevention and health. Quality improvement initiatives can be developed to address the specific issues of concern. Program effectiveness can be evaluated through an internal review of outcomes data by examining the changes observed between pre- and posttreatment and by comparing the outcomes with a predetermined goal. Program effectiveness also can be evaluated by comparing the outcomes of program with the aggregate outcomes of other programs. This often is referred to as “benchmarking” (external comparison). Because the purpose of the current document is to provide a practical approach to outcome evaluation for individual programs, the emphasis in this statement is directed at evaluating the effectiveness of a single program through internal review and comparison with therapy goals.
In an ideal setting, patients’ preintervention and follow-up measures are entered into a computerized database to facilitate ongoing and effective data management, and to generate both patient and program summary reports. Program summary reports reflect a “snapshot” picture of the program’s patient population and trends in outcomes achievement. Such information allows the manager and rehabilitation team to make data-driven decisions in prioritizing program activities and in identifying quality improvement initiatives specific to their program and patient population. Program summary reports are most helpful if they include descriptive information (demographic and clinical) about the enrolled patient population, followed by a summary of the changes (outcomes) achieved among patients who completed the program. A critical review of patient and program outcomes can help identify the program’s unique strengths and weaknesses as well as issues for quality improvement.
A sample report of information that can be used for internal evaluation of program effectiveness is illustrated in Figure 3.]This example is designed for internal critical review and may not be appropriate for external reporting, marketing, or other report purposes. Although numerous options exist for analyzing and displaying aggregate patient outcome data, it is important that the method chosen provide helpful and timely information to that specific program for quality improvement purposes. The sample form provides one option for describing and summarizing a program’s patient population and outcomes. This example shows outcomes results by comparing the distribution of patients “at goal” before and at follow-up assessment within each core component of care. This program summary format can help provide answers when potential quality issues are explored within each of the following outcome domains:
- Behavioral and clinical domains. At discharge, is the proportion of patients at goal higher than at entry in all the components of care? Does the program appear to be effective in contributing to the overall risk reduction observed in this group of patients? If improvement is not observed in all patients, what program changes are needed for the specific subgroup of patients (eg, women, elderly, or patients with a specific risk factor)? Is there a specific behavioral or clinical domain outcome that needs to be targeted for quality improvement actions? What are the current processes of care (assessment, education, counseling, and behavior interventions) that focus on this specific outcome? What can be done differently that may lead to improved outcomes?
- Health domain. Did patients perceive an overall improvement in health status or health-related quality of life at discharge as compared with scores at program entry? How do the scores compare from baseline to program end or with those of a reference population at the time of discharge? Is there a positive trending, or do the results raise issues that need to be addressed?
- Service domain. Does the program’s patient population have demographic characteristics similar to those of the geographic region or the primary referral source in acute care settings? Do the proportions reflect adequate access to the program for minority populations (women, nonwhite, and elderly)? Does the distribution of diagnoses reflect the proportion of diagnoses at discharge from the acute care setting? Are there opportunities to increase recruitment for underserved groups of patients? What is the distribution of patients within the risk stratification categories? Can this information help guide decisions in determining needs for staff training or resource allocation for the program? What is the program completion rate? Are there actions that can be implemented to address adherence issues? Do patients indicate a high level of satisfaction? If not, what can be done to improve patient satisfaction in a problematic area?
Despite the increased level of complexity involved in evaluating program effectiveness, all programs that consistently assess and document outcomes at the patient level are prepared to evaluate program effectiveness. The example provided in Figure 3 is a comprehensive summary report that includes multiple measures for each domain, which may not be feasible for some programs. However, programs must include at least a measure for each component of care (behavioral or clinical domain, ideally both), a measure of health status or health-related quality of life, and a measure in the service domain to help identify the program’s strengths and weaknesses in delivering care. Although more complex analyses are necessary for interpreting the statistical significance of the observed changes, these usually are not necessary at the program level in the provision of practical information to the rehabilitation team. Programs that have the available resources and expertise to perform outcomes research using more complex statistical analyses are encouraged to disseminate their findings through professional publications.
IMPROVING HEALTHCARE QUALITY
Quantification of healthcare quality is a complex and challenging process that often leads to frustration among clinicians who strive to provide the best possible patient care within their available resources. An outcomes approach to patient care and program evaluation can be compared with geology. It goes beyond “rock-collecting” or simply collecting data and generating reports. The numbers need to make sense from the patient and program perspective through data analyses and interpretation. Healthcare professionals must actively engage in the process to ensure that the data collected are relevant and valid. The results need to guide decision making in program activities and to help prioritize the selection of quality improvement plans. Ongoing evaluation is needed to determine whether the plans were effective, or whether different strategies are warranted.
Outcomes are influenced by factors other than the care provided to patients and include demographic and psychosocial characteristics, disease severity, clinical status, and the treatment setting. 23 Thus, risk adjustment is necessary to control the variability when patient outcomes are compared and interpreted among multiple programs. Without proper risk adjustment, the following question will always come up: “Are the differences in outcomes between programs attributable to quality of care or to the underlying differences in patient characteristics or provider settings?” Outcomes measurement serves a critical need in healthcare quality, but unless there is a method for discovering the reasons for observed differences in outcomes, the knowledge of results is useful only for judgment, not improvement. 24 Because of the complexity and sophistication of the analysis required for risk adjustment and because of the substantial possibility of misinterpretation, patient outcomes measurement and evaluation in clinical programs are considered most appropriate for internal quality improvement purposes. 25
Healthcare providers must acquire the knowledge to participate in the assessment of healthcare quality that includes outcomes evaluation. However, the current healthcare environment also demands performance evaluation because quality improvement goals are aimed at eliminating inappropriate variation in the processes, which should lead to improved outcomes. 26 Performance measures are quantifiable measures applied to the steps or processes that lead to an outcome. These performance measures are discrete measures of activities used to define optimal care for which providers are held accountable. For example, an outcome goal in CR and secondary prevention is smoking cessation. The outcome measure is smoking status, number of cigarettes smoked, or both. The performance measure is the method of smoking cessation counseling that was provided. Performance measures are derived from, but are not, practice guidelines. 25 They are intended to provide healthcare providers with tools for measuring the quality of care that they provide by defining specific, measurable elements in their delivery of care.
Interest in performance measures to provide comparative data for external review of healthcare quality is increasing among consumers, regulatory agencies, healthcare organizations, and health professionals. 27 Examples of cardiology projects using performance measures include the National Registry of Myocardial Infarction- II (NRMI-II), 28 Medicare’s Cooperative Cardiovascular Project, 29 the American College of Cardiology—National Cardiovascular Data Registry (ACC-NCDR), 30 and the American Heart Association (AHA)—Get With the Guidelines (GWTG). 31 More recently, evidence-based performance measures for practitioners in outpatient settings have been developed through the collaborative efforts of the AHA, the American College of Cardiology (ACC), and the American Medical Association (AMA) 32 for the diagnoses of stable coronary artery disease, heart failure, and hypertension. These performance measures will be pilot tested in physician offices over the next 2 years, and may continue to evolve as further evidence is gathered. Establishing evidence-based performance measures and outpatient tools specific to the outpatient CR setting that are modeled after the AHA/ACC outpatient performance measures may be a challenging yet progressive step for our professional organization.
Outcomes evaluation permits assessment of effectiveness in providing patient care and the subsequent improvement in quality of care. This is a simple concept, but it involves complex processes that may challenge a program’s resources to achieve the goal. The recently published Guidelines for CR and Secondary Prevention Programs define expectations in outcomes evaluation. 3 The intent of this report is to review the purposes of outcome evaluation and provide practical information and sample forms to help programs integrate an outcomes-directed approach within routine patient care and program functions.
Although it is imperative that individual programs take proactive steps to integrate outcomes evaluation processes within their routine functions, there is a critical need for direction and support from a national perspective to move outcomes efforts forward. Some lessons learned from innovative regional outcomes projects 32–35 have demonstrated the ability of multiple programs to collaborate and initiate standardized protocols, analyze data, and generate descriptive reports. However, leadership and consensus building are needed to help guide a unified national effort in outcomes evaluation that directly links activities and resources to quality improvement goals. From a national unified perspective, the greatest challenges in quantifying quality include confusing terminology, nonstandard definitions, measurement issues, changes in treatment recommendations, and limited resources. 29 With the current imperative to improve healthcare quality, each step taken to address these challenges in outcomes measurement and quality improvement represents advancement for our profession.
The authors acknowledge and thank members of the Outcomes Committee for their helpful suggestions during the preparation of this manuscript: Marjorie L. King, MD, Phillip Hoberty, EdD, RRT, Lynn Baker, RN, MS, Barb Fagan, MS, Valerie Kramer, RN, BSN, Helen Graham, RN, MSN, Lisa Medina, RN, BSN, PHN, Debbie Pacillas, RN, C, BSN, Greg Palevo, MSc, Laura Ann Peno-Green, MD, Terry Ray, RN, MN, and Mark Vitcenda, MS.
They also thank Mark A. Williams, PhD, Vera Bittner, MD, MSPH, and Pat Comoss, RN, BA, for their critical review of this manuscript.
1. James BC. Outcomes
measurement. In: Grenigar J, ed. National Quality of Care Forum: Bridging the Gap Between Theory and Practice: Exploring Outcomes Management.
Chicago, Ill: Hospital Research and Educational Trust; 1994:4–8.
2. Pashkow P, Ades PA, Emery CF, et al. Outcome measurement in cardiac and pulmonary rehabilitation by the AACVPR Outcomes
Committee. J Cardiopulm Rehabil.
3. American Association of Cardiovascular and Pulmonary Rehabilitation (AACVPR). Guidelines for
C ardiac Rehabilitation and Secondary Prevention Programs.
4th ed. Champaign, Ill: Human Kinetics; 2004.
4. Balady GJ, Ades PA, Bazzarre T, et al. AHA/AACVPR Scientific Statement: Core components of cardiac rehabilitation
/secondary prevention programs: a statement for healthcare professionals from the American Heart Association and the American Association of Cardiovascular and Pulmonary Rehabilitation. J Cardiopulm Rehabil.
5. Smith SC, Blair SN, Bonow RO, et al. AHA/ACC guidelines for preventing heart attack and death in patients with atherosclerotic cardiovascular disease: 2001 update: a statement for healthcare professionals from the American Heart Association and the American College of Cardiology. Circulation.
6. National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III). Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) final report. Circulation.
7. Joint National Committee (JNC) on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure. The JNC 7 Report. JAMA.
8. National Heart, Lung, and Blood Institute. Obesity Education Initiative Expert Panel. Clinical Guidelines on the Identification, Evaluation, and Treatment of Overweight and Obesity in Adults: The Evidence Report.
1998. NIH publication 98–4083.
9. American Diabetes Association. Position statement: standards of medical care for patients with diabetes mellitus. Diabetes Care.
10. Kaldyand J, Tarnove L. A clinical practice guideline approach to treating depression in long-term care. J Am Med Dir Assoc.
11. Linden V, Stossel C, Maurice J. Psychosocial interventions for patients with coronary artery disease. Arch Intern Med.
12. Dusseldorp E, van Elderen T, Meulman MS, Kraaif V. A meta-analysis of psychoeducational programs for coronary heart disease patients. Health Psychol.
13. McGee HM, Hevey D, Horgan JH. Psychosocial outcome assessments for use in cardiac rehabilitation
service evaluation: a 10-year systematic review. Soc Sci Med.
14. Oldridge NB. Outcome assessment in cardiac rehabilitation
: health-related quality of life and economic evaluation. J Cardiopulm Rehabil.
15. Wyman R, Vitcenda M, McBride P. The surveillance of cholesterol management in the cardiac rehabilitation
setting. J Cardiopulm Rehabil.
16. Sanderson BK, Phillips MM, Gerald L, DiLillo V, Bittner V. Factors associated with the failure of patients to complete cardiac rehabilitation
for medical and nonmedical reasons. J Cardiopulm Rehabil.
17. James BC. Good enough? Standards and measurement in continuous quality improvement
. In: Greniger J, ed. National Quality of Care Forum: Bridging the Gap between Theory and Practice: Exploring Continuous Quality Improvement.
Chicago, Ill: Hospital Research and Educational Trust; 1992:8–17.
18. Comoss P. The utility and viability of outcome measurement and monitoring in cardiac rehabilitation
[editorial]. J Cardiopulm Rehabil.
19. American Association of Cardiovascular and Pulmonary Rehabilitation (AACVPR). Outcome Tool Resource Guide, 2002.
Available (AACVPR members only) at: http://www.aacvpr.org/publications
. Accessed August 6, 2003.
20. Clancy C. AHRQ: coordinating a quantity of quality. Healthplan.
21. Sullivan M. The new subjective medicine: taking the patient’s point of view on healthcare and health. Soc Sci Med.
22. Kane RL. Approaching the outcomes
question. In: Kane RL, ed. Understanding Healthcare Outcomes Research
. Gaithersburg, Md: Aspen Publishers; 1997:1–15.
23. O’Malley JF. Ultimate Patient Satisfaction: Designing, Implementing, or Rejuvenating an Effective Patient Satisfaction and TQM Program.
New York, NY: McGraw–Hill; 1997.
24. Berwick DM. The double edge of knowledge [editorial]. JAMA
25. Quality of Care and Outcomes
Research Forum Executive Committee and Working Group. Krumholz HM (chair, steering committee). AHA/ACC Conference Proceedings. Measuring and improving quality of care. A Report from the American Heart Association/American College of Cardiology first scientific forum on assessment of healthcare quality in cardiovascular disease and stroke. Circulation.
26. Donabedian A. The quality of care: how can it be assessed?JAMA.
27. Skolnick AA. JCAHO, NCQA, and AMAP establish council to coordinate healthcare performance measurement. Joint Commission on Accreditation of Healthcare Organizations, National Committee for Quality Assurance, American Medical Accreditation Program. JAMA.
28. Every NR, Frederick PD, Robinson M, et al. A comparison of the national registry of myocardial infarction 2 with the cooperative cardiovascular project. J Am Coll Cardiol.
29. Marciniak TA, Ellerbeck EF, Radford MJ, et al. Improving the quality of care for Medicare patients with acute myocardial infarction: results from the Cooperative Cardiovascular Project. JAMA
30. Brandis RG, Fitzgerald S, Anderson HV, Shaw RE, Weintraub WS, Williams JF. The American College of Cardiology-National Cardiovascular Data Registry (ACC-NCDR): building a national clinical data repository. J Am Coll Cardiol.
31. American Heart Association (AHA). Get With the Guidelines.
Available at: http://www.americanheart.org
. Accessed August 6, 2003.
32. American Heart Association (AHA). Performance Measures, June, 2003.
Available at: http://www.americanheart.org
. Accessed August 6, 2003.
33. Jungbauer JS. Fuller B. Feasibility of a multisite outcomes
project for cardiopulmonary rehabilitation. J Cardiopulm Rehabil.
34. Verrill D, Barton C, Beasley W, Brennan M, Lippard M, King C. Quality of life measures and gender comparisons in North Carolina cardiac rehabilitation
programs. J Cardiopulm Rehabil
35. Gulanick M, Gavic AM, Kramer V, Rey J. Outcome in cardiac rehabilitation
programs across Illinois. J Cardiopulm Rehabil.
36. Vitcenda M, The Wisconsin outcome experience: baseline outcomes
of the WISCVPR Web-based outcomes
project. J Cardiopul Rehabil.
37. Wenger NK, Froelicher ES, Smith LK, et al. Cardiac Rehabilitation.
Clinical Practice Guideline No.17, Rockville Md: US Department of Health and Human Services, Public Health Service, Agency for Healthcare Policy and Research and the National Heart, Lung, and Blood Institute. AHCPR Publication 96–0672, October 1995.
38. American Association of Cardiovascular and Pulmonary Rehabilitation. Guidelines for
C ardiac Rehabilitation and Secondary Prevention Programs.
3rd ed. Champaign, Ill: Human Kinetics; 1999.