Assessment of the reliability of performance measure (PM) abstraction is an important step in PM validation. Reliability has not been previously assessed for abstracting PMs for the referral of patients to cardiac rehabilitation (CR) and secondary prevention (SP) programs. To help validate these PMs, we carried out a multicenter assessment of their reliability.
Hospitals and clinical practices from around the United States were invited to participate in the Cardiac Rehabilitation Referral Reliability (CR3) Project. Twenty-nine hospitals and 23 outpatient centers expressed interest in participating. Seven hospitals and 6 outpatient centers met participation criteria and submitted completed data. Site coordinators identified 35 patients whose charts were reviewed by 2 site abstractors twice, 1 week apart. Percent agreement and the Cohen κ statistic were used to describe intra- and interabstractor reliability for patient eligibility for CR/SP, patient exceptions for CR/SP referral, and documented referral to CR/SP.
Results were obtained from within-site data, as well as from pooled data of all inpatient and all outpatient sites. We found that intra-abstractor reliability reflected excellent repeatability (≥90% agreement; κ ≥ 0.75) for ratings of CR/SP eligibility, exceptions, and referral, both from pooled and site-specific analyses of inpatient and outpatient data. Similarly, the interabstractor agreement from pooled analysis ranged from good to excellent for the 3 items, although with slightly lower measures of reliability.
Abstraction of PMs for CR/SP referral has high reliability, supporting the use of these PMs in quality improvement initiatives aimed at increasing CR/SP delivery to patients with cardiovascular disease.
We carried out a multicenter assessment of the reliability of abstracting cardiac rehabilitation referral performance measures (PMs), an important step in PM validation. Intra- and interabstractor reliability was good to excellent, providing support for the use of these PMs in quality improvement initiatives aimed at increasing cardiac rehabilitation delivery.
Cardiovascular Health Clinic, Mayo Clinic and Foundation, Rochester, Minnesota (Drs Thomas and Pack and Mr Lahr); American College of Cardiology Foundation (Mr Chiu and Ms Shahriary) and American Heart Association (Mr Chiu), Washington, District of Columbia; Colorado School of Public Health, Aurora (Dr Goff); Helen Hayes Hospital, West Haverstraw, New York (Drs King and Lichtman); GRQ, LLC, Vienna, Virginia (Ms Lui); and Columbia University, New York, New York (Dr King).
Correspondence: Randal J. Thomas, MD, MS, Cardiovascular Health Clinic, Mayo Clinic and Foundation, 200 First St, SW, Rochester, MN 55905 (firstname.lastname@example.org).
Dr Goff is a member of (1) an operations committee for a clinical trial with sponsorship by Merck, (2) a speakers bureau for a talk sponsored by Merck, and (3) a data monitoring and safety board for a clinical trial with sponsorship by Takeda. Otherwise, the authors declare no conflicts of interest.