Introduction
The US Centers for Medicare and Medicaid Services (CMS) has added a measure of hospital readmissions for patients on dialysis to be included in the ESRD Quality Incentive Program (QIP) beginning with payment year 2017 (1). The performance period for the measure began January 1, 2015 (1). There can be no doubt that hospital readmission of patients on dialysis is an area of opportunity for which performance improvement activities should be focused. In 2012, 35.2% of United States patients on dialysis were readmitted to an acute care hospital within 30 days of an index hospitalization (2). This rate is higher than that observed for other serious chronic diseases: 25% in congestive heart failure (3) and 18% in chronic obstructive pulmonary disease (4).
It is highly plausible that improvement in ESRD readmissions is attainable, particularly through timely reevaluation postdischarge. A first step in performance improvement is the development of a well designed quality measure; in contrast, a flawed measure fails to drive meaningful change, frustrates users, and can lead to adverse unintended consequences (5). We believe that the measure developed for readmissions in dialysis, the standardized readmission ratio (SRR), is flawed and unlikely to drive true improvement. Moreover, the process used to push through the measure to a great extent bypassed substantive input to the CMS by stakeholders as required by the Social Security Act (added by section 3014[b] of the Affordable Care Act) (1). In this article, we will provide accounts of the endorsement process, we will critically evaluate the SRR measure and review the role of CMS and the National Quality Forum (NQF) in quality measure development, and we will suggest ways in which we believe hospital readmissions of patients with ESRD could be reduced.
A Process Run Amok
Performance measures that affect public reporting and payment are required by CMS rules to go through evaluation by a technical expert panel (TEP) convened by the CMS measure contractors (6) and then, are submitted to the NQF for assessment, stakeholder consensus building, and potential endorsement (7). Although the CMS is not required by legislation to follow the recommendations that emerge from this process, it would seem wise and collegial to do so, because this would drive the development of optimized measures, enhance buy in, assure transparency, and establish the credibility of the process.
A TEP on the SRR measure was convened May 1 and 2, 2012, “charged with providing recommendations to Arbor Research/UM-KECC (the contractor for the CMS) for specifications” (8). The TEP met, raised substantial issues, and noted flaws in the SRR measure’s design. The committee voted negatively on all but one aspect of the measure (the only positive vote was on feasibility; voting was negative on importance, reliability, and validity and usability).
Despite the unfavorable evaluation by the TEP, the SRR measure moved to the NQF for consideration for endorsement. A new NQF committee, the Standing Committee on All-Cause Admissions and Readmissions (SCAR), discussed the SRR measure in detail, and the contractor was given the opportunity to respond to multiple criticisms raised. After full deliberations, SCAR voted 12 to eight against the measure. Because only 40% of voting committee members voted in favor, by NQF policy, this placed the measure in the category of consensus not reached. This designation is made when a measure receives 40%–60% of positive votes.
The SRR measure was then sent out for public comment. The SRR received 10 comments, some extensively discussing the measure; nine of 10 comments were negative, including highly critical remarks from several entities. There were lengthy explanations of both technical and practical problems.
The process continued with the important step of voting by the actual NQF membership. A total of 30 NQF members voted, a relatively small proportion of the >400 total membership. However, the vote on the SRR was unambiguously negative, with 86% of votes against the measure and a final determination by membership to not recommend the measure.
The next step in the process would be the NQF Consensus Standards Approval Committee (CSAC) conducting a final vote on endorsement. The NQF states that the CSAC “…reviews the recommendations of Steering Committees and the results of NQF Member voting periods. After detailed review of a candidate standard, the CSAC determines if consensus has been reached...” (9). It would have seemed to be self-evident that the NQF process had been completed with no possible achievable consensus for the SRR measure. However, the roles of both the standing committee and the NQF membership are advisory and not binding to the NQF (and the CMS as well). On November 25, 2014, an email was sent by the NQF to the SCAR members that announced that the CSAC had voted positively to endorse the SRR measure, with no explanation provided. The NQF’s Board of Directors then approved the endorsement. The NQF process allows for appeals. On February 10, 2015, the Renal Physicians Association (RPA) on behalf of several renal stakeholder organizations appealed. The NQF’s CSAC rejected the appeal by a 92% vote. Then, in a stunning reversal, on March 5, 2015, the executive committee of the NQF, after hearing the RPA again explain the problems with the SRR measure, voted to withdraw endorsement. At the time of this writing, the endorsement’s fate remains in limbo.
Ultimately, the purpose of the NQF endorsement is to help the CMS comply with legislation that it obtains stakeholder input for quality measures used for public reporting and pay for performance. The NQF is the current contractor to accumulate information on whether consensus exists for measures. Given this, it is puzzling that, approximately 2 weeks before the CSAC vote, on November 6, 2014, the CMS had already published a final rule on the ESRD QIP that included the SRR as a pay-for-performance measure starting in payment year 2017; the performance period began January 1, 2015.
A Critical Appraisal of the SRR
As stated above, we believe that the number of readmissions of patients on dialysis is excessively high and that there is a clear opportunity for improvement. Quality measurement is a critically important first step in driving performance improvement. However, the metric must be properly designed to be effective.
The SRR is the ratio of unplanned readmissions that occurred within 30 days of hospital discharge for Medicare beneficiaries to the number of readmissions that would be expected. The expected number is calculated by considering characteristics of the discharging hospital and patients. The numerator of SRR is the dialysis facility’s actual number of discharges followed by an unplanned readmission within 30 days. The denominator is the expected number of readmissions derived from a model that uses regression adjustments for patients, random effects for hospitals characteristics, and fixed effects for facilities.
In a general sense, there is an inherent problem with hospital readmission measures. A falling total hospitalization rate may, in fact, occur, even if readmissions do not fall. This creates bias in the interpretation of readmissions, because healthier patients are managed more as outpatients with sicker patients who comprise a greater proportion of hospitalizations. The increase in disease severity of admitted patients (as well as the trend to shorter lengths of hospital stays) would be expected to lead to a higher rate of readmissions. This set of circumstances has, in fact, existed in the ESRD population. Over the last decade, the overall hospitalization rates have fallen from 1.94 (1999) to 1.72 (2012; an 11.5% drop) (2). The days hospitalized have also fallen significantly (2). This means that those hospitalized are likely sicker and that they may, in fact, be as or more likely to be readmitted. This confounds interpretation of the SRR, in that it does not even address trends over time. The following points are additional problems with the SRR measure as noted by several entities.
(1) The most important flaw with the SRR measure is its attribution or assigning ownership of the measure to the dialysis facility. We find it reasonable and almost self-evident that several care processes (Table 1) are likely to be the most important to reduce readmissions (10–12). In order of importance, the following parties have most control over these processes: the nephrologist, the discharging hospital, and at a distant last, the dialysis facility. It is a mistake to attribute the quality measure to the dialysis unit, especially because the measure does not include coattribution to the nephrologist or hospital. Evidence for the critical role of the nephrologist was reported recently in the study by Erickson et al. (13), which found reduced readmissions risk with more frequent nephrologist visits posthospitalization. As for the dialysis facility, how does its staffing relate to these processes?
- A dialysis facility nurse could help with medication reconciliation, but especially after a hospitalization, medication assessment requires a physician’s involvement.
- Detailed patient reassessment after a hospitalization with changes in health status and medication use is generally above and beyond the training, experience, and licensure of patient care technicians and most registered nurses, and it should be performed by a nephrologist. The facility’s medical director is a nephrologist, but his/her responsibilities are not for direct patient care of any facility patients other than his/her own patients.
- Dry weight assessment needs to be conducted by a clinician, arguably the most important activity in preventing readmissions.
- Determination of required medical and other follow-up is, in large, part a clinical function.
;)
Table 1: Processes likely to be most important for reducing rehospitalization of patients on dialysis
Because the processes of care that are most important for preventing readmissions are ones that the facility has little ability to directly influence, the dialysis facility should not be the sole owner of the SRR measure. The CMS has repeatedly countered this by putting forth the need to improve coordination between the dialysis facility, the hospital, and the nephrologist. We fully agree with this assertion; however, the use of a quality measure that is misdirected in attribution to achieve this goal is conceptually flawed. Changed regulation and payment structures are required for improved coordination.
(2) Approximately 17% of readmissions occur in the first 3 days after hospital discharge, a period during which the patient may not yet have returned to the dialysis unit. These readmissions occur too early for the dialysis facility to have any ability to prevent them, but they are included in the SRR numerator.
(3) Depending on the mix at a facility between patients on in-center dialysis and patients on home dialysis, the readmission rate may be significantly affected, but the SRR does not directly adjust for this factor.
(4) Sociodemographic factors are likely to be important in readmission risk, but the initial SRR measure does not account for them. It seems at the time of this writing that this issue will be evaluated with the rollout of the measure.
(5) Unplanned readmissions for any cause are included in the numerator of the SRR, including causes that the facility may have no control over, such as a motor vehicle accident. It has been estimated that 45% of dialysis readmissions are because of causes beyond the ability of the dialysis facility to prevent. Ideally, dialysis- and kidney disease–specific causes should be the only included admissions. Specifically, volume management is at the core of what the dialysis procedure addresses; admissions for volume overload should be included. Hypertension treatment and control and electrolyte regulation are other important management issues for dialysis units. Infection control and prevention are basic measures of a dialysis unit’s function, with limitation to vascular access infections, peritonitis and peritoneal dialysis catheter infections, and pneumonia and influenza prevention. These should all be included in the numerator as under the control of the dialysis facility and related to dialysis care. Another potentially unintended consequence of too broad a definition of rehospitalizations included in the numerator is a motivation to not rehospitalize a patient who is truly sick and requires hospitalization. We would not want to see the facility hesitate to refer a patient with chest pain postdischarge for hospital evaluation.
(6) Although certain planned readmissions are excluded from the SRR, those related to vascular access are not. These are usually necessary admissions and should not be included in the numerator for the SRR.
(7) Evaluations of reliability and validity are required by the NQF, and this was performed for the SRR; however, both were somewhat problematic. Reliability is a measure of test results consistency. The developers defined reliability by the interunit reliability (the proportion of variability attributable to between-facility variance). The interunit reliability was calculated to be 0.55, indicating a relatively low level of reliability. Validity (test accurately measures what it is supposed to measure) was evaluated by comparing with other quality measures. There were very low levels of correlation found for most, a cause for concern. The one measure that was highly correlated with the SRR was the standardized hospitalization rate (r = 0.53; P<0.001). Because hospitalization is required for rehospitalization, this relationship is with only limited meaning.
(8) The SRR risk adjustment model was criticized in public comments. In particular, although the characteristics of the discharging hospital are partially adjusted for, there is no adjustment for physician admitting patterns. This is particularly important, in that the physician both has the most control over a safe reentry to the dialysis facility if he/she sees the patient soon after discharge and makes the decision to send the patient back to the hospital if symptoms subsequently arise. Busy nephrologists may take the path of least resistance when a new symptom occurs and send the patient to the hospital rather than conduct an assessment.
(9) Commenters were also troubled by the SRR denominator definition by number of discharges rather than the number of patients at the facility. One or two complex patients at a smaller facility can have a large effect on the SRR.
A readmissions quality measure that creates joint responsibility among the nephrologist, the dialysis facility, and the hospital would most effectively target attribution. Changes in regulation and financial incentives could also be helpful, however, to fully drive improvement. We suggest that additional funding for a midlevel practitioner and/or case manager in a CMS-regulated role for care coordination would greatly improve quality. A separate carve-out payment to nephrologists for posthospitalization services would also help.
(10) Another problem with the SRR measure relates to the lack of evidence to support the measure focus. There can be no possible assertion that there is sufficient evidence. There simply are no published articles related to the readmissions of patients on dialysis in the current literature that, by the traditional hierarchy of evidence classification, would qualify as higher-order evidence. There is a retrospective observational analysis of risk factors. Although it is a good quality study, the results are somewhat trivial: that hemoglobin monitoring within 7 days after discharge with erythropoiesis stimulating agent dose adjustment and administration of vitamin D were both associated with lower risk for readmissions (14). This study does not constitute even the beginning of a reasonable body of evidence to indicate that process changes would reduce readmissions.
The NQF consideration of evidence works as follows. For process measures or intermediate outcomes, the threshold for evidence is quite stringent and reasonable. In contrast, for health outcome measures, the NQF simply requires that developers articulate a rationale for how the outcome is influenced by health care processes. There is no requirement for medical evidence. This subject is certainly controversial, and any detailed consideration of how much evidence should be required for outcome measures is beyond the scope of this article. One school of thought is that the threshold of evidence for quality intervention should generally be relaxed (15). In our opinion, any measure that is used for public reporting or payment for performance carries special weight that creates risk for unintended and potentially adverse consequences. As a result, we believe that there should be at least a strong basis in evidence for health outcome measures (one published retrospective study, which is the case for SRR, would not meet this threshold).
The NQF requirement for no evidence is also problematic, in that the representation to the public is that there was an assessment of evidence; there was not. Standing committee members were clearly instructed to vote simply on whether a rationale (as described above) could be articulated. The NQF description of the subcriterion as an assessment of “evidence to support measure focus” (16) is misleading. It leaves the public with a sense that medical evidence was evaluated. The problem should be addressed by clearly stating that medical evidence was not evaluated or voted on to ensure transparency and credibility of the process.
(11) The SRR, unlike the standardized hospitalization ratio and standardized mortality ratio, does not exclude patients during the first 90 days of dialysis. There should be harmonization with the other rates, because they will likely be used together for a general understanding of facility quality of care.
Reducing Dialysis Readmissions
The high rate of dialysis 30-day readmissions is at least, in part, because of the frailty, disease burden, and complexity of patients on dialysis. Readmissions for these reasons are probably not substantially addressable. In contrast, readmissions related to suboptimal care processes seem to be an important target for improvement. Flawed processes, such as resuming previous dialysis orders without a new assessment posthospitalization, suggest avenues for improvement. Although interventions to reduce dialysis readmissions have not been widely tested or reported on, there are some care processes highly likely to play a central role in reducing readmissions (Table 1).
- (1) Reassessment of dry weight. During hospitalizations, patients on dialysis frequently experience weight changes. With prolonged hospitalization, there may be a significant loss of lean body mass. Failure to readjust dry weight postdischarge may be a leading cause of readmissions related to volume overload. Instructions to simply resume previous orders are not acceptable.
- (2) Medication reconciliation. Chronic medications are often changed in the hospital. Inadequate reconciliation at discharge and on reentry to the dialysis unit may result in significant errors. These include therapeutic deletions, duplication, or dosing errors among others. Ideally, patients and/or their key caregivers should bring actual containers of medicines to the dialysis unit postdischarge for reconciliation.
- (3) Assessment of recovery from acute illness. After hospitalization, a medical assessment focused on medical stability and recovery from the acute illness is critically important. With today's relatively short hospitalizations, patients may be discharged before full stability is achieved. In some patients, additional acute care may still be required. In others, adjustments to the dialysis prescription on the basis of the patient’s medical status may be necessary.
- (4) Posthospitalization follow-up. Postdischarge follow-up with other physicians or related medical services may be indicated. Assessment of the need for follow-up and assurance that follow-up is completed may reduce readmission risk.
- (5) Because a number of issues require coordination and facilitation, a new dialysis facility role may prove to be beneficial. A transitions coordinator could interact with the discharging hospital to obtain hospitalization and discharge information; this individual can help coordinate an early postdischarge nephrologist visit. Subsequent required posthospital testing and physician appointments could be facilitated. Ideally, this role would be created with support of the conditions for coverage for dialysis facilities and adjustment to reimbursement.
- (6) The ability of the nephrologist to carry out the postdischarge processes discussed requires difficult adjustment of schedules to match the timing of return to dialysis. With the complicated schedules that nephrologists work under, a carve-out payment to incentivize early postdischarge care is likely to be successful, especially because the main processes that could prevent rehospitalization are mostly under the nephrologist’s control.
Context and Conclusion
The weaknesses of the SRR measure were reflected by the negative evaluation by the TEP, the failure of consensus by the SCAR of the NQF, the highly negative public comments, and the strong rejection by the NQF member vote. The surprising initial endorsement of the measure by the NQF suggests vulnerabilities in the process and a need for greater transparency.
The CMS is generally required by law to seek stakeholder input on quality measures used for public reporting or payment for performance. Not only was the SRR made official by the CMS before the NQF endorsement, but the agency was clearly aware of the multiple criticisms of the measure. The broadly negative assessment all occurred well before the CMS final rule, which included the SRR, was published on November 6, 2014. In this context, the difficult position that the NQF CSAC faced when it voted on or about the second week of November of 2014 becomes clearer.
The NQF enjoys special status as the “entity with a contract under section 1890(a) of the [Social Security] Act to convene multi-stakeholder groups to provide input to the Secretary on the selection of quality and efficiency measures” (17). This contract with the CMS is valuable for multiple reasons, and this creates a potential conflict that the NQF must manage with great care to continue its record of credibility. We believe that the parties acted with good intent as it related to the SRR measure. It may ultimately be that pressure created by the rapid rate with which the CMS pushed through the SRR measure exposed a vulnerable system susceptible to failure. We would suggest that the CMS should suspend planned use of the SRR measure for public reporting and payment for performance systems until an improved measure is developed. In addition, we would suggest that, going forward, the CMS should work more closely with the nephrology community to develop measures that will lead to improved quality and outcomes.
Disclosures
S.F. was a member of the National Quality Forum Standing Committee on Admissions and Readmissions that reviewed the standardized readmission ratio (SRR) measure. J.B.W. was a member of the US Centers for Medicare and Medicaid Services Technical Expert Panel that reviewed the SRR measure.
Published online ahead of print. Publication date available at www.cjasn.org.
See related article, “Brokering Compromise to Achieve Consensus,” on pages 195–197.
References
1. U. S. Government Printing Office—Federal Register. Available at:
http://www.gpo.gov/fdsys/pkg/FR-2014-11-06/pdf/2014-26182.pdf. Accessed January 12, 2015
2. U. S. Renal Data System—Section on
ESRD Hospitalizations. Available at:
http://www.usrds.org/2014/view/Default.aspx. Accessed May 20, 2015
3. Aggarwal S, Gupta V: Demographic parameters related to 30-day readmission of patients with congestive heart failure: Analysis of 2,536,439 hospitalizations. Int J Cardiol 176: 1343–1344, 2014
4. Nguyen HQ, Chu L, Amy Liu IL, Lee JS, Suh D, Korotzer B, Yuen G, Desai S, Coleman KJ, Xiang AH, Gould MK: Associations between physical activity and 30-day readmission risk in chronic obstructive pulmonary disease. Ann Am Thorac Soc 11: 695–705, 2014
5. Kanwar M, Brar N, Khatib R, Fakih MG: Misdiagnosis of community-acquired pneumonia and inappropriate utilization of antibiotics: Side effects of the 4-h antibiotic administration rule. Chest 131: 1865–1869, 2007
7. National Quality Forum. Available at:
http://www.qualityforum.org/what_we_do.aspx. Accessed January 23, 2015
10. Plantinga LC, Jaar BG: Preventing repeat hospitalizations in dialysis patients: A call for action. Kidney Int 76: 249–251, 2009
11. Wish JB: The role of 30-day readmission as a measure of quality. Clin J Am Soc Nephrol 9: 440–442, 2014
12. Hakim RM, Collins AJ: Reducing avoidable rehospitalization in
ESRD: A shared accountability. J Am Soc Nephrol 25: 1891–1893, 2014
13. Erickson KF, Winkelmayer WC, Chertow GM, Bhattacharya J: Physician visits and 30-day hospital readmissions in patients receiving hemodialysis. J Am Soc Nephrol 25: 2079–2087, 2014
14. Chan KE, Lazarus JM, Wingard RL, Hakim RM: Association between repeat hospitalization and early intervention in dialysis patients following hospital discharge. Kidney Int 76: 331–341, 2009
15. Berwick DM: The science of improvement. JAMA 299: 1182–1184, 2008
17. U. S. Government Printing Office—Federal Register. Available at:
http://www.gpo.gov/fdsys/pkg/FR-2013-12-02/pdf/2013-28451.pdf. Accessed January 28, 2015
