An estimated GFR (eGFR) <60 ml/min per 1.73 m2 (1) identifies the presence of CKD and is associated with a significantly higher risk of death, most commonly due to cardiovascular disease (2–4). Patients with CKD often do not receive optimal therapy (5–7), perhaps because physicians do not recognize earlier stages of CKD or are unaware that CKD confers elevated cardiovascular risk. This is particularly important given that treatments have been shown to delay or prevent ESRD and improve cardiovascular outcomes (8–13).
One strategy for transferring evidence from research into practice has been to combine clinical practice guidelines with point-of-care physician reminders (14,15). Similar to conditions such as dyslipidemia (for which information to aid physicians with interpretation often accompanies laboratory results), there has been widespread enthusiasm for the use of laboratory prompts to facilitate recognition of CKD (16). Accordingly, many jurisdictions have implemented eGFR reporting, meaning that eGFR is also reported when a provider requests serum creatinine measurements (16,17). A recent systematic review noted that eGFR reporting increases the rate of nephrologist referrals (16), but only a minority of studies assessed whether eGFR reporting influences the use of medications modulating the renin-angiotensin-aldosterone system (16), with the largest time-series analysis showing no change in use (17). No prior randomized studies have examined the effect of eGFR reporting on process-based outcomes (i.e., appropriate medication use) or clinical outcomes.
Despite the lack of evidence to support eGFR reporting on its own, there is considerable interest in adding additional information on optimal management of patients with CKD to the eGFR laboratory prompt to increase uptake of beneficial therapies and perhaps improve outcomes (18–20). However, no prior studies have evaluated the effect of an enhanced management-based eGFR prompt. Moreover, whether point-of-care reminders improve processes of care and clinical outcomes is uncertain (21–23).
We conducted a cluster randomized trial testing the effect of an enhanced laboratory prompt for patients with CKD managed by primary care physicians compared with a standard eGFR laboratory prompt.
Materials and Methods
We used a provincial repository of laboratory data to identify patients with CKD, defined as eGFR <60 ml/min per 1.73 m2 (24). Available data included the test result as well as the ordering physician, allowing patients to be randomized and analyzed within clusters. Provincial administrative data were used to determine nephrology visits, hospitalization, and vital status. Because drug coverage is only provided for Albertans aged >65 years by the provincial health ministry, medication use outcomes, including our primary outcome, could only be assessed in elderly individuals (defined as those aged ≥66 years, allowing 1 year to accrue baseline drug status). Other outcomes including all-cause mortality and cardiovascular outcomes could be assessed for all participants, regardless of age. Ethics approval was obtained from the institutional review boards of the Universities of Calgary and Alberta (trial registration no. ISRCTN26610787).
The primary study population included all elderly CKD patients with diabetes or proteinuria because our primary outcome, use of an ACE inhibitor (ACEi) or angiotensin receptor blocker (ARB), could only be assessed in elderly patients and because use of these medications in patients with diabetes or proteinuria is recommended by clinical practice guidelines (25,26). Secondary outcomes not requiring medication data were assessed in all CKD patients aged >18 years. Patients were identified as having CKD during the 1-year enrollment period, on the basis of an outpatient eGFR requested by primary care physicians in eligible practices that was <60 ml/min per 1.73 m2. Eligible practices were located in three Alberta Health Regions and included those with ≥2 full-time primary care physicians who worked at a single practice, reducing the risk of physicians being exposed to both the control and intervention laboratory prompt across multiple practices. On the basis of availability of laboratory data, the enrollment period varied by health regions: June 15, 2006 to June 15, 2007 in two regions, and January 1, 2007 to January 1, 2008 in the third. One region was largely urban (population serviced approximately 1 million), whereas the two others (populations serviced approximately 70,000 and 160,000) were a mixture of urban and rural communities. Subgroups considered included those in whom eGFR was <30 ml/min per 1.73 m2 (a group at higher risk of complications due to CKD in whom primary care physicians may be reluctant to prescribe an ACEi or ARB), and those with incident CKD (defined as absence of documented eGFR <60 ml/min per 1.73 m2 in the prior 2 years).
Control and Intervention Prompts
To reduce the risk of contamination between physicians co-managing patients with CKD, randomization occurred at the level of the group practice using a computer-generated randomization sequence and concealment. Physicians in the control group who ordered serum creatinine measurements continued to receive the standard eGFR prompt for patients with an eGFR <60 ml/min per 1.73 m2 (Supplemental Appendix 1). For physicians randomized to the enhanced eGFR prompt, additional education about the significance of CKD and specific management suggestions were reported in addition to eGFR. These suggestions included guideline-based (25,26) recommendations to measure urine albumin/creatinine ratio, prescribe an ACEi or ARB in patients with diabetes or albumin/creatinine ratio >35 mg/mmol, reduce BP to <130/80 mmHg, reduce LDL cholesterol to <2.5 mmol/L, and target hemoglobin A1C to <7% in patients with diabetes (Supplemental Material).
The primary outcome was the proportion of elderly CKD patients with diabetes or proteinuria who filled a prescription for an ACEi or ARB within 1 year of the first prompt being received by the physician (index eGFR measurement), irrespective of baseline ACEi/ARB use. We chose this primary outcome because ACEi or ARB delays progression to ESRD in those with proteinuria (11–13), improves survival (in those with diabetes) (11), is recommended by practice guidelines, and was explicitly recommended by the enhanced prompt. Secondary outcomes, measured in patients with CKD regardless of age, represented markers of good quality care, or relevant clinical outcomes for patients with CKD (Supplemental Material).
Other Variable Definitions
Demographic and baseline clinical data were determined from provincial administrative data during the 2-year period before the index eGFR measurement. We used validated algorithms to define hypertension (27) and diabetes (28), with other comorbid conditions based on the Deyo classification of Charlson comorbidities (29). The presence of proteinuria varied based on the laboratory testing that the patient had, but was defined by quantitative assessment of proteinuria (i.e., measured proteinuria or albuminuria >300 mg/d) if available or by urine dipstick results if quantitative testing was unavailable (≥1+ proteinuria on urine dipstick), using the testing done within 6 months of the index eGFR measurement (24).
Our sample size was estimated using a variety of assumptions based on the formula for cluster randomized trials with a binary outcome (30). On the basis of clinical expert consensus and for consistency with other studies, we deemed a 10% absolute increase in ACEi/ARB use to be the minimally important difference for this intervention. Because we had no a priori information on the intracluster correlation coefficient (ICC), we estimated power using ICC estimates of 0.01, 0.05, and 0.10. We estimated that we would have 75%–99% power to detect a 10% difference in the primary outcome with ICCs varying from 0.01 to 0.10. In post hoc analyses, given observed average cluster sizes of 58, an ICC of 0.02, and 90 primary care practices, we had >99% power to detect a 10% difference and 84% power to detect a 5% difference between groups.
We designed, conducted, and reported this trial according to published guidelines (31). We used a cluster randomized design in which the unit of observation was the patient, and the unit of randomization was the primary care practice. Randomization was concealed and stratified on the basis of the health region and whether the practice was located in a center with a population of <25,000 or >25,000. All analyses were intention to treat. Relative risk estimates were obtained using generalized estimating equations and binomial regression with a log-link function (30,32). The ICC was obtained from a random-intercept logistic regression model (33). To simultaneously control for individual-level and cluster-level covariates, we also estimated adjusted relative risks to obtain a more accurate variance associated with the odds of success for bivariate outcomes (30). The variables included were at the following levels: individual (age, sex, baseline kidney function, diabetes, congestive heart failure, cardiovascular disease), physician (sex, years in practice), and cluster (urban/rural, health region, and number of physicians per cluster). We conducted subgroup analyses for each outcome in patients in whom eGFR was <30 ml/min per 1.73 m2 on first measurement, and in those with incident CKD. We used Stata software (version 11; Stata Corp, College Station, TX) for all analyses.
The majority of primary care physicians and patients with CKD in the three health regions were included (Figure 1 and Supplemental Figure 1) and baseline characteristics of physician clusters were similar across study groups (Supplemental Table 1). Patients with CKD allocated to the enhanced prompt clusters were similar to those allocated to the standard prompt—both for elderly patients with diabetes or proteinuria, and for all patients aged ≥18 years (Table 1).
ACE or ARB Use in Elderly CKD Patients with Diabetes or Proteinuria.
Of the 5444 elderly CKD patients with diabetes or proteinuria who were eligible for assessment of the primary outcome (irrespective of baseline ACEi/ARB use), 483 had eGFR<30 ml/min per 1.73 m2. ACEi or ARB use was 77.1% and 76.9% in the standard and enhanced prompt groups, respectively (relative risk [RR], 1.00; 95% confidence interval [95% CI], 0.96–1.04]) (Table 2). We noted no difference in ACEi or ARB use between the standard and enhanced prompt groups when we repeated the analysis considering only patients who were not using an ACEi or ARB at baseline, or when we considered the subgroup of patients with significant proteinuria in whom ACEi or ARB use could be considered standard of care. When we considered the subgroup of 5055 elderly CKD patients with diabetes or proteinuria who had two eGFR measurements <60 ml/min per 1.73 m2, in whom the diagnosis of CKD was confirmed according to clinical practice guidelines (1), we also noted no difference in ACEi or ARB use (RR, 1.00; 95% CI, 0.96–1.04]).
We noted a significant interaction between the intervention (i.e., standard and enhanced prompt groups) and severity of CKD (eGFR 30–60 versus <30 ml/min per 1.73 m2) (P=0.015). Among patients with an eGFR<30 ml/min per 1.73 m2, ACEi or ARB use was 13% higher (RR, 1.13; 95% CI, 1.03–1.24) in the enhanced compared with the standard prompt group (Table 2). We found no difference between groups in the proportion of patients receiving ACEi or ARB in the subgroup with incident CKD.
For all outcomes, adjusted and unadjusted results were similar. No differential response to the enhanced prompt was noted for female and male physicians, or for physicians practicing for <10 years or >10 years.
Elderly CKD Cohort.
Among elderly CKD patients, there was no difference in the proportion of patients in whom a prescription for a cholesterol-lowering drug or an additional antihypertensive medication from a different therapeutic class was filled in the subsequent year between the two groups (Figure 2).
All Adults with CKD.
Among all patients aged ≥18 years with no measure of proteinuria in the prior 6 months, albuminuria was measured during the 6 months after randomization in 7.4% and 6.3% of patients in the enhanced and standard prompt groups, respectively (RR, 1.29; 95% CI, 1.03–1.62) (Figure 3). Among those with an eGFR <30 ml/min per 1.73 m2, corresponding proportions were 14.7% and 10.2%, respectively (RR, 1.50; 95% CI, 1.02–2.22). Results were similar when other measures of proteinuria (i.e., protein/creatinine ratio, urine dipstick) were considered. There were no differences between groups in the proportion of patients who had LDL cholesterol subsequently measured or the proportion of patients with diabetes who had hemoglobin A1C measured (Figure 3). Among patients with a baseline hemoglobin A1C >7.0%, we also noted no difference in the proportion of patients who had hemoglobin A1C of <7% during the observation period in the enhanced (35.5%) and standard (37.7%) laboratory prompt groups, respectively (RR, 0.94; 95% CI, 0.81–1.09). Finally, among the subgroup of patients with incident CKD, we found no difference between groups in the above measures (Supplemental Figure 2).
There was no difference in the proportion of patients with an eGFR <30 ml/min per 1.73 m2 who were seen by a nephrologist between the two groups (Figure 2). Results were similar when we considered referrals to internal medicine specialists. We also assessed whether the different prompts might result in referral of different types of patients (i.e., referral of more high-risk patients with proteinuria, or low-risk patients with an eGFR 30–60 ml/min per 1.73 m2 without proteinuria) but found no differences in the types of patients referred to nephrologists (Figure 2).
During a median follow-up of 2.1 years, there was no difference in the likelihood of a composite clinical outcome (death, ESRD, doubling of serum creatinine, or hospitalization for myocardial infarction, heart failure, or stroke) for patients receiving the enhanced prompt (14.0%) compared with the standard prompt (13.3%), overall (RR, 1.00; 95% CI, 0.89–1.12) or in subgroups (Figure 4).
eGFR reporting has been advocated based on its potential to improve the recognition and management of CKD by primary care physicians (16). A logical extension to eGFR reporting is providing additional information to the prompt with the goal of enhancing risk prediction and management. In this cluster randomized trial, we found that an enhanced eGFR laboratory prompt did not improve specific elements of care processes or clinical outcomes in patients with CKD.
Clinical decision support has been shown to change physician practice in many randomized trials, across a wide range of conditions and interventions (34,35), although only 7 of 52 studies included in a meta-analysis of clinical decision support noted an improvement in patient outcomes (34). Recent systematic reviews have shown that three features are strongly associated with effective clinical decision support: routinely providing guidance as part of clinician workflow, providing guidance at the time and location of decision making, and providing a recommendation rather than an assessment (21,35). In both the intervention and control groups of our study, guidance was provided as part of clinician workflow and recommendations were provided to the enhanced prompt group, although not at the time and location of decision making.
Why was the enhanced laboratory prompt not effective at improving processes of care in the management of CKD? It is possible that no further improvement in the primary outcome was possible, given the high baseline use of ACEi and ARB in both groups (approximately 77%). Alternatively, it is possible that the enhanced prompt was simply no more useful than an already effective standard prompt—physicians in the standard prompt group could access further information (including management recommendations) by visiting a website suggested in the laboratory report. However, because prior research suggests that eGFR reporting increases referral but does not seem to change prescribing of ACEi or ARB medication (16,17), we do not think that this would completely account for our findings. The enhanced eGFR prompt may have been too complex because it combined physician education on identification and significance of CKD with management suggestions. Because the enhanced laboratory prompt may have been more effective in patients with an eGFR <30 ml/min per 1.73 m2, we speculate that primary care physicians may not have considered all patients with an eGFR 30–60 ml/min per 1.73 m2 to require specific management of CKD. Finally, it is possible that physicians were overwhelmed with the number of patients receiving a prompt, because >10% of patients in whom serum creatinine was ordered had eGFR <60 ml/min per 1.73 m2.
Regardless of the explanation, our results suggest that adding clinical decision support to a laboratory prompt aimed at improving the identification and management of CKD does not provide meaningful benefit to patients (16,17). Data are urgently needed to clarify how routinely incorporating information on eGFR into routine care could improve outcomes rather than simply increasing physician workload. Because the effect of treatment recommendations delivered at point of care may be increased by accompanying them with endorsements from respected local physicians (22,36), future use of automated reminders should consider incorporating opinion leaders as signatories or other enabling strategies such as patient activation mechanisms and the involvement of allied health professionals (37).
Although it may have been more effective to provide the laboratory prompts within a functioning electronic medical record, which could prompt a physician to change practice during the clinical encounter), integrated electronic medical records are not yet widely available in Alberta. Indeed, <5% of American primary care physicians have access to electronic records with such advanced functionality (38) and the modality by which evidence prompts are delivered (computer-based versus paper-based) does not seem to influence effectiveness (39). Therefore, the interventions in our study were consistent with the usual mode of eGFR reporting throughout much of the world.
Our study had several limitations that should be considered. As noted, the high baseline ACEi/ARB use (approximately 77%) in our study participants may have affected our ability to detect a change in medication prescribing after exposure to the enhanced laboratory prompt. Not all patients are able to tolerate angiotensin blockade due to medication intolerance and adverse events, including hyperkalemia. However, it is important to note that in patients with an eGFR<30 ml/min per 1.73 m2 in whom baseline use was similarly high and adverse events might be expected to be higher, we were able to document a 13% increase in ACEi/ARB use, suggesting that high baseline use did not solely account for the lack of change in the primary outcome. Second, because we relied on pharmacy prescribing records, we do not know to what extent patient noncompliance led us to underestimate the efficacy of our intervention; however, it is unlikely that noncompliance occurred differentially. Third, we did not collect data on the acceptability of the enhanced laboratory prompt to primary care physicians, and thus cannot determine how this might have influenced our results. Although the outcomes assessed are clinically relevant, we did not have any information on changes in other cardiovascular risk factors such as BP, weight, or smoking status.
In elderly patients with reduced eGFR and an indication for ACEi or ARB medication, an enhanced laboratory prompt neither increased the use of either agent, nor improved clinically relevant outcomes among a broader cohort of CKD patients. These data suggest that enhanced management-based laboratory prompts cannot currently be recommended for routine use in all patients with CKD. The higher use of angiotensin blockade noted in patients with an eGFR<30 ml/min per 1.73 m2 who received the enhanced prompt requires confirmation. Future studies should examine the effects of using clinical decision support to target patients at higher risk of complications associated with CKD, including those with an eGFR <30 ml/min per 1.73 m2 or patients with proteinuria.
B.C. became an employee of Baxter Healthcare after this study was designed. K.G. became an employee of Abbott after this study was designed. Baxter Healthcare and Abbott provided no funding for the study, no input into the analysis or interpretation of the results, and no input into the drafting of the paper. The remaining authors have no financial conflicts of interest to report.
This study was funded by two peer-reviewed grants from the Alberta Heritage Foundation for Medical Research and Merck Canada and received support from an interdisciplinary research team grant from Alberta Innovates–Health Solutions. M.T., F.A.M., B.M., and B.H. are supported by Alberta Innovates–Health Solutions (formerly Alberta Heritage Foundation for Medical Research) career salary support awards. M.T. was also supported by a Government of Canada Research Chair in the optimal care of people with CKD. M.T., F.A.M., K.T., B.H., K.M., and B.M. were supported by an alternative funding plan from the Government of Alberta and the Universities of Alberta and Calgary.
None of the funding organizations had a role in the conception or design, conduct, analysis, interpretation, or reporting of the study, and none had access to the data.
Published online ahead of print. Publication date available at www.cjasn.org.
This article contains supplemental material online at http://cjasn.asnjournals.org/lookup/suppl/doi:10.2215/CJN.12391211/-/DCSupplemental.
See related editorial, “Decision Support and CKD: Not There Yet,” on pages 525–526.
1. National Kidney Foundation: K/DOQI clinical practice guidelines for chronic kidney disease: Evaluation, classification, and stratification. Am J Kidney Dis 39[Suppl 1]: S1–S266, 2002
2. Go AS, Chertow GM, Fan D, McCulloch CE, Hsu CY: Chronic kidney disease and the risks of death, cardiovascular events, and hospitalization. N Engl J Med 351: 1296–1305, 2004
3. James MT, Hemmelgarn BR, Wiebe N, Pannu N, Manns BJ, Klarenbach SW, Tonelli MAlberta Kidney Disease Network: Glomerular filtration rate, proteinuria, and the incidence and consequences of acute kidney injury: A cohort study. Lancet 376: 2096–2103, 2010
4. Hemmelgarn BR, Manns BJ, Lloyd A, James MT, Klarenbach S, Quinn RR, Wiebe N, Tonelli MAlberta Kidney Disease Network: Relation between kidney function, proteinuria, and adverse outcomes. JAMA 303: 423–429, 2010
5. Tonelli M, Bohm C, Pandeya S, Gill J, Levin A, Kiberd BA: Cardiac risk factors and the use of cardioprotective medications in patients with chronic renal insufficiency. Am J Kidney Dis 37: 484–489, 2001
6. Tonelli M, Gill J, Pandeya S, Bohm C, Levin A, Kiberd BA: Barriers to blood pressure control and angiotensin enzyme inhibitor use in Canadian patients with chronic renal insufficiency. Nephrol Dial Transplant 17: 1426–1433, 2002
7. Shlipak MG, Heidenreich PA, Noguchi H, Chertow GM, Browner WS, McClellan MB: Association of renal insufficiency with treatment and outcomes after myocardial infarction in elderly patients. Ann Intern Med 137: 555–562, 2002
8. Tonelli M, Isles C, Craven T, Tonkin A, Pfeffer MA, Shepherd J, Sacks FM, Furberg C, Cobbe SM, Simes J, West M, Packard C, Curhan GC: Effect of pravastatin on rate of kidney function loss in people with or at risk for coronary disease. Circulation 112: 171–178, 2005
9. Tonelli M, Isles C, Curhan GC, Tonkin A, Pfeffer MA, Shepherd J, Sacks FM, Furberg C, Cobbe SM, Simes J, Craven T, West M: Effect of pravastatin on cardiovascular events in people with chronic kidney disease. Circulation 110: 1557–1563, 2004
10. Tonelli M, Keech A, Shepherd J, Sacks F, Tonkin A, Packard C, Pfeffer M, Simes J, Isles C, Furberg C, West M, Craven T, Curhan G: Effect of pravastatin in people with diabetes and chronic kidney disease. J Am Soc Nephrol 16: 3748–3754, 2005
11. Strippoli GF, Craig M, Deeks JJ, Schena FP, Craig JC: Effects of angiotensin converting enzyme inhibitors and angiotensin II receptor antagonists on mortality and renal outcomes in diabetic nephropathy: systematic review. BMJ 329: 828, 2004
12. Giatras I, Lau J, Levey ASAngiotensin-Converting-Enzyme Inhibition and Progressive Renal Disease Study Group: Effect of angiotensin-converting enzyme inhibitors on the progression of nondiabetic renal disease: A meta-analysis of randomized trials. Ann Intern Med 127: 337–345, 1997
13. Jafar TH, Schmid CH, Landa M, Giatras I, Toto R, Remuzzi G, Maschio G, Brenner BM, Kamper A, Zucchelli P, Becker G, Himmelmann A, Bannister K, Landais P, Shahinfar S, de Jong PE, de Zeeuw D, Lau J, Levey AS: Angiotensin-converting enzyme inhibitors and progression of nondiabetic renal disease. A meta-analysis of patient-level data. Ann Intern Med 135: 73–87, 2001
14. Grol R, Grimshaw J: From best evidence to best practice: Effective implementation of change in patients’ care. Lancet 362: 1225–1230, 2003
15. Majumdar SR, McAlister FA, Furberg CD: From knowledge to practice in chronic cardiovascular disease: A long and winding road. J Am Coll Cardiol 43: 1738–1742, 2004
16. Kagoma YK, Weir MA, Iansavichus AV, Hemmelgarn BR, Akbari A, Patel UD, Garg AX, Jain AK: Impact of estimated GFR reporting on patients, clinicians, and health-care systems: A systematic review. Am J Kidney Dis 57: 592–601, 2011
17. Hemmelgarn BR, Zhang J, Manns BJ, James MT, Quinn RR, Ravani P, Klarenbach SW, Culleton BF, Krause R, Thorlacius L, Jain AK, Tonelli MAlberta Kidney Disease Network: Nephrology visits and health care resource use before and after reporting estimated glomerular filtration rate. JAMA 303: 1151–1158, 2010
18. Tangri N, Stevens LA, Griffith J, Tighiouart H, Djurdjev O, Naimark D, Levin A, Levey AS: A predictive model for progression of chronic kidney disease to kidney failure. JAMA 305: 1553–1559, 2011
19. Peralta CA, Shlipak MG, Judd S, Cushman M, McClellan W, Zakai NA, Safford MM, Zhang X, Muntner P, Warnock D: Detection of chronic kidney disease with creatinine, cystatin C, and urine albumin-to-creatinine ratio and association with progression to end-stage renal disease and mortality. JAMA 305: 1545–1552, 2011
20. Tonelli M, Manns B: Supplementing creatinine-based estimates of risk in chronic kidney disease: Is it time? JAMA 305: 1593–1595, 2011
21. Shojania KG, Jennings A, Mayhew A, Ramsay C, Eccles M, Grimshaw J: Effect of point-of-care computer reminders on physician behaviour: A systematic review. CMAJ 182: E216–E225, 2010
22. McAlister FA, Fradette M, Majumdar SR, Williams R, Graham M, McMeekin J, Ghali WA, Tsuyuki RT, Knudtson ML, Grimshaw J: The Enhancing Secondary Prevention in Coronary Artery Disease trial. CMAJ 181: 897–904, 2009
23. Thomas RE, Croal BL, Ramsay C, Eccles M, Grimshaw J: Effect of enhanced feedback and brief educational reminder messages on laboratory test requesting in primary care: A cluster randomised trial. Lancet 367: 1990–1996, 2006
24. Hemmelgarn BR, Clement F, Manns BJ, Klarenbach S, James MT, Ravani P, Pannu N, Ahmed SB, MacRae J, Scott-Douglas N, Jindal K, Quinn R, Culleton BF, Wiebe N, Krause R, Thorlacius L, Tonelli M: Overview of the Alberta Kidney Disease Network. BMC Nephrol 10: 30, 2009
25. Kidney Disease Outcomes Quality Initiative (K/DOQI): K/DOQI clinical practice guidelines on hypertension and antihypertensive agents in chronic kidney disease. Am J Kidney Dis 43[Suppl 1]: S1–S290, 2004
26. Levin A, Hemmelgarn B, Culleton B, Tobe S, McFarlane P, Ruzicka M, Burns K, Manns B, White C, Madore F, Moist L, Klarenbach S, Barrett B, Foley R, Jindal K, Senior P, Pannu N, Shurraw S, Akbari A, Cohn A, Reslerova M, Deved V, Mendelssohn D, Nesrallah G, Kappel J, Tonelli MCanadian Society of Nephrology: Guidelines for the management of chronic kidney disease. CMAJ 179: 1154–1162, 2008
27. Quan H, Khan N, Hemmelgarn BR, Tu K, Chen G, Campbell N, Hill MD, Ghali WA, McAlister FAHypertension Outcome and Surveillance Team of the Canadian Hypertension Education Programs: Validation of a case definition to define hypertension using administrative data. Hypertension 54: 1423–1428, 2009
28. Hux JE, Ivis F, Flintoft V, Bica A: Diabetes in Ontario: Determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care 25: 512–516, 2002
29. Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi JC, Saunders LD, Beck CA, Feasby TE, Ghali WA: Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care 43: 1130–1139, 2005
30. Donner A, Klar N: Design and Analysis of Cluster Randomization Trials in Health Research, edited by Donner A, Klar N, London, England, Arnold, 2000
31. Campbell MK, Elbourne DR, Altman DGCONSORT group: CONSORT statement: Extension to cluster randomised trials. BMJ 328: 702–708, 2004
32. Liang KY, Zeger SL: Longitudinal data analysis using generalized linear models. Biometrika 73: 13–22, 1986
33. Rabe-Hesketh S, Skrondal A: Multilevel and Longitudinal Modeling Using Stata, College Station, TX, Stata Press, 2007
34. Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, Haynes RB: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review. JAMA 293: 1223–1238, 2005
35. Kawamoto K, Houlihan CA, Balas EA, Lobach DF: Improving clinical practice using clinical decision support systems: A systematic review of trials to identify features critical to success. BMJ 330: 765, 2005
36. Majumdar SR, Johnson JA, McAlister FA, Bellerose D, Russell AS, Hanley DA, Morrish DW, Maksymowych WP, Rowe BH: Multifaceted intervention to improve diagnosis and treatment of osteoporosis in patients with recent wrist fracture: A randomized controlled trial. CMAJ 178: 569–575, 2008
37. McLean DL, McAlister FA, Johnson JA, King KM, Makowsky MJ, Jones CA, Tsuyuki RTSCRIP-HTN Investigators: A randomized trial of the effect of community pharmacist and nurse care on improving blood pressure management in patients with diabetes mellitus: study of cardiovascular risk intervention by pharmacists-hypertension (SCRIP-HTN). Arch Intern Med 168: 2355–2361, 2008
38. DesRoches CM, Campbell EG, Rao SR, Donelan K, Ferris TG, Jha A, Kaushal R, Levy DE, Rosenbaum S, Shields AE, Blumenthal D: Electronic health records in ambulatory care—a national survey of physicians. N Engl J Med 359: 50–60, 2008
39. Balas EA, Weingarten S, Garb CT, Blumenthal D, Boren SA, Brown GD: Improving preventive care by prompting physicians. Arch Intern Med 160: 301–308, 2000