The Society of Thoracic Surgeons (STS) and the Society of Cardiovascular Anesthesiologists (SCA) Clinical Practice Guideline for Perioperative Blood Transfusion and Blood Conservation in Cardiac Surgery was published in 20071 and has received much attention at both societies. A second iteration of the Perioperative Blood Transfusion and Blood Conservation Guidelines is currently in preparation. Such documents are thought to be a useful and important adjunct to the peer-reviewed literature, because they synthesize the evidence base supporting the management and treatment of cardiac surgical patients.2
Well-developed guidelines, based on a high level of evidence, have the potential to enhance the appropriateness of clinical practice, improve the quality of care, lead to better patient outcomes, improve cost effectiveness, and identify areas of further research needs.3 Despite the resource-intensive publication of evidence-based guidelines, their adoption into clinical practice is often suboptimal.4 The effectiveness and implementation of guidelines is variable and depends on how the guidelines are perceived by professional organizations, peer leaders, and individual practitioners.5 – 9 Without further interventions to encourage implementation, the effectiveness of guidelines is reduced over time.10
Because blood loss and transfusion-related events confer a substantial burden of cost, morbidity, and mortality to patients undergoing cardiac surgery,11,12 we assessed the current clinical practices of perfusion, anesthesia, and surgery, as recommended by the Guidelines, by using a multisociety survey of clinical practice. We also sought to determine the impact of the Guidelines in changing these practices. We hypothesized that the Guidelines would have made significant changes in these clinical practices. As additional objectives, we examined the current practice in the use of recombinant activated factor VII (rFVIIa) and leukoreduced blood products, because of the rapid increase in their use in cardiac surgery, and whether studies of the impact of aprotinin withdrawal had been conducted.
The author group developed a qualitative survey of clinical practice (Appendix I). Questions in the survey were based on the recommendations of the Perioperative Blood Transfusion and Blood Conservation in Cardiac Surgery Clinical Practice Guideline, available at http://ats.ctsnetjournals.org/cgi/content/abstract/83/5_Supplement/S27. There were 57 recommendations in the Guidelines. For level of evidence, 13 were level A, 27 were level B, and 17 were level C. For class of recommendation, 7 were class I, 18 were class IIa, 23 were class IIb, and 10 were class 3. We attempted to craft questions for all 57 recommendations; however, for 19 recommendations we were unable to create suitable language that survived trials on the test group or were not suitable for questioning because of the ambiguous construction of the recommendation.
Additional questions regarding rFVIIa use and aprotinin withdrawal were included because of recent changes in clinical practice. The questionnaire was pretested to assess face and content validity, comprehensibility, time to completion, and ambiguity. The questionnaire was initially pilot tested by 14 anesthesiologists and perfusionists who were either members of the Guidelines Taskforce or had leadership positions in their societies. Several modifications were made based on their feedback. Subsequently, the revised questionnaire was pilot tested on 8 individuals before being finalized (Supplemental Appendix I, see Supplemental Digital content 1, http://links.lww.com/AA/A154). The responses from pilot testing were not included in the analyses.
Study Cohort and Survey Distribution
E-mail addresses for nontrainee society members were provided by the SCA (n = 4140; http://www.scahq.org/), the American Academy of Cardiovascular Perfusion (n = 367; http://theaacp.addr.com/), the Canadian Society of Clinical Perfusion (n = 252; http://www.cscp.ca/), and the American Society of ExtraCorporeal Technology (n = 1423; http://www.amsect.org/). The Society of Thoracic Surgeons was approached for their participation in the survey but did not participate. Surveys were distributed using SurveyMonkey (http://www.surveymonkey.com/) and collected between February 2009 and April 2009. The survey was e-mailed to each member weekly, over a 3-week period. The authors were blinded to respondents’ contact information, except for institutional information provided by the respondents.
Survey responses were prespecified to be excluded from analysis if the respondent self-reported trainee status, was not a practicing anesthesiologist or perfusionist, or completed fewer than 20 questions in the survey. Summary data for each question excluded those who left the question unanswered. Responses to some questions were conditional on a response to a previous question. In these cases, the percentage responses for the follow-up question were reported based on the total number of responses to the first question. The approximate precision for a point estimate of 50% based on 1412 responses is 1.3%, based on the half-length of a 95% confidence interval.
To assess the accuracy of responses, we determined the extent of consensus in responses from respondents within each of 11 Canadian and 19 United States (US) institutions, from which 5 or more responses were obtained. The percentage of respondents reporting the most frequent response from the institution was estimated. Respondents who reported “do not know” to a question were excluded from analysis for that question.
Analysis was performed using JMP 8.0.1 (SAS Institute, Cary, NC). Categorical comparisons were made using χ2 or Fisher exact test, where appropriate. Significance was assessed using a 2-sided P value <0.01.
Surveys were sent to 6161 members of the participating societies. Surveys were not received by 442 members because they had opted out of SurveyMonkey mailings (n = 62) or the e-mail address was incorrect (n = 380). Of 5719 mailings received by members, 1828 surveys (32%) were returned by 29% of anesthesiologists and 47% of perfusionists. Responses were excluded from analysis if the respondent self-reported trainee status (n = 16), was not a practicing anesthesiologist or perfusionist (n = 39), or completed fewer than 20 questions in the survey (n = 386). A total of 1402 returned surveys were used for subsequent analysis. The demographic and professional characteristics of the respondents are reported in Table 1. Responses were received from 891 institutions, principally in the US (677 institutions) and Canada (34 institutions). Detailed categorized responses are provided as supplementary material at the Journal's Web site (www.anesthesia-analgesia.org).
There was wide distribution of the Guidelines with 78% of anesthesiologists and 67% of perfusionists reporting having read all, part, or a summary of the Guidelines. However, only 20% of respondents reported that an institutional discussion had taken place as a result of the Guidelines, and only 14% of respondents reported that an institutional monitoring group had been formed (Table 2). Only 10% reported that both discussion and monitoring groups had been formed. There was wide variability in current preoperative testing, anesthesia, perfusion, surgical, and pharmacological practices reported by respondents (Figs. 1–8). Specifically, only 3 practices were reported by >75% of respondents: the routine use of intraoperative red cell saving and transfusion of all pump blood; the use of lysine analog antifibrinolytics; and the cessation of antiplatelet agents before surgery, with the exception of aspirin. Seven practices were reported by <25% of respondents: an increased use of off-pump coronary artery bypass surgery, the use of erythropoietin or desmopressin, the use of point-of-care coagulation testing, the use of lowered heparin or activated clotting time levels, the use of a higher hemoglobin level cutoff for a transfusion trigger, and routine transfusion of washed shed mediastinal blood. Additionally, many practices were differentially reported by anesthesiologists and perfusionists (Appendix II, Tables 3–9). Sentinel examples are the use of an open venous reservoir reported by 81% of perfusionists, but only 35% of anesthesiologists, and the use of preoperative bleeding time or other equivalent test in all patients reported by 61% of perfusionists but 18% of anesthesiologists.
Twenty-six percent of respondents reported 1 or more practice changes in response to the Guidelines. The changes made were reported to be highly (9%) or somewhat (31%) effective in reducing overall transfusion rates. Individuals who described themselves as institutional leaders were more likely to report the Guidelines as somewhat or highly effective at reducing transfusion and less likely to report they did not know the effect of the Guidelines on transfusion at their institution. The changes were also reported to be highly (25%) or somewhat (54%) embraced by the respondent's specialty (anesthesiologist or perfusionist) and highly (11%) or somewhat (59%) embraced by the other specialties (anesthesiologist, perfusionist, or surgeon). Only 4 Guideline recommendations were reported to have motivated a change in clinical practice by >5% of respondents. They were a reduced transfusion trigger, use of reduced cardiopulmonary bypass circuit prime volume, use of retrograde autologous cardiopulmonary bypass circuit priming, and routine use of an intraoperative point-of-care hemostasis or platelet function test in all patients who are bleeding.
There were significant (P < 0.01) differences in practices and changes in practice between institutions that conducted an institutional discussion of the Guidelines and those that did not. In general, the percentage of respondents reporting a change in practice within institutions that conducted an institutional discussion of the Guidelines was higher for many of the recommendations (Supplemental Appendix II, Tables 15, 16, 20, and 21, see Supplemental Digital Content 2, http://links.lww.com/AA/A155). The percentage of respondents reporting highly or somewhat effective reductions in transfusion was significantly higher among institutions that conducted an institutional discussion of the Guidelines than those that did not (Supplemental Appendix II, Table 13A, http://links.lww.com/AA/A155). The percentage of respondents in institutions that conducted an institutional discussion of the Guidelines reporting highly or somewhat effective “embracing” of the changes was significantly higher than that in institutions that did not conduct an institutional discussion (Supplemental Appendix II, Table 13A, http://links.lww.com/AA/A155). These changes were generally not affected by the institution's academic affiliation (Supplemental Appendix II, Tables 13C, 22–24, 27, and 28, http://links.lww.com/AA/A155).
The use of rFVIIa as a rescue therapy in the setting of life-threatening bleeding unresponsive to routine therapy was reported by 65% of respondents. Use of rFVIIa as a first-line therapy for bleeding was reported by 2% of respondents. Only 87 of respondents (6%) reported that their institution had systematically examined the effect of rFVIIa on renal failure and mortality. The most common effect of such institutional examination was restriction in its use, reported by 78% of the 87 respondents. There were no significant differences in rFVIIa use within Western world countries (Supplemental Appendix II, Table 12, http://links.lww.com/AA/A155).
The use of aprotinin was reported to have been examined in the institutions of 15% of respondents, with a higher rate of examination in institutions that also had an institutional discussion of the guidelines (Supplemental Appendix II, Tables 6 and 18, http://links.lww.com/AA/A155). Respondents from different types of institutions did not differ in their reported rate of examination of aprotinin use (Supplemental Appendix II, Table 25, http://links.lww.com/AA/A155).
We determined the extent of consensus for responses from respondents within each of the 11 Canadian and 19 US institutions, from which 5 or more responses were obtained. Most questions had good (≥75%) consistency between respondents from an institution. For several questions, there was low (<75%) agreement from respondents within the institution regarding current institutional practice. These included whether there had been an institutional discussion of the Guidelines, whether a monitoring group had been formed, whether changes in clinical practice were implemented as a result of the Guidelines, effectiveness of transfusion guidelines, and how well the changes had been embraced by their and other specialties (Supplemental Appendix III, see Supplemental Digital Content 3, http://links.lww.com/AA/A156).
In this survey of SCA, American Academy of Cardiovascular Perfusion, Canadian Society of Clinical Perfusion, and American Society of ExtraCorporeal Technology members, we observed wide variation in reported clinical practices of cardiac surgery, as related to blood conservation and transfusion. Furthermore, many practices were reported at different frequencies between perfusionists and anesthesiologists, and even among individuals from the same institution. There appeared to be little change in clinical practices attributed to the STS/SCA Guidelines. Respondents from institutions where an institutional discussion of the Guidelines was held were more likely to report changes in practices in response to the Guidelines and to report that the changes were somewhat or highly effective in reducing overall transfusion rates. This observation likely stems from both the credibility of the guideline process as well as the institutional support for improvement of care. The responses to the Guidelines were not affected by the type of academic affiliation of the respondent's institution.
The wide variation in reported perfusion, anesthesia, and surgical practices is an important observation not previously reported in this cohort. Importantly, many of the Guideline recommendations are practiced by <50% of the respondents’ institutions. Such variation is the rationale for the Guidelines and emphasizes the importance of their promulgation and implementation. The key components for implementation lie in senior leadership and their endorsement of the guidelines, peer review, and involvement in implementation, along with frequent reinforcement and even reward, for continued adherence to the guidelines.13 Moreover, continued and practice-specific feedback of performance is a key component of incorporating clinical practice guidelines into actionable changes within institutions. A sentinel example of effective and widely practiced guidelines is those developed and promulgated by the American Heart Association (AHA), notable for their extensive peer review by all interested stakeholders, wide dissemination, and institution-specific feedback at the annual meeting.3 Importantly, the AHA “Get with the Guidelines” program is based on the importance of continued education to improve implementation. Similar methods have also been successfully used for assessing cardiovascular perfusion care relative to evidence-based guidelines in northern New England.14 Similarly, Berry et al.15 recently reported on the redesign of cardiac surgical care by implementing 40 measurable process variables that were rooted in the American College of Cardiology/AHA coronary artery bypass graft guidelines, with reported significant reduction in ICU readmissions and blood product use.
Implementation of guidelines in the cardiac surgery arena has been previously reported to be poor,16,17 but focused implementation of guidelines at single institutions has been reported to be successful.15,18 Important features of successful implementation of guidelines likely include the quality and supporting evidence for the guidelines. The organization(s) promulgating the guidelines may also affect the adoption rate, especially when multiple practitioners from several specialties are affected. Implementation at the institutional level will be highly affected by organizational and peer leadership and the organizational structure within the institution.19
There was low level of concordance for some questions from each of the institutions having 5 or more respondents. Sixty percent or fewer respondents agreed on whether a formal institutional discussion had been held, whether an institutional monitoring group had been formed, or whether any changes in clinical practice had been implemented. Similarly, respondents failed to agree on whether there had been an institutional examination of the effects of aprotinin withdrawal or rFVIIa use, and whether the changes made were effective in reducing overall transfusion rates. The reasons for such lack of concordance are not clear, but may reflect a lack of involvement or communication among all parties, or inaccuracy of the survey methodology.
This survey found infrequent use of technology for timely and definitive monitoring of a patient's coagulation, hemostasis, and platelet function. Additionally, <40% of respondents reported using heparin concentration monitoring. The overall low use of technology that may guide specific coagulation product administration may reflect issues of cost and technology effectiveness. However, the use of coagulation assessment may be strongly warranted, notably because the use of platelet inhibitors is frequent, the incidence of coagulopathy is high, and the dose-response relationships for coagulation products are unknown for individual patients.
The formation of an institutional monitoring group did not appear to be completely effective in providing feedback to practitioners. Comparing responses from institutions that did or did not form an institutional monitoring group, 25% of respondents in the “monitoring group formed” category did not receive feedback on overall transfusion rates (Supplemental Appendix II, Table 13B, http://links.lww.com/AA/A155) compared with 55% in the “monitoring group not formed” category. Incorporating evidence-based practice guidelines into practice may be the rate-limiting step for practice change because of broader organizational implications such as displacement of resources, budgetary restrictions, and peer pressure.20 Furthermore, many of the changes resulting from the STS/SCA guidelines and other such documents require system-level modifications with complex implementation. Quality-improvement initiatives used in industry such as Total Quality Management and Six Sigma, which appear applicable to surgical processes, are dependent on the collection and distribution of quantitative feedback to all individuals involved in the process.21,22 The notion that we can improve cardiac surgical care and the performance of individual members of the team without proper timely feedback is not reasonable. In fact, real-time feedback has been demonstrated to be particularly effective.23
Guidelines are a valuable method for reducing practice variation, errors, and unnecessary and potentially harmful use of limited health care resources.14 The ever-rising cost of health care suggests the need for a more methodical approach to redesigning care as well as allocating human and material resources. Guidelines may be an effective tool for minimizing unwarranted variation in practice and may provide reference material for obtaining the resources that are needed to improve care, but we did not observe this for the STS/SCA Guidelines. These findings are in agreement with prior observations that guidelines are frequently poorly, and slowly, adopted.2,24 Reasons for low adoption include practitioner views of outside control of clinical practice, low level of strength of the evidence supporting a given recommendation, appropriateness and usefulness of the specific guidelines, and availability of resources to implement the guidelines.25 – 27 These factors may be important in the reported low adoption of the STS/SCA Guidelines.
This survey has limitations. One of the most serious was the absence of participation by surgeons. Furthermore, only 32% of those polled responded to the survey. Although this is a common response rate for professional surveys, the overall accuracy would have been enhanced by a greater response rate. Individual society members who had not provided a current e-mail address were excluded from the survey because of the cost associated with individual mailings. Nevertheless, >90% of SCA members and 80% of perfusion society members had provided e-mail addresses and were polled. Surveys are subject to 2 major sources of error: incomplete sampling and misinterpretation of the survey questions by respondents. The former results in sampling bias error and the latter produces responses that do not accurately reflect clinical practice or respondent opinions. It is possible that individuals with the greatest interest in blood conservation measures, and who therefore may have instituted more changes in institutional practices, may have responded at a greater rate than those with less interest, and may have been more optimistic about their positive effect. Thus, the low reported rate of clinical practice change and adoption of the Guideline recommendations may be even worse than that reported by the responders to our survey. Furthermore, both types of error could result in ascribing inappropriate inferences from the responses. Similarly, surveys may not reflect actual clinician or institutional behavior and may be prone to bias. Specifically, we do not know the basis or validity for a respondent's response that changes made had been effective at reducing overall transfusion rate. Therefore, these results should be regarded with caution until validated by direct institutional observation.28 Nor does the survey define or directly assess effectiveness of the Guidelines or any specific guideline.
Adoption of the Guideline's recommendations may also have been hindered by the sometimes ambiguous nature of the recommendations, especially the use of the AHA guideline phrases “it is not unreasonable” and “it is reasonable.” The distinctions in this phraseology are not well understood by clinical practitioners and may not provide sufficient imperative for adoption. Common to many guidelines, the relatively low level of evidence for many recommendations (40 of 57 had only class B or lower level of evidence) may not have provided sufficient impetus to institutional change. Furthermore, many of the recommendations require redesigning complex institution-wide processes to implement, which may further reduce compliance.
In summary, this survey emphasizes the obligations of clinical groups and all individuals involved in the management of cardiac surgical patients. At the level of those formulating practice guidelines, there is a mandate to involve all interested parties at every level of formulation and promulgation of evidence-based guidelines. Societies and other organizations have a key role in endorsement, dissemination, and use of guidelines to improve care. At the institutional level, those with leadership roles have an obligation to involve hospital leadership, practitioners, those with support roles, and even practitioners beyond the hospital environs with the tools and mandate to practice patient-centered, cost-effective, and guideline-oriented care of the cardiac surgical patient.
From the *Department of Surgery, The Center for Leadership and Improvement, Dartmouth Medical School, The Dartmouth Institute for Health Policy and Clinical Practice, Dartmouth-Hitchcock Medical Center Dartmouth College, Lebanon, New Hampshire; †Division of Cardiac Surgery, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts; ‡Cardiac Services, Maine Medical Center, Portland, Maine; §Department of Cardiovascular Perfusion, St. Mary's Regional Cardiac Program, Kitchener, Ontario, Canada; ∥Cardiac Surgery Research and Perfusion, Cardiothoracic Surgical Unit, Flinders Medical Centre and Flinders University, Adelaide, Australia; ¶Department of Cardiothoracic Surgery, Montefiore-Einstein Heart Center, New York, New York; #Department of Anesthesia, Keenan Research Center in the Li Ka Shing Knowledge Institute, St. Michael's Hospital, University of Toronto, Toronto, Ontario, Canada; **Department of Anesthesiology and Emergency Medicine, Virginia Commonwealth University Reanimation Engineering Shock Center, Richmond, Virginia; and ††Department of Anesthesiology, Perioperative and Pain Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts.
1. Ferraris VA, Ferraris SP, Saha SP, Hessel EA II, Haan CK, Royston BD, Bridges CR, Higgins RS, Despotis G, Brown JR, Spiess BD, Shore-Lesserson L, Stafford-Smith M, Mazer CD, Bennett-Guerrero E, Hill SE, Body S. Perioperative blood transfusion and blood conservation in cardiac surgery: the Society of Thoracic Surgeons and The Society of Cardiovascular Anesthesiologists clinical practice guideline. Ann Thorac Surg 2007;83:S27–86
2. Committee on Quality of Health Care in America, Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press, 2001
3. ACC/AHA Task Force on Practice Guidelines. Manual for ACC/AHA Guideline Writing Committees Methodologies and Policies. Dallas, TX: American Heart Association, 2009
4. Hunt SA, Baker DW, Chin MH, Cinquegrani MP, Feldman AM, Francis GS, Ganiats TG, Goldstein S, Gregoratos G, Jessup ML, Noble RJ, Packer M, Silver MA, Stevenson LW, Gibbons RJ, Antman EM, Alpert JS, Faxon DP, Fuster V, Jacobs AK, Hiratzka LF, Russell RO, Smith SC Jr. ACC/AHA guidelines for the evaluation and management of chronic heart failure in the adult: executive summary. A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee to Revise the 1995 Guidelines for the Evaluation and Management of Heart Failure). J Am Coll Cardiol 2001;38:2101–13
5. Tinmouth A, Macdougall L, Fergusson D, Amin M, Graham ID, Hebert PC, Wilson K. Reducing the amount of blood transfused: a systematic review of behavioral interventions to change physicians’ transfusion practices. Arch Intern Med 2005;165:845–52
6. Brandt MM, Rubinfeld I, Jordan J, Trivedi D, Horst HM. Transfusion insurgency: practice change through education and evidence-based recommendations. Am J Surg 2009;197: 279–83
7. Freedman J, Luke K, Monga N, Lincoln S, Koen R, Escobar M, Chiavetta J. A provincial program of blood conservation: The Ontario Transfusion Coordinators (ONTraC). Transfus Apher Sci 2005;33:343–9
8. O'Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard-Jensen J, Kristoffersen DT, Forsetlund L, Bainbridge D, Freemantle N, Davis DA, Haynes RB, Harvey EL. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2007: CD000409
9. Freedman J, Luke K, Escobar M, Vernich L, Chiavetta JA. Experience of a network of transfusion coordinators for blood conservation (Ontario Transfusion Coordinators [ONTraC]). Transfusion 2008;48:237–50
10. Body SC, Fanikos J, DePeiro D, Philip JH, Segal BS. Individualized feedback of volatile agent use reduces fresh gas flow rate, but fails to favorably affect agent choice. Anesthesiology 1999;90:1171–5
11. Christensen MC, Krapf S, Kempel A, von Heymann C. Costs of excessive postoperative hemorrhage in cardiac surgery. J Thorac Cardiovasc Surg 2009;138:687–93
12. Reeves BC, Murphy GJ. Increased mortality, morbidity, and cost associated with red blood cell transfusion after cardiac surgery. Curr Opin Cardiol 2008;23:607–12
13. Fleming M, Wentzell N. Patient safety culture improvement tool: development and guidelines for use. Healthc Q 2008;11:10–5
14. DioDato CP, Likosky DS, DeFoe GR, Groom RC, Shann KG, Krumholz CF, Warren CS, Pieroni JW, Benak A, McCusker K, Olmstead EM, Ross CS, O'Connor GT. Cardiopulmonary bypass recommendations in adults: the northern New England experience. J Extra Corpor Technol 2008;40:16–20
15. Berry SA, Doll MC, McKinley KE, Casale AS, Bothe A Jr. ProvenCare: quality improvement model for designing highly reliable care in cardiac surgery. Qual Saf Health Care 2009;18:360–8
16. Hiratzka LF, Eagle KA, Liang L, Fonarow GC, LaBresh KA, Peterson ED. Atherosclerosis secondary prevention performance measures after coronary bypass graft surgery compared with percutaneous catheter intervention and nonintervention patients in the Get With the Guidelines database. Circulation 2007;116:I207–12
17. Steinberg BA, Steg PG, Bhatt DL, Fonarow GC, Zeymer U, Cannon CP. Comparisons of guideline-recommended therapies in patients with documented coronary artery disease having percutaneous coronary intervention versus coronary artery bypass grafting versus medical therapy only (from the REACH International Registry). Am J Cardiol 2007;99:1212–5
18. Yam FK, Akers WS, Ferraris VA, Smith K, Ramaiah C, Camp P, Flynn JD. Interventions to improve guideline compliance following coronary artery bypass grafting. Surgery 2006; 140:541–7
19. Gagnon MP, Sanchez E, Pons JM. Integration of health technology assessment recommendations into organizational and clinical practice: a case study in Catalonia. Int J Technol Assess Health Care 2006;22:169–76
20. Lemieux-Charles L, McGuire W, Blidner I. Building interorganizational knowledge for evidence-based health system change. Health Care Manage Rev 2002;27:48–59
21. Grimshaw JM, Thomson MA. What have new efforts to change professional practice achieved? Cochrane Effective Practice and Organization of Care Group. J R Soc Med 1998; 91 (Suppl 35): 20–5
22. Mowatt G, Grimshaw JM, Davis DA, Mazmanian PE. Getting evidence into practice: the work of the Cochrane Effective Practice and Organization of Care Group (EPOC). J Contin Educ Health Prof 2001;21:55–60
23. Vasaiwala S, Nolan E, Ramanath VS, Fang J, Kearly G, Van Riper S, Kline-Rogers E, Otten R, Cody RA, Eagle KA. A quality guarantee in acute coronary syndromes: the American College of Cardiology's Guidelines Applied in Practice program taken real-time. Am Heart J 2007;153:16–21
24. Timmermans S. From autonomy to accountability: the role of clinical practice guidelines in professional power. Perspect Biol Med 2005;48:490–501
25. Jamal A, McKenzie K, Clark M. The impact of health information technology on the quality of medical and health care: a systematic review. HIM J 2009;38:26–37
26. Lugtenberg M, Burgers JS, Westert GP. Effects of evidence-based clinical practice guidelines on quality of care: a systematic review. Qual Saf Health Care 2009;18:385–92
27. Rawlins M. The NICE way of influencing health spending: a conversation with Sir Michael Rawlins. Interview by Nicholas Timmins. Health Aff (Millwood) 2009;28:1360–5
28. Hartley RM, Charlton JR, Jarman B, Harris CM. Case history questionnaires in the study of doctors’ use of resources. Are they measuring what we want? Med Care 1985;23:1163–70