The Society of Critical Care Medicine’s (SCCM) revised version of the pain, agitation, and delirium (PAD) guidelines was published in Critical Care Medicine (CCM) in January 2013 (1). This article describes the methodology used to develop the PAD guidelines in more detail to give readers a better understanding of the unique processes used to develop the PAD guideline statements and recommendations and to provide a template for future clinical practice guideline development. The methodology used to develop the PAD guidelines differs considerably from that of previous versions and included the use of: 1) the Grading of Recommendations Assessment, Development and Evaluation (GRADE) method for evaluating the literature and developing statements and recommendations; 2) a professional librarian to conduct literature searches and to develop and maintain the PAD guideline reference database; 3) RefWorks software (Bethesda, MD) to create a single, electronic, web-based database accessible to all task force members; 4) rigorous analyses of the psychometric properties of pain, sedation/agitation, and delirium assessment tools for use in ICU patients; 5) anonymous, electronic polling of all task force members, with predefined voting thresholds to achieve group consensus for all guideline statements and recommendations; and 6) creation of an ICU PAD care bundle to facilitate guideline implementation and widespread adoption. We believe that this approach helped us to create a set of clinical practice guidelines that are more evidence based, robust, practical, and transparent to readers. The purpose of this article is to provide more details about the methods used to develop the PAD guidelines and to help guide the development process for clinical practice guidelines.
Assembling the PAD Guideline Task Force
A 20-member task force was appointed by the American College of Critical Care Medicine (ACCM) in 2006 to revise the ACCM’s 2002 version of the clinical practice guidelines for the management of sedation and analgesia in adults admitted to the ICU (2). This interdisciplinary task force included physicians, nurses, and pharmacists with expertise in pain, sedation, delirium, and related clinical outcomes in critically ill patients. Six of the seven task force members with previous guideline development experience had served on the task force responsible for creating the 2002 version of these guidelines. The PAD task force included a liaison to the ACCM’s Board of Regents. Over a 6-year period, only two individuals were replaced because of their inability to continue to serve as task force members. A member of the GRADE Working Group ( http://www.gradeworkinggroup.org) and a panel of three psychometric testing experts served as task force consultants.
The PAD task force initially met in January 2006 at the SCCM’s Annual Congress. The task force was divided into four subcommittees: pain, sedation, delirium, and related clinical outcomes, with team leaders for each subcommittee appointed by the task force chair. Subcommittees were responsible for developing corresponding clinical questions, identifying and evaluating the relevant clinical evidence, developing guideline statements and recommendations using the GRADE methodology ( http://www.gradeworkinggroup.org), and drafting their section of the PAD guidelines. Committee members collaborated over a 6-year period in person (PAD task force meetings held at the annual SCCM Congress) and via teleconferences and through e-mail correspondence.
Developing Clinical Questions
Subcommittees developed relevant PAD topic outlines and corresponding clinical questions, which were then reviewed and approved by all PAD task force members. Questions were categorized as either “descriptive” or “actionable.” In general, descriptive questions address issues such as prevalence, prognosis, or effectiveness of a test or treatment, but do not include a specific recommendation (i.e., Question: Do adult ICU patients experience nonprocedural pain in the ICU? Answer: Adult medical, surgical, and trauma ICU patients routinely experience pain, both at rest and with routine ICU care). Actionable questions address issues related to a specific intervention and form the basis of a recommendation (i.e., Question: Should pain assessments be routinely performed in adult ICU patients? Answer: We recommend that pain be routinely monitored in all adult ICU patients). The answers to descriptive questions provide important background for actionable questions and for formulating recommendations and interpreting the evidence about the effects of treatments.
All actionable questions in the PAD guidelines were phrased and analyzed using the population, intervention, comparison, and outcomes (PICO) format (3). For each actionable question, subcommittees identified relevant clinical outcomes, and then ranked them on a scale, based on their relative importance in terms of their impact on clinical decision making (i.e., ranking: not important [1–3], important [4–6], or critical [7–9] to clinical decision making). Only critical and important clinical outcomes were included in literature review process for each question, and only critical outcomes were used to develop corresponding recommendations (4).
Creating the PAD Database
Subcommittees developed a comprehensive list of related key words. A professional medical librarian was employed to expand and organize this key word list, identify corresponding medical subject heading terms (Supplemental Digital Content 1, http://links.lww.com/CCM/A590), search relevant clinical databases, and to create and maintain an electronic, web-based PAD database using RefWorks software (v1.0 and v2.0) (1). Eight clinical databases were included in all literature searches: PubMed, MEDLINE, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, CINAHL, Scopus, ISI Web of Science, and the International Pharmaceutical Abstracts. Search variables included published (or in press) English-only articles on adult humans (> 18 yr), published anytime between December 1999 (i.e., the search limit for the 2002 guidelines) and December 2010. Studies with n less than 30 patients, editorials, narrative reviews, case reports, animal or in vitro studies, and letters to the editor were excluded. Foreign language texts with English abstracts were also excluded because the full article could not be evaluated without the additional expense of translating it to English. Repeat searches were performed automatically on a biweekly basis beyond December 2010, and relevant articles published between December 2010 and July 2012 were incorporated into the PAD guidelines. Studies that were published after December 2010 were not included in either the GRADE evidence review or the voting process to develop guideline statements and recommendations. All references included in the 2002 version of the guidelines, as well as targeted searches of the literature published before December 1999, were also included in the PAD database. Readers are referred to an article by Davidson et al (5) for a more detailed discussion of the PAD guideline search methodology.
The PAD database was password protected with a group login through the RefWorks website ( http://www.refworks.com). Multiple task force members were able to log into the database simultaneously to perform literature searches, retrieve references, create folders, and view portable document format (PDF) attachments. However, the medical librarian served as the database “gatekeeper” in order to maintain the integrity of the PAD database. The librarian was solely responsible for the addition of any new references, the elimination of duplicate references, and uploading PDF attachments to references (which were retrieved by individual task force members). File size restrictions were waived by Refworks management for this project in order to allow for uploading of an unlimited number of attachments (i.e., the average PDF file is approximately 500 kb). The PAD database currently contains 19,402 references, including 599 PDF attachments, with a total file size of 5.1 GB.
Reviewing and Evaluating the Literature
All task force members were involved in reviewing, evaluating, and summarizing the literature using the GRADE methodology, which is a structured system for rating quality of evidence and grading strength of recommendations in clinical practice (6–8). Only one task force member had any prior experience using GRADE, which meant that considerable time and effort had to be invested initially in teaching all members how to correctly apply GRADE to evaluate the literature. Task force members received GRADE instruction primarily through reading online references ( http://www.gradeworkinggroup.org) and in close consultation with a GRADE Working Group content expert via conference calls and e-mails. Four task force members attended a 1-day GRADE workshop conducted by the Grade Working Group held at the SCCM Congress in Nashville in 2009.
Guideline questions were distributed among subcommittee members who were responsible for performing primary literature searches using the RefWorks PAD database. Task force members were given RefWorks database instruction using online tutorials ( http://www.refworks.com). The body of literature behind each question had a primary and secondary reviewer. Primary reviewers were responsible for the initial literature searches and for identifying relevant articles using the PAD database. They recorded their individual RefWorks search strategies used for each question in the corresponding RefWorks folder and placed the relevant studies for review in RefWorks folders labeled “Final Cut” for each question. They were then responsible for retrieving PDFs for all references included in these Final Cut folders and forwarding these to the medical librarian to upload to the PAD database as attachments.
Both the primary and secondary reviewers were responsible for independently reading and evaluating each study, completing a GRADE evidence profile (Appendix 1), and determining the overall quality of evidence behind each question (Note: task force members who had coauthored articles related to a particular question were recused from evaluating the body of literature related to that topic) (4). Team leaders were responsible for resolving any discrepancies in the quality of evidence assessments between reviewers.
Each study was evaluated as it related to specific questions and in the case of actionable questions, relevant clinical outcomes as well (4). The overall quality of the evidence concerning particular outcomes across multiple studies was influenced not only by individual study design but also by other factors and was ranked as high, moderate, or low/very low.
Generally, randomized controlled trials (RCTs) yield the highest quality of evidence, followed by well-controlled observational studies (OS) and other types of study design. But the quality of evidence in individual studies can be influenced by a number of factors related to study methodology (4). Evidence quality derived from RCTs may be downgraded by: limitations of study design or implementation, imprecision of estimates (i.e., wide CIs or generally small number of events), variability in results (heterogeneity), indirectness of evidence, or publication bias. Conversely, evidence quality in OS may be strengthened by a large magnitude of effect, a dose-response gradient, or if all plausible biases would reduce an apparent treatment effect. If multiple studies related to a particular outcome demonstrated disparate results, and no published systematic reviews on the topic existed, a meta-analysis of the relevant literature was performed by a member of the GRADE Working Group.
Psychometric Analyses of PAD Assessment Tools
The PAD guidelines task force also performed rigorous analyses of the psychometric properties of tools for assessing pain (six scales), agitation/sedation, and delirium in critically ill patients. Scale reliability, validity, and feasibility were assessed using scoring systems adapted from previous work, and developed with input from three psychometric testing experts (9, 10). Pain, sedation, and delirium assessment tools were compared in terms of their: 1) item selection and content validation; 2) reliability; 3) validity; 4) feasibility; and 5) their relevance or impact of implementation on ICU patient outcomes. Different scoring systems were developed for purposes of evaluating scales in each domain (i.e., pain, sedation, and delirium), to account for differences in the validation process used for each of these domains (1, 11) (Appendix 2). Raw scores ranged from 0 to 25 for pain scales, 0 to 18 for sedation scales, and 0 to 21 for delirium scales. Weighted scores were established for each criterion to address variations in scores and to facilitate the interpretation of results, resulting in a total weighted score of 0–20 for all three domains. The overall psychometric properties of scales were then qualitatively assessed based on their weighted score. Those scales with weighted scores of 15–20 were considered to have very good psychometric properties, 12–14.9 had moderate psychometric properties, 10–11.9 had some acceptable psychometric properties which required validation in additional studies, and 0–9.9 either had very few psychometric properties reported or unacceptable results. Those scales with moderate to very good psychometric properties (i.e., a weighted score ≥ 12) were considered to be acceptable for use in adult ICU patients. The quality of evidence for each pain, sedation, and delirium assessment tool was also evaluated using GRADE with evidence tables adapted to these psychometric criteria. The evidence review and the scoring of the psychometric properties for each scale were performed independently by two reviewers. A more detailed discussion of the psychometric methodology used to analyze pain scales has been previously published (11). Readers are also referred to the article by Robinson et al (12) in this supplement for a more detailed discussion of the psychometric methodology used to analyze sedation scales.
Developing Guideline Statements and Recommendations
Each subcommittee was responsible for developing corresponding statements and recommendations for all questions within their domain. The primary reviewer for each question was responsible for drafting a brief (i.e., one page) evidence summary, using the PICO format for actionable questions (Table 1). Using the nominal group technique, subcommittees reviewed their GRADE evidence profiles and summaries, drafted initial statements and recommendations for each question, and determined the overall quality of evidence (for both descriptive and actionable questions) and the strength of recommendation (for actionable questions) (13).
Applying the GRADE method, the overall quality of evidence for each statement and recommendation was ranked as high (A), moderate (B), or low/very low (C). For actionable questions with recommendations, the overall quality of evidence was based on an assessment of the body of evidence related only to critical outcomes. The strength of a recommendation was defined as either strong (1) or weak (2), and either for (+) or against (–) an intervention, based on both the quality of evidence and the risks and benefits across all critical outcomes (8). A strong recommendation either in favor of (+1) or against (–1) an intervention implied that the majority of task force members (discussed later) believed that the benefits of the intervention significantly outweighed the risks (or vice versa) and that the majority of patients and providers would pursue this course of action (or not), given the choice. A weak recommendation either in favor of (+2) or against (–2) an intervention implied that the benefits of the intervention likely outweighed the risks (or vice versa), but that task force members were not confident about these trade-offs, either because of a low quality of evidence or because the trade-offs between risks and benefits were closely balanced. On the basis of this information, most people might pursue this course of action (or not), but a significant number of patients and providers would choose an alternative course of action (4, 8, 14). A no recommendation (0) determination could also be made due to either a lack of evidence or a lack of consensus among subcommittee members. Consensus statements based on expert opinion alone were not used when evidence could not support a recommendation. Once consensus was achieved for all questions within each subcommittee, these evidence summaries and statements and recommendations were then circulated to all task force members for review.
Group consensus for all statements and recommendations was achieved using a modified Delphi method with anonymous voting (15–18). All task force members reviewed the subcommittees’ GRADE Evidence Summaries, statements and recommendations, then voted and commented anonymously using the online E-Survey tool ( http://www.esurvey.com, Scottsdale, AZ). All polling was managed by an SCCM staff person to maintain anonymity. Consensus on the quality of evidence for each question required a majority (> 50%) vote. Consensus on the strength of recommendations was defined as: a recommendation in favor of an intervention (or the comparator) required greater than or equal to 50% of all task force members voting in favor, with less than 20% voting against; failure to meet these voting thresholds resulted in no recommendation being made. For a recommendation to be graded as strong rather than weak, greater than or equal to 70% of those voting had to vote for a strong recommendation, otherwise it received a weak recommendation. This method for reaching formal consensus has been proposed by the GRADE Working Group and was adopted by the 2008 Sepsis Guidelines Panel to ensure fairness, transparency, and anonymity in the creation of guideline recommendations (19, 20). Polling results and comments were then compiled and distributed to all PAD guideline task force members for review. When one round of voting failed to produce group consensus, additional group discussion was followed by a second and third round of voting, if needed. Polling for all questions was completed by December 2010. Distribution of the final voting tallies for each statement and recommendation is summarized in a digital appendix of the PAD guidelines ( http://download.lww.com/wolterskluwer_vitalstream_com/PermaLink/CCM/A/CCM_41_1_2012_10_10_BARR_204220_SDC2.pdf) (1). For all strong recommendations, the phrase “We recommend …” was used, and for all weak recommendations, “We suggest …” was used for the final wording of all recommendations.
An ICU PAD care bundle was also created as part of the PAD guidelines in order to facilitate their translation into clinical practice (1). This care bundle includes evidence-based strategies for assessing, treating, and preventing pain, agitation, oversedation, and delirium in adult ICU patients in an interdisciplinary and integrated fashion. The care bundle also includes corresponding metrics to enable critical care providers to measure PAD performance and to improve both PAD processes and outcomes in order to improve ICU patient care. The care bundle also links PAD management to other evidence-based strategies for improving the care of ICU patients, such as spontaneous awakening trials, spontaneous breathing trials, early mobility protocols, and environmental sleep management strategies. This allows for additional synergistic benefits to ICU patient care by linking the PAD guidelines with other evidence-based ICU protocols.
Writing and Publishing the Article
The PAD guidelines writing team consisted of the task force chair, the team leaders, and the ACCM liaison. Each team leader worked with their subcommittee members to draft a background section on their topic, along with the rationale sections for each statement and recommendation (the rationale sections were based primarily on the previously written evidence summaries for each question). Team leaders referenced their respective sections (with unformatted references) using the RefWorks Write-N-Cite Tool (Write-N-Cite III) in conjunction with MS Word software (Microsoft, Redmond, WA). The task force chair was responsible for drafting the abstract, introduction, methods, and summary sections of the final article, as well as an Executive Summary of the PAD guidelines that included all statements and recommendations, which was published in the American Journal of Health-Systems Pharmacy (21). The entire PAD writing team was ultimately responsible for merging and editing all portions of the final article and for formatting the bibliography in RefWorks. Prior to submitting the article for external review, all task force members were able to review and comment on the final version of the article online using Google Docs ( http://www.Google.com, Mountain View, CA). All team members received Google Docs training using online Google tutorials. Authorship on the final article was determined by the chair with input from the team leaders and was based on both individual and team contributions throughout the entire PAD guidelines process.
The PAD guideline external review process included reviews by members of the ACCM’s Board of Regents, the SCCM’s Executive Council, the American Society of Health-System Pharmacists (ASHP), and the editorial staff of the journal CCM. A final draft of the article was submitted to ACCM, SCCM, and ASHP in July 2011. Reviewers’ comments were received by the PAD task force from the ACCM and SCCM in November 2011 and from the ASHP reviewers in December 2011. The PAD task force submitted a revised version of the article along with responses to the reviewers’ comments in January 2012, which was accepted for publication by the ACCM Board of Regents, the SCCM, and the ASHP in February 2012. The article was then forwarded to the CCM editorial staff for review. The first round of CCM reviewers’ comments was received in June 2012, which included a request to reduce the size of the PAD guidelines article by 10–20%. With the help of a professional medical editor, the article was edited and resubmitted to CCM in August 2012. The PAD guidelines were accepted by CCM for publication in October 2012, with an electronic version published in December 2012, and a print version published in January 2013. The timeline to complete the external review process and to publish the PAD guidelines took 18 months.
Managing Conflicts of Interest
All PAD task force members were required to complete annual SCCM conflict of interest (COI) statements. To minimize the perception of bias in these guidelines, individuals with a significant COI on a specific topic (e.g., article coauthorship) were recused from grading the literature, writing evidence summaries, and developing specific statements and recommendations on that topic. However, all task force members were allowed to vote anonymously on the final wording, strength of evidence, and strength of recommendations for all questions. These concerns applied only to the guideline development, not to the articles in this supplement. No industry funding or support was used to develop any aspect of the PAD guidelines.
The 2013 ICU PAD guidelines include a total of 54 evidence-based statements (n = 22) and recommendations (n = 32) on the management of pain, sedation, and delirium in adult ICU patients and are the largest set of critical care clinical practice guidelines published to date (1). Six of the 54 statements and recommendations (11%) were based on high quality evidence, 29 (54%) were based on moderate quality of evidence, and 16 (30%) were based on low/very low quality of evidence. Three (5%) had no evidence. Of the 32 recommendations included in the PAD guidelines, 14 were strong, 11 were weak, and 7 yielded no recommendation. Subsequent analysis of the polling results demonstrated that task force members agreed with each other 76–83% of the time when rating the quality of evidence, and agreed 72–74% when rating the strength of recommendations, with a high degree of interrater reliability (quality of evidence, κ = 0.66–0.79; strength of recommendations, κ = 0.66–0.69) (5).
The PAD guidelines included psychometric analyses of 6 behavioral pain scales, 10 sedation/agitation scales, and 5 delirium monitoring tools. Based on these analyses, the most valid and reliable tools for assessing pain in ICU patients who cannot self-report pain are the Behavioral Pain Scale (BPS) and the Critical-Care Pain Observation Tool (CPOT). The most valid and reliable ICU sedation assessment tools are the Richmond Agitation-Sedation Scale (RASS) and the Sedation-Agitation Scale (SAS). The Confusion Assessment Method for the ICU (CAM-ICU) and the Intensive Care Delirium Screening Checklist (ICDSC) are the most valid and reliable delirium assessment tools for use in adult ICU patients.
A meta-analysis was performed to assess the overall impact of sedation with either benzodiazepines (midazolam and lorazepam) or nonbenzodiazepines (propofol and dexmedetomidine) on ICU clinical outcomes. The results of this meta-analysis of 13 studies (n = 1,551 patients) with moderate- to high-quality evidence showed that benzodiazepine sedation is associated with an increased ICU length of stay and a longer duration of mechanical ventilation in ICU patients (1). A more detailed description and discussion of this meta-analysis can be found in the article by Fraser et al (22) in this supplement.
The ICU PAD care bundle serves as a roadmap for implementing the PAD guidelines in an integrated and interdisciplinary fashion (1) (Fig. 1). The PAD care bundle focuses on how to assess, treat, and prevent pain, agitation, oversedation, and delirium in ICU patients and includes metrics for PAD process and outcome measures to assess performance. It also links PAD management with other ICU interventions (i.e., spontaneous awakening and breathing trials, early mobility protocols, and environmental sleep management strategies) to achieve additional improvements in ICU patient outcomes.
The methodology used to develop the 2013 ICU PAD guidelines differs considerably from the previous versions and included the use of: 1) the GRADE method for evaluating the literature and developing all statements and recommendations; 2) a professional librarian to conduct literature searches and to develop and maintain the PAD guideline database; 3) RefWorks software to create a single, electronic, web-based database accessible to all task force members; 4) rigorous psychometric analyses of pain, sedation/agitation, and delirium assessment tools for use in ICU patients; 5) anonymous, electronic polling of all task force members, with predefined voting thresholds to achieve group consensus for all guideline statements and recommendations; and 6) creation of an ICU PAD care bundle to facilitate guideline implementation and widespread adoption.
The Use of GRADE
The 2002 version of these guidelines used the Cochrane methodology for grading the literature and developing recommendations (23). The Cochrane method grades recommendations strictly on the strength of the clinical evidence (i.e., strong evidence yields only strong recommendations, and weak evidence yields only weak recommendations) and allows for the substitution of expert consensus opinion in the absence of evidence (24). The GRADE methodology was developed in 2004 (25) ( http://www.gradeworkinggroup.org) to create “a common, sensible, and transparent approach to grading quality of evidence and strength of recommendations (for clinical practice guidelines).”
GRADE defines specific criteria for rigorously evaluating the literature, but it also takes into account not only the quality of evidence but also the impact of interventions in terms of their relative risks and benefits on important clinical outcomes. As a result, weak recommendations for an intervention may be made based on strong evidence, and vice versa, making guideline recommendations more practical and applicable to clinical practice. In addition, GRADE does not allow for expert consensus opinion to be used as a substitute for either an absence of evidence or conflicting evidence, resulting in more robust guideline recommendations (4, 8, 14). The rigor and transparency of GRADE help to minimize the potential effects of bias and COI on the guideline development process. GRADE has recently been adopted by the Cochrane Collaborative as the standard method for guideline development ( http://www.cochrane.org/training/cochrane-handbook), and GRADE complies with the Institute of Medicine’s (IOM) recently revised standards for developing clinical practice guidelines (26).
The Use of a Professional Librarian
A professional medical librarian was used to conduct literature searches and to develop and maintain the PAD guideline database. By contrast, for the 2002 version of these guidelines, individual task force members independently conducted their own topic searches, using their own search criteria and selection of clinical databases to be searched. Identifying the best available evidence for clinical decision making is essential to creating robust and applicable clinical practice guidelines. But the literature search process itself requires not only adequate knowledge of a subject but also the ability to identify and search appropriate resources, understand database retrieval system interfaces, and the ability to appropriately evaluate search results. Medical librarians with database expertise can significantly improve the efficiency, quality, and yield of database searches for relevant references (27–29). When compared with literature searches performed by clinicians, librarians improve the search yield of relevant references by up to 50% and significantly reduce the number of search errors (30–32). Medical librarians can also identify appropriate clinical databases to include in the search, recognize the strengths and weaknesses of each clinical database, and can perform similar searches across multiple databases (33).
The use of a professional librarian to help develop the PAD guidelines ensured that all literature searches were performed thoroughly and consistently. The explosion of medical information available online has created a false sense of security that clinicians can perform comprehensive, relevant literature searches on their own in order to develop clinical practice guidelines. But search results obtained by individuals who lack the database expertise of medical librarians can be of “questionable quality at best and dangerous at worst” (33). Following publication of the PAD guidelines, PAD task force members were surveyed about the role of the medical librarian in the development of these guidelines (5). Ninety-two percent of task force members felt that the medical librarian had been an invaluable asset to the guideline development process. The addition of a medical librarian to the guidelines development process can help to ensure that the IOM standards for establishing evidence foundations for clinical practice guidelines are being met (34).
The Use of an Electronic Database
The use of RefWorks database software facilitated the creation and utilization of the PAD guideline database, which was maintained by the PAD medical librarian. RefWorks allowed for the creation of a single, web-based, password-protected database that was simultaneously accessible to all PAD task force members. In addition, once an individual reference or PDF was added to the database, it was accessible to all other members of the task force for review. The PAD database could also be referenced by individual task force members (using Write-N-Cite) to draft all segments of the final article, which could then be easily merged into a single document with a bibliography that could be formatted in one step. Having the medical librarian oversee the management of the PAD database ensured its overall integrity throughout the process.
By contrast, there was no single database created for the 2002 version of these guidelines. Task force members would conduct their own individual literature searches, creating multiple individual database libraries with overlapping references that were not necessarily shared among all task force members. It was also impossible to automatically merge these libraries into a single database, in order to format a bibliography for the final article, requiring a time-consuming manual reconciliation of references. No single database of all references included in the bibliography for the 2002 guidelines was ever created. However, the current PAD database will continue to be maintained by the SCCM to facilitate future revisions to the PAD guidelines.
Psychometric Analyses of Pain, Sedation, and Delirium Assessment Tools
Psychometric analyses of pain, sedation, and delirium assessment tools were also unique to the 2013 PAD guidelines. In the 2002 version of the guidelines, routine pain assessment was given a weak recommendation, based on weak (C) evidence, and a numerical rating scale to assess pain was recommended for patients who are able to self-report their pain, based on moderate (B) evidence (2). But there was no clear recommendation for a preferred pain assessment tool(s) for use in ICU patients who cannot self-report their pain. In the 2013 PAD guidelines, pain assessment was given a strong (1+) recommendation, based on moderate (B) evidence (1). And based on rigorous psychometric analyses of six behavioral pain scales, the BPS and CPOT scales were considered to be the most valid and reliable pain scales for use in ICU patients who cannot self-report, based on moderate (B) evidence.
In the 2002 version of the guidelines, regular assessment of the depth of sedation was a weak recommendation, based on weak (C) evidence. The SAS, the Motor Activity Assessment Scale (MAAS), and the Vancouver Interaction and Calmness Scale (VICS) were all recommended as possible sedation monitoring tools, based on moderate (B) evidence. For the 2013 PAD guidelines, routine sedation monitoring specifically targeting a light level of sedation was a strong (1+) recommendation, based on moderate (B) evidence. Based on the psychometric analysis of 10 sedation scales, including the three scales recommended previously in the 2002 guidelines, the RASS and SAS were considered to be the most valid and reliable sedation assessment tools for measuring quality and depth of sedation in adult ICU patients, based on moderate (B) evidence. By contrast, the MAAS and VICS scales were considered to have only low and moderate psychometric properties, respectively.
For the 2002 guidelines, the routine assessment of delirium in ICU patients was advocated, based on moderate (B) evidence. In the 2013 PAD guidelines, routine delirium monitoring was given a strong (1+) recommendation, based on moderate (B) evidence, with both the CAM-ICU and the ICDSC considered to be the most valid and reliable of the five delirium assessment tools, based on strong (A) evidence.
The psychometric analyses of existing pain, sedation, and delirium assessment tools included in the 2013 PAD guidelines give readers a clearer idea of which tools they should use to reliably detect significant pain, assess depth of sedation, and to detect delirium in ICU patients. The ability to distinguish between pain, anxiety, and delirium as the cause of agitation in a critically ill patient enables clinicians to better identify appropriate treatment strategies for these patients.
Achieving Consensus on Guideline Recommendations
The IOM has recently published a set of standards for developing clinical practice guidelines, with the goal of making them more “trustworthy,” “relevant,” and “accessible” to both providers and consumers, using “transparent and objective” processes (26, 34). The use of GRADE together with anonymous voting by all task force members, along with using predefined voting thresholds to develop all statements and recommendations, helped to ensure that the PAD guidelines were both objective and transparent. To our knowledge, using this approach for achieving group consensus for all guideline statements and recommendations is unique to the PAD guidelines. The benefits of this approach were that: 1) there were predetermined criteria for achieving group consensus for all statements and recommendations, which minimized unnecessary discussion and disagreements; 2) all task force members had an “equal voice” in the process; and 3) this approach increased the level of agreement and the degree of interrater reliability for assessing the strength of evidence and recommendations in the PAD guidelines (5, 35). It also helped to focus the subsequent group discussions on how to modify the language used for statements and recommendations in order to making them clearer and more explicit to readers. Providing readers with the polling distribution results for all statements and recommendations is also unique to the PAD guidelines and further increases the transparency of the guideline development process. The use of consistent language for all recommendations (i.e., “We suggest …” for weak recommendations and “We recommend …” for strong recommendations) was new to the PAD guidelines as well. This wording for recommendations was adopted in parallel by the Surviving Sepsis Campaign for developing the 2012 sepsis guidelines (36). The Surviving Sepsis Campaign has used a similar anonymous polling and voting threshold system for achieving group consensus for their guidelines, but has limited its use only to those recommendations where group consensus could not be achieved through open discussions either via e-mail or in person (19, 36). The limited use of this polling methodology and the GRADE grid by the Surviving Sepsis Campaign did facilitate rapid achievement of group consensus and brought closure to topics that had “previously generated extended but apparently inconclusive discussion among expert participants with vigorous views on both the science and the interpretation of research evidence.” For previous versions of the PAD guidelines, the process for achieving group consensus on statements and recommendations was not clearly defined by any predetermined rules, was not anonymous, and was not transparent to readers.
Creating an ICU PAD Care Bundle
The statements and recommendations of the 2013 PAD guidelines have been incorporated into the ICU PAD care bundle in order to facilitate the implementation and widespread adoption of the PAD guidelines. The PAD care bundle takes a systematic, interdisciplinary, team-based approach to assessing and managing pain, depth of sedation, and delirium in ICU patients in an integrated fashion, using both pharmacologic and nonpharmacologic strategies. It applies to all ICU patients, not just those who are mechanically ventilated. However, the PAD care bundle contains very few specific recommendations for the types of analgesics and sedatives to be used in ICU patients, which reflects the relative lack of evidence favoring one drug over another across a variety of clinical circumstances. The most notable medication-related recommendations in the PAD care bundle are to avoid the use of benzodiazepines in ICU patients who are either delirious or at high risk for delirium. The PAD care bundle focuses on identifying and treating pain first, then determining the need for sedation, with the goal of keeping patients lightly sedated whenever possible. It highlights the need for identifying ICU patients who are at risk for delirium, and managing delirious patients with both pharmacologic and nonpharmacologic interventions, using antipsychotics judiciously. It also links PAD management strategies to other evidence-based management strategies to achieve synergistic benefits in ICU patient outcomes, such as: 1) linking PAD protocols with spontaneous awakening trials and spontaneous breathing trials to facilitate weaning from mechanical ventilation; 2) linking spontaneous awakening trials to early mobility and exercise protocols to reduce delirium and to improve strength; 3) implementing environmental controls to protect patients’ sleep-wake cycles to reduce delirium and to improve sleep. A more detailed discussion of the potential benefits of implementing the ICU PAD care bundle can be found in the article by Barr and Pandharipande (37) in this supplement.
By contrast, the treatment algorithm proposed for managing sedation and analgesia in the 2002 version of the guidelines was almost exclusively drug based, making recommendations for using specific analgesics and sedatives in various clinical circumstances (i.e., fentanyl for analgesia in hemodynamically unstable patients and morphine for hemodynamically stable patients; midazolam for acute agitation and lorazepam for maintenance and long-term sedation; and propofol for neurosurgical patients) (2). The 2002 guidelines also recommended that sedation should be monitored using a sedation scale, and that sedatives should be titrated to a “defined endpoint,” but light sedation was neither defined nor promoted in these guidelines as the ideal level of sedation to be maintained in most ICU patients.
Clinical practice guidelines are crucial in helping clinicians make informed and evidence-based decisions in managing critically ill patients, and the development of high-quality guidelines is imperative to this process. The overall approach used for the 2013 PAD guidelines is consistent with the current IOM recommendations for clinical practice guideline development and has helped to define a more contemporary and rigorous approach for developing high-quality clinical practice guidelines.
We thank and acknowledge psychometric experts David Streiner, PhD (University of Toronto, Department of Psychiatry, Toronto, ON, Canada; and McMaster University, Department of Clinical Epidemiology and Biostatistics, Hamilton, ON, Canada), Celeste Johnston, RN, DEd (School of Nursing, McGill University, Montreal, QC, Canada), and Carolyn Waltz, RN, PhD, FAAN (School of Nursing, University of Maryland, Baltimore, MD), for their help in developing the psychometric assessment tools used to develop the pain, agitation, and delirium guidelines; and Patricia Rohr, Professional Medical Editor (Stanford University School of Medicine, Stanford, CA), for her help with editing the final PAD guidelines article.
1. Barr J, Fraser GL, Puntillo K, et al. American College of Critical Care Medicine Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit. Crit Care Med. 2013; 41:263–306
2. Jacobi J, Fraser GL, Coursin DB, et al. Task Force of the American College of Critical Care Medicine (ACCM) of the Society of Critical Care Medicine (SCCM), American Society of Health-System Pharmacists (ASHP), American College of Chest Physicians Clinical practice guidelines for the sustained use of sedatives and analgesics in the critically ill adult. Crit Care Med. 2002; 30:119–141
3. Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol. 2011; 64:395–400
4. Guyatt GH, Oxman AD, Kunz R, et al. GRADE Working Group What is “quality of evidence” and why is it important to clinicians? BMJ. 2008; 336:995–998
5. Davidson JE, Tung A, Kishman CP Jr, et al. Reflecting on use of the GRADE process for development of the 2013 PAD guidelines. Semin Respir Crit Care Med. 2013; 34:262–272
6. Schünemann HJ, Jaeschke R, Cook DJ, et al. ATS Documents Development and Implementation Committee An official ATS statement: Grading the quality of evidence and strength of recommendations in ATS guidelines and recommendations. Am J Respir Crit Care Med. 2006; 174:605–614
7. Guyatt G, Gutterman D, Baumann MH, et al. Grading strength of recommendations and quality of evidence in clinical guidelines: Report from an American college of chest physicians task force. Chest. 2006; 129:174–181
8. Guyatt GH, Oxman AD, Vist GE, et al. GRADE Working Group GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008; 336:924–926
9. Pudas-Tähkä SM, Axelin A, Aantaa R, et al. Pain assessment tools for unconscious or sedated intensive care patients: A systematic review. J Adv Nurs. 2009; 65:946–956
10. Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. 2008; Fourth Edition Oxford Oxford University Press
11. Gélinas C, Puntillo KA, Joffe AM, et al. A validated approach to evaluating psychometric properties of pain assessment tools for use in non-verbal critically ill adults. Semin Respir Crit Care Med. 2013; 34:153–168
12. Robinson BR, Berube M, Barr J, et al. Psychometric analysis of subjective sedation scales in critically ill adults. Crit Care Med. 2013; 41:S16–S29
13. Allen J, Dyas J, Jones M. Building consensus in health care: A guide to using the nominal group technique. Br J Community Nurs. 2004; 9:110–114
14. Guyatt GH, Oxman AD, Kunz R, et al. GRADE Working Group Going from evidence to recommendations. BMJ. 2008; 336:1049–1051
15. Cross H. Consensus methods: A bridge between clinical reasoning and clinical research? Int J Lepr Other Mycobact Dis. 2005; 73:28–32
16. McGinnis PQ, Wainwright SF, Hack LM, et al. Use of a Delphi panel to establish consensus for recommended uses of selected balance assessment approaches. Physiother Theory Pract. 2010; 26:358–373
17. Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998; 2:i–iv, 1
18. Linstone HA, Turoff M. The Delphi Method: Techniques and Applications. 2002; Reading, MA Addison-Wesley
19. Jaeschke R, Guyatt GH, Dellinger P, et al. GRADE Working Group Use of GRADE grid to reach decisions on clinical practice guidelines when consensus is elusive. BMJ. 2008; 337:a744
20. Dellinger RP, Levy MM, Carlet JM, et al. International Surviving Sepsis Campaign Guidelines Committee; American Association of Critical-Care Nurses; American College of Chest Physicians; American College of Emergency Physicians; Canadian Critical Care Society; European Society of Clinical Microbiology and Infectious Diseases; European Society of Intensive Care Medicine; European Respiratory Society; International Sepsis Forum; Japanese Association for Acute Medicine; Japanese Society of Intensive Care Medicine; Society of Critical Care Medicine; Society of Hospital Medicine; Surgical Infection Society; World Federation of Societies of Intensive and Critical Care Medicine Surviving Sepsis Campaign: International guidelines for management of severe sepsis and septic shock: 2008. Crit Care Med. 2008; 36:296–327
21. Barr J, Fraser GL, Puntillo K, et al. Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit: Executive summary. Am J Health-Syst Pharm. 2013; 70:53–58
22. Fraser GL, Devlin JW, Worby CP, et al. Benzodiazepine versus nonbenzodiazepine-based sedation for mechanically ventilated, critically ill adults: A systematic review and meta-analysis of randomized trials. Crit Care Med. 2013; 41:S30–S38
23. Nasraway SA Jr, Jacobi J, Murray MJ, et al. Task Force of the American College of Critical Care Medicine of the Society of Critical Care Medicine and the American Society of Health-System Pharmacists, American College of Chest Physicians Sedation, analgesia, and neuromuscular blockade of the critically ill adult: Revised clinical practice guidelines for 2002. Crit Care Med. 2002; 30:117–118
24. Field MJ, Lohr KN, et al. Institute of Medicine (U.S.). Clinical Practice Guidelines: Directions for a New Program. 1990; Washington, DC National Academy Press
25. Atkins D, Best D, Briss PA, et al. GRADE Working Group Grading quality of evidence and strength of recommendations. BMJ. 2004; 328:1490
26. Graham R. Institute of Medicine (U.S.): Committee on Standards for Developing Trustworthy Clinical Practice Guidelines: Clinical Practice Guidelines We Can Trust. 2011; Washington, DC National Academies Press
27. Klem ML, Weiss PM. Evidence-based resources and the role of librarians in developing evidence-based practice curricula. J Prof Nurs. 2005; 21:380–387
28. Rankin JA, Grefsheim SF, Canto CC. The emerging informationist specialty: A systematic review of the literature. J Med Libr Assoc. 2008; 96:194–206
29. Wade CA, Turner HM, Rothstein HR, et al. Information retrieval and the role of the information specialist in producing high-quality systematic reviews in the social, behavioural and education sciences. Evid Policy. 2006; 2:89–108
30. Gardois P, Calabrese R, Colombi N, et al. Effectiveness of bibliographic searches performed by paediatric residents and interns assisted by librarians. A randomised controlled trial. Health Info Libr J. 2011; 28:273–284
31. Gruppen LD, Rana GK, Arndt TS. A controlled comparison study of the efficacy of training medical students in evidence-based medicine literature searching skills. Acad Med. 2005; 80:940–944
32. Sampson M, McGowan J. Errors in search strategies were identified by type and frequency. J Clin Epidemiol. 2006; 59:1057–1063
33. Medical Library Association Role of expert searching in health sciences libraries. J Med Libr Assoc. 2005; 93:42–44
34. Kuehn BM. IOM sets out “gold standard” practices for creating guidelines, systematic reviews. JAMA. 2011; 305:1846–1848
35. Atkins D, Briss PA, Eccles M, et al. GRADE Working Group Systems for grading the quality of evidence and the strength of recommendations II: Pilot study of a new system. BMC Health Serv Res. 2005; 5:25
36. Dellinger RP, Levy MM, Rhodes A, et al. Surviving Sepsis Campaign Guidelines Committee including the Pediatric Subgroup Surviving sepsis campaign: International guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013; 41:580–637
37. Barr J, Pandharipande P. The pain, agitation, and delirium care bundle: Synergistic benefits of implementing the 2013 pain, agitation, and delirium guidelines in an integrated and interdisciplinary fashion. Crit Care Med. 2013; 41:S99–S115
Copyright © 2013 by the Society of Critical Care Medicine and Lippincott Williams & Wilkins