Effectiveness of strategies to implement urinary incontinence clinical practice guidelines on healthcare delivery and patient outcomes : JBI Evidence Synthesis

Secondary Logo

Journal Logo

Advanced Search
Systematic Review Protocol

Effectiveness of strategies to implement urinary incontinence clinical practice guidelines on healthcare delivery and patient outcomes

JBI Library of Systematic Reviews 9(32):p 1-24, | DOI: 10.11124/jbisrir-2011-411
  • Free

Centre conducting review

The Spanish Centre for Evidence Based Healthcare, Institute of Health Carlos III. Madrid: a Collaborating Centre of the Joanna Briggs Institute.

Contact for review

Name: Gema Escobar-Aguilar RN, MSc, PhD candidate

Phone: +34 918222557

Fax: +34 913877897

Email: [email protected]

Review panel

Principal reviewer: Gema Escobar-Aguilar RN, MSc, PhD candidate

The Spanish Centre for Evidence Based Healthcare, Institute of Health Carlos III. Madrid: a Collaborating Centre of the Joanna Briggs Institute.

Email: [email protected]

Secondary reviewer: Ana Barderas-Manchado MLSc

Email: [email protected]

Secondary reviewer: María Taltavull.-Aparicio RN, MSc

Email: [email protected]

Secondary reviewer: Pilar Córcoles-Jimenez RN, MSc, PhD.

Email: [email protected]

Background

Urinary incontinence

Urinary incontinence (UI) is a common problem among adults. It is more frequent among women, increases with age and level of functional dependence1,2 and is a fairly generalised problem in nursing homes and hospitals for the chronically ill 3, 4.

The most common types of UI are urge urinary incontinence, characterized by an urgent need to void which cannot be delayed; stress urinary incontinence, associated with involuntary loss of urine when intra-abdominal pressure increases, such as during coughing, sneezing, laughing, or physical exertion; mixed urinary incontinence, which combines the symptoms of stress and urge UI; and functional incontinence, in which there is no demonstrable alteration of the urination control mechanisms but, due to physical (reduced mobility), cognitive (dementia) disorders or the use of certain drugs, the individual is unable to reach the bathroom in time5,6. It has been established7, 8, 9 that prevalence of urinary incontinence in women varies from 23 to 40%. According to Moehrer7, close to 20% of women, between the ages of 25-64 years, experience symptoms related to UI. These symptoms increase during perimenopause, when almost 31% of women report experiencing urine loss at least once a month. Although UI occurs in nulliparous women, evidence identifies vaginal birth as a significant predictor of UI. Even women who have had just one vaginal birth are 2.5 times more likely to report incontinence than are their nulliparous counterparts10.

The prevalence is even greater in the population of 65 and over. One of the largest transversal studies on urinary incontinence in women (27.936 Norwegian women) indicate a gradual increase in prevalence with age, peaking at maturity (50 to 54 years), followed by a decline and then a further peak in old age (70 years)11. In Spain, Gavira and cols, reported a prevalence of 31.7% among women between 60-69 years, 35% among women between 70-79 years and 53.3% among women over the age of 809.

Functional incontinence is the most frequent form among people of 65 and over and has the worst prognosis. The risk factors associated to the development of functional incontinence include, institutionalisation, confusion and the use of wheelchairs or other devices to move around prior to ambulation following surgery. Palmer et al12, have established that there is an increase in the prevalence of urinary incontinence in people of 65 and over who have undergone surgical hip fracture repair. This study identified an increase in the rate of urinary incontinence pre and post op in 20 to 43% of cases respectively, the risk was greater among men and people with some form of cognitive disorder, although the sample reviewed was small. In 2002, a further study found that 21% of women hospitalised due to hip fracture developed urinary incontinence during their hospital stay13.

A recent cohort study conducted in 2009 among 163 continent hip repair subjects over the age of 65, with no cognitive impairment and independent in performing daily life activities14, has shown that after three months, 11.4% presented signs of urinary incontinence and 18.1% suffered from occasional urinary incontinence. At 6 months, 12.4 and 17.9% suffered from total incontinence and occasional incontinence; and 8.3 and 19.5% by twelve months respectively, indicating that once incontinence has set in, it does not appear to improve with time, in spite of enhanced mobility.

UI has a profound impact on daily and social activity15, it affects the mental health of those who suffer from it16 and it also represents an increasing cost to healthcare systems world-wide. Treatment of UI can range from the use of medications and surgical intervention to more or less conservative approaches. Studies into these therapeutic options show that bladder training and pelvic floor exercises are effective in increasing bladder volume capacity and improving bladder voiding cycles and can also be effective in stress urinary incontinence by enhancing support of the urethra10.

Although there is no high evidence that pelvic floor exercises prevent post-partum incontinence, the evidence is more encouraging for women at risk of developing incontinence after childbirth17. One meta-analysis has shown that practising pelvic floor exercises during pregnancy is an effective way to prevent and treat UI, as is doing these exercises post-partum18. Other systematic reviews have established that pelvic floor exercises should be incorporated into conservative treatment schedules as a first line treatment in women with stress, urge or mixed urinary incontinence11, 17.

Despite its high prevalence and the existing evidence, UI remains underreported, under-diagnosed and consequently, undertreated1, 2, 19. This has been attributed to the poor attitudes and misconceptions by both, patients and practitioners, to a lack of knowledge and education regarding its assessment and treatment and to the wide variations in the quality of care provided by regions and countries around the world20. As a result, fewer than 50% of affected people seek advice from their healthcare providers2.

Consequently, the translation of research outcomes on continence care into daily practice is essential for improving health care delivery and patient outcomes.

Use of Urinary Incontinence Clinical Practice Guidelines

In recent years there has been growing interest in how the outcomes of research are disseminated and implemented in clinical practice, as health service planning and delivery should be based on valid and current evidence.

This interest stems from the Evidence Based Practice (EBP) movement -defined as the “conscientious, explicit and judicious use of current best evidence in making decisions about care of individual patients…”-, from the ongoing debate around how to best guarantee effectiveness and efficiency of care, as well as from changes in policy related to user involvement and the concept of practitioner accountability21,22.

According to Sackett22, EBP is a 5 step process that includes 1) Translation of uncertainty to answerable questions, 2) Systematic retrieval of the best evidence available, 3) critical appraisal of the evidence, 4) Results implementation in practice, and 5) Evaluation of clinical practice. The key to more widespread application of research hinges on the fourth step, the implementation of results in clinical practice23-25.

One of the most common strategies used world-wide, to bring evidence into clinical practice is the development, of evidence based clinical practice guidelines (CPGs). According to the American Institute of Medicine, clinical practice guidelines can be defined as “A set of systematically developed statements, usually based on scientific evidence, to assist practitioners and patient decision making about appropriate healthcare for specific clinical circumstances” (IOM, 1990). Thus, the purpose of the clinical practice guidelines is to modify clinical practices to enhance patient outcomes, reduce variability in clinical practice, make more cost efficient use of services, improve collaboration between professionals and ensure that the interventions implemented are the most appropriate26. In order to achieve these goals, it is necessary to use CPGs dissemination and implementations strategies. The objective of disseminating research is to raise awareness of its outcomes, so that its findings are implemented into clinical practice, which is what is understood as implementation27. Grol et al.28 provide an overview about approaches to introducing evidence and CPGs into routine daily practice. Among the dissemination strategies mentioned were the following: continuing medical education, mass media campaigns, newsletters, videos, etc. Some of the implementation strategies reviewed included: educational outreach visits, local opinion leaders, audit and feedback, use of reminders and computers, substitution of task, multiprofessional collaboration and patient-mediated interventions. The study concluded that there are many different approaches to changing practice, which all claim to be effective, but none of them is superior for all changes in all situations.

During the bibliographic search, the reviewers found several systematic reviews that assessed the effects of clinical practice guidelines implementation in healthcare settings, however none of them was specifically for urinary incontinence. A systematic review that identified 91 studies examining the evidence on whether CPGs can change behaviour of health professionals and how best to introduce them into clinical practice, concluded that guidelines are more likely to be effective if they take into account local circumstances, are disseminated by an active educational intervention, and implemented by patient specific reminders relating directly to professional activity29. Grimshaw and Russell26, performed a review of the implementation of CPGs for medical staff, whilst Thomas et al.30 studied the implementation of guidelines for nurses and other related professions. Both reviews concluded that care based on adequately developed CPGs can change clinical practice and modify patient outcomes, although further studies using a higher quality methodology are required to assess the various strategies used to disseminate and implement clinical practice guidelines. Thus, the literature shows that implementation strategies which are nearer the end user and integrated into the process of health care delivery are more likely to be effective.

According to a literature review which evaluated the application of clinical practice guidelines, protocols and clinical pathways on urinary incontinence31, the implementation of CPGs on urinary incontinence seems to be effective in improving patient outcomes, albeit substantially increasing costs; is effective in pre-incontinent women but not in women of 65 and over; that practitioners have identified many barriers to the implementation of these guidelines; that educational initiatives alone are not particularly effective; and other approaches are needed. To date, no systematic review could be found that evaluated the effectiveness of dissemination and/or implementation of urinary incontinence clinical practice guidelines, thus it was decided to conduct such a systematic review to assess the effectiveness of this approach in improving professional practice and patient outcomes.

Review Question:

Are strategies to disseminate and/or implement urinary incontinence clinical practice guidelines effective in modifying healthcare delivery and improving patient outcomes?

Objective:

The overall aim of this review is to establish the effectiveness of strategies to disseminate and/or implement urinary incontinence clinical practice guidelines in modifying healthcare delivery and patient outcomes.

More specifically, the objectives are to:

  • To establish the effectiveness of strategies to disseminate and/or implement urinary incontinence clinical practice guidelines in modifying healthcare delivery.
  • To establish the effectiveness of strategies to disseminate and/or implement urinary incontinence clinical practice guidelines in improving health outcomes of patients with urinary incontinence.
  • To describe the quality of the clinical practice guidelines on urinary incontinence used in the included studies.

Inclusion Criteria

Types of participants

The review will consider studies in which the units of analysis are:

  • Healthcare providers treating patients with urinary incontinence. Physicians, nurses, midwives and physical therapists will be included; and/or
  • Patients with some form of urinary incontinence: urge, stress, mixed incontinence or functional incontinence; and/or
  • Healthcare institutions.

Types of interventions

Studies which evaluate the effectiveness of strategies to disseminate and/or implement urinary incontinence clinical practice guidelines will be included:

The following groups will be used for comparison:

  • Urinary incontinence clinical practice guidelines plus a strategy to disseminate and/or implement vs. no guideline.
  • Urinary incontinence clinical practice guidelines plus a strategy to disseminate and/or implement vs. guidelines plus another strategy to disseminate and implement.

For the purpose of this review:

  • Clinical practice guidelines will be defined as: “A set of systematically developed statements, usually based on scientific evidence, to assist practitioners and patient decision making about appropriate healthcare for specific clinical circumstances”, and taking into account the following criteria:
    • That they be based on reviews that systematically identify, critically appraise, summarize and interpret the evidence.
    • That they clearly identify the recommendations and data sources and the quality of these.
    • That they clearly indicate by whom and how often the guideline will be reviewed.
  • Dissemination is defined as any process by which information is transmitted (made available or accessible) to intended audiences or target groups (e.g., continuing medical education, mass media campaigns, newsletters, videos…).
  • Implementation is defined as promoting the use of research findings and other evidence-based practices into routine practice (e.g., educational outreach visits, local opinion leaders, audit and feedback, use of reminders and computers, substitution of task, mulltiprofessional collaboration, patiet-mediated interventions…)

Type of outcome measures

The review will include those studies which assess the effect of implementing urinary incontinence clinical practice guidelines on the performance of healthcare providers and on patient outcomes.

The following will be considered specific outcome measures:

  • Related with healthcare delivery:
    • Degree of compliance with the clinical practice guidelines
    • Number of Patients treated according to the clinical practice guidelines
    • Indicators of quality of care
    • Changes in clinical practice: Evaluation of the presence or evolution of urinary incontinence throughout the patient’s history (% of patients identified, % of patients in follow-up), physical examination, request for tests, % of treatment prescriptions.
  • Related to patient outcomes:
    • Degree of Urinary incontinence
    • Number of episodes of Urinary incontinence
    • Amount of urine lost
    • Type and number of absorbent pads used
    • Changes in bladder voiding patterns
    • Impact of Urinary incontinence on health
    • Level of satisfaction with healthcare delivered
    • Quality of life

Types of studies

This review will consider randomised clinical trials, before and after controlled studies, cohort studies and quasi experimental studies without a control group.

Search strategy

The literature review will focus on both a search for studies published and indexed in national and international databases as well as a search for unpublished studies.

In an initial phase an electronic search has been conducted in the MEDLINE and CINAHL databases to identify publications from 2008-2009. The objective was to analyse the words contained in title, abstract and Mesh terms in the most relevant articles identified which will then permit an electronic search in the second phase.

The words located in this initial phase are:

  • Urinary incontinence/ incontinence/ female urinary incontinence/ urge urinary incontinence/ stress urinary incontinence/ mixed urinary incontinence/ functional urinary incontinence/ implementing incontinence research/ continence nurse/ bladder training/ Pelvic Muscle Exercise/ voiding pattern/ voiding intervals
  • Clinical practice guidelines/ guidelines/ protocols/ care pathways/ evidence based guidelines/ evidence based protocols/ health planning guideline/ care map/ professional standard/ Practice Guidelines as Topic
  • Evidence Based Practice/ Best practice/ Evidence implementation/ Changing practice/
  • Applied research/ decision aids/ decision making
  • Research utilization/ Knowledge utilization/ Knowledge translation/Knowledge uptake/ Research uptake/ implementation of research findings into practice/ translating research into practice
  • Randomized control trials/ Controlled clinical trials/ Quasi-experimental studies/ Cohort studies/ Interrupted time series/ Controlled before and after designs/
  • Evaluation of the Efficacy-Effectiveness of Interventions/ Efficacy/ Effectiveness/ Efficiency

In the second phase, an electronic search will be conducted using both indexing and “free text” terms, incorporating the terms extracted from the initial search as well as the key words specific to each database.

The search strategy will encompass studies published in English, French, Portuguese and Spanish. It will not be limited in time (from the creation of the data bases until the performance of the search) and will be carried out in the following data bases: MEDLINE, COCHRANE, CINAHL, PSYCINFO, EMBASE, DHSS-DATA, SUMSEARCH, TRIPDATABASE, IME, CUIDEN, PBSC, DARE, LILACS and IBECS. The SCIELO platform as well as other sources of clinical practice guidelines from different countries for example National Guidelines Clearinghouse, Health Services/Technology Assessment Text (HSTAT) will also be consulted.

Searches in the Spanish databases will be performed using the same terms used to search the English language databases as well as terms specific to word indexes. The same will apply for the French and Portuguese databases.

Likewise a secondary search of articles will be conducted based on the references cited in the studies identified. Additionally, a manual search will also be conducted in journals in which according to the electronic search an important number of documents have been located.

The third phase will consist of the identification of unpublished studies or Grey literature using the following sources: Dissertation abstracts international, Proceedings database, SIGLE (System for information on Grey Literature database), The New York Academy of Medicine and contacts with “key” individuals in the review field, Mednar Database, as well as a manual search of conference proceedings and work done by foundations, etc. will be conducted.

Selection of studies for critical assessment

Two independent Reviewers will assess how relevant the studies are by title and abstract to determine whether or not they meet the inclusion criteria and will eliminate those which are obviously not eligible. Where there is disagreement between reviewers the full article will be read and both reviewers will then discuss and agree upon the relevance of the article.

In the absence of an abstract the full text of the document will be retrieved. The process to select studies will be performed using a purpose-designed form containing all the selection criteria (APPENDIX I).

Critical Appraisal

The full texts of the articles selected for appraisal will be retrieved and will undergo critical appraisal according to their methodological quality, by two independent reviewers.

For the purpose of critical appraisal the Joanna Briggs Institute critical appraisal tools will be applied according to the type of study being appraised.

  • Experimental Studies (APPENDIX II)
  • Cohort Studies (APPENDIX III)

The possible answers to each item contained in the critical appraisal form are “Yes”, “No” or “unclear”. The studies in which the all the items are answered “Yes” will be considered “High Quality”, those in which some items are answered “Unclear” will be considered “Moderate Quality” and those studies in which any of the items are answered “No” will be considered “Low Quality”. If according to the study design, some questions are not applicable these will not be taken into account in the quality assessment (see Appendix II)

Should any discrepancies arise with regards the quality of the study, the reviewers will meet to discuss these and reach an agreement. In the event no agreement can be reached a third reviewer will arbitrate.

Classification of Studies according critical appraisal

TU1-6
Table:
No Caption available.

Quality of the clinical practice guidelines used in the studies included in the review

The clinical practice guideline appraisal tool “AGREE II instrument” (available from http://www.agreetrust.org/about-agree/introduction0/) will be used to analyse the quality of the clinical practice guidelines used in the studies selected for this review.

Data extraction

Data from the studies in the review are extracted using an adaptation of JBI data extraction tool (APPENDIX IV). Prior to extracting the data a pilot will be conducted to evaluate the appropriateness of the data extraction form. Data will be extracted independently by two reviewers and then crosschecked for verification. Any difference of opinion regarding data extraction will be discussed and resolved by the reviewers. The study authors may be requested to provide further clarification where information is lacking or if the existing information is insufficient for the formal comparisons.

In order to perform a quality control of the data and avoid errors in transcription, there will be a double entry of at least 20% of the data.

Data synthesis

1°) Firstly, a description will be provided of the studies selected for evaluation, setting out the rationale for their inclusion or exclusion and the study characteristics.

2°) Secondly, the clinical heterogeneity of the studies included in the review will be evaluated according to the study population, the type of intervention and the outcome variables measured. Likewise, the methodological heterogeneity will be assessed taking into account the study design and quality.

If comparable studies are found (with clinically and methodological heterogeneity) these will be combined in a meta-analysis. The degree of statistical heterogeneity will be calculated using chi-squared and I2, heterogeneity of p values will be taken to be p<0.10 and I2>50%.

Where there is heterogeneity a random effects model analysis will be performed as well as a subgroup analysis according to the methodological quality of the studies.

The corresponding effect measures will be calculated using odds ratio (for dichotomous outcomes) or weighted mean differences (for continuous outcomes) with a confidence interval (CI) of 95%. In order to evaluate the possible publication bias a sensitivity analysis will be performed and the results will be presented on a funnel plot.

Where is not appropriate or feasible to perform the statistical analysis as described above the data will be presented in a narrative synthesis.

The review will be conducted using the JBI MASTARI software.

Potential conflicts of interest:

The authors have declared they have no conflicts of interest with any of the aspects examined in this systematic review.

REFERENCES

1. Ueda T, Tamaki M, Kageyama S, Yoshimura N, Yoshida O. Urinary incontinence among community-dwelling people aged 40 years or older in Japan: prevalence, risk factors, knowledge and self-perception. Int J Urol. 2000; 7:95-103.
2. Liu Ch, Andrews GR. Prevalence and incidence of urinary incontinence in the elderly: a longitudinal study in South Australia. Chin Med J. 2002; 115:119-22.
3. Fleishman RH, Heilbrun G, Mandelson J, Shirazi V. Improving the quality of institutional care of urinary incontienence among the elderly: a challenge for governmental regulation. Int J Health Care.1999; 12:105-19.
4. Durrant J, Snape J. Urinary incontinence in nursing homes for older people. Ageing. 2003; 32:12-8.
5. Abrams P, Cardozo L, Fall M, Griffiths D, Rosier P, Ulmsten U et al. The standardisation of terminology of lower urinary tract function: report from the standardisation sub-committee of the International Continence Society. Neurourol Urodyn. 2002;21(2):167-78.
6. Roqueta C, Miralles R, Cervera AM, Mendo M, Esperanza A. Incontinencia urinaria funcional de inicio reciente. Revista multidisciplinar de Gerontología. 2004; 14(5):289-294.
7. Hunskaar S, Lose G, Sykes D, Voss S. The prevalence of urinary incontinence in women in four European countries. BJU Int.2004; 93:324-30.
8. Moehrer B, Hextall A, Jackson S. Estrógenos para la incontinencia urinaria en mujeres. En: LaBiblioteca Cochrane Plus, 2006 Número 1. Oxford: Update Software Ltd. Disponible en: http://www.update-software.com. (Traducida de The Cochrane Library, 2006 Issue 1. Chichester, UK: John Wiley & Sons, Ltd.).
9. Gavira F, Caridad J, Pérez J, Valderrama E, López M, Romero M, et al. Prevalence and psychosocial impact of urinary incontinence in older people of a Spanish rural population. J Gerontol Ser A Biol Sci Med Sci.2000; 55 A(4):207-14.
10. Sampselle CM, Burns PA, Dougherty MC, Newman DK, Thomas KK, Wyman JF. Continence for Women: Evidence-Based Practice. JOGNN.1997; 26(4):375-85.
11. Hay-Smith EJC, Dumoulin C. Entrenamiento muscular del suelo pelviano versus ningún tratamiento, o tratamientos de control inactivo, para la incontinencia urinaria en mujeres (Revisión Cochrane traducida). En: La Biblioteca Cochrane Plus, 2007 Número 4. Oxford: Update Software Ltd. Disponible en: http://www.updatesoftware.com. (Traducida de The Cochrane Library, 2007 Issue 4. Chichester, UK: John Wiley & Sons, Ltd.).
12. Palmer MH, Myers AH, Fedenko KM. Urinary continence changes after hip-fracture repair. Clin Nurs Res. 1997; 6(1):8-21.
13. Palmer MH, Baumgarten M, Langenberg P, Carson JL. Risk factors for hospital-acquired incontinence in elderly female hip fracture patients. J Gerontol A Biol Sci Med Sci. 2002; 57(10):672-677.
14. Del Egido Fernández MA, Villada Munera A, Jiménez Sánchez D, Córcoles Jiménez MP, Candel Parra E, Moreno Moreno M, Carrión González M, Denia Cortés A. Incontinencia funcional sobrevenida en pacientes ancianos tras fractura de cadera secundaria a una caída. Enfuro Asociación Española de Enfermería en Urología. 2009; 109: 27-31
15. Hagglund D, Walker-Engstrom ML, Larsson G, Leppert J. Changes in urinary incontinence and quality of life in four years. A population-based of women aged years. Scand J Prim Health Care. 2004; 22(2):112-17.
16. Hajjar R. Psychosocial impact of urinary incontinence in the elderly population. Clin Geriatr Med. 2004; 20(3):5523-64.
17. Hay-Smith EJC, Herbison P,ved S. Fisioterapia para la prevención de la incontinencia urinaria y fecal en adultos. En: La Biblioteca Cochrane Plus, 2006 Número 1. Oxford: Update Software Ltd. Available in: http://www.update-software.com (Traducida de The Cochrane Library, 2006 Issue 1. Chichester, UK: John Wiley& Sons Ltd).
18. Haddow G, Watts R, Robertson J. Effectiveness of a pelvic floor muscle exercise program on urinary incontinence following childbirth. Int J Evid Based Healthc. 2005; 3:103-46.
19. Adedokun AO, Wilson MM. Urinary incontinence: historical, global, and epidemiologic perspectives. Clin Geriatr Med. 2004; 20(3):399-407.
20. Knight R, Procter S. Implementation of clinical guidelines for female urinary incontinence: a comparative analysis of organizational structures and service delivery. Health and Social Care in the Community.1999; 7(4):280-90.
21. Kitson A., Ahmed L.B., Harvey G., Seers K., & Thompson D.R. From research to practice: one organizational model for promoting research-based practice. Journal of Advanced Nursing. 1996:23; 430-40.
22. Sackett DL, Richardson S, Rosenberg W, Haynes RB. Evidence based medicine: How to practice and how to teach. EBM. Edinburgh: Churchill Livingstone. 2000.
23. Bero L.A., Grilli R., Grimshaw JM., Harvey E., Oxman H. & Thomson, M.A. Getting research findings into practice: Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. British Medical Journal. 1998; 317:465-468.
24. Estabrooks C.A. Modelling the individual determinants of research utilization. Western Journal of Nursing Research. 1999; 21 (6): 758-772.
25. Thomson DS, Estabrooks CA, Scott-Findlay S, Moore K, Wallin L. Interventions aimed at increasing research use in nursing: a systematic review. Implementation Science. 2007; 2: 15. [on line]. Available in: http://www.implementationscience.com/content/2/1/15
26. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluation. Lancet. 1993; 342:1317-22.
27. NHS Centre for Reviews and Dissemination. Getting evidence into practice. Effective Health Care. 1999; 5(1):1-16.
28. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients care. The lancet. 2003;362:1225-30
29. NHS Centre for Reviews and Dissemination. Implementing clinical practice guidelines: can guidelines be used to improve clinical practice? Effective Health Care. 1994; 1(8):1-12.
30. Thomas L, Cullum N, Mccoll E, Rousseau N, Soutter J, Steen N. (1998). Guías para profesionales relacionados con la medicina. (Revisión Cochrane traducida). En: La biblioteca Cochrane Plus, 2008 Número 1. Oxford: Update Software Ltd. Available in: http://www.update-software.com (Traducida de The Cochrane Library, 2008 Issue 1. Chichester, UK: John Wiley & Sons, Ltd.).
31. Roe B, Watson NM, Palmer MH, Mueller C, Vinsnes AG, Wells M. Translating research on incontinence into practice. Feature articles. 2004; 53(6S):S56-S60.

APPENDIX I. Study selection criteria

TU2-6
Table:
No Caption available.

Note: In order for the study to be included in the review both reviewers must answer YES to all of the items

APPENDIX II. Critical appraisal tool for experimental studies.

TU3-6
Table:
No Caption available.

Note

  • Question 1 is not applicable to nonrandomised experimental studies.
  • For uncontrolled experimental studies questions 1, 2, 3, 6 and 7 are not applicable. Pre and Post comparisons are required for question 8.

1. Was the assignment to treatment groups truly random?

There are three broad types of randomisation within trials, randomisation, quasi- (or pseudo) and stratified randomisation. True randomisation occurs when every patient has a truly equal chance of being in any group included in the trial. This may involve using computer generated allocation methods to ensure allocation is truly random. The Consort criteria for randomisation (http://www.consortstatement.org/index.aspx?o=1025, accessed 21st Sept, 2007) state that: With random allocation, each participant has a known probability of receiving each treatment before one is assigned, but the actual treatment is determined by a chance process and cannot be predicted. Consort emphasises that true randomisation will minimise selection bias, thus identification of the method of randomisation provides reviewers with a good indication of study quality. In the presence of true randomisation, the sample is said to be representative of the population of interest, with homogeneity of characteristics at baseline. Hence any variation between groups in the trial would be expected to reflect similar differences in the relevant population. In quasi-randomisation, allocation is not truly random, being based on a sequential method of allocation such as birth date, medical record number, or order of entry in to the study (alternate allocation). These methods may not conceal allocation effectively; hence there is an increased risk of selection bias associated with their usage. The third type of randomisation commonly utilised in randomised trials is stratification. Stratification may be used where a confounding factor (a characteristic that is considered likely to influence the study results, i.e. medications or co-morbidities) needs to be evenly distributed across groups. Whichever approach to randomisation was used, it should be described with sufficient detail to enable reviewers to determine whether the method used was sufficient to minimise selection bias. Authors of primary studies have competing interests in describing their methods, the need to be descriptive at times conflicts with the need to fit within word limits. However, brevity in the methods often leaves reviewers unable to determine the actual method of randomisation. Generalist phrases such as “random”, “random allocation” or “randomisation” are not sufficient detail for a reviewer to conclude randomisation was “truly random”, it is then up to the reviewer to determine how to rank such papers. This should be raised in initial discussion between the primary and secondary reviewers before they commence their independent critical appraisal.

2. Were participants blinded to treatment allocation?

Blinding of participants is considered optimal as patients who know which arm of a study they have been allocated to may inadvertently influence the study by developing anxiety or conversely, being overly optimistic, attempting to “please” the researchers. This means under- or over-reporting outcomes such as pain or analgesic usage; lack of blinding may also increase loss to follow-up depending on the nature of the intervention being investigated.

3. Was allocation to treatment groups concealed from the allocator?

Allocation is the process by which individuals (or groups if stratified allocation was used) are entered in to one of the study arms following randomisation. The Cochrane Systematic Review handbook states: When assessing a potential participant’s eligibility for a trial, those who are recruiting participants … should remain unaware of the next assignment in the sequence until after the decision about eligibility has been made. Then, after assignment has been revealed, they should not be able to alter the assignment or the decision about eligibility. The ideal is for the process to be impervious to any influence by the individuals making the allocation. Allocator concealment of group allocation is intended to reduce the risk of selection bias. Selection bias is a risk where the allocator may influence the specific treatment arm an individual is allocated to, thus optimally, trials will report the allocator was unaware of which group all study participants were randomised to, and had no subsequent influence on any changes in allocation.

4. Were the outcomes of people who withdrew described and included in the analysis?

Commonly intention to treat analysis is utilised where losses to follow-up are included in the analysis. Intention to treat (ITT) analysis may reduce bias due to changes in the characteristics between control and treatment groups that can occur if people either drop out, or if there is a significant level of mortality in one particular group. The Cochrane Systematic Review handbook identifies two related criteria for ITT analysis, although it is equally clear that how these criteria are applied remains an issue of debate:

  • Trial participants should be analysed in the groups to which they were randomised regardless of which (or how much) treatment they actually received, and regardless of other protocol irregularities, such as ineligibility
  • All participants should be included regardless of whether their outcomes were actually collected.

5. Were those assessing the outcomes blind to the treatment allocation?

In randomised controlled trials, allocation by a third party not otherwise directly involved in the implementation of the study is preferred. Where these resources are not available, electronic assignment systems may be described in trials. Inadequate blinding of allocation is associated with more favourable outcomes for the primary intervention of interest in RCTs. Reviewers should seek to establish whether those assessing outcomes were truly blinded to allocation. Some sources suggest blinded assessment reduces the risk of detection bias. Note that studies reporting multiple outcomes may be at risk of detection bias for some outcomes within a study, but not others. Therefore, attempts should be made to establish if outcomes assessors were blinded to all outcomes of interest to the review.

6. Were the control and treatment groups comparable at entry?

Homogeneity or comparability at entry is related to the method of allocation. If allocation was truly random, groups are more likely to be comparable as characteristics are considered to be randomly distributed across both groups. However, randomisation does not guarantee comparability. Primary studies should report on the baseline characteristics of all groups, with an emphasis on any differences between groups that reach statistical probability.

7. Were groups treated identically other than for the named intervention?

Studies need to be read carefully to determine if there were any differences in how the groups were treated - other than the intervention of interest. If there was a difference in how the groups were treated that arose from flaws in the trial design, or conduct, this is known as a systematic difference and is a form of bias which will skew study results away from the accuracy the primary authors would otherwise have intended. Randomisation, blinding and allocation concealment are intended to reduce the effects of unintentional differences in treatment between groups.

8. Were outcomes measured in the same way for all groups?

In identifying how robust the outcomes for a study are, the definitions, scales and their values as well as methods of implementation of scales needs to be the same for all groups. This question should include consideration of the assessors, were they the same people or trained in the same way, or were there differences such as different type of health professionals involved in measurement of group outcomes.

9. Were outcomes measured in a reliable way?

Were the instruments used to measure outcomes adequately described, and had they been previously validated, or piloted within the trial? These types of questions inform reviewers of this risk to detection bias. Give consideration to the quality of reporting of findings. If an RCT reports percentage of change but gave no baseline data, it is not possible to determine the relevance of the reported value between groups (or within a single group). If a P value is reported but no confidence interval given, the significance has been established, but the degree of certainty in the finding has not.

10. Was appropriate statistical analysis used?

As with any consideration of statistical analysis, consideration should be given to whether there was a more appropriate alternate statistical method that could have been used. Advice from a statistician may be needed to establish if the methods of analysis were appropriate.

APPENDIX III. Critical appraisal tool for cohort studies.

TU4-6
Table:
No Caption available.

1. Is the sample representative of patients in the population as a whole?

This question relies upon knowledge of the broader characteristics of the population of interest. If the study is of women undergoing chemotherapy for breast cancer knowledge of at least the characteristics, demographics, medical history is needed. The term population as a whole should not be taken to infer every individual from everywhere subject to a similar intervention or with similar disease or exposure characteristics. Instead, give consideration to specific population characteristics in the study, including age range, gender, morbidities, medications, and other potentially influential factors.

2. Are the patients at a similar point in the course of their condition/illness?

Check the paper carefully for descriptions of diagnosis and prognosis to determine if patients within and across groups have similar characteristics in relation to disease or exposure, for example tobacco use.

3. Has bias been minimised in relation to selection of cases and controls?

It is useful to determine if patients were included in the study based on either a specified diagnosis or definition. This is more likely to decrease the risk of bias. Characteristics are another useful approach to matching groups, and studies that did not use specified diagnostic methods or definitions should provide evidence on matching by key characteristics.

4. Are confounding factors identified and strategies to deal with them stated?

Confounding has occurred where the estimated intervention effect is biased by the presence of some difference between the comparison groups (apart from the intended intervention/s). Typical confounders include baseline characteristics, prognostic factors, or concomitant interventions. A confounder is a difference between the comparison groups and it influences the direction of the study results. A high quality study at the level of cohort or case-control design will identify the potential confounders and measure them (where possible). This is difficult for studies where behavioural, attitudinal or lifestyle factors may impact on the results.

5. Are outcomes assessed using objective criteria?

Refer back to item three of this appraisal scale and read the methods section of the paper again. If the outcomes were assessed based on existing definitions or diagnostic criteria, then the answer to this question is likely to be yes. If the outcomes were assessed using observer reported, or self reported scales, the risk of over- or under-reporting is increased, and objectivity is compromised. Importantly, determine if the measurement tools used were validated instruments as this has a significant impact on outcome assessment validity.

6. Was follow-up carried out over a sufficient time period?

The appropriate length of time for follow-up will vary with the nature and characteristics of the population of interest and/or the intervention, disease or exposure. To estimate an appropriate duration of follow-up, read across multiple papers and take note of the range for duration of follow-up. The opinions of experts in clinical practice or clinical research may also assist in determining an appropriate duration of follow-up.

7. Were the outcomes of people who withdrew described and included in the analysis?

Commonly intention to treat analysis is utilised where losses to follow-up are included in the analysis. Intention to treat (ITT) analysis may reduce bias due to changes in the characteristics between control and treatment groups that can occur if people either drop out, or if there is a significant level of mortality in one particular group. The Cochrane Systematic Review handbook identifies two related criteria for ITT analysis, although how these criteria are applied remains somewhat contentious:

  • Trial participants should be analysed in the groups to which they were randomised regardless of which (or how much) treatment they actually received, and regardless of other protocol irregularities, such as ineligibility
  • All participants should be included regardless of whether their outcomes were actually collected
  • ??

8. Were outcomes measured in a reliable way?

Having established the objectivity of the outcome measurement instrument (see item 5 of this scale), it’s important to establish how the measurement was conducted. Were those involved in collecting data trained or educated in the use of the instrument/s? If there was more than one data collector, were they similar in terms of level of education, clinical or research experience, or level of responsibility in the piece of research being appraised?

9. Was appropriate statistical analysis used?

As with any consideration of statistical analysis, consideration should be given to whether there was a more appropriate alternate statistical method that could have been used. The methods section of cohort or case-control studies should be detailed enough for reviewers to identify which analytical technique was used (in particular, regression or stratification) and how specific confounders were measured. For studies utilising regression analysis, it is useful to identify if the study identified which variables were included and how they related to the outcome. If stratification was the analytical approach used, were the strata of analysis defined by the specified variables? Additionally, it is also important to assess the appropriateness of the analytical strategy in terms of the assumptions associated with the approach as differing methods of analysis are based on differing assumptions about the data and how it will respond.

APPENDIX IV. Data Extraction Form

TU5-6
Table:
No Caption available.
TU6-6
Table:
No Caption available.
© 2011 by Lippincott Williams & Wilkins, Inc.