Skip Navigation LinksHome > March 2014 - Volume 89 - Issue 3 > Literature Reviews Should Reveal the Reviewers’ Rationale to...
Academic Medicine:
doi: 10.1097/ACM.0000000000000128
Letters to the Editor

Literature Reviews Should Reveal the Reviewers’ Rationale to Opt for Particular Quality Assessment Criteria

Hannes, Karin PhD; Aertgeerts, Bert PhD

Free Access
Article Outline
Collapse Box

Author Information

Assistant professor, Methodology of Educational Sciences Research Group, Faculty of Psychology and Educational Sciences, KU Leuven, Leuven, Belgium; e-mail: Karin.hannes@ppw.kuleuven.be.

Professor, Academic Centre for General Practice, Faculty of Medicine, KU Leuven, Leuven, Belgium.

Disclosures: Dr. Hannes is co-convenor of the Cochrane Qualitative and Implementation Methods Group. Her scientific work concerns the development of methods for qualitative evidence synthesis. Dr. Aertgeerts chaired the Belgian Centre for Evidence-Based Medicine and the Belgian Branch of the Cochrane Collaboration.

Back to Top | Article Outline

To the Editor:

Qualitative evidence synthesis has become increasingly popular amongst researchers.1 Swennen and colleagues2 produced a thematic synthesis, describing major themes that cut across a selection of studies on barriers and facilitators to evidence-based practice. However, the authors may have missed many high-quality studies because of the three inclusion criteria for quality assessment they chose: (1) showing proof of methodological triangulation, (2) describing the approach to data analysis, and (3) using purposeful or theoretical sampling.

The first criterion has been adopted from Mays and Pope,3 most likely to ensure validity, and has been critiqued by these authors as being “controversial as a genuine test of validity because it assumes that any weaknesses in one method will be compensated by strengths in another, and that it is always possible to adjudicate between different accounts.” Eighty-one of 133 potentially relevant studies were excluded because they did not triangulate methods. Establishing accuracy in what researchers collect and report is important. However, such accuracy could also be achieved by the use of multiple observers or data collectors to allow for cross-checking of observations. This was applied in several excluded studies.

Second, it is not fully clear whether the authors excluded primary studies for (1) not describing their approach because of the lack of descriptive information or (2) their own evaluation of the degree of methodological congruence between the chosen approach to analysis and what was outlined in the Results section. We agree with Sandelowski and colleagues’4 statement that “there are many instances in which investigators have produced findings worthy of note, but have used the ‘wrong’ language and method citations to describe their work.” In other words, a study presented as grounded theory, using a hybrid deductive–inductive approach, may still conceptually contribute to the synthesis.

Third, the authors included studies using purposeful sampling variants that led to conceptual saturation. Many health-care-related qualitative studies do not reach a saturation point or the level of depth that would allow for a generalization beyond the setting that has been researched, but again may conceptually contribute to a synthesis.

Swennen’s specific quality criteria provide a clear explanation for why certain papers have been excluded, but they do little to identify the quality of the decisions made or the rationale behind the methodological requirements outlined. It is important to shift the focus from whether or not to appraise studies to what criteria to use for an assessment of methodological quality. The choice can seriously affect the final outcome of the synthesis.1

Karin Hannes, PhD

Assistant professor, Methodology of Educational Sciences Research Group, Faculty of Psychology and Educational Sciences, KU Leuven, Leuven, Belgium; e-mail: Karin.hannes@ppw.kuleuven.be.

Bert Aertgeerts, PhD

Professor, Academic Centre for General Practice, Faculty of Medicine, KU Leuven, Leuven, Belgium.

Back to Top | Article Outline

References

1. Hannes K, Macaitis K. A move to more transparent and systematic approaches of qualitative evidence synthesis: Update of a review on published papers. Qual Res. 2012;12:402–442

2. Swennen MHJ, Van der Heijden GJMG, Boeije HR, et al. Doctors’ perceptions and use of evidence-based medicine: A systematic review and thematic synthesis of qualitative studies. Acad Med. 2013;88:1384–1396

3. Mays N, Pope C. Qualitative research in health care. Assessing quality in qualitative research. BMJ. 2000;320:50–52

4. Sandelowski M, Docherty S, Emden C. Qualitative metasynthesis: Issues and techniques. Res Nurs Health. 1997;20:365–371

© 2014 by the Association of American Medical Colleges

Login

Article Tools

Share