Share this article on:

Developing Good Clinical Questions and Finding the Best Evidence to Answer Those Questions

Burns, Patricia B. M.P.H.; Chung, Kevin C. M.D., M.S.

Plastic & Reconstructive Surgery: August 2010 - Volume 126 - Issue 2 - pp 613-618
doi: 10.1097/PRS.0b013e3181de24a7
Ebm Special Topics

Summary: Evidence-based medicine involves the integration of the best scientific evidence available with physician experience and patient preferences. One of the first steps in evidence-based medicine involves developing a question based on issues encountered with patients or a disease and finding evidence to answer the question. This article is a guide to plastic surgeons in how to begin the practice of evidence-based medicine by developing clinical questions and finding evidence to answer these questions.

Ann Arbor, Mich.

From the Section of Plastic Surgery, Department of Surgery, University of Michigan Health System.

Received for publication October 28, 2009; accepted December 22, 2009.

Disclosure: The authors have no financial interest to declare in relation to the content of this article.

Kevin C. Chung, M.D., M.S.; Section of Plastic Surgery; University of Michigan Health System; 2130 Taubman Center, SPC 5340; 1500 East Medical Center Drive; Ann Arbor, Mich. 48109-5340;

Back to Top | Article Outline


You are a surgeon and your patient is a 25-year-old man with a severe type III open tibial fracture as a result of a motorcycle accident. The options for treatment are reconstruction or amputation. Understandably, this is an emotional issue for the patient and he wants the leg to be saved using reconstructive surgery. Your preference is for flap reconstruction of the limb. Patient and physician preferences must be taken into account when making the decision of whether to reconstruct the leg or amputate, but you question whether the decision is based on the best available scientific evidence.

In another scenario, you are a surgeon who is interested in researching outcomes in rheumatoid arthritis patients who elect to have arthroplasty to replace deformed joints in their hands. You have performed arthroplasty on numerous patients and find your patients to be very satisfied with how their hands appear after surgery and report good outcomes. However, you question whether their hands actually function better after surgery. You speculate that patients are reporting better outcomes because their fingers are less deformed and the improved aesthetic appearance prompts the patients to report being satisfied. You are interested in researching this topic to apply evidence to your clinical experience but are not sure how to start.

In both cases, you have identified a gap in your knowledge, and need to identify research evidence to fill that gap. You could answer your clinical questions by reading an expert opinion article or consulting a colleague. However, the current environment of evidence-based medicine stresses the need to develop a good question and finding high-quality research to answer clinical questions. Sackett, one of the founding fathers of evidence-based medicine, states that “studies show that busy clinicians who devote their scarce reading time to selective, efficient, patient driven searching, appraisal, and incorporation of the best available evidence can practice evidence based medicine.”1 As part of the emphasis to introduce the concepts and practices of evidence-based medicine to readers of Plastic and Reconstructive Surgery, we present this article as a guide to developing a clinical question and how to find the best available evidence to answer the question.

Back to Top | Article Outline


Before developing a specific question, it is important to think about the type of question you have. Clinical questions can be narrowed down to four types: therapy, prognosis, diagnosis, and economic/decision analysis. For example, if you are comparing two treatments such as amputation and reconstructive surgery, this is a therapy question. If you are interested in understanding the outcomes for older patients after breast reconstruction, this is a prognosis question. A diagnosis question might inquire about the accuracy of a new diagnostic test such as the use of ultrasonography to detect silicone breast implant ruptures. If you are asking whether a treatment results in lower costs and improved quality of life, this is an economic/decision analysis question.

The type of question also determines the type of study that you should look for when searching for evidence. For example, in the scenario for arthroplasty, you may want to determine whether arthroplasty on the joints in the hand results in better outcomes than no surgery on the affected joints. A randomized controlled trial is the preferred study type for answering treatment questions. However, because of feasibility and ethical concerns, many surgical procedures cannot be randomly allocated or blinded. In this case, prospective cohort studies would be a better choice. When questioning the prognosis for a disease, a prospective cohort study would be more appropriate. Patients are enrolled at the time of diagnosis or treatment and followed over time to determine whether any factors are associated with prognostic outcomes. Diagnostic studies that evaluate new diagnostic tests are more likely to be cross-sectional or prospective cohort studies. Economic and decision analysis questions arise when the best available evidence concerning treatments is inconclusive and a randomized controlled trial is not ethically practical or feasible.

Back to Top | Article Outline


The next step is developing a question. You want an answer to your question but you also want to avoid spending precious free time aimlessly searching through journals. Investing a few moments to craft a question can help to narrow the search for answers (Fig. 1). The question must be clinically relevant, well thought out, and answerable. For example, a treatment (e.g., anticoagulant treatment for deep venous thrombosis) or operation (e.g., replanting an amputated thumb) that is widely accepted with outstanding results would not be a good subject for a clinical question. In contrast, researching whether the use of costly tissue sealant can decrease the incidence of postoperative seroma is an important study question, as is whether a particular face-lift technique is associated with better outcomes and a lower complication rate.

PICO (patient/problem, intervention, comparison intervention, and outcomes) is a tool to help develop focused clinical questions.2–4 The question will be more focused if you can be specific for each element. The first element in the question is a description of the patient population or problem of interest. The second and third elements contain the intervention and comparison intervention of interest, respectively. An intervention is often a treatment but could also be a prognostic factor or a diagnostic test. In some cases, there will not be a comparison intervention. Lastly, the question should consider the outcomes from the intervention. For example, will the intervention lead to fewer complications and less morbidity?

Using our scenarios as examples, we identify the pieces that will be used to formulate a question. In the case of tibial fractures, you consider the most severe types because these types of fractures are more likely to be considered for amputation. Based on the Gustilo classification system,5 you decide to restrict your patient population to type IIIB and IIIC because of the lack of soft-tissue coverage of the fracture that requires flap reconstruction. The intervention of interest in your question is limb salvage using reconstructive surgery. This surgical procedure will then be compared with amputation. You consider what outcomes you would like to compare between these two treatments. After reviewing several of your cases, you decide to concentrate on improved limb function and quality of life after surgery. Putting the four elements together, we construct a focused question: Do patients with severe type IIIB and IIIC open tibial fractures have better limb function and quality of life than patients who have amputations?

In the arthroplasty scenario, you have decided to concentrate your research efforts on rheumatoid arthritis patients who elect to replace joints in their hands. Based on your personal experience with rheumatoid hand deformities, you further decide to concentrate on arthroplasty at the metacarpophalangeal joint. After considering the different types of implants that can be used for metacarpophalangeal arthroplasty, you select silicone metacarpophalangeal arthroplasty because it is has been used for over 40 years and is not experimental. You consider alternate operations but decide to compare silicone metacarpophalangeal arthroplasty patients with those who do not undergo surgery. The outcome that you decide on is improvement in hand function as shown in patient-reported questionnaires and biomechanical measurements. We put the pieces of the question together: “Do rheumatoid arthritis patients who undergo silicone metacarpophalangeal arthroplasty have better functional outcomes and quality of life than patients who do not have silicone metacarpophalangeal arthroplasty or any other type of arthroplasty”?

Back to Top | Article Outline


Before searching for evidence, you will need to consider the hierarchy of evidence (Fig. 2). The hierarchy of evidence ranks study types from those with the highest evidence to those with the lowest evidence. Randomized controlled trials, systematic reviews, and meta-analyses are at the top (highest evidence) and expert opinions and case reports at the bottom (lowest evidence). Systematic reviews and meta-analyses combine or pool data from multiple individual studies using systematic, predetermined methods. This pooling of data, if performed correctly, allows the reader an unbiased summary of the available research. A systematic review of randomized controlled trials is considered higher in the hierarchy than a systematic review of observational studies. A narrative review or expert opinion is also a summary of studies but is conducted in a nonsystematic manner and therefore more likely to be biased.

Back to Top | Article Outline


The next step in practicing evidence-based medicine is finding the best available literature to answer your question. This is accomplished by searching through electronic databases such as MEDLINE, EMBASE, CINAHL, and the Cochrane Library.

The Cochrane Collaboration was founded in 1993 and produces the Cochrane Library.6 The library consists of several evidence-based medicine databases. Most notable is the Cochrane Database of Systematic Reviews, which consists of systematic reviews and meta-analyses. The library is updated on a quarterly basis and considered a reliable source of evidence-based medicine. All systematic reviews must meet rigorous standards before publication. Because not every condition has been investigated with a systematic review, it is often necessary to look at other databases.

MEDLINE is maintained by the National Library of Medicine, contains over 11 million citations from journals dating back to 1950, and is updated weekly.7 Articles are referenced using medical subject headings. PubMed has over 19 million citations from life science journals and MEDLINE. PubMed has several search options that can help simplify your literature search. You can search specific journals or a single citation. Another search option is called Clinical Queries, which allows the user to narrow their search by subject and clinical study category (e.g., therapy, diagnosis, prognosis) or limit the search to systematic reviews. EMBASE contains biomedical and pharmacologic articles from 1947 to the present.8 It contains more than 19 million records from more than 7000 journals. Although EMBASE and MEDLINE overlap in their coverage of the literature, EMBASE has a better coverage of European journals.9 The CINAHL database covers 2960 journals and other citations related to nursing and allied health from 1981 to the present.10 The CINAHL database has over 2 million records.

It is important when searching for evidence to start at the top of the hierarchy and work your way down. You should begin by searching for systematic reviews or meta-analyses. The Cochrane Library is an excellent source for this type of evidence.6 Another choice is the American College of Physicians Journal Club, which preappraises evidence for physicians.11 If there is no systematic review or meta-analysis, the next step involves searching MEDLINE or one of the other databases for randomized controlled trials. Many questions are not appropriately answered by randomized controlled trials and therefore few if any randomized controlled trials exist. This is often the case for surgical procedures.12,13 It will then be necessary to move down the hierarchy and gather evidence from prospective cohort studies, case-control studies, and lastly case reports. Case reports are best for generating hypotheses. The reader needs to be aware of the lower quality of these types of studies and take that into account when evaluating the evidence.

Back to Top | Article Outline


Although there may be abundant research on your question, it is necessary for expediency to restrict your search such that you have a manageable number of articles to read. When asking a clinical question that needs a timely answer, a few high-quality articles are adequate. However, in the research setting, it is important to find all the relevant articles. Developing accurate search terms will aid in finding the evidence you need to answer your question. The search terms need to be specific enough to narrow your search but not so narrow so that you miss important articles. In MEDLINE, each article is assigned Medical Subject Headings. Medical Subject Headings terms can then be used to search for pertinent research. MEDLINE gives the option to explode on terms (to include more narrowly defined terms) to gather the most research. In addition to Medical Subject Headings terms, it is sometimes helpful to include text words to gather evidence. Medical Subject Headings terms are usually combined with Boolean operators (AND, OR) to maximize the number of articles selected. It may take several attempts using different combinations of search terms to get the most comprehensive and relevant list of articles.

All databases give you the option to put limits on your search. For example, you can limit your search by language (e.g., English), age (e.g., adults older than 18 years), humans, or type of article (e.g., review). If your search results in more than 200 articles, you may need to refine your search terms to reduce the number of articles. Medical librarians have a great deal of experience with developing search terms, and it may be useful to consult one.

Once you have your list of possible articles, you can scan the titles and abstracts to determine whether the article is relevant to your search. You can access the articles electronically or have them pulled and copied if necessary. If you find an article of particular interest in MEDLINE or PubMed, you have the option of finding related articles that may not have been identified by your search.

Using the tibial fracture question as an example, we search MEDLINE using PubMed and select Clinical Queries as a tool. We select the systematic review option and use the following Medical Subject Headings terms: tibial fractures or lower leg injuries. We further limit our search to humans and English language. A total of 176 articles are selected based on these search items. This is a reasonable number of articles, and searching titles and abstracts should reduce the number further. An alternate search would be to access MEDLINE (in this case using the interface Ovid) and enter the terms (expTibial Fracture or Lower Leg Injuries) and (expAmputation or expLimb Salvage or expReconstructive Surgical Procedures). The exp in the search denotes that these terms were exploded. A total of 1659 articles were selected based on the search terms and limiting the search to English language, humans, and review articles brought the number down to 97 articles. After reviewing the titles and abstracts, we found a systematic review comparing the outcomes for amputation or reconstructive surgery after severe tibial fractures.14 The systematic review article did not find any difference in outcomes for the two treatments.

In the research setting, the search items should be broad enough to find as many articles as possible. Using the scenario for arthroplasty in rheumatoid arthritis patients, we decide to search the Cochrane Library first to see whether there are any systematic reviews of this topic. We enter the terms rheumatoid arthritis and arthroplasty. The search yielded 25 systematic reviews. A quick search of the titles and abstracts of the articles selected by our search did not find any relevant review articles. Many of the review articles selected for this search concern knee replacement surgery. We would like to find more relevant research on this topic and therefore restrict the search to surgery on the metacarpophalangeal joint. Using MEDLINE, we enter the terms (expRheumatoid Arthritis and (expArthroplasty/ or expArthroplasty, Replacement, Finger)), and (expMetacarpophalangeal Joint/Surgery). The search results in 112 articles, a number that drops to 99 when we limit articles to English. Again, we search the titles and abstracts, and identify a systematic review.15 This systematic review is based mostly on retrospective studies and found improvement in hand function after silicone metacarpophalangeal arthroplasty. However, none of the clinical trials or prospective studies included in the review compared surgical cases to nonsurgical cases. We decide to widen our search using the same search terms but using the Boolean operator OR instead of AND in the search. For example, we change the search to: (expRheumatoid Arthritis AND (expArthroplasty/ OR expArthroplasty, Replacement, Finger)) OR (expMetacarpophalangeal Joint/Surgery). This results in considerably more articles (2811). Restricting articles to the English language drops the number of articles to 2322. Although this combination of search terms results in articles on other joints such as the knee and shoulder, we do find one prospective cohort study of silicone metacarpophalangeal arthroplasty patients who compared surgical cases to nonsurgical cases.16 The article along with the systematic review has the information available to help answer your question. In addition, both articles provide a high level of evidence.

Back to Top | Article Outline


Although you may not be publishing the results of your literature search, it is important to know the elements needed for a good search. Standards for reporting literature searches were developed with the mnemonic STARLITE (sampling strategy, type of study, approach, range of years, limits, inclusion, and exclusion criteria, terms, and electronic sources).17 Standards have been developed for reporting randomized controlled trials, such as Consolidated Standards of Reporting Trials (CONSORT)18; meta-analysis of randomized controlled trials, such as Quality of Reports of Meta-analysis (QUOROM)19; and meta-analysis of observational studies, such as Meta-analysis of Observational Studies in Epidemiology (MOOSE).20 In a research article, it is important to report the elements of the literature search so that your results can be reproduced.

STARLITE was developed for qualitative studies but could also be useful when searching for quantitative studies (Table 1). The sampling strategy should state whether the researchers attempted to find all the available research or restricted the search in some format. The types of studies included in the search such as randomized controlled trials or prospective cohort studies should be reported. This allows others to know the level of evidence that was included in your search. The approach states how you found the research (e.g., electronic databases or other methods such as hand searching). The range of years, search limits or filters, inclusion and exclusion criteria, and terms used in the search should also be reported. If you used electronic databases, they need to be listed.

Back to Top | Article Outline


As is often the case, one question leads to another. For example, in the tibia fracture question, the best available evidence found no difference in outcomes between amputation and reconstructive surgery. You could continue your practice of performing reconstructive surgery, confident that you are providing a good outcome to your patient by saving the limb. You may still question whether there are any other studies that can distinguish the outcomes for the two treatments. For example, economically, does limb reconstruction have better outcomes than amputation? If searching for evidence provides no answer, you could propose your own research to answer the question.

This article presents the initial steps for those interested in pursuing evidence-based medicine. We have provided guidance and examples to help with developing a focused clinical question and finding the highest level of research to answer the question. It may be difficult to start the process, but it is a skill that improves with practice and persistence. Evidence-based medicine is a new direction in American medicine21 and will be used for quality improvement efforts in the United States.22 By applying evidence-based medicine principles to integrate with physician experiences, we should be able to provide better care for our patients by eliminating idiosyncrasies in our treatments.

Back to Top | Article Outline


This work was supported in part by a Midcareer Investigator Award in Patient-Oriented Research (K24 AR053120) from the National Institute of Arthritis and Musculoskeletal and Skin Diseases (to K.C.C.).

Back to Top | Article Outline


1. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: What it is and what it isn't. BMJ. 1996;312:71–72.
2. Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: A key to evidence-based decisions. ACP J Club 1995;123:A12–A13.
3. Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak. 2007;7:16.
4. Centre for Evidence Based Medicine. Available at: Accessed September 28, 2009.
5. Gustilo RB, Mendoza RM, Williams DN. Problems in the management of type III (severe) open fractures: A new classification of type III open fractures. J Trauma 1984;24:42–46.
6. Cochrane Library. Available at: Accessed September 30, 2009.
7. National Library of Medicine. MEDLINE Fact Sheet. Available at: Accessed October 7, 2009.
8. STN Database Summary Sheet EMBASE. Available at: Accessed October 7, 2009.
9. Egger M, Davey Smith G. Principles of and Procedures for Systematic Reviews. 2nd ed. London: BMJ Publishing Group; 2001.
10. CINAHL Information Systems. Available at: Accessed October 6, 2009.
11. American College of Physicians Journal Club. Available at: Accessed October 6, 2009.
12. Chung KC, Burns PB, Davis Sears E. Outcomes research in hand surgery: Where have we been and where should we go? J Hand Surg (Am.) 2006;31:1373–1379.
13. Chung KC, Burns PB. A guide to planning and executing a surgical randomized controlled trial. J Hand Surg (Am.) 2008;33:407–412.
14. Saddawi-Konefka D, Kim HM, Chung KC. A systematic review of outcomes and complications of reconstruction and amputation for type IIIB and IIIC fractures of the tibia. Plast Reconstr Surg. 2008;122:1796–1805.
15. Chung KC, Kowalski CP, Myra Kim H, Kazmers IS. Patient outcomes following Swanson silastic metacarpophalangeal joint arthroplasty in the rheumatoid hand: A systematic overview. J Rheumatol. 2000;27:1395–1402.
16. Chung KC, Burns PB, Wilgis EF, et al. A multicenter clinical trial in rheumatoid arthritis comparing silicone metacarpophalangeal joint arthroplasty with medical treatment. J Hand Surg (Am.) 2009;34:815–823.
17. Booth A. “Brimful of STARLITE”: Toward standards for reporting literature searches. J Med Libr Assoc. 2006:94:421–429, e205.
18. Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials: The CONSORT statement. JAMA. 1996;276:637–639.
19. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM statement. Quality of Reporting of Meta-analyses. Lancet 1999;354:1896–1900.
20. Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: A proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000;283:2008–2012.
21. Chung KC, Ram AN. Evidence-based medicine: The fourth revolution in American medicine? Plast Reconstr Surg. 2009;123:389–398.
22. Chung KC, Rohrich RJ. Measuring quality of surgical care: Is it attainable? Plast Reconstr Surg. 2009;123:741–749.
©2010American Society of Plastic Surgeons