Secondary Logo

Journal Logo

Reply: The Quality of Systematic Reviews in Hand Surgery An Analysis Using AMSTAR

Momeni, Arash M.D.; Talley, John R. M.D.; Lee, Gordon K. M.D.

Plastic and Reconstructive Surgery: September 2014 - Volume 134 - Issue 3 - p 483e–484e
doi: 10.1097/PRS.0000000000000470

Division of Plastic and Reconstructive Surgery, Stanford University Medical Center, Palo Alto, Calif.

Correspondence to Dr. Momeni, Division of Plastic and Reconstructive Surgery, Stanford University Medical Center, 770 Welch Road, Suite 400, Palo Alto, Calif. 94304,

Back to Top | Article Outline


We thank Samargandi and Hasan for their interest in our work and appreciate the opportunity to further clarify our analysis. We agree that “researchers and clinicians have the responsibility of providing the highest possible level of evidence” and to “ensure that the best quality and highest level of evidence is being used.” However, we disagree with their conclusion that we provided a “false encouraging image” or a “misleading” conclusion regarding the quality of research within our specialty.1

The importance of transparent reporting cannot be overstated and represents a key element of quality. With respect to criterion 5 (i.e., list of included and excluded studies), we reported that providing a list of included studies was deemed sufficient to meet this criterion. It is true that we deviated from the strict definition that AMSTAR uses.2 The reader, however, is informed about this modification in a transparent manner, thus honoring an important principle of scientific reporting.3 An essential component facilitating critical appraisal is transparent reporting, and we feel we have adhered to this principle. We certainly welcome any discussion of our decision to modify this criterion, particularly as quality assessment tools, such as AMSTAR, should not be regarded as static instruments. A constructive discussion regarding design and content is a prerequisite for improvement of any quality assessment instrument. AMSTAR is no exception as it, despite all its strengths, represents an instrument that was originally designed and tested on systematic reviews of randomized controlled trials.4 In fact, a new AMSTAR tool is currently being developed and validated against observational studies and thus may represent a more appropriate instrument for reviews in plastic surgery.

As outlined in our article, none of the reviews analyzed provided a separate list of excluded studies. It is interesting, however, that AMSTAR does not state specifically which excluded studies need to be listed. Does a list of all excluded studies (i.e., including those excluded following the initial screening, which can include up to several hundred articles, depending on the research question), or only those excluded following full-text review, need to be provided? Lack of clarity regarding this criterion may have, in fact, been the reason why this criterion has been found to have only moderate interrater agreement.4 Therefore, we felt our modification was indeed justified, particularly as this decision was clearly communicated to the reader. We feel, in fact, that not considering this criterion as fulfilled solely because of a lack of a list of excluded studies, although the total number of articles retrieved following the initial search, the number and justification of why articles were excluded, and a list of included studies is reported, represents a too strict and potentially counterproductive interpretation of this AMSTAR criterion.

To address Samargandi and Hasan’s concern regarding the validity of our conclusions, we reanalyzed the articles applying the more strict definition of this criterion. Of note, although none of the studies provided a separate list of included and excluded studies, we did consider this criterion as met if the excluded studies were referenced in the respective article, as the critical information was contained within the article. Reanalysis of our data using this more strict definition resulted in 11.9 percent of studies meeting this criterion. This resulted in a median AMSTAR score of 5 (from 7 as reported previously), even when considering that none of the articles met criterion 11 and continues to reflect a fair to good score, thus not changing our previous conclusion. Similarly, as reported previously, a significant increase in the median AMSTAR score was noted over time after reanalysis (p = 0.004).

We are thankful for Samargandi and Hasan’s thoughts and concerns, yet hope to have demonstrated that our conclusions remain valid and certainly do not provide a “false encouraging image” of the quality of systematic reviews in hand surgery. Certainly, improvements are necessary along with further development of our instruments with which we analyze the quality of plastic surgery research.

Back to Top | Article Outline


The authors have no financial interest to declare in relation to the content of this communication.

Arash Momeni, M.D.

John R. Talley, M.D.

Gordon K. Lee, M.D.

Division of Plastic and Reconstructive Surgery

Stanford University Medical Center

Palo Alto, Calif.

Back to Top | Article Outline


1. Samargandi O, Hasan H.. The quality of systematic reviews in hand surgery: An analysis using AMSTAR (Letter). Plast Reconstr Surg. 2014;134:000
2. Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10
3. Momeni A, Lee GK, Talley JR. The quality of systematic reviews in hand surgery: An analysis using AMSTAR. Plast Reconstr Surg. 2013;131:831–837
4. Shea BJ, Hamel C, Wells GA, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62:1013–1020
Back to Top | Article Outline


Letters to the Editor, discussing material recently published in the Journal, are welcome. They will have the best chance of acceptance if they are received within 8 weeks of an article’s publication. Letters to the Editor may be published with a response from the authors of the article being discussed. Discussions beyond the initial letter and response will not be published. Letters submitted pertaining to published Discussions of articles will not be printed. Letters to the Editor are not usually peer reviewed, but the Journal may invite replies from the authors of the original publication. All Letters are published at the discretion of the Editor.

Letters submitted should pose a specific question that clarifies a point that either was not made in the article or was unclear, and therefore a response from the corresponding author of the article is requested.

Authors will be listed in the order in which they appear in the submission. Letters should be submitted electronically via PRS’ enkwell, at

We reserve the right to edit Letters to meet requirements of space and format. Any financial interests relevant to the content of the correspondence must be disclosed. Submission of a Letter constitutes permission for the American Society of Plastic Surgeons and its licensees and asignees to publish it in the Journal and in any other form or medium.

The views, opinions, and conclusions expressed in the Letters to the Editor represent the personal opinions of the individual writers and not those of the publisher, the Editorial Board, or the sponsors of the Journal. Any stated views, opinions, and conclusions do not reflect the policy of any of the sponsoring organizations or of the institutions with which the writer is affiliated, and the publisher, the Editorial Board, and the sponsoring organizations assume no responsibility for the content of such correspondence.

The Journal requests that individuals submit no more than five (5) letters to Plastic and Reconstructive Surgery in a calendar year.

©2014American Society of Plastic Surgeons