Journal Logo


Defining Outcomes Articles for the Journal

Lin, Samuel J. M.D., M.B.A.; Chung, Kevin C. M.D., M.S.; Rohrich, Rod J. M.D.

Author Information
Plastic and Reconstructive Surgery: January 2018 - Volume 141 - Issue 1 - p 239-244
doi: 10.1097/PRS.0000000000003964
  • Free


Since its introduction several years ago in the Journal, Outcomes articles have become more numerous over time. As outcomes research becomes more broadly defined, the goals and objectives of this article are severalfold: to review the history of outcomes research in the literature, to define what criteria an article needs to meet to be labeled as an outcomes article, to state what the “outcomes” designation means to the readership, to suggest potential venues for those in the field, and to shape research to be outcomes work. We strive to illustrate what the outcomes articles may mean to the members of the public through tracing the development and history of outcomes research (Fig. 1).

Fig. 1.
Fig. 1.:
Outlining the development of outcomes studies.


In the 1950s and 1960s, the roots of the outcomes movement stemmed from the expansion of medical services shortly after World War II. This movement, “the third revolution of medical care,” resulted in third-party payers demanding cost containment, raising concerns about unknown end results related to these medical services, and questioning whether cost reduction or geographic differences would affect health care quality.1 In prior years, the majority of studies focused on medical service differences with regard to mortality, readmission, complications, and other traditional measures of clinical outcomes.2 The current era has seen expansion of the outcomes that physicians and policy-makers consider as valid markers of health, including nontraditional clinical indexes of functional status, emotional health, social interaction, cognitive function, degree of disability, and other factors.2

By the late 1980s, additional categories of medical care and determination of results had further been defined. Assessment of medical effectiveness and improvement of clinical practice was primarily described as a four-step process, consisting of (1) monitoring, (2) analysis of variations, (3) assessment of interventions, and (4) feedback and education.3 These steps examined outcomes of various interventions, medical care variations, assessment of interventions, and feedback and education, respectively. During this period, Torrance and Feeney reviewed options of evaluating metrics in quality of life.4 This article provided a perspective on two techniques of utilities and quality-adjusted life-years and their interrelationship. Utilities are specifically designed for individual decision-making under uncertainty and can be aggregated across individuals to provide a group utility function. Quality-adjusted life-years are designed to aggregate in a single summary measure the total health improvement for a group of individuals.4 These measures may capture improvements from impacts on both quantity and quality of life. Utilities can be used as the quality-adjustment weights for quality-adjusted life-years, with a combination of the two allowing for powerful and highly useful variation on cost-effectiveness analysis known as cost-utility analysis.4

In the 1990s, when discussing general health status and a broader concept of quality of life, Ware noted a consensus regarding the inclusion of physical, mental, social, and role functioning and general health perceptions measures for comprehensive assessments of health.5 Laine and Davidoff noted another perspective on physician versus patient-centered care. Historically, medicine has been largely physician driven, but physicians have begun to incorporate patients’ perspectives in ways that increasingly are at the forefront.6 This shift has been recognized as “patient-centered” care, applying to many areas of medicine.6 As growth of outcomes research continues, it is essential to capture specific variables of interest reflecting diseases, patients, providers, or organizations.7 These nontraditional measures extend beyond the scope of mortality and morbidity, by including physical/clinical, performance/function, economic, and humanistic measurements.7 The variables serve to relay relevant information back to the respective parties, by relying on outcome measures focused on improving the patient perspective of care through “patient-reported outcomes.” There is precedence in other surgical fields, such as neurosurgery, related to the importance of systematic patient outcomes reporting through aspects of patient symptoms, functional status, quality of life, satisfaction with treatment, and health care costs for specific medical conditions or treatments.8 The described subclassifications set the groundwork for assessment of what patients truly find most important to their care. Moreover, they enable recognition of what aspects of patient care surgeons can work on improving.

It is no longer appropriate to assess outcomes under an umbrella term. The true value of health care can be determined only by systematic examination of patient outcomes. This trend was eventually noted in higher impact journals. Clancy and Eisenberg noted clinical success traditionally was appraised in terms of mortality, physiologic measures, and definable clinical events, and they commented on the progression to assess other patient preferences of outcomes through health perceptions, functional measures, preference-based measures, and patient satisfaction.9 The quality of outcomes measures must also change to keep health care moving in a forward motion, with patient-reported outcomes being one step in that direction.

By the mid 1990s, the initial classification schemes for outcome measures were described. Wilson and Cleary proposed a classification scheme for different measures of health outcomes, divided into five levels: biological and physiological factors, symptoms, functioning, general health perceptions, and overall quality of life.10 However, because of the interlinking nature of the five levels, there was difficulty in independently assessing the factors and defining the approaches to doing so. This scheme brought forth an understanding that health outcomes were invariably intertwined, and that future measurement tools would need to account for this fact. This was the starting point for recognition and implementation of the current-day measures of health care outcomes.

By the turn of the century, the term “outcomes” had extended to encompass a number of factors all related to the care and treatment of patients. Lee et al. stated that the key classic feature distinguishing clinical research from outcomes research is the emphasis on efficacy in clinical trials (the effect of an intervention measured under controlled circumstances, as in a clinical trial) rather than on effectiveness in outcomes research (the effect of an intervention as applied to broad populations in real practice). However, with the streamlining of outcomes research and its development as an umbrella term, it now includes studies other than those looking at effectiveness.11 Considering this coverage of a broad range of study questions, it is more important now than ever to have clear definitions of what constitutes “outcomes research.”


The Agency for Healthcare Research and Quality has defined 11 categories of outcomes studies that will be used for identifying outcomes articles for the Journal (Table 1).12Table 1 lists the 11 categories and an example of each type of category. Although broad in scope, the reality of outcomes measurements requires that each field varies in data/metrics and that several categories need be present to be all inclusive. Interestingly, Stryer et al. reviewed the actual outcomes of this categorization of outcomes studies, conducting a query of outcomes studies based on the 11 categories defined by Agency for Healthcare Research and Quality.13 This survey revealed that descriptive epidemiology was the most common (24 percent), followed by comparative effectiveness (17 percent) and economic assessments (12 percent).13 Chung et al. performed a similar study specifically for plastic surgery, finding that quality of health care was the most represented category (76 percent), followed by descriptive epidemiology (14 percent); surprisingly, cost-benefit analysis and studies involving legal, legislative, or regulatory matters constituted less than 1 percent of studies.14 Although there is now a research-heavy focus on quality-of-life measures, patient-reported outcomes in plastic surgery are only one of the 11 categories set forth by the Agency for Healthcare Research and Quality. It is critical at this juncture to study the other aforementioned 10 categories to at least the same extent, to develop future experts in these fields for the Journal, and at the same time to ensure that plastic surgery adheres to the highest standards set forth in health care. Outcomes research can be further defined by four levels of impact (Table 2).15,16 Chung et al. found that, in plastic surgery, 90 percent of studies were at level 1 that did not change practice, 0.2 percent studies were at level 2, no studies were at level 3, and 10 percent were at level 4.16 Given the sparse literature at levels 2, 3, and 4, apart from recognition of the 11 different outcomes categories, it is crucial to understand both what outcomes research encompasses and the impact of these articles to influence practice.

Table 1.
Table 1.:
Eleven Agency for Healthcare Research and Quality Categories
Table 2.
Table 2.:
Four Levels of Impact for Outcomes Research


It has taken time for the adoption of outcomes measurements across medicine, but not long after development of Agency for Healthcare Research and Quality categories, disciplines ranging from internal medicine, to pediatrics, to otolaryngology have addressed outcomes research.17–19 These studies were useful in identifying lacking outcomes categories. Plastic surgeons have also noted the importance of outcomes studies as well in the early 2000s. Chung et al. described the history and predicted a path forward for outcomes research in hand surgery in their systematic review of hand surgery outcomes studies, finding that much of the research is confined to testing new or existing surgical techniques.14 Alderman and Chung described specific tools as well, stating that outcomes research in hand surgery provides patients and providers with objective, reliable information to assist in making medical decisions.20 In this article, the authors reviewed the research techniques available to study the following surgical outcomes of the hand: national trends in surgical care, surgical complications, objective measures of hand function, patient-reported measures of hand function, and economic burden.20 In a systematic review of outcomes studies in plastic surgery by Sears et al., the main outcome categories were defined as quality of health care/comparative effectiveness; descriptive epidemiology; methodologic development/severity indices; modeling/outcome prediction; practice variation; sociology of health care; meta-analysis/systematic review; cost-benefit analysis; and legal, legislative, or regulatory.16 They noted that outcomes research can have a greater impact on policy and patient outcomes in the future through pursuing studies that demonstrate direct improvement in cost and other patient outcome measures with one treatment compared with clinical standards, especially taking into consideration patient preference and morbidity.16 Valentini et al. also studied trends in outcomes research in breast cancer, breaking studies down into subgroups of clinical, humanistic, and economic outcomes.21 Outcomes research serves to drive health policy in the improvement of overall patient care quality. As such, there must be a consistent effort to both reflect on the current scheme and challenge the paradigm. With all this talk of the different outcomes that should be addressed, the key messages to remember when conducting outcomes studies should be pertinent to health policy implications and the influence on practice.

We outline tools available to measure outcomes in plastic surgery and subsequently submit objective measurements of plastic surgical conditions. Six major outcome categories, with four validated categorical endpoint outcome tools, are discussed (Tables 3 and 4).22 Considering the inherent bias in each outcome measure, it can be advantageous to use a minimum of three different tools to complement one another.22 The authors have used various utility outcomes measures for a number of plastic surgical conditions, ranging from nipple-areola complex, to nasal deformity, to facial paralysis.23–35 However, there should be continued future studies on other conditions given the breadth of plastic surgery.

Table 3.
Table 3.:
Six Major Outcome Categories
Table 4.
Table 4.:
Four Validated Endpoint Outcome Tools


Outcomes research drives decision-making, and it is necessary to first establish guidelines for any intervention, limiting premature adoption of surgical approaches or treatments without initially conducting required comparative studies.36 The Agency for Healthcare Research and Quality suggests that the goal of outcomes research is to understand the end results of particular health care practices and interventions, consisting of what the general public, or patients, emphasize.15 Although plastic surgery has done remarkably well in addressing quality-of-life studies, there is a relatively smaller number of other end results, such as those pertaining to economic costs. Economic outcomes are growing in importance in accordance with diminishing health care funding. In the field of plastic surgery, the complex mix of payers makes collecting economic outcomes more difficult and more important at the same time. Notably in today’s changing reimbursement in health care, plastic surgeons must include economic outcomes in analyses to maintain a position of national policy relevance.37

The Medicare Access and CHIP Reauthorization Act of 2015 program for reimbursement and the Quality Payment Program as the next payment system is on us from the Centers for Medicare and Medicaid Services. The Medicare Access and CHIP Reauthorization Act of 2015 has repealed the Sustainable Growth Rate and provides a payment plan based on Alternative Payment Models and the Merit-based Incentive Payment System. Indeed, quality care and outcomes studies are keys to continuing the pursuit of delivery of the highest quality of care to our patients.

The “outcomes” designation carries the broad definitions of the 11 Agency for Healthcare Research and Quality criteria (Table 1) for our Journal and is important for increasing the impact of our work in the care of our patients. The outcomes article assignment is also important as the members of the public who have increasing access to our Journal articles become better informed in evidence-based outcomes, which further drives the impact of our Journal. The essence of outcomes research is in improving outcomes with better public and patient understanding of treatment through education.


Outcomes research provides a significant perspective on how care is delivered and measured. It is quantified across a broad variety of topics and gathered by different means. Providing the readership with the background and underpinnings of distinguishing outcomes studies in the Journal will potentially prepare practitioners with the data as health care reimbursement evolves.


Research reported in this publication was supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases of the National Institutes of Health under Award Number 2 K24-AR053120-06. The authors thank Austin D. Chen for contributions in the development of this article.


The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.


1. Relman AS. Assessment and accountability: The third revolution in medical care. N Engl J Med. 1988;319:12201222.
2. Epstein AM. The outcomes movement: Will it get us where we want to go? N Engl J Med. 1990;323:266270.
3. Roper WL, Winkenwerder W, Hackbarth GM, Krakauer H. Effectiveness in health care: An initiative to evaluate and improve medical practice. N Engl J Med. 1988;319:11971202.
4. Torrance GW, Feeny D. Utilities and quality-adjusted life years. Int J Technol Assess Health Care 1989;5:559575.
5. Ware JE Jr. The status of health assessment 1994. Annu Rev Public Health 1995;16:327354.
6. Laine C, Davidoff F. Patient-centered medicine: A professional evolution. JAMA 1996;275:152156.
7. Maloney K, Chaiken BP. An overview of outcomes research and measurement. J Healthc Qual. 1999;21:49; quiz 9.
8. Gerszten PC. Outcomes research: A review. Neurosurgery 1998;43:11461156.
9. Clancy CM, Eisenberg JM. Outcomes research: Measuring the end results of health care. Science 1998;282:245246.
10. Wilson IB, Cleary PD. Linking clinical variables with health-related quality of life: A conceptual model of patient outcomes. JAMA 1995;273:5965.
11. Lee SJ, Earle CC, Weeks JC. Outcomes research in oncology: History, conceptual framework, and trends in the literature. J Natl Cancer Inst. 2000;92:195204.
12. Health Literacy Interventions and Outcomes: An Updated Systematic Review. Rockville, Md: Agency for Healthcare Research and Quality. Available at: Accessed May 11, 2017.
13. Stryer D, Tunis S, Hubbard H, Clancy C. The outcomes of outcomes and effectiveness research: Impacts and lessons from the first decade. Health Serv Res. 2000;35:977993.
14. Chung KC, Burns PB, Sears ED. Outcomes research in hand surgery: Where have we been and where should we go? J Hand Surg Am. 2006;31:13731379.
15. The outcome of the outcomes research at AHCPR: Final report. Rockville, Md: Agency for Healthcare Research and Quality. Available at: Accessed May 11, 2017.
16. Sears ED, Burns PB, Chung KC. The outcomes of outcome studies in plastic surgery: A systematic review of 17 years of plastic surgery research. Plast Reconstr Surg. 2007;120:20592065.
17. Jefford M, Stockler MR, Tattersall MH. Outcomes research: What is it and why does it matter? Intern Med J. 2003;33:110118.
18. Forrest CB, Shipman SA, Dougherty D, Miller MR. Outcomes research in pediatric settings: Recent trends and future directions. Pediatrics 2003;111:171178.
19. Stewart MG. Outcomes research: An overview. ORL J Otorhinolaryngol Relat Spec. 2004;66:163166.
20. Alderman AK, Chung KC. Measuring outcomes in hand surgery. Clin Plast Surg. 2008;35:239250.
21. Valentini M, Milesi A, Bettini A, Tondini C, Nicolucci A. Type and trends in outcomes research in breast cancer between 2000 and 2007. Ann Oncol. 2011;22:21602165.
22. Sinno H, Dionisopoulos T, Slavin SA, Ibrahim AM, Chung KC, Lin SJ. The utility of outcome studies in plastic surgery. Plast Reconstr Surg Glob Open 2014;2:e189.
23. Ibrahim AM, Sinno HH, Izadpanah A, et al. Nipple-areolar complex reconstruction following postmastectomy breast reconstruction: A comparative utility assessment study. Plast Reconstr Surg Glob Open 2015;3:e380.
24. Ibrahim AM, Sinno HH, Izadpanah A, et al. Population preferences of undergoing brachioplasty for arm laxity. Ann Plast Surg. 2014;73(Suppl 2):S149S152.
25. Ibrahim AM, Sinno HH, Izadpanah A, et al. Mastopexy for breast ptosis: Utility outcomes of population preferences. Plast Surg (Oakv.) 2015;23:103107.
26. Izadpanah A, Sinno H, Vorstenbosch J, Lee BT, Lin SJ. Thigh laxity after massive weight loss: A utilities outcomes assessment. Ann Plast Surg. 2013;71:304307.
27. Sinno H, Izadpanah A, Thibaudeau S, et al. The impact of living with a functional and aesthetic nasal deformity after primary rhinoplasty: A utility outcomes score assessment. Ann Plast Surg. 2012;69:431434.
28. Sinno H, Thibaudeau S, Izadpanah A, et al. Utility outcome scores for unilateral facial paralysis. Ann Plast Surg. 2012;69:435438.
29. Sinno H, Thibaudeau S, Tahiri Y, et al. Utility assessment of body contouring after massive weight loss. Aesthetic Plast Surg. 2011;35:724730.
30. Sinno H, Izadpanah A, Tahiri Y, et al. The impact of living with severe lower extremity lymphedema: A utility outcomes score assessment. Ann Plast Surg. 2014;73:210214.
31. Sinno H, Dionisopoulos T, Slavin SA, Ibrahim AM, Chung KC, Lin SJ. The utility of outcome studies in plastic surgery. Plast Reconstr Surg Glob Open 2014;2:e189.
32. Sinno H, Izadpanah A, Vorstenbosch J, et al. Living with a unilateral mastectomy defect: A utility assessment and outcomes study. J Reconstr Microsurg. 2014;30:313318.
33. Sinno HH, Ibrahim AM, Izadpanah A, et al. Utility outcome assessment of the aging neck following massive weight loss. Otolaryngol Head Neck Surg. 2012;147:2632.
34. Sinno H, Tahiri Y, Thibaudeau S, et al. Cleft lip and palate: An objective measure outcome study. Plast Reconstr Surg. 2012;130:408414.
35. Sinno H, Izadpanah A, Thibaudeau S, Christodoulou G, Lin SJ, Dionisopoulos T. An objective assessment of the perceived quality of life of living with bilateral mastectomy defect. Breast 2013;22:168172.
36. Yuan F, Chung KC. Defining quality in health care and measuring quality in surgery. Plast Reconstr Surg. 2016;137:16351644.
37. Shauver MJ, Chung KC. Applying economic principles to outcomes analysis. Clin Plast Surg. 2013;40:281285.
Copyright © 2017 by the American Society of Plastic Surgeons