In the past several decades, techniques and technology in plastic surgery have advanced tremendously. Traditionally, the discussion of outcomes in plastic surgery has centered on the provider's perspective, focusing on measuring complications and considering photographic analyses. Today, however, such data alone are no longer sufficient to support the progress being made in the field. As the specialty of plastic surgery continues to develop, more sophisticated ways of examining outcomes are required.
Recently, the expression “quality of life” has caught the attention of patients, the public, health care payers, and policymakers. In the current environment of health care industry restrictions and performance metrics, quality-of-life outcomes have become ever more important to clinical practice and research.1,2 In plastic surgery, there has been a natural inclination to consider outcomes from the provider's perspective; today, attention is increasingly being placed on understanding the patient's perception of the surgical result and the impact that surgery has on quality of life.
Patient satisfaction and quality of life may be measured using specially designed questionnaires, known as patient-reported outcome measures. To adequately appraise and demonstrate the benefits of a chosen surgical technique, it is important for plastic surgeons to understand the available approaches to measurement of patient-reported outcomes. To this end, this article describes patient-reported outcome measures, explains their uses in clinical practice and research, and provides plastic surgeons with the necessary critical appraisal skills to interpret and apply evidence from patient-reported outcomes research in their own clinical practice.
DEFINITION OF TERMS
“Patient-reported outcome measures” is a term that applies specifically to a questionnaire used in a clinical or research setting where responses are collected directly from patients. These questionnaires quantify quality of life and/or significant outcome variables (e.g., patient satisfaction, symptoms) from the patient's perspective.3 Patient-reported outcome measures provide a means of quantifying the way patients perceive their health and the impact treatments have on their quality of life. A good patient-reported outcome measure should capture the impact of disease, trauma, or health care intervention on various aspects of a patient's day-to-day life in a manner that is scientifically sound and clinically meaningful.3
In plastic surgery research, many of the patient-reported outcome measures frequently used in studies to evaluate surgical outcomes have not been developed and validated using acknowledged guidelines.3–7 Such patient-reported outcome measures are considered ad hoc questionnaires, and although they may pose clinically reasonable questions, one cannot be confident about their reliability (i.e., ability to produce consistent and reproducible scores) or validity (i.e., ability to measure what is intended to be measured). However, the use of such measures has been extensive in plastic surgery. As an example, in a systematic review to identify patient-reported outcome measures developed for use with breast surgery patients, we identified 65 studies using an ad hoc questionnaire.4
Patient-reported outcome measures that can be used in any patient group regardless of their health condition are called “generic questionnaires” and allow direct comparisons across disease groups and/or healthy populations. These measures can provide important information for health policy decisions. For example, research using the Medical Outcome Study Short Form 36-Item Health Survey (the most widely used generic measure in the world8) with breast reduction patients shows that women report clinically important differences in quality of life compared with women in the general population.9–12 In the United States, breast reduction is a procedure for which health insurance companies frequently dispute payment; it is thus useful to place the health deficits experienced by women with hypermastia into the context of patients with other medical conditions (e.g., women with hip or knee osteoarthritis seeking joint replacement).
However, there are limitations associated with the use of generic measures. Given their broad nature, they potentially lack sensitivity to the specific disease-related issues in a particular patient group. For instance, the Medical Outcome Study Short Form 36-Item Health Survey, which measures physical, emotional, and social functioning, does not include questions about sexuality, body image, or satisfaction with breast appearance, which are clearly important concerns of breast reduction patients.13
Generic measures such as the Medical Outcome Study Short Form 36-Item Health Survey may then not capture the most relevant issues for a particular patient group. Disease- or condition-specific measures address problems specific to a single disease or treatment group. Such measures, when developed through in-depth patient interviews, can help to identify specific issues of importance. As these measures include content areas that are more relevant to a given patient group, they are more likely than generic measures to be sensitive to measuring change in key aspects of health following health care interventions. However, in general, they cannot be used to make comparisons across different patient groups.
An exception to this rule is patient-reported outcome measures that are developed with multiple modules that make it possible to evaluate outcomes in different patient groups with related problems. For example, the recently developed BREAST-Q is a patient-reported outcome measure that measures satisfaction and quality-of-life issues important to breast surgery patients.14,15 This measure has separate procedure-specific modules for breast augmentation, reduction, reconstruction, and mastectomy without reconstruction, and a common core set of items relevant to all breast surgery patients. The separate modules can be used to examine concerns unique to each patient group, and the common items allow comparisons between surgical groups.
To address the limitations associated with both generic and disease-specific measures, a common approach is to supplement generic measures with a condition or disease-specific measure (or vice versa) to capture information on the major domains common to all diseases and the ones unique to the particular condition of interest. However, in plastic surgery, this approach is not always possible, as there are many areas for which disease- or condition-specific questionnaires have not been developed. For example, in a review of patient-reported outcome measures developed for pediatric plastic surgery,6 only seven condition-specific measures were identified covering only three main areas of surgery (i.e., burns, craniofacial anomalies, and pectus excavatum). The same situation exists in facial aesthetic surgery. Recently, Kosowski et al. conducted a systematic review of published patient-reported outcome measures for facial cosmetic surgery and found only nine measures, all of them being limited by their development, validation, or content.5
When there is no measure available for a particular patient group, the options are to either find a generic measure that covers the issues relevant to the patient group of interest as closely as possible, or to undertake the work of developing a new measure. If a generic measure is used, the prospective user must consider carefully the process used to develop patient-reported outcome measures (described below) and look closely at the items that compose the scales to choose a measure that best addresses the issues most important to the patient group of interest.
OVERVIEW OF PATIENT-REPORTED OUTCOME MEASURE DEVELOPMENT
To choose a patient-reported outcome measure for use in a study or in clinical practice, prospective users should take into account the process used to develop patient-reported outcome measures (Fig. 1). Knowing how a patient-reported outcome measure was developed is crucial to choosing an appropriate measure. Not infrequently, researchers choose questionnaires based simply on whether or not they have been “validated” and ask few questions about how the items for the questionnaire were constructed and tested. A poorly constructed, or ad hoc questionnaire, may be deemed validated simply because it has been used in various studies and some basic statistical evidence has been supplied. Frequent use of a questionnaire does not equate with quality or improve its psychometric properties. Rather, to ensure that a patient-reported outcome measure will ultimately prove to be reliable, valid, and responsive, a rigorous step-by-step development process is essential. During the development process, careful qualitative work is necessary to conceptualize, map out, and operationalize the variables most relevant to patients. Patient-reported outcome measures that are developed based on expert opinion alone cannot be expected to address all satisfaction and quality-of-life issues that patients find relevant. Thus, although expert opinion is clearly valuable, patient interviews and focus groups are essential sources of information.
Since the early 1990s, there has been increasing international consensus regarding appropriate methods for the development and validation of quality-of-life measures, culminating with the 2002 report by the Scientific Advisory Committee of the Medical Outcomes Trust16 and more recently with the U.S. Food and Drug Administration recommendations.17 Cano et al.3 summarized these guidelines in a three-step approach to patient-reported outcome measure development that involves procedures for item generation, item reduction, and psychometric evaluation.
In the first step, the conceptual model to be measured is formally defined, and a pool of items is generated. These items for a patient-reported outcome measure are developed through the following three sources: review of the literature, qualitative patient interviews, and expert opinion. The item pool is developed into a questionnaire, which is pretested on a small sample of patients to clarify ambiguities in item wording, confirm appropriateness, and determine acceptability and completion time.
In the second step, the patient-reported outcome measure is field-tested using a large sample of patients. The questions that represent the best indicators of outcome are then retained in a shortened version of the patient-reported outcome measure based on their performance against a standardized set of psychometric criteria. The research team's objective is to choose the best items from the field-test measure for inclusion in the final measure. The item-reduction process results in a field test version of the questionnaire.
In the third step, a psychometric evaluation study is performed. This step involves the administration of the field test version of the questionnaire to a large population of patients to determine acceptability, reliability, validity, and responsiveness.
ESSENTIAL ELEMENTS FOR PATIENT-REPORTED OUTCOME MEASURES
Patient-reported outcome measures must be clinically meaningful and scientifically sound. A questionnaire that is clinically meaningful addresses those issues considered important to patients and their surgeons. Scientific soundness refers to the demonstration of reliable, valid, and responsive measurement of the outcome of interest. A reliable measure yields consistent and reproducible results of the same measure. Reliability is an important property of a patient-reported outcome measure because it is essential to establish that any changes observed in patient groups are attributable to the intervention or disease and not to problems in the measure. Test-retest reliability may be evaluated by having individuals complete a questionnaire on more than one occasion over a time period when no changes in outcome are expected to have occurred. Commonly reported reliability statistics include Cronbach's α18 and intraclass correlation coefficients.19
Validity is the ability of an instrument to measure what is intended to be measured. Establishment of validity may be considered an ongoing process. A patient-reported outcome measure is examined from various angles, which include an assessment of the development process, consideration of known group differences, evaluation of internal consistency, and evaluation of both convergent and discriminant validity relative to other existing related measures.
Responsiveness is defined as the ability of a measure to accurately detect change. Responsiveness is an important psychometric property when evaluating changes as the result of a health care intervention or when following patients over time. Responsiveness is usually examined by comparing preintervention and postintervention scores using standardized change indicators, such as effect size statistics.20
MODERN PSYCHOMETRIC METHODS
The science underpinning the testing of the attributes of reliability, validity, and responsiveness is known as psychometrics. The most commonly used form of scale evaluation is based on traditional psychometric methods.16 However, in recent times, researchers have become aware of the limitations of these methods and are now moving to newer techniques. As such, modern psychometric methods, such as Rasch measurement21 and item response theory,22 are increasingly being used in the development of patient-reported outcome measures. Although traditional psychometric methods provide ordinal-level data, Rasch analysis provides interval-level data21 and improves the accuracy with which clinical change can be measured. In addition, these methods provide estimates for patients (and items) that are independent of the sampling distribution of items (and patients). Among other benefits, interval-level data allow for accurate estimates suitable for individual person measurement. Such data can help to directly inform patient monitoring, management, and treatment. Other advantages of using patient-reported outcome measures developed with modern psychometric methods include the potential for item banking, scale equating, computerized scale administration, and handling missing data.23,24
In the coming years, we expect that these newer, more clinically amenable techniques, with their own tests and criteria (e.g., scale-to-person targeting, item locations, thresholds, fit, residual correlations, and differential item functioning) will supersede traditional approaches to developing and testing patient-reported outcome measures. In the meantime, we recommend that surgeons become familiar with the traditional methods, as the central tenets of reliability, validity, and responsiveness will remain the same.
PLASTIC SURGERY EXAMPLE OF THE DEVELOPMENT OF A PATIENT-REPORTED OUTCOME MEASURE
Our research group's development of the BREAST-Q illustrates how new patient-reported outcome measures can be constructed.15 In phase I, a conceptual framework to understand patient satisfaction and quality of life in breast surgery patients was developed and included six domains: satisfaction with breasts; overall outcome and process of care; and psychosocial, physical, and sexual well-being. Our team developed this conceptual framework and set of items to measure each component of the framework using in-depth interviews and focus groups with patients, expert panels with plastic surgeons, and a review of the literature.4 We then performed pilot testing and cognitive debriefing to ensure that our patient-reported outcome measure addressed all relevant issues and that the items were acceptable to patients and easily understood. In phase II, we performed an extensive multicenter field test with 1950 patients. Based on these data, item reduction and scale development were performed using modern psychometric methods. In phase III, the item-reduced measure (BREAST-Q) was completed by a large sample of breast surgery patients (n = 1283), and the data were examined to determine reliability, validity, and responsiveness.
HOW TO USE PATIENT-REPORTED OUTCOME MEASURE DATA IN RESEARCH AND CLINICAL PRACTICE
Data gathered through the use of patient-reported outcome measures are important whenever the goal of a health care intervention is to make a patient feel or function better.25 Patient-reported outcome measure data are also crucial when patient satisfaction is a key determinant of success.26 For example, when a woman undergoes rhinoplasty, the objective of surgery is to satisfy her expectations in terms of the final aesthetic result and to help her feel better about her appearance and perhaps even function better socially. An additional goal is to improve or maintain her nasal airflow (i.e., improve physical function and thus quality of life).
It should be noted that patient-reported outcome measures can be used in different types of study designs. For example, a patient-reported outcome measure can be used in a randomized controlled trial (i.e., level I evidence) or in a cross-sectional survey (i.e., level III evidence). Patient-reported outcome measure data are helpful when comparing different surgical options with equivalent survival or when the risk of complications of different surgical procedures is the same. Patient-reported outcome measure data make it possible to consider patient morbidity and understand which option might be preferable from the patient's perspective. As an example, when a woman seeks implant breast reconstruction, a surgeon may consider placement of either saline or silicone implants. With either technique, cancer survival is the same and the overall risk of complications is similar. It is important to understand expected differences in how a woman's breasts will feel to her and to communicate this information in the process of shared medical decision-making.
In addition, routine collection of patient-reported outcome measure data may provide valuable information on patient satisfaction and quality of life that can be incorporated into comprehensive cost-effectiveness analyses of surgical procedures (e.g., cost per quality-adjusted life-year). This information may help surgeons advocate for patients to help increase access to advanced surgical procedures that are considered superior from a patient's perspective.27
Patient-reported outcome measure data can be used as the basis for making choices about treatment. For instance, when discussing the implications of a transverse rectus abdominis musculocutaneous flap with a woman contemplating breast reconstruction, a plastic surgeon can use results from clinical trials that report patients' perspectives of postoperative abdominal function and satisfaction with both breast and abdominal appearance. This information can inform the patient's decision-making process and also plays an important role in creating realistic expectations toward surgical outcomes.
Routine collection of patient-reported outcome measure data in a surgeon's individual practice can help him or her to assess change in individual patients over time. For example, using a patient-reported outcome measure for facial aesthetic patients, a plastic surgeon would be able to reliably measure patient satisfaction with his or her facial appearance before and after a rhytidectomy. Patient-reported outcome measure data collected on individual patients may also provide surgeons with valuable insight into the patient's concerns. For example, using patient-reported outcome measure data in clinical practice can help a surgeon determine whether a dissatisfied patient is upset about the surgical outcome or something else, such as an aspect of care received (e.g., interactions with office employees). Using a patient-reported outcome measure in clinical practice can help to identify problems, facilitate communication, and direct appropriate treatment of underappreciated symptoms. It is important to note that only those patient-reported outcome measures developed using modern psychometric methods (Rasch and item response theory) are appropriate for use with individual patients.
There are several potential pitfalls to consider when using patient-reported outcome measure data in evidence-based practice. When used in research, a low response rate might indicate selection bias (e.g., Did only satisfied patients respond to the questionnaire?). When a study reports no difference in outcome following treatment or fails to show a difference between treatment groups in a head-to-head comparison, the sensitivity (responsiveness) of the patient-reported outcome measure used should be considered. If a measure is not adequately calibrated to detect a difference in the target population (e.g., a generic measure was used), no statistical difference may be observed even though there may in fact be an important clinical difference in outcome. For example, in a study comparing vertical breast reduction to inverted-T pattern reduction, a patient-reported outcome measure that does not report patients' perception of breast shape or scars may fail to reveal any difference between surgical groups. Equally important to consider is that when a statistically significant difference is noted, it should be interpreted with caution because a statistically significant difference is not necessarily clinically meaningful.28 The clinical importance of findings using patient-reported outcome measure data should be interpreted using either clinical anchors, such as minimally important differences,29,30 or distribution-based methods, such as effect sizes.31,32 Finally, when interpreting clinical trial data, methodologic issues (e.g., missing data) should be evaluated using established criteria.33,34
As reconstructive and cosmetic surgery techniques continue to advance, treatment options continue to increase. At the same time, there is increased scrutiny of health care cost. In addition, patients are becoming increasingly involved in their own medical care and are demanding meaningful data to help them better understand expected outcomes. With these changes and advancements, a greater emphasis is being placed on evidence-based practice. As patient satisfaction and quality of life are among the most important outcomes in plastic surgery, patient-reported outcome measure data are essential to evaluate the benefits of newly developed surgical techniques. Reliable, valid, and clinically meaningful patient-reported outcome measure data can help surgeons, patients, and policymakers better quantify and understand the benefits of plastic surgery procedures. As a central component to comparative effectiveness studies, patient-reported outcome measure data will support patient advocacy and facilitate patient access to procedures that demonstrate a positive effect on their overall health and well-being.
1. Fitzpatrick R, Jenkinson C, Klassen A, Goodacre T. Methods of assessing health-related quality of life and outcome for plastic surgery. Br J Plast Surg
2. Cano SJ, Browne JP, Lamping DL. Patient-based measures of outcome in plastic surgery: Current approaches and future directions. Br J Plast Surg
3. Cano SJ, Klassen A, Pusic AL. The science behind quality-of-life measurement: A primer for plastic surgeons. Plast Reconstr Surg
4. Pusic AL, Chen CM, Cano S, et al. Measuring quality of life in cosmetic and reconstructive breast surgery: A systematic review of patient-reported outcomes instruments. Plast Reconstr Surg
. 2007;120:823–837; discussion 838–839.
5. Kosowski TR, McCarthy C, Reavey PL, et al. A systematic review of patient-reported outcome measures after facial cosmetic surgery and/or nonsurgical facial rejuvenation. Plast Reconstr Surg
6. Klassen AF, Stotland MA, Skarsgard ED, Pusic AL. Clinical research in pediatric plastic surgery and systematic review of quality-of-life questionnaires. Clin Plast Surg
7. Branski RC, Cukier-Blaj S, Pusic A, et al. Measuring quality of life in dysphonic patients: A systematic review of content development in patient-reported outcomes measures. J Voice
8. Garratt A, Schmidt L, Mackintosh A, Fitzpatrick R. Quality of life measurement: Bibliographic study of patient assessed health outcome measures. BMJ
9. Kerrigan CL, Collins ED, Striplin D, et al. The health burden of breast hypertrophy. Plast Reconstr Surg
10. Blomqvist L, Eriksson A, Brandberg Y. Reduction mammaplasty provides long-term improvement in health status and quality of life. Plast Reconstr Surg
11. Hermans BJ, Boeckx WD, De Lorenzi F, van der Hulst RR. Quality of life after breast reduction. Ann Plast Surg
12. Klassen A, Fitzpatrick R, Jenkinson C, Goodacre T. Should breast reduction surgery be rationed? A comparison of the health status of patients before and after treatment: Postal questionnaire survey. BMJ
13. Klassen AF, Pusic AL, Scott A, Klok J, Cano SJ. Satisfaction and quality of life in women who undergo breast surgery: A qualitative study. BMC Womens Health
14. Pusic AL, Reavey PL, Klassen AF, Scott A, McCarthy C, Cano SJ. Measuring patient outcomes in breast augmentation: Introducing the BREAST-Q Augmentation module. Clin Plast Surg
. 2009;36:23–32, v.
15. Pusic AL, Klassen AF, Scott AM, Klok JA, Cordeiro PG, Cano AJ. Development of a new patient-reported outcome measure for breast surgery: The BREAST-Q. Plast Reconstr Surg
16. Scientific Advisory Committee of the Medical Outcomes Trust. Assessing health status and quality-of-life instruments: Attributes and review criteria. Qual Life Res
17. U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research; U.S. Department of Health and Human Services FDA Center for Biologics Evaluation and Research; U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health. Guidance for industry. Patient-reported outcome measures: Use in medical product development to support labeling claims. Health Qual Life Outcomes
18. Cronbach L. Coefficient alpha and the internal structure of tests. Psychometrika
19. Hays RD, Anderson R, Revicki D. Psychometric considerations in evaluating health-related quality of life measures. Qual Life Res
20. Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Med Care
21. Rasch G. Probabilistic Models for Some Intelligence and Attainment Tests
. Copenhagen:Danish Institute for Education Research; 1960.
22. Lord FM, Novick MR. Statistical Theories of Mental Test Scores
. Reading, Mass: Addison-Wesley; 1968.
23. Wright B, Stone M. Best Test Design: Rasch Measurement
. Chicago: MESA Press; 1979.
24. Wright BD, Linacre JM. Observations are always ordinal; measurements, however, must be interval. Arch Phys Med Rehabil
25. Guyatt GH, Veldhuyzen Van Zanten SJ, Feeny DH, Patrick DL. Measuring quality of life in clinical trials: A taxonomy and review. CMAJ
26. Carr-Hill RA. The measurement of patient satisfaction. J Public Health Med
27. Thoma A, Veltri K, Khuthaila D, Rockwell G, Duku E. Comparison of the deep inferior epigastric perforator flap and free transverse rectus abdominis myocutaneous flap in postmastectomy reconstruction: A cost-effectiveness analysis. Plast Reconstr Surg
28. Osoba D, Rodrigues G, Myles J, Zee B, Pater J. Interpreting the significance of changes in health-related quality-of-life scores. J Clin Oncol
29. Schunemann HJ, Guyatt GH. Commentary: Goodbye M(C)ID! Hello MID, where do you come from? Health Serv Res
30. Norman GR, Sridhar FG, Guyatt GH, Walter SD. Relation of distribution- and anchor-based approaches in interpretation of changes in health-related quality of life. Med Care
31. King MT. Cohen confirmed? Empirical effect sizes for the QLQ-C30. In: Proceedings of the 8th Annual Conference of the International Society for Quality of Life Research (ISOQOL)
. Vol. 10. Amsterdam, The Netherlands: Quality of Life Research; 2001:278–279.
32. Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: The remarkable universality of half a standard deviation. Med Care
33. Efficace F, Bottomley A, Osoba D, et al. Beyond the development of health-related quality-of-life (HRQOL) measures: A checklist for evaluating HRQOL outcomes in cancer clinical trials. Does HRQOL evaluation in prostate cancer research inform clinical decision making? J Clin Oncol
34. Cohen J. Statistical Power Analysis for the Behavioral Sciences
. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum; 1988.
Submit your manuscript today through PRS' Enkwell. The Enkwell submission and review Web site helps make the submission process easier, more efficient, and less expensive for authors, and makes the review process quicker, more accessible, and less expensive for reviewers. If you are a first-time user, be sure to register on the system.