Secondary Logo

Journal Logo

CLINICAL RESEARCH

Responsiveness of the PROMIS and its Concurrent Validity with Other Region- and Condition-specific PROMs in Patients Undergoing Carpal Tunnel Release

Bernstein, David N. MBA, MA; Houck, Jeff R. PhD; Mahmood, Bilal MD; Hammert, Warren C. MD

Author Information
Clinical Orthopaedics and Related Research: November 2019 - Volume 477 - Issue 11 - p 2544-2551
doi: 10.1097/CORR.0000000000000773

Abstract

Introduction

Recently, there has been an interest in using patient-reported outcomes measures (PROMs) [1] as the US healthcare system shifts towards focusing on value—defined as dollar spent per health outcome achieved [26]. The use of PROMs may help better align orthopaedic surgeons’ expectations with clinical outcomes [8] and help surgeons incorporate measures of patient-reported health into treatment decisions to understand patients’ responses to different treatments and understand who might clinically benefit from a given intervention [23, 29]. Within orthopaedic surgery, a number of validated, region-specific PROMs have frequently been used in patient care [7, 18, 22, 30, 34]. However, more recently, there has been a new movement towards universal PROMs as the push towards population health increases. The Patient-reported Outcome Measurement Information System (PROMIS) is a generic PROM that incorporates item response theory and computerized adaptive testing [6]. One benefit of the PROMIS is that it captures constructs that are similar to those of specialty-specific PROMs [2, 9, 10, 25, 28]. It is also responsive; that is, the PROMIS’s domains can be used detect change over time [13, 14, 16, 17, 19]. Although studies in certain areas of orthopaedic surgery (such as ACL reconstruction and spine disorders) show that the PROMIS domains and region-specific scales have comparable responsiveness [19, 32], there is still concern that the PROMIS Physical Function (PF), Upper Extremity (UE), and Pain Interference (PI) domains are not sufficiently specific to the hand or certain hand procedures such as carpal tunnel release (CTR). Because of concern that the PROMIS PF domain is not responsive to hand conditions, a PROMIS UE physical function scale was developed.

Studies documenting the responsiveness and concurrent validity of the PROMIS in patients seeking hand care are needed. An evaluation of responsiveness and concurrent validity in a subset of patients seeking hand care, such as those undergoing CTR, will offer much better insight into common hand conditions. Patients undergoing CTR are an ideal population to begin such an inquiry in because CTR is common, with a lifetime prevalence of 3.1% [27], and CTR is the only hand condition that can be evaluated with both a region-specific PROM (Michigan Hand Questionnaire [MHQ]) and a condition-specific PROM (Boston Carpal Tunnel Questionnaire [BCTQ]). If the PROMIS is equally responsive and valid as region-specific PROMs, the financial investment in potentially improved patient care may be worthwhile, because the PROMIS measures a patient’s overall health and the impact of carpal tunnel syndrome (CTS) instead of simply disease-specific effects [15].

Therefore, we asked: (1) Which PROMs demonstrate high responsiveness among patients undergoing CTR? (2) Which of the PROMIS domains (PF, UE, and PI) demonstrate concurrent validity with the MHQ and BCTQ domains?

Patients and Methods

Study Design and Patient Sample

Between November 2014 and October 2016, patients presenting to a single hand surgeon at a large, urban, academic medical center who had isolated CTS and wanted to undergo CTR were asked to participate in this prospective, institutional review board-approved study. The presence of CTS was determined by taking the patient’s history and performing a physical examination, and was confirmed with EMG testing. A total of 101 patients were willing to participate. Of these, 31 patients (31%) were removed from the final sample because they did not attend a followup clinic visit at least 6 weeks after undergoing CTR. This left a final sample of 70 patients (69%) for all analyses (Fig. 1). The average age was 61 years (range, 27-86 years) and the majority were women (46; 66%) (Table 1). Most patients (43; 61%) had a 6-week followup visit, while the remainder (27 patients; 39%) had a 3-month followup visit. The majority of operations were on the right wrist (43 patients; 61%).

Fig. 1
Fig. 1:
This diagram illustrates the pathway from the total possible patient sample agreeing to participate in this prospective study to the final patient sample used for all analyses.
Table 1.
Table 1.:
Patient characteristics (n = 70)

Apple iPads (Apple, Cupertino, CA, USA) were used to gather PROM data; this has been shown to be an efficient approach in a hand-surgery population [35]. Each patient completed the following PROMs: the PROMIS PF, PI, and UE domains; the BCTQ; and the MHQ, and the results were collated using REDCap (Vanderbilt University, Nashville, TN, USA). Patients completed all PROMs at each visit during the study period. To allow for recovery after CTR, we only included patients with PROM scores from initial preoperative clinic visits and final followup clinic visits at 6 weeks or 3 months postoperatively.

Higher PROMIS PF and UE scores indicate better physical function, while higher PROMIS PI scores denote increased pain interference or physical limitations secondary to pain [5]. PROMIS domains follow a population-based normal distribution, with a mean t-score of 50 and standard deviation of 10 [11]. Unlike the other PROMs used in this study, there is no total PROMIS score, because each domain evaluates a different construct. For the BCTQ–Total Score (average of the two BCTQ domains–Functional Status Scale and Symptom Severity Scale), which has a score ranging from 1 to 5, higher scores indicate worse symptom severity and physical function. For the MHQ, which has a score ranging from 0 to 100, lower scores indicate worse overall hand health.

Statistical Analysis

We calculated the patients’ characteristics. We also evaluated the three PROMIS domains (UE, PF, PI); eight MHQ domains (Overall Hand Function, Activities of Daily Living–One Hand, Activities of Daily Living–Two Hands, Overall Activities of Daily Living, Ability to Work, Pain, Aesthetic, and Satisfaction), in addition to the total score; and two BCTQ domains (Symptom Severity Scale and Functional Status Scale), in addition to the average of the two domains (that is, the BCTQ–Total Score).

To evaluate the responsiveness of each PROM domain, we used Cohen’s d, or the effect-size index (ESI), for all changes in scores. The absolute value of each ESI was taken because the size of the effect was of the most interest, not the direction. The ESI is a frequently used index of change to judge responsiveness, with higher values indicating improved responsiveness or better ability to detect change [20]. Responsiveness is important to evaluate for any PROM; for example, if the instrument does not effectively identify clear clinical improvement after CTR, then we cannot use that particular PROM to evaluate the success or failure of CTR. Values of 0.2, 0.5, and greater than 0.8 are considered small, medium, and large effect sizes, respectively [20]. To put these values into a clinical perspective, an ESI of 0.5 (that is, a medium effect size) may be thought of as representing a true change in patient status that is appreciated by a careful observer [33]. In the current study, we evaluated the change between the preoperative visit and longest followup postoperatively, either 6 weeks or 3 months.

Concurrent validity was evaluated using Spearman’s correlation coefficients (ρ) because the assumption of normality (Shapiro-Wilk test, skewness, kurtosis, and visual assessment of histograms) was not met for all variables. Absolute values were used because the strength, not the direction of the correlation, indicates concurrent validity. Spearman’s correlation coefficients were calculated for each preoperative PROMIS domain and MHQ domain (as well as MHQ–Total Score) and each preoperative PROMIS domain and BCTQ domain (as well as BCTQ–Total Score). Spearman’s correlation coefficients were also calculated for each postoperative PROMIS domain and MHQ domain (as well as MHQ–Total Score) and each postoperative PROMIS and BCTQ domain (as well as BCTQ–Total Score). Using guidelines from a previous study using the PROMIS [31], we designated the strength of the Spearman’s correlations for the total PROM scores as follows: excellent (> 0.7), excellent-good (0.61-0.70), good (0.4-0.6), and poor (< 0.4). Significance was set a priori at p < 0.05.

Results

Responsiveness of Outcomes Tools

The PI domain of the PROMIS demonstrated the best responsiveness (ESI = 0.74; 95% CI, 0.39-1.08), followed by the UE domain (ESI = -0.66; 95% CI, -1.00 to -0.31). For the MHQ, the Satisfaction domain demonstrated the largest effect size (ESI = -1.48; 95% CI, -1.85 to -1.09), followed by the Total Score (ESI = -1.22; 95% CI, -1.58 to -0.84). For the BCTQ, the Symptom Severity domain demonstrated the best responsiveness (ESI = 1.54; 95% CI, 1.14-1.91). All PROMs except for the PROMIS PF domain (ESI = -0.25; 95% CI, -0.59 to 0.08) and MHQ Aesthetic domain (ESI = 0.02; 95% CI, -0.32 to 0.36) demonstrated at least a medium effect size at the final followup examination (Table 2).

Table 2.
Table 2.:
ESI values at final followup

Concurrent Validity Among Outcomes Tools

Among the preoperative PROM scores, the PROMIS UE and PI scores demonstrated concurrent validity (excellent-good and excellent correlations, respectively) with the MHQ Total Score (UE: ρ = 0.68; p < 0.001; PI: ρ = 0.74; p < 0.001, but the PROMIS PF score demonstrated a poor correlation with the MHQ Total Score (ρ = 0.33; p = 0.006) (Table 3). Preoperative PROMIS UE scores exhibited good-to-excellent correlations with all preoperative MHQ domain scores (range, UE: ρ = 0.41-0.80; all p < 0.001) except for the MHQ Aesthetics score (ρ = 0.24; p = 0.04) (Table 3). Likewise, preoperative PROMIS PI scores exhibited good to excellent-good correlations with all preoperative MHQ domain scores (UE: ρ = 0.53-0.69; all p < 0.001) except for the MHQ Aesthetics score (ρ = 0.18; p = 0.15) (Table 3). For the PROMIS PF, correlations with the MHQ domains ranged from poor to good (PF: ρ = 0.007-0.48; p = 0.95 to < 0.001) (Table 3). Among preoperative PROM scores, the PROMIS UE and PI demonstrated excellent and excellent-good correlations, respectively, with the BCTQ–Functional Status domain (UE: ρ = 0.74; p < 0.001; PI: ρ = 0.67; p < 0.001), while the PF demonstrated a poor correlation (PF: ρ = 0.39; p = 0.001) (Table 3). The PROMIS UE and PI domains also demonstrated good and excellent-good correlations, respectively, with the BCTQ–Symptom Severity scale (UE: ρ = 0.49; p < 0.001; PI: ρ = 0.69; p < 0.001), while the PF had a poor correlation (PF: ρ = 0.04; p = 0.73) (Table 3).

Table 3.
Table 3.:
Spearman’s correlation coefficient for the preoperative PROMIS, MHQ, and BCTQ domain scores (n = 70)

Among postoperative PROM scores, the PROMIS UE and PI scores demonstrated concurrent validity (excellent-good to excellent correlations, respectively) with the MHQ Total Score (UE: ρ = 0.65; p < 0.001; PI: ρ = 0.72; p < 0.001) (Table 4). The PROMIS PF scores had a poor correlation with the MHQ Total Score (ρ = 0.36; p = 0.002) (Table 4). Except for the MHQ Aesthetics domain (ρ = 0.007; p = 0.96), postoperative PROMIS UE scores exhibited good to excellent-good correlations with all other preoperative MHQ domain scores (UE: ρ = 0.53-0.69; all p < 0.001) (Table 4). Except for the MHQ Aesthetics domain (ρ = 0.09; p = 0.48), postoperative PROMIS PI scores exhibited good-to-excellent correlations with all other preoperative MHQ domain scores (UE: ρ = 0.54-0.72; all p < 0.001) (Table 4). For the PROMIS PF, only the correlation with MHQ Overall Hand Function, Activities of Daily Living–Two Hands, and Overall Activities of Daily Living was above poor (PF: ρ = 0.40-0.44; all p < 0.05) (Table 4). The PROMIS UE and PI domains also demonstrated excellent and excellent-good correlations, respectively, with the BCTQ–Symptom Severity domain (UE: ρ = 0.74; p < 0.001; PI: ρ = 0.62; p < 0.001), while the PF domain had a good correlation (PF: ρ = 0.40; p = 0.008). Similarly, for the BCTQ–Functional Status scale postoperatively, the PROMIS UE and PI also demonstrated excellent and excellent-good correlations, respectively (UE: ρ = 0.75; p < 0.001; PI: ρ = 0.66; p < 0.001). The PROMIS PF domain had a poor correlation with the BCTQ–Functional Status domain postoperatively (PF: ρ = 0.37; p = 0.002) (Table 4).

Table 4.
Table 4.:
Spearman’s correlation coefficients for the postoperative PROMIS, MHQ, and BCTQ domain scores (n = 70)

Discussion

PROMs, especially generic instruments such as the PROMIS, are becoming common in orthopaedic clinics with an increasing interest in value-based health care. In many orthopaedic settings, the PROMIS, which may add more value than other PROMs because of its ability to provide insight into a patient’s overall health in the setting of illness, has been shown to capture constructs that are similar to those of more focused PROMs (such as region-specific PROMs) [2, 9, 10, 25, 28] with comparable responsiveness [13, 16, 17, 19]. However, it is unclear if this is consistent within a patient population that has a single, common hand condition (that is, CTS) treated with the same procedure (CTR). The findings of this study suggest that the PROMIS UE and PI scores effectively detect change in patients undergoing CTR; however, region- and disease-specific scales demonstrated a greater ability to detect change than the PROMIS did. Overall, we found the PROMIS UE and PI scales to be moderately responsive and valid in capturing information on the outcomes of patients undergoing CTR, while the PROMIS PF is neither responsive nor valid for this purpose. Thus, we recommend that surgeons use the PROMIS UE and PI scales rather than the PF when electing to use the PROMIS for evaluating patients.

Our study’s limitations should be considered when evaluating our findings. First, the data come from a single institution and surgeon. While there may be a question of generalizability, we feel our findings may be applicable to clinics with similar characteristics (such as an urban tertiary care center). Nonetheless, validating our findings with an independent sample would be valuable. Further work may also evaluate whether our results are consistent with more rural, community-based centers.

Second, we had a selection bias for two key reasons: (1) We only included patients who presented preoperatively and at 6 weeks or 3 months postoperatively and (2) although 101 patients were ultimately enrolled in our study, we approached every patient with a diagnosis of CTS and not everyone was willing to participate. It is possible that those willing to enroll were more invested in their care and/or had other traits or resources that would impact PROMs. Additionally, because patients who returned to the clinic before 6 weeks after surgery were not included, we could be missing patients who recovered to a greater extent. However, we feel our study was not substantially affected by these factors because this bias is consistent across all PROMs, and we did not evaluate clinical outcomes but the performance of one PROM against others at the final timepoint at which patients felt they needed to see a hand surgeon.

Third, the study had two postoperative timepoints. While this may have affected the size of the ESI for each PROM because more or less time had passed for a patient to recover, it does not substantially affect our study findings. This is because our goal was not to evaluate clinical outcomes over time but to compare the ability of each PROM to capture change against one other. Indeed, although the times differ, the length of followup affects all PROMs equally; thus, comparing responsiveness is still valid.

Fourth, the severity of CTS may vary, which could impact the amount of possible clinical improvement. However, similar to the third limitation, we are not as worried about this limitation because this study did not evaluate clinical outcomes, and all PROMs used for comparison were from the same patient population.

Lastly, our patients were informed that the PROMs would be used for research. Although this may have impacted how the patients completed the questionnaires, previous research on this subject has shown that patients do not tend to change PROM answers, even if they are aware that the scores will be used for research or clinical care [4].

Although the PROMIS UE and PI domains had acceptable responsiveness in evaluating patients with CTS undergoing CTR at the final followup, they were not as responsive as the MHQ and BCTQ. The PROMIS PF was not reasonably responsive in the same patient population. Previous research has shown that the PROMIS PF, UE, and PI domains in patients with hand and upper-extremity disorders are responsive [17]. Indeed, the study showed large effect sizes (ESI, 0.80-1.48) in patients with hand and upper-extremity disorders [17]. These findings demonstrate that the PROMIS domains were slightly more responsive in a general hand and upper-extremity population than in a specific population of patients with CTS undergoing CTR. This suggests that not all hand and upper-extremity disorders are the same. Another study has examined how PROMIS domains respond to CTR compared with the MHQ and BCTQ at different timepoints postoperatively [21]. The authors noted that the PROMIS UE had a similar but weaker responsiveness at each postoperative timepoint (that is, the 1-week, 6-week, and 3-month timepoints), while the PROMIS PI did not respond at 1 week postoperatively but at the 6-week and 3-month followup timepoints [21]. The PROMIS PF did not demonstrate good responsiveness at any of these timepoints [21]. Although our study differs because we assessed responsiveness at least 6 weeks after CTR to allow for at least moderate recovery and only at the patient’s final followup visit, we found similar results. Indeed, based on the ESI values, the PROMIS UE and PI show a lower ability to detect change in patients with CTS than the MHQ and BCTQ did, although they were comparable. This is to be expected, because the MHQ and BCTQ were specifically designed to evaluate an anatomic region and disease process, while PROMIS domains are meant to capture data on overall health. The finding that the PROMIS UE and PI can detect change at the final followup visit after CTR—even moderately—in patients with such a limited, neuropathic hand condition raises the question of how specific a disease process can be before a PROMIS domain does not capture change.

The PROMIS UE and PI domains demonstrated stronger correlations, in general, with the MHQ and BCTQ domains in both the preoperative and postoperative time periods than the PF did. Studies of patients with hand and upper-extremity conditions showed there was concurrent validity between the PROMIS PF and UE and DASH and QuickDASH (the validated shortened version of the DASH) [2, 24]. However, we did not find that the PROMIS PF had concurrent validity with many of the MHQ domains, which are region-specific and similar to the DASH and QuickDASH. This difference may be because of our focus on a single condition and its treatment (that is, patients with CTS undergoing CTR) or because of the type of hand conditions we evaluated (such as neuropathic). Given our results, we suggest using the PROMIS UE instead of the PROMIS PF in patients with CTS whenever possible. If a generic PROM is preferred, we feel it is also reasonable to use the PROMIS UE instead of the PROMIS PF in patients with all types of hand and wrist conditions until additional research is performed to determine any differences between various hand conditions. For example, future research is needed to determine whether our findings would be consistent in patients with basal joint arthritis or traumatic conditions (such as distal radius fracture) as well.

This study demonstrated that the PROMIS UE and PI are responsive and show concurrent validity in patients undergoing CTR. Region- and disease-specific scales both showed better responsiveness, with the MHQ Satisfaction and BCTQ Symptom Severity scales demonstrating the highest ESI scores. Our evaluation of concurrent validity showed that the PROMIS UE and PI scales are acceptable for evaluating patients undergoing CTR. However, we do not recommend using the PROMIS PF for patients undergoing CTR because it is not responsive and does not correlate strongly with region- or condition-specific PROMs. As expected, region- and condition-specific PROMs tend to capture data on a specific anatomic area or condition slightly better than general PROMs do; however, it is promising that a global PROM competes well with these instruments. Ultimately, using a global PROM such as the PROMIS may be more advantageous than region- and condition-specific PROMs as we shift our focus to population health, general wellbeing, and value-based medicine. This is because the PROMIS is normalized to the general population and offers a more complete picture of a patient’s overall health. There is a great opportunity in day-to-day clinical medicine to use these PROMIS domains, as well as others that capture more psychosocial elements of health such as self-resiliency, to inform surgeons on a patient’s overall health status and how best to council or treat him or her.

Acknowledgments

We thank Allison McIntyre, MPH, Chris DaSilva, BS, and the University of Rochester Health Laboratory for their assistance with collecting data and reporting patient-reported outcomes via the UR VOICE platform.

References

1. Baumhauer JF. Patient-Reported Outcomes - Are They Living Up to Their Potential? N Engl J Med. 2017;377:6-9.
2. Beckmann JT, Hung M, Voss MW, Crum AB, Bounsanga J, Tyser AR. Evaluation of the Patient-Reported Outcomes Measurement Information System Upper Extremity Computer Adaptive Test. J Hand Surg Am. 2016;41:739-744.e734.
3. Bernstein DN, Houck JR, Gonzalez RM, Wilbur DM, Miller RJ, Mitten DJ, Hammert WC. Preoperative PROMIS Scores Predict Postoperative PROMIS Score Improvement for Patients Undergoing Hand Surgery. Hand (N Y). 2018. [Epub ahead of print].
4. Bernstein DN, Mahmood B, Ketonis C, Hammert WC. A Comparison of PROMIS Physical Function and Pain Interference Scores in Patients With Carpal Tunnel Syndrome: Research Collection Versus Routine Clinical Collection. Hand (N Y). 2019. [Epub ahead of print].
5. Brodke DJ, Saltzman CL, Brodke DS. PROMIS for Orthopaedic Outcomes Measurement. J Am Acad Orthop Surg. 2016;24:744-749.
6. Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, Ader D, Fries JF, Bruce B, Rose M. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care. 2007;45:S3-s11.
7. Fairbank JC, Couper J, Davies JB, O'Brien JP. The Oswestry low back pain disability questionnaire. Physiotherapy. 1980;66:271-273.
8. Ghomrawi HMK, Mancuso CA, Dunning A, Gonzalez Della Valle A, Alexiades M, Cornell C, Sculco T, Bostrom M, Mayman D, Marx RG, Westrich G, O'Dell M, Mushlin AI. Do Surgeon Expectations Predict Clinically Important Improvements in WOMAC Scores After THA and TKA? Clin Orthop Relat Res. 2017;475:2150-2158.
9. Hancock KJ, Glass N, Anthony CA, Hettrich CM, Albright J, Amendola A, Wolf BR, Bollier M. Performance of PROMIS for Healthy Patients Undergoing Meniscal Surgery. J Bone Joint Surg Am. 2017;99:954-958.
10. Hancock KJ, Glass N, Anthony CA, Wolf BR, Hettrich CM, Albright J, Bollier M, Amendola A. PROMIS: a valid and efficient outcomes instrument for patients with ACL tears. Knee Surg Sports Traumatol Arthrosc. 2019;27:100-104.
11. HealthMeasures. PROMIS. 2018. Available at: http://www.healthmeasures.net/score-and-interpret/interpret-scores/promis. Accessed August 11, 2018.
12. Ho B, Houck JR, Flemister AS, Ketz J, Oh I, DiGiovanni BF, Baumhauer JF. Preoperative PROMIS Scores Predict Postoperative Success in Foot and Ankle Patients. Foot Ankle Int. 2016;37:911-918.
    13. Hung M, Baumhauer J, Licari FW, Bounsanga J, Voss MW, Saltzman CL. Responsiveness of the PROMIS and FAAM Instruments in Foot and Ankle Orthopedic Population. Foot Ankle Int. 2019;40:56-64.
    14. Hung M, Baumhauer JF, Licari FW, Voss MW, Bounsanga J, Saltzman CL. PROMIS and FAAM Minimal Clinically Important Differences in Foot and Ankle Orthopedics. Foot Ankle Int. 2019;40:65-73.
    15. Hung M, Franklin JD, Hon SD, Cheng C, Conrad J, Saltzman CL. Time for a paradigm shift with computerized adaptive testing of general physical function outcomes measurements. Foot Ankle Int. 2014;35:1-7.
    16. Hung M, Saltzman CL, Greene T, Voss MW, Bounsanga J, Gu Y, Anderson MB, Peters CL, Gililland J, Pelt CE. Evaluating instrument responsiveness in joint function: The HOOS JR, the KOOS JR, and the PROMIS PF CAT. J Orthop Res. 2018;36:1178-1184.
    17. Hung M, Saltzman CL, Greene T, Voss MW, Bounsanga J, Gu Y, Wang AA, Hutchinson D, Tyser AR. The responsiveness of the PROMIS instruments and the qDASH in an upper extremity population. J Patient Rep Outcomes. 2017;1:12.
    18. Hung M, Saltzman CL, Kendall R, Bounsanga J, Voss MW, Lawrence B, Spiker R, Brodke D. What Are the MCIDs for PROMIS, NDI, and ODI Instruments Among Patients With Spinal Conditions? Clin Orthop Relat Res. 2018;476:2027-2036.
    19. Hung M, Saltzman CL, Voss MW, Bounsanga J, Kendall R, Spiker R, Lawrence B, Brodke D. Responsiveness of the PROMIS, NDI and ODI Instruments in Patients with Spinal Disorders. Spine J. 2019;19:34-40.
    20. Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol. 2000;53:459-468.
    21. Mahmood B, Chongshu C, Qiu X, Messing S, Hammert WC. Comparison of the Michigan Hand Outcomes Questionnaire, Boston Carpal Tunnel Questionnaire, and PROMIS Instruments in Carpal Tunnel Syndrome. J Hand Surg Am. 2018. [Epub ahead of print].
    22. Nilsdotter AK, Lohmander LS, Klassbo M, Roos EM. Hip disability and osteoarthritis outcome score (HOOS)--validity and responsiveness in total hip replacement. BMC Musculoskelet Disord. 2003;4:10.
    23. Noble PC, Dwyer M, Brekke A. Commonalities, differences, and challenges with patient-derived outcome measurement tools: function/activity scales. Clin Orthop Relat Res. 2013;471:3457-3465.
    24. Overbeek CL, Nota SP, Jayakumar P, Hageman MG, Ring D. The PROMIS physical function correlates with the QuickDASH in patients with upper extremity illness. Clin Orthop Relat Res. 2015;473:311-317.
    25. Papuga MO, Mesfin A, Molinari R, Rubery PT. Correlation of PROMIS Physical Function and Pain CAT Instruments With Oswestry Disability Index and Neck Disability Index in Spine Patients. Spine (Phila Pa 1976). 2016;41:1153-1159.
    26. Porter ME. What Is Value in Health Care? N Engl J Med.2010;363:2477-2481.
    27. Pourmemari MH, Heliovaara M, Viikari-Juntura E, Shiri R. Carpal tunnel release: Lifetime prevalence, annual incidence, and risk factors. Muscle Nerve. 2018;58:497-502.
    28. Purvis TE, Neuman BJ, Riley LH 3rd, Skolasky RL. Discriminant Ability, Concurrent Validity, and Responsiveness of PROMIS Health Domains Among Patients With Lumbar Degenerative Disease Undergoing Decompression With or Without Arthrodesis. Spine (Phila Pa 1976). 2018;43:1512-1520.
    29. Riddle DL, Jensen MP, Ang D, Slover J, Perera R, Dumenci L. Do Pain Coping and Pain Beliefs Associate With Outcome Measures Before Knee Arthroplasty in Patients Who Catastrophize About Pain? A Cross-sectional Analysis From a Randomized Clinical Trial. Clin Orthop Relat Res. 2018;476:778-786.
    30. Roos EM, Roos HP, Lohmander LS, Ekdahl C, Beynnon BD. Knee Injury and Osteoarthritis Outcome Score (KOOS)--development of a self-administered outcome measure. J Orthop Sports Phys Ther. 1998;28:88-96.
    31. Saad MA, Kassam HF, Suriani RJ Jr., Pan SD, Blaine TA, Kovacevic D. Performance of PROMIS Global-10 compared with legacy instruments in patients with shoulder arthritis. J Shoulder Elbow Surg. 2018;27:2249-2256.
    32. Scott EJ, Westermann R, Glass NA, Hettrich C, Wolf BR, Bollier MJ. Performance of the PROMIS in Patients After Anterior Cruciate Ligament Reconstruction. Orthop J Sports Med. 2018;6(5).
    33. Sullivan GM, Feinn R. Using Effect Size-or Why the P Value Is Not Enough. J Grad Med Educ. 2012;4:279-282.
    34. Vernon H, Mior S. The Neck Disability Index: a study of reliability and validity. J Manipulative Physiol Ther. 1991;14:409-415.
    35. Yaffe M, Goyal N, Kokmeyer D, Merrell GA. The use of an iPad to collect patient-reported functional outcome measures in hand surgery. Hand (N Y). 2015;10:522-528.
    © 2019 by the Association of Bone and Joint Surgeons