In 2016, 69.8 million adults in the United States (US) reported some difficulty in functioning and another 20.2 million reported a great deal of difficulty in functioning or could not function at all.1 If health is truly “a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity,”2 then “measurements must better capture outcomes that people consider central components of well-being.”3 To improve the quality of patients’ lives and well-being, there must be more focus on the use of functional outcomes/measurements – to capture the impact of the condition and treatment plan on patients’ lives, including their participation in work, home life, and society. These are major elements of a patient-centered model. Evaluating and optimizing patient function should be a central focus of all clinical encounters not only to improve an individual patient's care, but also to improve population health,4 thus reducing the economic burden on both individuals and society as a whole due to functional loss.
The importance of helping patients achieve functional improvement has become embodied in federal US statues. For example, the US Social Security Act (42 U.S.C. 1396d(a)13) regards improvements in functioning as a definitional component of treatment, defining the broad treatment term “medical assistance” in part as: “[Medical assistance is] … any medical or remedial services for the maximum reduction of physical or mental disability and restoration of an individual to the best possible functional level.”5 Federal statues have also recognized that determinations regarding change in functioning are not possible without methods of assessing function. To this end, a section of the Affordable Care Act entitled “Prioritization in the Development of Quality Measures” states: “…[The] Secretary shall give priority to the development of quality measures that allow the assessment of – (A) health outcomes and functional status of patients….”6 This need for assessing function has been further documented in whitepapers published by the Department of Health and Human Services,7 Centers for Disease Control and Prevention (CDC),7 Institute of Medicine,8 and the Substance Abuse and Mental Health Services Administration.9 The centrality of assessing functional gains resulting from medical treatment also has growing acceptance in scientific literature and has been noted in standards for outcome measures recommended for use in surgical trials,10 and in guidelines including those for chronic pain,11 substance abuse,7,9 and mental health disorders.12–14 While these documents advocate for the importance of assessing functional gains, they do not suggest a detailed plan for how to do so.
People seek medical attention for periodic health examinations and when an illness or injury disrupts their ability to carry on as usual, either due to the actual impact of the condition or due to concerns about it. However, the US health care system focuses on using diagnostic tests to attempt to measure objective signs of disease that would explain patient symptoms, while paying little attention to their ability to pursue a quality lifestyle. This belief in the power of objective data has led to the establishment of a complicated system of outcome metrics involving numbers representing objective data of interest. As a result, laboratory values may receive more outcome emphasis than patients desire to improve their function in specific areas. However, patients do not generally initiate medical treatment to change their lab values, rather they seek care to address medical or psychological conditions that affect their ability to function in life.
The health care system would benefit greatly by aligning the goals of population health with a patient-centered model and related tools.15 Current medical care places little value on the quality of life or functional measures.16 Clinical visits generally focus on disease identification and treatment.17 The current approach may result in chasing diagnoses and providing unnecessary and/or harmful care (aka “low-value care”).18 The medical literature has shown improved health care outcomes and reduced financial burden when providers focus not only on patient treatment from a disease perspective, but also from a quality of life and function perspective (ie, emotional, mental, spiritual, social, and financial characteristics).4
Focusing on short-term outcomes in health care does not assist in creating a healthier population nor does it support appropriate allocation of financial resources. Focusing on measuring, documenting, and improving function is an essential next step toward improving the health and lives of patients. To address this issue, the American College of Occupational and Environmental Medicine (ACOEM) and the American Psychological Association (APA) in collaboration with the National Institute for Occupational Safety and Health (NIOSH) convened a Patient-reported Functional Status Symposium on June 28, 2019, at ACOEM headquarters. The Symposium brought together key stakeholders, including medical/health specialty societies, insurers, employers, and electronic health record (EHR) developers to discuss and promote the use of functional patient-reported outcomes measures (fPROMs) in all medical practice settings. Working together as a group representing multiple specialties and health care organizations can more powerfully promote efficient and effective health care than by acting alone.
Disability, impairment, and health care costs are increasing and unsustainable in the US. People are living longer and, with the rise in chronic diseases, living with more problems impacting function in their activities of daily living (ADLs).19 Chronic conditions include any physical or mental health condition that lasts more than 1 year and limits ability or requires ongoing treatment (eg, high cholesterol, high blood pressure, anxiety, arthritis, heart disease, diabetes, back and neck pain, and depression).20 As of 2014, 60% of the US adult population (150 million individuals) had at least one chronic medical condition; 42% had multiple conditions; and 12% had five or more chronic health conditions.21 The number of Social Security Disability Insurance beneficiaries has risen from 1,812,786 in 1970 to 9,919,094 in 2018, driven predominantly by an increase in the number of disabled workers.22 US health care costs are projected to increase at an average rate of 5.5% per year from 2018 to 2027, and reach nearly $6.0 trillion per year by 2027.23
It is essential to recognize that physician/clinical services constitute only about 20% of the total health care dollars spent in the US.24,25 However, medical clinicians are in control of virtually all the additional health care dollars spent on hospitalizations, surgeries, prescriptions, and durable medical equipment. Reducing health care costs requires managing efficient and effective use of health care resources. To assess effectiveness, clinicians need data that can help them better understand which activities and services will have the greatest impact on outcomes without causing harm.26,27 Clinicians are also frequently involved in decisions impacting return to work (RTW), which in turn can greatly impact disability costs. Therefore, changing how clinicians interact with and treat patients, will have the largest impact on both national health care and national disability costs.
In 2001, the World Health Organization (WHO), recognizing that function incorporates both biological health and lived health—how health affects activities of daily life,19 endorsed another health indicator, the International Classification of Functioning, Disability, and Health (ICF). This ICF model examines what the patient does and his or her participation in activities versus only reviewing how the patient has performed on physical assessments.28,29
Functional outcome measures quantify how well patients can perform activities in their day-to-day lives. Patient-reported outcome measures (PROMS) are also used extensively in scientific peer-reviewed literature to determine a successful outcome.16,30–32 Current medical literature supports the use of patient-reported functional status assessments as an important outcome measure. In addition, the majority of studies used to establish recommendations for evidence in evidence-based guidelines include patient-reported function measures as either the primary or secondary outcome in the study. Some studies have found patient-specific functional scales are better at predicting response to low back treatment than physical performance measures.33 Another study found that self-reported data on swelling and morning stiffness showed comparable efficacy to magnetic resonance imaging (MRI) in predicting radiographic knee osteoarthritis.34
Several medical specialties already use functional assessments as part of routine care including occupational medicine, physical medicine and rehabilitation, physical therapy, occupational therapy, pediatrics, geriatrics,35 sports medicine, and to some extent cardiology, pulmonary medicine, and psychology. Despite this, multidisciplinary knowledge of, and access to, the use of functional status assessment tools and protocols is limited. These fPROMs, referred to as functional status assessment tools, are validated measurements/medical care outcomes that health care providers can include in their patient care “toolbox.” Currently, many incentives such as value-based payment are connected to activities identified by evidence-based medicine. This includes process measures such as eye examinations and laboratory measures such as A1C levels for people with a diagnosis of diabetes.
While these outcomes are important, they may not be the most effective means to guide care toward achieving the highest quality of life for many patients. Health care providers are missing important information about their patients if they are not including fPROMs when treatment goals are established and as continual assessment tools to monitor treatment progress. Opportunities to prevent disability are thus lost as patients are shuffled through a system in which existing, effective tools are underutilized.
FUNCTIONAL STATUS TOOLS
The framework for measuring function includes assessing both higher levels of physical, cognitive, and psychosocial functioning along with basic-level ADLs and instrumental activities of daily living (IADLs). Table 1 provides examples of ADLs and IADLs though more detailed lists are available.36 There are many functional assessment tools available that include these elements. Specific tools, developed for different purposes, may be most useful for initial screening, treatment follow up, treatment outcomes assessment, ability to work, general capacity to perform ADLs and/or IADLs, and tracking functional status changes over time. Some tools are more useful for populations with specific physical or mental impairments (eg, rheumatoid arthritis patients,37 while others are designed to apply to the general adult population who can live independently,37 the elderly,38 and others).
Studies have shown that mental health (eg, depression) and behavioral issues limit the performance of ADLs and IADLs.42,43 Mental health is inextricably linked to function. It is important to identify any mental health barriers that would affect a patient's functional progress.44 Therefore, when choosing patient-reported functional tools, these tools should include elements that address psychological function or a separate tool should be used to facilitate earlier identification of psychological function that are likely to affect quality of life outcomes.
Clinicians are frequently involved in decisions impacting return to employment. While the primary goal of fPROMs is not aimed at establishing work functional ability, clinicians can assist in assessing ability to work. Decreased independence in many ADLs and IADLs can have vast implications for work including getting ready for work, going to and from work, interacting with coworkers, and being able to complete work tasks. Work is an essential part of quality of life for adults, whether that work is related to salary production, volunteerism, or family assistance. The establishment of purposeful activities improves quality of life and employment is beneficial for health, particularly for depression and general mental health.45
ADVANTAGES OF USING PATIENT-REPORTED FUNCTIONAL OUTCOMES (fPROMs)
There are several advantages to implementing fPROMs into medical practice (Fig. 1). The use of a patient-reported functional status tool can facilitate a more meaningful discussion, improve communication and decision making between physicians and patients,46 and improve patients’ satisfaction with their care.47–49 One systematic review showed that a “collection of PROs with feedback was associated with a strong evidence of enhancing patient-provider communication and improved patient satisfaction and moderate strength of evidence for monitoring treatment response and detecting unrecognized problems.”49 It can demonstrate areas of concern or functional deficit efficiently, thus providing patient-reported information reflecting the quality-of-life status and guiding the meeting toward the actual issues confronting the patient. Patients are more actively engaged in their treatment and functional recovery plan. Clinicians may also be able to more quickly identify mental health complications, physical complications affecting recovery, or social issues interfering with care. Such tools can assist physicians by helping to set patient expectations, following effectiveness of treatment, and increasing RTW.50 They serve as a record of objective data and, when administered routinely, patient concerns will not go unnoticed.51 Treatment goals will be clearly oriented toward the best functional outcomes for the patient as determined by the collaboration between the patient and clinician, and there may be a higher overall satisfaction both for the clinician and the patient regarding their visit. If providers feel more satisfied with the communication they have with their patients, it may also help decrease physician burnout.52,53
fPROMs can provide a record of improvement, decline, or document ups and downs in conditions over time. If these measures are available in readily graphable formats and easy to visualize, they can be efficiently incorporated into practice and used to validate care, look at treatment value for the patient, and more directly assess patient concerns. Providers can identify earlier patients with delayed recovery or patients receiving medical care that is not necessarily beneficial. If the clinician identifies a flat line in the patient's progress, this may uncover issues earlier that are interfering with their care. In one study, “the ability to perform daily activities, pain, and general health were important predictors of sickness absence >2 weeks.”54 Earlier identification of these functional deficits may increase the likelihood that the clinician could improve the patient's progress in treatment. fPROMS can serve as a trigger for discussing the important role of the patient in achieving improved function and help manage the patient's expectations. In the end, it can improve patient care satisfaction.49 Results over time can provide an educational/motivational tool for patients.
fPROMs can also support communication among health care providers in different specialties and direct more appropriate referrals to needed providers. fPROMs would promote coordination of interdisciplinary care as all providers could see the progress of the patient using the same tools. “Measures associated with best practices and outcomes for integrated care can increase accountability across health care settings, diminish disincentives to serve and treat these complex patients, broaden dissemination of research-proven models that improve patient outcomes, and enhance the efficiency of the health care system as a whole.”55 The consistent use of scientifically validated tools can provide documentation of severity of disease that is easily understandable to a wider audience than is currently available from clinical descriptions only.
Many systems, such as workers’ compensation (WC), require functional improvement documentation to support continued treatment authorization. fPROM metrics showing slow improvement may support authorization for additional care. fPROMs are also useful to chart disease progression or the occurrence of a patient's good and bad days in the cases of relapsing-remitting diseases, such as multiple sclerosis. If a patient is showing no improvement, the results may justify the need for further diagnostic testing or a change in treatment. In addition, fPROMs may support a finding of Maximum Medical Improvement (MMI). Functional measures support evidence-based care where improvement in function can provide medical evidence that a given line of treatment is necessary to maintain or improve function (ie, if treatment is withdrawn, function deteriorates). For example, the WC system in Colorado uses this approach. Clinicians can demonstrate to the payer, using validated fPROMs results, that the patient is not improving and thus obtain authorization for more extensive or different treatment. fPROMs can also be useful in utilization review (UR) because it may provide support for a physician ordering a particular treatment, decreasing the burden of UR on the clinician.
fPROMs can assist in establishing level of impairment or answering RTW questions by identifying areas in which the patient may need accommodation or other assistance. Tracking functional status and associated treatment over time can also be used to evaluate outcomes of treatment and programs.56
CURRENT EXAMPLES RELATED TO ASSESSING FUNCTION
There are several examples of programs that are currently assessing a patient's function. These include:
- There are some evidence-based treatment guidelines that focus primarily on the functional outcome of the patient, such as requirements for assessing whether opioid use is medically indicated.11,57 The CDC opioid guidelines recommend that health care providers address the risk for abuse, mental health problems, drug screening, the opioid dose, and what kinds of activities the patient can perform safely.58 The provider is always required to include a functional assessment of the patient to demonstrate compliance with any opioid guideline. Providers need to record function if they are going to continue or change treatment. Currently, there are software platforms that can be added to EHRs to assist with prescribing and monitoring patients on opioids. They include pain and function scores.
- The concept of functional outcome tools fits within NIOSH's Total Worker Health® Model which includes policies, programs, and practices that integrate protection from work-related safety and health hazards with promotion of injury and illness prevention efforts to advance worker well-being. This concept specifically relates to the worker level outcome category which includes health-related quality of life and functional status as sample constructs. The idea of considering the patient's emotional, mental, spiritual, and social characteristics is consistent with the Total Worker Health® Model.59,60
- The Centers for Medicare and Medicaid Services’ (CMS) Merit-based Incentive Payment System (MIPS) includes one measure related to this—Functional Outcome Assessment—which looks at the percentage of visits for adult patients that include documentation of a functional outcome assessment using a standardized tool on the date of the encounter and documentation of a care plan based on identified functional deficiencies.61 CMS also began the Meaningful Measures Initiative in 2017 which focuses health care quality efforts on what is important to patients, families, and caregivers including making informed decisions about their care, aligning care with the patient's goals and preferences, and improving quality of life and patient outcomes.62 For example, one meaningful measure includes patient-reported functional outcomes indicating that “slightly more than 15% of adults report physical functioning difficulties,” and urges the clinician to “improve or maintain patient's quality of life by addressing physical functioning that affects their ability to undertake daily activities most important to them.”63
- Health Level 7 (HL7), an American National Standards Institute (ANSI)-accredited standard developing organization, includes one measure that directly addresses function in its new proposed EHR domain of Work and Health Functional Profile—CP 3.2: “Provide the ability to capture mood, behavior and daily functioning as structured or non-structured data.” However, this functional profile is not ready for universal use and does not yet support interoperability across EHR systems. The work would require clear definitions of functional measures of interest, value sets that capture those definitions, and then integration with interoperability standards.
- The Integrated Care Resource Center (ICRC) offered by CMS helps states develop integrated care programs that coordinate medical, behavioral health, long-term services, and supports for individuals who are dually eligible for Medicare and Medicaid where hopefully all providers will be using the same type of tools to examine patients.
- Colorado's WC model reimburses its providers when they use fPROMs. Following training, participating providers choose a tool appropriate for their patient based on a list of validated fPROMs and administer a validated patient-reported mental health screening tool. Providers are paid for the initial screening and then reimbursed intermittently when they use the same patient report tool to manage care. As an example, there is a software tool available which records the measures from the patient, graphs them, and interprets the scoring for the provider with red flags if certain results or changes occur. The results are a resource to guide the discussion with the patient on what issues he or she may be experiencing and quickly identify any barriers to effective treatment. The biggest obstacle to the use of this model is the lack of inclusion of the fPROM in most EHRs. An informal survey of users revealed that many providers began using the fPROMs on their patients, even if they were not getting paid, because they found it assisted in patient communication and treatment planning. There were a few incidences where complications from surgery were identified earlier because patients initially improved but their function later flat-lined. Providers explored the concept of medical complications earlier than they otherwise might have done and resolved the problems. (Per communication with the Colorado Division of Workers’ Compensation).
- In the State of Washington's WC system there is a RTW program featuring Centers for Occupational Health and Education (COHE). COHEs are independent community-based entities that are affiliated with health care delivery organizations. The state contracts with COHEs to work with providers, employers, and workers after an injury. COHE providers develop an activity prescription, discuss function with the patient, and talk about function on the phone with health care service coordinators. The health care service coordinators then work with employers and supervisors to institute the RTW plan. This began with a few pilot studies in individual facilities. The state decided to continue the program due to the positive impact on health outcomes and case costs.64 “By coordinating care, training providers, coordinating communication among the different parties, Washington has been successful in improving injured worker outcomes, reducing the risk of long-term disability and possible premature exit from the workforce, and promoting the workers’ RTW.”65 Physicians are reducing the impact of WC on people's lives and reduced entry to Social Security disability by 30%.66
- After extensive testing, Kaiser Permanente incorporated a RTW model, based on functional restrictions, into its EHR. They began by identifying how many of their physicians of all specialties spent time on “work notes” (RTW, restrictions, and limitations). California providers complete approximately 50,000 state disability insurance forms per month. The new model allows Kaiser physicians to use their computer screen to select a box for functional restrictions and then check all the ones relevant to the patient. After this procedure was implemented in the EHR, Kaiser Permanente was able to reduce the labor time to complete these forms from 20 to 2 minutes resulting in significant improvement in efficiency.67,68 In addition, supporting the development and communication of evidence-based activity prescriptions could help improve the patients’ quality of life by preventing unnecessary work disability.67
- For several years, the Veterans Health Administration (VA) has implemented the use of routine measurement of functional status in primary care clinics—either at their Patient Aligned Care Team (PACT) or Geriatric (GeriPACT) clinics. The VA encouraged clinics to measure functional status among older adults but did not make it mandatory. The clinics used a variety of approaches to collect functional status data on patients—routine collection via a standardized clinical reminder, routine collection via other methods (eg, a pre-visit telephone screen or electronic note), or ad hoc (no standard or routine approach). The inconsistency among the clinics showed the difficulties in developing a routine and standardized approach to measuring functional status. Current clinic processes offer some observations into features that might need to be considered when choosing a functional status assessment instrument. For example, the instrument might need to be validated for telephone and in-person use and include categorical responses as well as an area for free text to document any additional notes about the patient. The VA is continuing to study the barriers and facilitators to assist them with a design and implementation of a standardized tool for assessing function across primary care settings.35
- Many clinical data registries require the use of some type of functional outcome (eg, some registries use a diagnosis-specific functional outcome while some use a more global functional outcome). A Qualified Clinical Data Registry (QCDR) is approved by CMS to develop its own quality measures so that providers who report through the registry could get credit for Merit-Based Incentive Payment System (MIPS) using the measures developed. The measures are approved by CMS after review to ensure they meet certain standards. The developer of the measures is responsible for implementing and testing those measures and providing performance data on those measures. For example, the American Psychological Association (APA) Mental and Behavioral Health Registry (MBHR), a CMS-approved QCDR since 2018, provides a cloud-based platform to report quality measures to the MIPS. As a QCDR, APA can define and propose to CMS the quality performance measures that are the most meaningful and appropriate for mental and behavioral health providers and patients. This work is being led by an appointed advisory committee with expertise in measurement-based care. The most recent measures accepted in 2019 focused on transdiagnostic (intended to be relevant to any patient regardless of mental health diagnosis or setting) and functional aspects of care, and included measures of pain interference, sleep quality, and social role functioning. MBHR is the only registry to include these specific quality measures.69
BARRIERS TO USING PATIENT-REPORTED FUNCTIONAL OUTCOMES (fPROMs)
There are several barriers that need to be addressed before fPROMs can be widely implemented in clinical practice. First, these measures do not exist in most EHRs, and fPROM responses should be integrated into EHRs in real-time to save time and to provide better clinical care to the patient.4 Clinicians may view the addition of fPROMs as an administrative burden that adds to their workload rather than shortening it. However, if the fPROMs were added to the EHRs, and presented like lab values, this may improve ease of use. Clinicians would then be able to view a graph that would alert them to problems needing to be addressed.
At the time of this writing, there is no nationally-accepted payment incentive in place for clinicians to use fPROMs even though there are multiple ways that a provider could be compensated or rewarded for tracking fPROMs (eg, a provider codes for it or health care systems choose to reward providers for documenting these measures as in Colorado and Washington). In the absence of reimbursement for the use of fPROMs, payers could offer other incentives for their use, such as decreased UR. If functional benefit is shown from a specific care pattern, formal UR may not be necessary. UR is costly both in money for insurers and time for providers and is driven by systems that require providers to perform certain tests and documentation before they can order additional tests or procedures. Large self-insurers could add this functional status measure as one criterion for entering a quality network or as a bonus to providers. Not having to do UR is not direct compensation but is a reduction of one time-consuming and burdensome aspect of practice that would likely decrease costs.
The Symposium Panel was concerned about the idea of adopting fPROMs directly from the current value outcome measure as presented with the CMS Merit-based Incentive Payment System (MIPS) measure for all visits—Functional Outcome Assessment—which looks at the percentage of visits for adult patients that include documentation of a functional outcome assessment using a standardized tool on the date of the encounter and documentation of a care plan based on identified functional deficiencies.61 The Functional Outcome Assessment measures dictates that a tool be given at every visit, which may or may not be feasible for certain providers and settings. However, the Panel agreed that the patient's functional status should determine the frequency of reassessing patient-reported functional measures.
Several barriers from the patient's perspective that might interfere with the use of fPROMs include additional response burden,48 concerns with confidentiality, and patient's symptoms (eg, cognitive deficits) or literacy/language interfering with the patient being capable of completing the tool.70 Subjective tools may also offer the opportunity for self-reporting bias. In addition, issues of fraud and abuse of subjective tools could be used to magnify symptoms of pain or function.71–75 Therefore, appropriate safeguards must be put into place to avoid those pitfalls. Clinician education on the appropriate use and interpretation of the tools can provide additional safeguards against fraud and abuse.
RECOMMENDATIONS REGARDING REQUIREMENTS FOR PATIENT-REPORTED FUNCTIONAL OUTCOMES
The consensus of the Panel was that a consistent general patient-reported functional tool could be useful but may not be appropriate for use in every type of patient encounter. The Panel did not determine which population should be assessed, though several potential types of patients were suggested (eg, people with a chronic problem, people who are not getting better, any child who is unable to go to school, any working person whose job is interrupted, people who have conditions that typically have a broad range of impact on people's lives, etc). The patient self-reported tool should be used with caution in patients with cognitive impairments including those from brain injury or stroke. This population may have greater difficulty reliably completing the tool. There was also no determination as to how often the patient should be checked during treatment with a patient-reported function tool. There was agreement that it should at least be done at the beginning of treatment and some periods of time in between but no specific timeline was identified. Ideally, the general functional status tool should meet the following criteria:
- Be of a 4th grade reading level (current patient education standard; the group did acknowledge that there still will be problems with literacy and language);
- Have an equivalent version in the Spanish language;
- Be a measure completed by the patient and not the provider;
- Be possible to administer on a cell phone/mobile device;
- Provide graphable results that plot change over time (similar to a lab value);
- Have a delivery system that allows viewing results at the time of the visit immediately available to the provider;
- Either be formally standardized or have published scientific evidence of validity, reliability, norms or adequate basis for any cutoff scores, and evidence that the measure was developed with a diverse population;
- Include items on mood, behavior, ADLs, IADLs, and work and/or school (preferably);
- Be brief in length (12 or less questions);
- Contain items useful for follow-up;
- Be easy for the provider to understand and use; and
- Be available for use without additional monetary charge.
There are several pathways to promote the implementation of fPROMs into all medical practice settings.
- Urge health care organizations to implement fPROMs into their own EHRs and to reward providers for their use to improve quality care.
- Encourage providers to use fPROMs through clinical registries, most of which require fPROMs as a portion of the results reported. Many specialty societies allow providers who are not members of their organizations to use the clinical registries.
- Explore how fPROMs may be used for patients who are illiterate or those who are uncomfortable with how the questions are presented in order to develop appropriate assistance for patient completion.
- Convene a work group to examine current available measures and consider which general status functional tool might have acceptance among a variety of providers and specialties. The tool should address the criteria listed above. This group could also consider the use of a screening tool (eg, a single question on what impact a patient's condition has on his/her everyday life and function) to triage patients who would benefit from completing the longer tool.
- Collaborate with HL7 to organize a work group to establish interoperability standards for fPROMs in its proposed EHR domain of Work and Health Functional Profile. Though this is a complex task, it is more likely to guarantee entry into a greater number of EHRs.
- Develop education and training programs on the appropriate implementation and interpretation of fPROMs into clinical practice. Training will be essential to educate providers since fPROM use is not included in current medical education, except for education regarding opioid protocols.
- Encourage further research to develop short tools and/or computer adaptive tests that include the requirements outlined above. While fPROMs have been used for evidence and validated for outcomes, there is no research evidence for consistent use by providers to change or adjust treatment. Therefore, there is a need for research to show that if a provider uses fPROMs at the beginning, middle, and end of treatment, it changes outcomes for the patient (ie, return on investment, improved clinical outcomes).
Providers should be documenting each patient's report of what activities they can and cannot do, and then determine treatment and outcome goals related to functioning. This may include adjusting the patient's medication that is interfering with function or recommending changes to the patient's daily activities. The use of patient-reported assessments of function, as reported in peer-reviewed literature, can lead to more effective, efficient, patient-centered care that reduces impairment and work disability and results in an improved culture of health for patients and society.
Symposium Participants (in-person or online) Government
National Institute for Occupational Safety and Health
Medical/Health Specialty Societies
American College of Occupational and Environmental Medicine
American Psychological Association
American College of Physicians
American Academy of Physical Medicine & Rehabilitation
American Academy of Orthopaedic Surgeons
American Physical Therapy Association
California Academy of Family Physicians
AMA Integrated Health Model Initiative
American Occupational Therapy Association
American Association of Occupational Health Nurses
The Joint Commission
Electronic Health Record Developers
APPENDIX 1: SOCIETIES ACTIVITIES AROUND PATIENT-REPORTED FUNCTIONAL TOOLS
American College of Occupational and Environmental Medicine (ACOEM)
ACOEM published a paper calling for more function-based treatment,16 advocated for the advantages of this treatment approach to Congress and the Social Security Administration and added new content to its function-based clinical guidelines that can be used by specialties outside occupational health. It has also called for changes in the nation's WC system intended to promote better quality care—an arena where functional treatment could have tremendous impact, and where primary care physicians are key stakeholders. In addition, ACOEM worked with the National Institute for Occupational Safety and Health (NIOSH) to develop the knowledge to support a clinical decision support (CDS) tool to provide guidance to primary care practitioners regarding activity associated with acute LBP. The recommendation suggested embedding the proposed tool in electronic health record (EHR) systems, expanding its use to other conditions and use to support both nonwork-related and work-related disability.67 A tool to support this approach awaits development.
American Psychological Association (APA)
The APA has a CMS-approved qualified clinical data registry (QCDR) named the Mental and Behavioral Health Registry (MBHR). The MBHR is overseen by an APA governance-appointed advisory committee consisting of nine experts in the field, whose mission is to identify quality measures for mental and behavioral health and substance use. In collaboration with APA staff, the advisory committee identify measurement needs, review the literature, and identify psychometrically sound measurement tools, with attention to measures with evidence of reliability and validity for the intended purpose and population to be studied.69
Based on a gap analysis, the advisory committee selected anxiety disorders as the domain most in need of measure development and selected the General Anxiety Disorder-7 (GAD-7) for this purpose. Additional measures identified were: (a) pain interference response utilizing Patient-Reported Outcomes Measurement Information System (PROMIS)76; (b) sleep quality screening and outcome response at 3 months utilizing the Insomnia Severity Index77; (c) social role functioning outcome utilizing PROMIS (Ability to Participate in Social Roles and Activities—Short Form 8a); and (d) screening and monitoring for psychosocial problems among children and youth utilizing the Pediatric Symptom Checklist.78
The advisory committee identified future measurement priorities, prioritizing the identification of transdiagnostic measures (eg, measures relevant to all patients regardless of psychiatric diagnosis). The advisory committee prioritized developing functioning measures for mental and behavioral health care.69 The advisory committee also advocated for more rigorous measurement development standards.69 These recommendations included the utilization of methods such as item response theory, and basing determinations on standards such as sample invariant reliability, scale invariance, measurement bandwidth, and item and scale bias across diverse subgroups (race, ethnicity, age, language, education, etc).76,79
American Academy of Physical Medicine & Rehabilitation (AAPMR)
AAPM&R members and staff have been engaged in a number of national initiatives related to defining and measuring functional status. As the primary medical specialty focusing on function, AAPM&R strives to move healthcare to measure longitudinal functional status outcomes. The AAPM&R has made three distinct advances in evaluating patient reported functional tools—one related to post-acute care data standardization, one related to developing clinical quality measures, and another related to standardizing patient reported outcomes for AAPM&R's clinical data registry.
- In 2016, the AAPM&R Board of Governors adopted recommendations on data standardization for post-acute care settings in light of the IMPACT Act of 2014 and many policy discussions on a unified post-acute care payment system. The AAPM&R's recommendations state self-care and mobility are two priority data elements to measure functional status.
- AAPM&R developed, via a multi-disciplinary panel, five clinical quality measures. These measures were developed out of the need for measures in the Post-Acute Care space that focus on functional status of stroke and traumatic brain injury patients.
- AAPM&R launched a clinical data registry that is evolving to collect patient reported outcomes. AAPM&R identified PROMIS® (Patient-Reported Outcomes Measurement Information System) as the ideal patient reported outcomes measurement tool to incorporate into its registry. PROMIS® is a set of person-centered measures that evaluates and monitors physical, mental, and social health in adults and children. It can be used with the general population and with individuals living with chronic conditions. PROMIS® has many formats and versions. AAPM&R is closely looking at the PROMIS® 29 instrument for inclusion into its registry. It must be noted that for each clinical population that physical medicine and rehabilitation physicians treat, there may be other patient reported outcome tools that are pertinent to collect. AAPM&R is currently working on defining those patient-reported outcomes, beyond the PROMIS® instrument. For example, the AAPM&R registry will begin to collect the Visual Analog Scale for Back and Leg Pain for the low back pain population entered into the registry.
American Occupational Therapy Association (AOTA)
AOTA staff and members have been involved with a number of activities related to measuring patient outcomes with particular interest on the client's (individual, group, or population) ability to participate in everyday activities and occupations that he/she needs and wants to do. Examples include:
- Many occupational therapy practitioners use the Canadian Occupational Performance Measure (COPM) as a patient-reported outcome tool. COPM is relatively unique in measurement as it allows the patient to set individualized goals.
- In 2014, published the 4th edition of Occupational Therapy Assessment Tools which is an annotated index with nearly 600 instruments reviewed including self-report rating scales and questionnaires. An assessment profile for each tool provides useful information about format, purpose, population, brief description, interpretation, reliability and validity, source, additional references, etc.
- In 2016, AOTA's Mental Health Special Interest Section (MHSIS) sponsored the development of a resource of Occupational Therapy Service Outcome Measures for Certified Community Behavioral Health Centers (CCBHCs) which is a framework for occupational therapy service with rationale for outcome measures selection and listing of occupational therapy outcome measure tools.
- As payment systems shift from volume to value, AOTA's Quality Team conducted a national survey to understand which assessment tools, instruments, and measures occupational therapy practitioners are using most frequently. Using these responses, a Quality Toolkit identifies some of the most frequently used standardized tools across key areas of OT practice. Initial analysis identifies two patient-reported instruments: Lawton-Brody Instrumental Activities of Daily Living Scale and Canadian Occupational Performance Measure.
- AOTA's evidence-based practice project has sponsored systematic reviews on selected topics of interest to occupational therapy practice, education, and research. The results of the reviews are disseminated in a variety of formats and have included content on outcomes measures identified and utilized to assess results of the various interventions that met the reviews inclusion criteria. Commonly utilized outcome measures include the following: Disabilities of the Arm, Shoulder, and Hand (DASH) and QuickDASH, Functional Assessment Scale, Health Status Questionnaire (SF-36; SF-12), Work and Social Adjustment Scale, WHO Disability Assessment Schedule 2.0 (WHODAS 2.0), and WHOQOL-BREF.
- While not a specific PROM, occupational therapy practitioners use the occupational profile as a formalized way to gather information to inform patient (or client) centered care as a part of the evaluation. AOTA has developed the AOTA Occupational Profile Template to encourage the comprehensive and consistent completion of the profile.
American Association of Occupational Health Nurses (AAOHN)
AAOHN supports evidence-based practice guidelines in an effort to improve worker health. It is imperative that injured workers are treated from a global perspective that includes all of the WHO-ICF domains. While AAOHN has not tackled this issue directly, a current member has published an article in Workplace Health & Safety on “Occupational Functionality,” and the topic of functional outcomes will be a featured topic at the 2020 AAOHN National Conference. AAOHN is proud to support NIOSH as a Total Worker Health Affiliate and actively promoted their webinar on “Integrating Functional Outcomes with Clinical Measures.”
American College of Physicians (ACP)
The Performance Measurement Committee of the ACP reviews internal medicine-relevant performance measures across several clinical topic areas including functional outcomes. The Performance Measurement Committee supports the implementation of meaningful measures for functional status/PROs, but the benefit of tying these measures to payment is unclear; measures should include valid and reliable tools for assessing functional status/PROs that are consistent across care settings. The committee requests data on the efficacy, validity, and reliability of these measures prior to considering their candidacy for use beyond research and internal quality improvement (QI) purposes.
American Physical Therapy Association (APTA)
Movement is complex and purposeful behavior which results from the coordinated interaction of multiple body systems and is critical to human functioning and thriving. Whether measured in terms of mortality, morbidity, or quality of life, it is increasingly evident in the research that movement, as it relates to function, predicts future health. Preventing and mitigating functional decline through improved movement quality and capacity has therefore become a desired intervention among providers who recognize this. Physical therapists, who play a central role in preventing and reversing functional decline associated with movement-related impairment and pain, utilize a variety of data and clinical tests as part of their evaluation, treatment and consultation strategies. APTA supports clinical examination measures and population level screening tools that test gross movement, muscular and aerobic power, as well as balance and coordination; all which have been correlated to future health. Specifically, measurement of an individual's ability to transfer (which provides an understanding of their general mobility, strength, and even cardiorespiratory fitness when time-bound), stabilize, change direction and walk quickly (which provide an understanding of balance and coordination), are robust in their ability to predict health and yet grossly underutilized clinically.
Given that both the meaningfulness and ease of testing become paramount in determining health status when using dynamic movement testing, APTA suggests the following tests as a minimum battery: grip strength, in-office functional cardiorespiratory fitness tests (eg, chair stand test for time), gross movement testing (eg, floor to stand transfer or floor to shoulder lift test), and balance screening (ie, STEDI or 4-stage balance test). In addition, the APTA supports data collection and analysis efforts designed to enable clinicians to make meaningful insights related to function. The Physical Therapy Outcomes Registry (www.ptoutcomes.com) which to date includes several validated functional measures that utilize patient reported outcomes such as the Modified Low Back Disability Questionnaire (MDQ), the Neck Disability Index, the Lower Extremity Functional Scale (LEFS), the Disabilities of the Shoulder, Arm and Hand (DASH), and the Knee Outcome Score (KOS) can be a powerful tool. With plans to expand into collection of measures such as those listed above, physical therapists will be well positioned to serve society and the care team as an emphasis on function becomes central to practice.
1. CDC, National Center for Health Statistics. Table 42. Functional limitation among adults aged 18 and over, by selected characteristics: United States, selected years 2010-2016; 2017. Available at: https://www.cdc.gov/nchs/data/hus/2017/042.pdf
. Accessed May 20, 2020.
2. World Health OrganizationConstitution of the World Health Organization as adopted by the International Health Conference, New York, 19-22 June 1946; signed on 22 July 1946 by the representatives of 61 States (Official Records of the World Health Organization, no. 2, p. 100) and entered into force on 7 April 1948. In Grad FP. The Preamble of the Constitution of the World Health Organization. Bull WHO
3. vanderWeele T, McNeely E, Koh H. Reimagining heatlh -- flourishing. JAMA
4. Wagle N. Implementing patient-reported outcome measures. NEJM Catalyst; 2017. Available at: https://catalyst.nejm.org/implementing-proms-patient-reported-outcome-measures/
. Accessed June 3, 2019.
5. US Social Security Act. Available at: https://www.ssa.gov/OP_Home/ssact/ssact-toc.htm
. Accessed January 13, 2020.
6. Affordable Care Act. Available at: http://housedocs.house.gov/energycommerce/ppacacon.pdf
. Accessed May 1, 2020.
7. US Department of Health and Human Services. Pain Management Best Practices Inter-Agency Task Force Report: Updates, Gaps, Inconsistencies, and Recommendations. Washington, DC: DHHS; 2019. Available at: https://www.hhs.gov/ash/advisory-committees/pain/reports/index.html
. Accessed January 13, 2020.
8. Institute of, MedicineRelieving Pain in America: A Blueprint for Transforming Prevention, Care, Education and Research. Washington, DC:National Academies of Science; 2011.
9. Substance Abuse, Mental Health Services AdministrationFederal Guidelines for Opioid Treatment Programs. Rockville, MD:Department of Health and Human Services; 2015.
10. Dworkin RH, Turk DC, Farrar JT, et al. Core outcome measures for chronic pain clinical trials: IMMPACT recommendations. Pain
11. American College of Occupational and Environmental Medicine. ACOEM Practice Guidelines: Chronic Pain; 2017. Available at: https://www.mdguidelines.com
. Accessed May 17, 2020.
12. American Psychiatric Association. Practice Guideline for the Treatment of Patients with Major Depressive Disorder; 2010.
13. American Psychological Association. Clinical Practice Guideline for the Treatment of PTSD; 2017.
14. American College of Occupational and Environmental Medicine. ACOEM Practice Guidelines: Workplace Mental Health – Introduction; 2019. Available at: https://www.mdguidelines.com
. Accessed March 13, 2020.
16. Mueller K, Konicki D, Larson P, Hudson TW, Yarborough C. ACOEM Expert Panel on functional outcomes. Advancing value-based medicine: why integrating functional outcomes with clinical measures is critical to our health care future. J Occup Environ Med
17. Jonas W, Schoomaker E, Marzolf J, Gaudet T. Finding the cause of the crises: opioids, pain, suicide, obesity, and other “epidemics." NEJM Catalyst; 2019. Available at: https://catalyst.nejm.org/cause-crises-whole-health-whole-person/
. Accessed October 9, 2019.
18. Schwartz AL, Landon BE, Elshaug AG, Chernew ME, McWilliams JM. Measuring low-value care in Medicare. JAMA Intern Med
19. Stucki G, Bickenbach J, Melvin J. Strengthening rehabilitation in health systems worldwide by integrating information on functioning in national health information systems. Am J Phys Med Rehabil
20. Basu J, Avila R, Ricciardi R. Hospital readmission rates in U.S. states: are readmissions higher where more patients with multiple chronic conditions cluster? Health Serv Res
21. RAND Corporation. Multiple Chronic Conditions in the United States. Santa Monica, Calif.; 2017. Available at: https://www.rand.org/pubs/tools/TL221.html
. Accessed October 9, 2019.
22. Social Security Administration. Annual Statistical Report on the Social Security Disability Insurance Program. Washington, DC: SSA; 2018. Publ No. 13-11826. Available at: https://www.ssa.gov/policy/docs/statcomps/di_asr/2018/di_asr18.pdf
. Accessed May 20, 2020.
23. Centers for Medicare and Medicaid Services (CMS). National Health Expenditure Projections 2018-2027. Available at: https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/NationalHealthExpendData/Downloads/ForecastSummary.pdf
. Accessed September 13, 2019.
24. Maddox KEJ, Bauchner H, Fontanarosa PB. US health policy: 2020 and beyond. JAMA
25. Centers for Medicare & Medicaid Services. National Health Expenditure Fact Sheet; 2018. Available at: https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/NationalHealthExpendData/NHE-Fact-Sheet.html
. Accessed December 26, 2019.
26. Stowell C, Akerman C. Better value in health care requires focusing on outcomes. Harvard Bus Rev; 2015. Available at: https://hbr.org/2015/09/better-value-in-health-care-requires-focusing-on-outcomes
. Accessed May 1, 2020.
27. Berwick DM. Elusive waste: the Fermi Paradox in US Health Care. JAMA
28. Escorpizo R, Stucki G. Disability evaluation, social security, and the international classification of functioning, disability and health: the time is now. J Occup Environ Med
29. Fairbairn K, May K, Yang Y, Balasundar S, Hefford C, Abbott JH. Mapping Patient-Specific Functional Scale (PSFS) items to the International Classification of Functioning, Disability and Health (ICF). Phys Ther
30. Beckmann JT, Hung M, Bounsanga J, Wylie JD, Granger EK, Tashjian RZ. Psychometric evaluation of the PROMIS Physical Function Computerized Adaptive Test in comparison to the American Shoulder and Elbow Surgeons score and Simple Shoulder Test in patients with rotator cuff disease. J Shoulder Elbow Surg
31. Krebs EE, Gravely A, Nugent S, et al. Effect of opioid vs nonopioid medications on pain-related function in patients with chronic back pain or hip or knee osteoarthritis pain: the SPACE Randomized Clinical Trial. JAMA
32. Cheung EC, Moore LK, Flores SE, Lansdown DA, Feeley BT, Zhang AL. Correlation of PROMIS with orthopaedic patient-reported outcome measures. JBJS Rev
33. Pengel LH, Refshauge KM, Maher CG. Responsiveness of pain, disability, and physical impairment outcomes in patients with low back pain. Spine (Phila Pa 1976)
34. Landsmeer ML, Runhaar J, van Middelkoop M, et al. Abstract #44. Can we use synovitis-related clinical questions instead of synovitis on MRI to predict incident knee OA? Presented OARSI 2019 World Congress on Osteoarthritis; May 2-5; Toronto, Canada.
35. Spar M, Nicosia F, Steinman M, Brown R. Current approaches to measuring functional status among older adults in VA primary care clinics. Fed Pract
36. Rondinelli R. Guides to Evaluation of Permanent Impairment. 6th ed.Chicago, IL:American Medical Association; 2007.
37. Ward MM. Interpreting measurements of physical function in clinical trials. Ann Rheum Dis
2007; 66: (suppl): 32–34. iii.
38. Sherman SE, Reuben D. Measures of functional status in community-dwelling elders. J Gen Intern Med
39. Katz S. Assessing self-maintenance: activities of daily living, mobility, and instrumental activities of daily living. J Am Geriatr Soc
40. Lawton MP, Brody EM. Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist
41. Katz S, Ford A, Moskowitz R, Jackson B, Jaffe M. Studies of illness in the aged. The index of ADL: a standardized measure of biological and psychosocial function. JAMA
42. Meltzer H, Bebbington P, Brugha T, et al. Physical ill health, disability, dependence and depression: results from the 2007 national survey of psychiatric morbidity among adults in England. Disabil Health J
43. National Academies of Sciences Engineering MedicineFunctional Assessment for Adults with Disabilities. Washington, DC:The National Academies Press; 2019.
44. Tyser A, Gaffney C, Zhang C, Presson A. The association of patient satisfaction with pain, anxiety, and self-reported physical function. J Bone Joint Surg Am
45. van der Noordt M, Jzelenberg HI, Droomers M, Proper KI. Health effects of employment: a systematic review of prospective studies. Occup Environ Med
46. Kotronoulas G, Kearney N, Maguire R, et al. What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J CLin Oncol
47. Nelson EC, Eftimovska E, Lind C, Hager A, Wasson JH, Lindblad S. Patient reported outcome measures in practice. BMJ
48. Warsame R, D'Souza A. Patient reported outcomes have arrived: a practical overview for clinicians in using patient reported outcomes in oncology. Mayo Clin Proc
49. Chen J, Ou L, Hollis S. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res
50. Jurisic M, Bean M, Harbaugh J, et al. The personal physician's role in helping patients with medical conditions stay at work or return to work. J Occup Environ Med
51. Stojanovic MP, Higgins DM, Popescu A, Bogduk N. COMBI: a convenient tool for clinical outcome assessment in conventional practice. Pain Med
52. Lapointe R, Bhesania S, Tanner T, Peruri A, Mehta P. An innovative approach to improve communication and reduce physician stress and burnout in a university affiliated residency program. J Med Syst
53. Ratanawongsa N, Roter D, Beach MC, et al. Physician burnout and patient-physician communication during primary care encounters. J Gen Intern Med
54. Laaksonen M, Kaaria SM, Leino-Arjas P, Lahelma E. Different domains of health functioning as predictors of sickness absence--a prospective cohort study. Scand J Work Environ Health
55. Goldman ML, Spaeth-Rublee B, Pincus HA. Quality indicators for physical and behavioral health care integration. JAMA
56. Fulton-Kehoe D, Gluck J, Wu R, Mootz R, Wickizer TM, Franklin GM. Measuring work disability: what can administrative data tell us about patient outcomes? J Occup Environ Med
57. American College of Occupational and Environmental Medicine. ACOEM Practice Guidelines: Opioids; 2017. Available at: https://www.mdguidelines.com/acoem/disorders/opioids
. Accessed September 17, 2019.
58. Dowell D, Haegerich TM, Chou R. CDC Guideline for Prescribing Opioids for Chronic Pain - United States, 2016. MMWR Recomm Rep
59. Lee MP, Hudson H, Richards R, et al. Fundamentals of Total Worker Health Approaches: Essential Elements for Advancing Worker Safety, Health, and Well-Being DHHS (NIOSH) Publication No. 2017-112. Cincinnati, OH:Department of Health and Human Services; 2016.
60. Tamers SL, Chosewood LC, Childress A, Hudson H, Nigam J, Chang CC. Total Worker Health® 2014-2018: the novel approach to worker safety, health, and well-being evolves. Int J Environ Res Public Health
61. Center for Medicare and Medicaid Services. Functional Outcome Assessment. Available at: https://qpp.cms.gov/docs/QPP_quality_measure_specifications/Claims-Registry-Measures/2018_Measure_182_Claims.pdf
. Accessed December 26, 2019.
62. Centers for Medicare and Medicaid Services (CMS). The CMS Meaningful Measures Initiative: What It Means to Patients, Families, Clinicians and Providers; 2017. Available at: https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityInitiativesGenInfo/Downloads/CMS-Meaningful-Measures_What-It-Means-Fact-Sheet_508_2018-02-28.pdf
. Accessed January 31, 2020.
64. Ben-Shalom Y. Steps States Can Take to Help Workers Keep Their Jobs after Injury, Illness, or Disability. Mathematica Policy Research; 2016. Available at: https://www.dol.gov/odep/topics/pdf/SAW-RTW_PAP_States.pdf
. Accessed February 5, 2020.
65. Sung D, Lore H, Magill K, eds. Improving Occupational Healthcare Delivery to Support Workers’ Compensation Return to Work: Building on Evidence-Based Practice from Washington State's Centers of Occupational Health & Education (COHE) Experience. For SAW/RTW Policy Collaborative, U.S. Department of Labor, Office of Disability Employment Policy, IMPAQ International, LLC; 2017.
66. Wickizer TM, Franklin GM, Fulton-Kehoe D. Innovations in occupational health care delivery can prevent entry into permanent disability: 8-year follow-up of the Washington State Centers for Occupational Health and Education. Med Care
67. McLellan RK, Haas NS, Kownacki RP, Pransky GS, Talmage JB, Dreger M. Using electronic health records and clinical decision support to provide return-to-work guidance for primary care practitioners for patients with low back pain. J Occup Environ Med
69. Wright CV, Goodheart C, Bard D, et al. Promoting measurement-based care and quality measure development: the APA mental and behavioral health registry initiative. Psychol Serv
70. Lewis CC, Boyd M, Puspitasari A, et al. Implementing measurement-based care in behavioral health: a review. JAMA Psychiatry
71. Bass C, Halligan P. Factitious disorders and malingering: challenges for clinical assessment and management. Lancet
72. Bianchini KJ, Greve KW, Glynn G. On the diagnosis of malingered pain-related disability: lessons from cognitive malingering research. Spine J
73. Crighton AH, Wygant DB, Applegate KC, Umlauf RL, Granacher RP. Can brief measures effectively screen for pain and somatic malingering? Examination of the Modified Somatic Perception Questionnaire and Pain Disability Index. Spine J
74. Sellbom M, Wygant D, Bagby M. Utility of the MMPI-2-RF in detecting non-credible somatic complaints. Psychiatry Res
75. Silver JM. Effort exaggeration and malingering after concussion. J Neurol Neurosurg Psychiatry
76. Cella D, Hahn EA, Jensen SE, et al. Patient-Reported Outcomes in Performance Measurement. NC:Research Triangle Park; 2015.
77. Bastien CH, Vallieres A, Morin CM. Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Med
78. Jellinek MS, Murphy JM, Little M, Pagano ME, Comer DM, Kelleher KJ. Use of the Pediatric Symptom Checklist to screen for psychosocial problems in pediatric primary care: a national feasibility study. Arch Pediatr Adolesc Med
79. Schmidt KM, Embreson SE. Schinka J, Velicer W. Item response theory and measuring abilities. Handbook of Psychology: Vol 2 Research Methods in Psychology Hoboken
. NJ:Wiley; 2003. 429–445.