In a time of challenges that call for mission-oriented research  such as those in the healthcare, the future of sustainability requires new multistakeholder and multidisciplinary organizational models of cooperation that guarantee a long-term return on investment, not only economic. An important driver of this change is enabling science with and of patient input. What began as an extension of patient advocacy has now evolving into an emerging discipline aimed at understanding and incorporating patient needs and perspectives into the processes of developing, regulating, and delivering new therapies as well as improving care: the science with and of patient input .
This is also the thinking behind Public Engagement within the Responsible Research Innovation (RRI) European Union's Horizon 2020 program. Within this frame, Patient Advocacy Organizations are playing an important role, as boundary organizations, to define and implement the ‘how to’ of patient engagement . Within this strategic frame, the RRI EU-funded MULTI-ACT project , coordinate by the Italian MS Society, is attracting the interest of the health-research community [5▪▪]. The innovative model proposed by the MULTI-ACT project argues that excellence, validity, and relevance are connected by engaging patients in the research continuum as key stakeholder (science with patient input) and by evaluating impact of research on the outcomes that matter to patients (science of patient input). The power of the science with and of patient input relies on the innovative framework used to engage patients and on the sources of patient data. In particular, the patient-reported outcomes (PROs) and their measurement instruments (patient-reported outcome measures [PROMs]) data are supporting every aspect of the healthcare continuum from research to clinical trials and practice through to public health. PROs refer to direct patient reports of their experience with a disease and its treatment [6,7]. PROs are playing an increasing role in multiple sclerosis clinical trials and practice and are essential for understanding the effects that multiple sclerosis and multiple sclerosis treatments have on patients’ lives. This review discusses the role of PRO in measuring the impact of health research, treatment and care on outcomes that matter most to people with multiple sclerosis and propose directions for the future.
Evolution of patient-reported outcomes use in clinical trials
There are enough PRO to warrant entire collections of measures . Most PRO measures are categorized as either generic or targeted. However so far, PROs have been mainly used in postmarketing, observational studies. PROMs are increasingly used as secondary or tertiary outcomes in multiple sclerosis clinical trials on disease-modifying therapies and symptomatic treatments, whereas in rehabilitation trials are used as primary or coprimary outcomes. There are currently 14 disease-modifying agents that have been approved by the appropriate regulatory agencies. Although some clinical trials show promising results , based on a recent literature search of PROs used in randomized phase III trials of multiple sclerosis disease modifying drugs (DMDs), it is not possible to reach any conclusion about the overall benefit of DMDs or the relative benefit of one over another on the well-being of individuals living with multiple sclerosis . The Food and Drug Administration (FDA) and the European Medicines Agency (EMA) both consider that the voices of patients in medicine regulation are essential, as they bring the unique perspective of someone living with a disease . This perspective complements the medical and scientific information that is used when evaluating medicines for regulatory approval. For example, patients can highlight unmet needs for their condition that they consider to be particularly important, which might differ from the standard end points evaluated in clinical trials and patient perceptions frequently differ from those of clinicians [12–15]. However, the uptake of PROs as primary outcome in the decision-making processes of regulatory agencies for drug approvals is still challenging and controversial. The outcome of the Multiple Sclerosis Outcome Assessments Consortium qualification processes of a Clinical Outcome Assessment instrument, developed to assess treatment benefit in clinical trials of therapies for multiple sclerosis, reinforces the EMA's guidance on the design of multiple sclerosis clinical trials for which the Expanded Disability Status Scale (EDSS) is still the gold standard .
There are several limitations to the use of PROMs  in clinical trials. Based on the variability of PRO data from these trials and the variety of measures implemented it is not possible to reach any conclusion about the overall benefit of DMDs or the relative benefit of one over another on the well-being of individuals living with multiple sclerosis. The lack of a set of standard measures has significant disadvantages and some available measures are of uncertain validity and were created without using modern test development methodology . These aspects represent some of the challenges that prevent the use of PROs as primary outcomes in clinical trials. To help meeting the challenge, different standard sets of PRO measures have been developed. Among these, the National Institutes of Health (NIH) supported the creation of two standard sets of PRO measures, one appropriate for use across neurological conditions (Neuro-QoL)  and one for use across a broad range of chronic health conditions (PROMIS) . The above standard sets of PRO measures are consistent with FDA guidance regarding PROs and the EMA Reflection Paper on the use of health-related quality of life measures the NIH . The ‘SymptoMScreen’ PRO measurement tool is another valuable addition to existing PRO measures that are available in multiple sclerosis and could be highly useful in studies of large populations seeking to minimize respondent burden .
Evolution of patient-reported outcomes use in clinical practice
Accompanying acceptance of the need to integrate patient perspectives into clinical trials is an increase in the demand for research-based methods and tools to measure the effectiveness of incorporating patient input into the clinical practice and, ultimately, its impact on patients’ health and quality of life [23▪▪]. Given the plethora of functional domains affected in patients with multiple sclerosis, a multidisciplinary care team, such as the recently proposed multiple sclerosis care unit model [24▪▪], is needed to improve patient engagement and the uptake of PRO in clinical practice. There was broad agreement that the patient voice should be heard more in discussions around multiple sclerosis care . Many of the existing PROs were not developed in partnership with patients and with insufficient understanding about what matters to patients. This makes also challenging to prove the clinically meaningfulness of PROs, one of the main requirement for the uptake of PRO in the decision-making drug-approval process. Within this frame, iConquerMS is a multiple sclerosis people-powered research network established in 2014 with funding by the Patient-Centered Outcomes Research Institute (USA) . It is dedicated to engaging all multiple sclerosis stakeholders to enable research on topics that matter most to people affected by multiple sclerosis. The initiative is based on the patient-reported outcomes measurement information system Global Health Survey, and the Neuro-QoL Adult Short Forms. To date, iConquerMS over 5000 PwMS have joined the initiative. Another initiative, based in Europe, that could help meeting the challenge is the A new functional PROfile to MOnitor the PROgression of disability in Multiple Sclerosis [27▪▪] project promoted and funded by the Italian MS Society. The PROs, related to domains that most care to multiple sclerosis patients (case study) defined by the PROMOPROMS initiative (manual ability, bladder functions, motor, cognitive, psychosocial fatigue, anxiety, and depression, quality of life), are currently used to measure disease progression and the efficacy of treatments (drugs and/or rehabilitation) in clinical practice and could be potentially measured via digital health technologies.
The ideal would be to find a happy medium between PROs that work at population level in clinical trials  and PROs that can be individualized for use in clinical practice. The ideal would be to find a happy medium between PROs that work at population level in clinical trials and PROs that can be individualized for use in clinical practice. Electronic health technologies (e-Health) could help meeting this challenge and could play an increased role in filling the gaps between PROs use in clinical trials versus clinical practice in multiple sclerosis . Therefore, data about patients’ experiences outside the clinic are not only ‘nice to have’ but also critical to understanding and improving those outcomes. A great deal of momentum surrounds the application of new technologies, such as mobile devices and other digital platforms, to both deliver care and generate real-world data on patients’ experiences. Within this frame, e-Health is becoming increasingly relevant in multiple sclerosis clinical management . e-Health describes an emerging field of technologies designed to be worn on the body or embedded into mobile and portable solutions (e.g., smartphones, watches, bracelets, and clothing) and able to provide individual's data such as those from sensors (passive monitoring) as well as self-reported data or questionnaires, such as electronic PRO (active monitoring) . There is a clear drive to collect PROs electronically using electronic ePRO systems. In this context, the use of ePROs instead of paper formats in clinical trials could facilitate the robust analysis and reporting of PRO data. However, the user interfaces of ePRO systems need to be adequately assessed to ensure they are not only fit for purpose but also acceptable to patients who are the end users  and effective for the different context of use, that is, clinical trials  and clinical practice. The advancement of ePRO is likely to also enable successful development of therapeutics in progressive multiple sclerosis that depends not only on identification of relevant pathways and effective therapies but also on available outcome assessments capable of demonstrating treatment effects on disease progression [34,35].
Stakeholder engagement and cooperation
Despite differences in viewpoint, the different stakeholders agree that PRO and PROMs have not reached their full potential of delivering benefits to people with multiple sclerosis. In particular people with multiple sclerosis are frustrated that functional domains that matter most to them are not addressed by current PROs; clinicians acknowledge the importance to include patient-reported outcome measures (PROMs) to understand treatment effects or compare treatment options; regulatory authorities recognize that current outcomes (EDSS, relapse rate, Magnetic Resonance Imaging features) do not fully capture the experience of people with multiple sclerosis; Healthcare Technology Assessment Agencies aim to integrate the ‘patient voice’ in coverage decisions; Pharma acknowledge the importance to include patient PROs in Medicine Life Cycle Management and consumer-driven healthcare is gaining ground. Efforts to enable the uptake of PROs and PROMs into clinical trials, regulatory agencies/Health Technology Sector decision-making processes and practice could be greatly enhanced and informed by a commonly held strategic PRO agenda and roadmap shared by all relevant stakeholders. The newly born multistakeholder Patient Reported Outcome Initiative for multiple sclerosis (PROMS) [5▪▪] aims at achieving a unified vision PRO that meets the expectations of multiple sclerosis patients and that can serve the Healthcare Providers, Regulatory Agencies and Health Technology Sector Agencies. The initiative is jointly led and coordinate by the European Charcot Foundation and Multiple Sclerosis International Federation, with the Italian Multiple Sclerosis Society acting as lead agency on behalf and for the multiple sclerosis movement. The strategic priorities of PROMS are centered around the validation and development of PROs that matter most to people with multiple sclerosis. However, an important driver of the multistakeholder initiative will be to show that PRO will also measure outcomes that correspond to all the stakeholder needs. In this context, the EU-funded MULTI-ACT project  provides a new Collective Research Impact Framework (CRIF) which will be translated into an online toolbox and a set of guidelines to improve the governance of multistakeholder research initiatives and stakeholder coaccountability in health research and innovation. The core component of such CRIF applies innovative guidelines for patient-engagement and a multistakeholder perspective to assess the impact of health research considering five dimensions of accountability (efficacy/mission, social, economic, excellence and patient-reported). Among the MULTI-ACT CRIF indicators, the PROMOPROMS [27▪▪] set of outcomes will be used to evaluate impact of health research and care on people with multiple sclerosis-reported dimension.
We have reviewed the main challenges concerning the use of PROs in multiple sclerosis and the initiatives that are developing the field. We have attempted to stimulate discussion and encourage critical reflection that an important driver to meet the challenges is enabling science with and of patient input via multistakeholder and multidisciplinary organizational models. We argued the potential of the EU-funded MULTI-ACT project to enable the development of these models.
We would like to thank Dr. Roberta Guglielmino for useful discussion and help in manuscript editing.
Financial support and sponsorship
The authors receive funding from the EU-funded project MULTI-ACT (https://www.multiact.eu/) within the European Union's Horizon 2020 Research and Innovation Programme under the Grant Agreement No. 787570.
Conflicts of interest
There are no conflicts of interest.
REFERENCES AND RECOMMENDED READING
Papers of particular interest, published within the annual period of review, have been highlighted as:
1. Mazuccato M. Mission-Oriented Research & Innovation in the European Union. European Union, 2018. doi:10.2777/360325. https://ec.europa.eu/info/sites/info/files/mazzucato_report_2018.pdf
2. Anderson M, McCleary KK. On the path to a science of patient input. Sci Transl Med 2016; 8:336ps11doi: 10.1126/scitranslmed.aaf6730. https://www.fastercures.org/programs/patients-count/science-of-patient-input-resources/
. Lancet Neurology Editorial quotes the Patient Reported Outcome Initiative for Multiple Sclerosis
(PROMS) launched on Sept 12, 2019, at the 35th Congress of the European Committee for Treatment and Research in Multiple Sclerosis
, held in Stockholm, Sweden.
3. Schneeman K, Barton V, Huneycutt B. Advancing Models of Patient Engagement
: Patient Organizations as Research and Data Partners.©2019 Milken Institute. https://milkeninstitute.org/reports/advancing-models-patient-engagement-patient-organizations-research-and-data-partners
5▪▪. The Lancet. Patient-reported outcomes in the spotlight. Lancet Neurol 2019; 18:981doi:10.1016/S1474-4422(19)30357-6.
6. European Medicines Agency (EMA). Reflection article on the regulatory guidance for the use of health related quality of life (HRQL) measures in the evaluation of medicinal products. London: European Medicines Agency; 2005.
7. U.S. Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research (CDER) Center for Biologics Evaluation and Research (CBER) Center for Devices and Radiological Health (CDRH) December 2009 Clinical/Medical. Guidance for Industry Patient-Reported Outcome
Measures: Use in Medical Product Development to Support Labeling Claim. December 2009 Clinical/Medical. https://www.fda.gov/media/77832/download
8. Nowinski CJ, Miller DM, David Cella D. Evolution of patient-reported outcomes and their role in multiple sclerosis
Clinical Trials. Neurotherapeutics 2017; 14:934–944.
9. Berger T, Brochet B, Brambilla L, et al. Effectiveness of delayed-release dimethyl fumarate on patient-reported outcomes and clinical measures in patients with relapsing-remitting multiple sclerosis
in a real-world clinical setting: PROTEC. Mult Scler J Exp Transl Clin 2019; 5:2055217319887191.
10. van Munster CE, Uitdehaag BM. Outcome measures in clinical trials for multiple sclerosis
. CNS Drugs 2017; 31:217–236.
11. Mavris M, Furia Helms A, Bere N, et al. Engaging patients in medicines regulation: a tale of two agencies. Nat Rev Drug Discov 2019; 18:885–886.
12. Edelman EJ, Gordon K, Justice AC. Patient and provider reported symptoms in the postcart era. AIDS Behav 2011; 15:853–861.
13. Janse AJ, Gemke R, Uiterwaal C, et al. Quality of life: patients and doctors don’t always agree: a meta-analysis. J Clin Epidemiol 2004; 57:653–661.
14. Kremenchutzky M, Walt L. Perceptions of health status in multiple sclerosis
patients and their doctors. Can J Neurol Sci 2013; 40:210–218.
15. Rothwell PM, McDowell Z, Wong CK, Dorman PJ. Doctors and patients don’t agree: cross sectional study of patients’ and doctors’ perceptions and assessments of disability in multiple sclerosis
. BMJ 1997; 314:1580.
17. Caspar EP, van Munster, Bernard MJ. Uitdehaag outcome measures in clinical trials for multiple sclerosis
. CNS Drugs 2017; 31:217–236.
18. Powers JH, Patrick DL, Walton MK, et al. Clinician-reported outcome assessments of treatment benefit: report of the ISPOR clinical outcome assessment emerging good practices task force. Value Health 2017; 20:2–14. doi:10.1016/j.jval.2016.11.005.
19. Cella D, Yount S, Rothrock N, et al. The patient-reported outcomes measurement information system (PROMIS): progress of an NIH roadmap cooperative group during its first two years. Med Care 2007; 45:S3–S11.
20. Cella D, Lai JS, Nowinski C, et al. Neuro-QoL: brief measures of health-related quality of life for clinical research in neurology. Neurology 2012; 78:1860–1867.
21. European Medicines Agency Committee for Medicinal Products for Human Use. Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products. Available at https://www.ema.europa.eu/en/documents/scientific-guideline/reflection-paper-regulatory-guidance-use-healthrelated-quality-life-hrql-measures-evaluation_en.pdf
22. Fitzgerald KC, Salter A, Tyry T, et al. Validation of the SymptoMScreen with performance-based or clinician-assessed outcomes. Mult Scler Relat Disord 2019; 29:86–93.
23▪▪. D’Amico E, Haase R, Ziemssen T. Review: Patient-reported outcomes in multiple sclerosis
care; Mult Scler Relat Disord. Mult Scler Relat Disord 2019; 33:61–66.
24▪▪. Soelberg Sorensen P, Giovannoni G, Montalban X, et al. The multiple sclerosis
care unit. Mult Scler 2019; 25:627–636.
25. Yeandle D, Rieckmann P, Giovannoni G, et al. Patient power revolution in multiple sclerosis
: navigating the new frontier. Neurol Ther 2018; 7:179–187.
27▪▪. Brichetto G, Monti Bragadin M, Fiorini S, et al. The hidden information in patient-reported outcomes and clinician-assessed outcomes: multiple sclerosis
as a proof of concept of a machine learning approach. Neurol Sci 2020; 41:459–462.
28. Hobart J, Ziemssen T, Feys P, et al. Assessment of clinically meaningful improvements in self reported walking ability in participants with multiple sclerosis
: results from the randomized, double-blind, phase III ENHANCE trial of prolonged-release fampridine. CNS Drugs 2019; 33:61–79.
29. Lavorgna L, Brigo F, Moccia M, et al. e-Health and multiple sclerosis
: an update. Mult Scler 2018; 24:1657–1664.
30. Marziniak M, Brichetto G, Feys P, et al. The use of digital and remote communication technologies as a tool for multiple sclerosis
management: narrative review. JMIR Rehabil Assist Technol 2018; 5:e5.
31. Brichetto G, Pedullà L, Podda J, et al. Beyond center-based testing: understanding and improving functioning with wearable technology in MS. Mult Scler 2019; 25:1402–1411.
32. Olalekan LA. Key methodological considerations for usability testing of electronic patient-reported outcome
(ePRO) systems 2020; 29:325–333.
33. Zbrozek A, Hebert J, Gogates G, et al. Validation of electronic systems to collect patient-reported outcome
(PRO) data-recommendations for clinical trial teams: report of the ISPOR ePRO systems validation good research practices task force. Value Health 2013; 16:480–489.
34. Simblett SK, Evans J, Greer B, et al. Engaging across dimensions of diversity: a cross-national perspective on mHealth tools for managing relapsing remitting and progressive multiple sclerosis
. Mult Scler Relat Disord 2019; 32:123–132.
35. Zhang Y, Taylor BV, Simpson S Jr. Patient-reported outcomes are worse for progressive-onset multiple sclerosis
than relapse-onset multiple sclerosis
, particularly early in the disease process. Eur J Neurol 2019; 26:155–161.