Round 1 Delphi Survey
The first round ran from May 2 to June 6, 2018. This was longer than proposed (5 vs. 3 wk) in order to send further reminder emails to meet the proposed response rate of at least 40%.20 Ninety-three (42%) of the 223 invited participants completed the survey. Five participants declined participation, and 4 emails were not delivered (Fig. 1). The response rates of each stakeholder group can be seen in Table 3.
The results of the participants’ rating on the 35 outcome domains are presented in Figure 2. Two domains, “Pain Biomarkers” and “Informal Care,” did not meet the a priori criteria and were excluded from round 2. Two domains, “Number of Deaths” and “Health Care Services” marginally met the criteria for retention but only in 1 stakeholder group each. These 2 domains were presented to the International Steering Committee for consideration of exclusion from round 2, with the final consensus (100% agreement) being the removal of “Number of Deaths” and retention of “Health Care Services.”
Twenty-one participants nominated additional domains to be included. Fourteen of the nominated domains were rejected, as they were duplicates, irrelevant, or inapplicable as a COS domain. The remaining 7 domains were presented to the International Steering Committee for final decision for inclusion in round 2, with the final consensus of adding “Perceived Injustice,” “Beliefs/Expectations,” and “Optimism” under the domain “Psychological Functioning,” “Intimacy” added under “Quality of Life,” “Physical Activity” added under “Physical Functioning,” and an additional domain of “Time to Recovery” (Appendix A, Supplemental Digital Content 1, http://links.lww.com/CJP/A581).
Ninety-two participants answered the question on the optimal number of domains, and 91 participants indicated a specific number ranging from 3 to 30. One participant suggested an excessive number of 69 domains, which was excluded from consideration. Twenty-three (25%) participants suggested “5” as the optimal number, and 22 (23.9%) participants suggested “10” (Table 4), with the majority in favor of a small COS.
Fifty-five (60.4%) participants indicated generally that some domains should be combined, and 53 (58.2%) participants identified specific domains that could be combined. These proposed combinations and the International Steering Committee decision are depicted in Table 5.
Twenty-five outcome domains were included in the second round of the Delphi survey. These were as follows: Satisfaction with Care/Treatment, Physical Functioning, Perceived Recovery, General Health, Work Related Factors, Satisfaction with Social Roles and Activities, Fatigue, Autonomy, Psychological Functioning, Self-Efficacy, Cognitive Functioning, Sleep, Quality of Life, Pain, Use of Pain Medication, Dizziness, Visual Disturbances, Swallowing Difficulties, Sensory Symptoms, Range of Motion, Neck Posture, Muscle Function, Proprioception, Health Care Services, and Time to Recovery.
Round 2 Delphi Survey
Round 2 ran from June 18 to July 16, 2018. The survey was extended by an additional week to improve response rates in order to meet the a priori criteria. Two hundred twenty-two participants were emailed the results from round 1 and invited to participate in the second round. Figure 1 and Table 3 depict the participant responses.
Eight domains, namely (1) “General Health,” (2) “Dizziness,” (3) “Visual Disturbances,” (4) “Swallowing Difficulties,” (5) “Sensory Symptoms,” (6) “Muscle Function,” (7) “Proprioception,” and (8) “Health Care Services,” did not meet the a priori criteria for retention and were therefore excluded (Fig. 3).
Round 3 Delphi Survey
Round 3 ran from July 17 to August 10, 2018. Two hundred twenty-one participants were emailed the results from round 2 and invited to participate in the final round. Figure 1 and Table 3 depict the participant responses. Eleven domains, namely (1) “Physical Functioning,” (2) “Perceived Recovery,” (3) “Work Related Factors,” (4) “Satisfaction with Social Roles and Activities,” (5) “Autonomy,” (6) “Psychological Functioning,” (7) “Self-Efficacy,” (8) “Cognitive Functioning.” (9) “Sleep,” (10) “Quality of Life” and (11) “Pain,” met the a priori criteria for retention (Fig. 4).
Final Consensus Meeting
Before the meeting, the results of all 3 rounds of the Delphi survey were collated and sent to all participants. A 2-day face-to-face consensus meeting was held at the conclusion of the IASP World Congress in Boston 2018. There were 19 attendees, and, for those who were unable to attend in person (n=1), the results of discussion and voting were emailed, and their support or otherwise was gained before the final decisions were made. At the commencement of the meeting, the Delphi results were presented.
Discussions were first held about the number of core domains to be included. There was full agreement that the ideal number was ∼5, but the final decision on this would be made after discussion and voting on the core domains. The final number of core domains was eventually determined as 6.
Discussions were then held around the 11 core domains, and the following recommendations were voted by a majority of the attending committee:
- 100% agreement to include “Self-Efficacy” under the domain of “Psychological Functioning.”
- 100% agreement to include “Autonomy” under the domain of “Satisfaction with social roles and activities.”
- 100% agreement to merge “Work Related Factors” and “Satisfaction with social roles and activities” into 1 domain “Work and Social Factors.”
- 100% agreement to remove “Cognitive Functioning” due to broadness and difficulties in measurement of this domain.
- 100% agreement to include “Sleep” under the domain of “Physical Functioning.” The committee recognized that “Sleep” was rated highly by all stakeholder groups, (Fig. 4) and there was much discussion about whether or not it should be a domain on its own or incorporated under a broader domain. After discussion, there was 100% agreement to include it under “Physical Functioning,” but it was acknowledged that sleep is not only a physical construct.
- 89% agreement to include “Perceived Recovery.” There was a discussion that this construct may be subject to recall bias and influenced by current health status.26
At the conclusion of the meeting, the domains “Physical Functioning,” “Perceived Recovery,” “Work and Social Functioning,” “Psychological Functioning,” “Quality of Life,” and “Pain” were deemed the final set with 100% consensus.
We undertook a 3-step process consistent with recommendations of the COMET initiative9 and with a priori established protocol20 to develop a set of core outcome domains for use in clinical trials for WAD. International, multidisciplinary, and multistakeholder consensus was achieved. Six core domains were recommended for inclusion in all clinical trials of WAD. These were: “Physical Functioning,” “Perceived Recovery,” “Work and Social Functioning,” “Psychological Functioning,” “Quality of Life,” and “Pain.”
A common objective of most interventions is to improve physical functioning. Patients with WAD also report this as being important to them.27,28 This core domain is recommended by IMMPACT for chronic pain and by low back pain initiatives.12,15,16 Physical function is a broad domain that can encompass various aspects of a person’s life, ranging from ability to carry out daily activities, for example, household tasks, recreational activities, or self-care, to specific strength, endurance, and functional capacity.
We included “sleep problems” under this domain. Sleep problems have not been included in other COSs including those by IMMPACT, or low back pain, but, as it was rated very highly by all stakeholder groups in our Delphi survey, the consensus meeting participants felt its inclusion was necessary. Recent studies show that sleep disruption is an important aspect of various musculoskeletal conditions.29 It would seem to be also important for patients with WAD.30 After discussion at the consensus meeting, the final decision was not to include it as a separate domain. Reasons for this decision included that, at this stage, the research of sleep and its role in WAD is limited and that there would be difficulties in measuring this construct to warrant inclusion as a stand-alone domain. It was agreed to include it under the domain of Physical Functioning, but it was acknowledged that sleep is not only a physical construct.
The question as to whether or not a patient feels that his or her condition has improved or deteriorated as a result of an intervention is fundamental to both clinical research and practice. However, this domain attracted much discussion with respect to limitations such as participant recall bias and that it could be strongly influenced by the patient’s current health status.26 Despite the limitations, the concept of clinical relevance to an individual is identified as critical in the choice of outcome measures.31 Including measures of perceived global improvement allows the patient to aggregate the effect of the intervention on all aspects of their condition, or at least those important to that individual. The patient may include pain relief, physical and psychological function, and side effects among other factors into an overall perception of the benefit of the intervention for them.11 Participant rating of global improvement usually involves just a single item question on a Likert scale, allowing for an outcome domain that is neither time-consuming nor arduous.
Work and Social Functioning
This is a broad domain and encompasses the health condition’s impact on an individual’s ability to interact with others, ability to build and maintain relationships, engage in their desired community social life and work environment, ability to work or return to work, and efficacy during work. This domain was not included as a core domain for nonspecific low back pain,15,16 but was recommended by IMMPACT as a supplemental domain.12 The consensus decision of this study was informed by the high rating of importance (>85%) by all 4 stakeholder groups for both domains “Work Related Factors” and “Satisfaction with Social Roles and Activities.” This result is consistent with findings from qualitative studies of patients with WAD.32 There was a discussion of paid and unpaid work roles and how these vary by gender and age. The group acknowledged that return to work does not necessarily mean paid work for all people. During discussions, it was noted that some available instruments such as health-related quality of life questionnaires include items about social functioning and work-related factors, potentially supporting a short list of measurement instruments required at the next stage of development of the COS.
A recent systematic review indicated greater impact of psychological distress in patients with WAD compared with spinal cord injury and traumatic brain injury.33 Various psychological factors have been shown to be associated with WAD, including posttraumatic stress symptoms,34 pain catastrophizing,35 fear of movement,36 and depression37 among others. In our study, the description accompanying the domain “Psychological Functioning” includes subdomains fear of movement, stress associated with the crash, general stress, anxiety, depression, hyperarousal, perceived injustice, beliefs, self-efficacy, and optimism. Researchers could adopt any of these subdomains should the clinical trial require a specific psychological outcome to be measured, depending on the design and nature of the intervention conducted.
Quality of Life
This domain includes patients’ perceptions on their position in life, taking into consideration the culture and value systems in which they live, and in relation to their personal values, goals, expectations, standards, and concerns.38 It is a broad-ranging concept incorporating components of a patient’s self-reported physical heath, psychological state, social functioning, interaction with salient features of their environment, and level of independence.39 These components have been found to be of importance from a patient’s perspective.40 This core domain has been included in COS for low back pain and shoulder disorders.13–16
The inclusion of Pain as a core domain is consistent with other outcome recommendations for chronic pain in general12 and for chronic regional pain syndrome.41 Similar initiatives for low back pain were more specific and recommended “pain intensity.”15,16 Managing pain has been nominated as being important by patients with WAD,42 and the measurement of pain intensity is commonly used in clinical trials.43 Patients with WAD also commonly report widespread pain, pain that changes over time, and different pain qualities such as burning or throbbing pain.44 It has been argued that it is not pain intensity per se that is important but how much the pain bothers or interferes with patients’ daily lives that is more relevant.45 These factors may need to be considered at the next stage of the COS development—the recommendation of core outcome measurement set.
There was a discussion about the importance of economic analyses in clinical trials for WAD. The committee recommended that an economic or cost-effectiveness analysis is a critical component of phase III and IV trials. However, the committee agreed that economic evaluations should not be included in Phase I and II trials.
Discussion was also held around the acute versus chronic stages of WAD and whether the different outcome measures should be recommended on the basis of time frame after injury. For discussion purposes, the IASP definition of chronic pain as persistent or recurrent pain lasting >3 months was used.46 It was agreed that the core domains recommended by the committee are relevant for both acute and chronic WAD and that no differentiation between recovery stages was made.
The management of patients with WAD often occurs in an environment where health and compensation systems intersect. Compensation systems vary greatly between jurisdictions, for example, “no fault” schemes versus “tort-based” schemes with evidence indicating that compensation-related factors can influence health outcomes.47 The committee agreed that clinical trials for WAD should specify and describe the nature of the compensation and/or health system in which the jurisdiction operates.
This is the first study to identify core outcome domains for use in clinical trials of WAD. Strengths include a previously developed and published protocol, methods that follow recommendations of COMET, and being consistent with other initiatives including IMMPACT and OMERACT. We included international, multidisciplinary, and multistakeholder participants in both the Delphi surveys and the consensus meeting. There are several limitations to our study. Response rates of patients and insurers in each Delphi round was low, which could have led to an underestimation or overestimation of the importance of certain domains. However, despite this, the response from these stakeholder groups was in line with the other groups. Furthermore, the primary objective of this study was to develop a core set for inclusion for all clinical trials, and not to develop a comprehensive set of domains important to all stakeholders. In addition, all of our insurer and patient participants were from Australia and may not be representative of the views of these stakeholders in other countries. Nevertheless, these participant groups were from the 3 largest states—New South Wales, Queensland, and Victoria, where different insurance schemes operate. Queensland and New South Wales are tort-based and fault-based systems, respectively, whereas Victoria operates under a “no fault” system; hence, we do have views from the 2 main types of compulsory third-party schemes.
Following this study, the next step of the development of COS for clinical trials on WAD is to determine the measurement instruments for each of these domains. Selection of outcome measurement instruments will be guided by the published methodological guideline.9
1. Vos T, Allen C, Arora M, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries: a systematic analysis for the Global Burden of Disease Study 2015. Lancet. 2015;388:1545–1602.
2. Connelly L, Supangan R. The economic costs of road traffic crashes: Australia, staes and territories. Accid Anal Prev. 2006;38:1087–1093.
3. Anstey R, Kongsted A, Kamper S, et al. Are people with whiplash
-associated neck pain
different from people with nonspecific neck pain
? J Orthop Sports Phys Ther. 2016;46:894–901.
4. Sarrami P, Armstrong E, Naylor JM, et al. Factors predicting outcome
injury: a systematic meta-review of prognostic factors. J Orthop Traumatol. 2017;18:9–16.
5. Walton D, Macdermid J, Giorgianni A, et al. Risk factors for persistent problems following acute whiplash
injury: update of a systematic review and meta-analysis. J Orthop Sports Phys Ther. 2013;43:31–43.
6. Meeus M, Nijs J, Hamers V, et al. The efficacy of patient education in whiplash
associated disorders: a systematic review. Pain
7. Wiangkham T, Duda J, Haque S, et al. The effectiveness of conservative management for acute whiplash
associated disorder (WAD) II: a systematic review and meta-analysis of randomised controlled trials. PLoS ONE. 2015;10:e0133415.
8. Williamson PR, Altman DG, Blazeby JM, et al. Developing core outcome
sets for clinical trials: issues to consider. Trials. 2012;13:132.
9. Prinsen CA, Vohra S, Rose MR, et al. How to select outcome
measurement instruments for outcomes included in a “Core Outcome
Set”—a practical guideline. Trials. 2016;17:449.
10. Tugwell P, Boers M, Brooks P, et al. OMERACT: an international initiative to improve outcome
measurement in rheumatology. Trials. 2007;8:38.
11. Dworkin RH, Turk DC, Farrar JT, et al. Core outcome
measures for chronic pain
clinical trials: IMMPACT recommendations. Pain
12. Turk DC, Dworkin RH, Allen RR, et al. Core outcome domains
for chronic pain
clinical trials: IMMPACT recommendations. Pain
13. Buchbinder R, Page MJ, Huang H, et al. A preliminary core domain set for clinical trials of shoulder disorders: a report from the OMERACT 2016 Shoulder Core Outcome
Set Special Interest Group. J Rheumatol. 2017;44:1880–1883.
14. Ramiro S, Page MJ, Whittle SL, et al. The OMERACT core domain set for clinical trials of shoulder disorders. J Rheumatol. 2019. [Epub ahead of print].
15. Chiarotto A, Deyo RA, Terwee CB, et al. Core outcome domains
for clinical trials in non-specific low back pain
. Eur Spine J. 2015;24:1127–1142.
16. Chiarotto A, Boers M, Deyo RA, et al. Core outcome
measurement instruments for clinical trials in nonspecific low back pain
17. Laxe S, Zasler N, Selb M, et al. Development of the International Classification of Functioning, Disability and Health core sets for traumatic brain injury: an International consensus process. Brain Inj. 2013;27:379–387.
18. Roe C, Bautz-Holter E, Cieza A. Low back pain
in 17 countries, a Rasch analysis of the ICF core set for low back pain
. Int J Rehabil Res. 2013;36:38–47.
19. Scott D, Jull G, Sterling M. Widespread sensory hypersensitivity is a feature of chronic whiplash
-associated disorder but not chronic idiopathic neck pain
. Clin J Pain
20. Maujean A, Carroll L, Curatolo M, et al. A core outcome
set for clinical trials in whiplash
associated disorders (WAD): a study protocol. Trials. 2018;19:635.
21. Spitzer W, Skovron M, Salmi L, et al. Scientific Monograph of Quebec Task Force on Whiplash
Associated Disorders: redefining “Whiplash
” and its management. Spine. 1995;20(suppl 8):1–73.
22. Boers M, Idzerda L, Kirwan JR, et al. Toward a generalized framework of core measurement areas in clinical trials: a position paper for OMERACT 11. J Rheumatol. 2014;41:978–985.
23. Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med. 2011;8:e1000393.
24. Williamson PR, Altman DG, Bagley H, et al. The COMET Handbook: version 1.0. Trials. 2017;18(suppl 3):280.
25. Wylde V, MacKichan F, Bruce J, et al. Assessment of chronic post-surgical pain
after knee replacement: development of a core outcome
set. Eur J Pain
26. Kamper S, Ostelo R, Knol D, et al. Global perceived effect scales provided reliable assessments of health transition in people with musculoskeletal disorders, but ratings are strongly influenced by current status. J Clin Epidemiol. 2010;63:760–766.
27. Maujean A, Sterling J, Sterling M. What information do patients need following a whiplash
injury? The perspectives of patients and physiotherapists. Disabil Rehabil. 2018;40:1135–1141.
28. Walton DM, Macdermid JC, Taylor T. What does ‘recovery’ mean to people with neck pain
? Results of a descriptive thematic analysis. Open Orthop J. 2013;7:420–427.
29. Kelly GA, Blake C, Power CK, et al. The association between chronic low back pain
and sleep: a systematic review. Clin J Pain
30. Valenza MC, Valenza G, Gonzalez-Jimenez E, et al. Alteration in sleep quality in patients with mechanical insidious neck pain
-associated neck pain
. Am J Phys Med Rehabil. 2012;91:584–591.
31. Middel B, Stewart R, Bouma J, et al. How to validate clinically important change in health-related functional status. Is the magnitude of the effect size consistently related to magnitude of change as indicated by a global question rating? J Eval Clin Pract. 2001;7:399–410.
32. Soderlund A, Nordgren L, Sterling M, et al. Exploring patients’ experiences of the whiplash
injury-recovery process—a meta-synthesis. J Pain
33. Craig A, Tran Y, Guest R, et al. Psychological impact of injuries sustained in motor vehicle crashes: systematic review and meta-analysis. BMJ Open. 2016;6:e011993.
34. Sterling M, Hendrikz J, Kenardy J. Similar factors predict disability and PTSD trajectories following whiplash
35. Sullivan M, Adams H, Martel M, et al. Catastrophizing and perceived injustice: risk factors for the transition to chronicity after whiplash
injury. Spine. 2011;36:s244–s249.
36. de Jong JR, Vangronsveld K, Peters ML, et al. Reduction of pain
-related fear and disability in post-traumatic neck pain
: a replicated single-case experimental study of exposure in vivo. J Pain
37. Phillips L, Carroll L, Cassidy D, et al. Whiplash
-associated disorders: who gets depressed? Who stays depressed? Eur Spine J. 2010;19:945–956.
38. Carlesso LC, Walton DM, MacDermid JC. Reflecting on whiplash
associated disorder through a QoL lens: an option to advance practice and research. Disabil Rehabil. 2012;34:1131–1139.
39. Kuyken W, Orley JKuyken W, Orley J. The Development of the World Health Organisation Quality of Life Assessment Instrument (the WHOQOL). Quality of Life Assessment: International Perspectives, Vol 53. Berlin, Germany: Springer; 1994:41–57.
40. Hoving JL, O’Leary EF, Niere KR, et al. Validity of the Neck
Disability Index, Northwick Park Neck Pain
Questionnaire, and problem elicitation technique for measuring disability associated with whiplash
-associated disorders. Pain
41. Grieve S, Perez R, Birklein F, et al. Recommendations for a first Core Outcome
Measurement set for complex regional PAin
syndrome Clinical sTudies (COMPACT). Pain
42. Rydstad M, Schult ML, Lofgren M. Whiplash
patients’ experience of a multimodal rehabilitation programme and its usefulness one year later. Disabil Rehabil. 2010;32:1810–1818.
43. Walton D. A review of the definitions of ‘recovery’ used in prognostic studies on whiplash
using an ICF framework. Disabil Rehabil. 2009;31:943–957.
44. Sterling M, Pedler A. A neuropathic pain
component is common in acute whiplash
and associated with a more complex clinical presentation. Man Ther. 2009;14:173–179.
45. Dunn KM, Croft PR. Classification of low back pain
in primary care: using “bothersomeness” to identify the most severe cases. Spine (Phila Pa 1976). 2005;30:1887–1892.
46. Treede R-D, Rief W, Barke A, et al. A classification of chronic pain
for ICD-11. Pain
47. Murgatroyd DF, Casey PP, Cameron ID, et al. The effect of financial compensation on health outcomes following musculoskeletal injury: systematic review. PLoS ONE. 2015;10:e0117597.
whiplash; outcome; domains; neck; pain
Supplemental Digital Content
Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.