Secondary Logo

Journal Logo

HEALTH SERVICES RESEARCH

Reliability and Validity of the English Version of the AOSpine PROST (Patient Reported Outcome Spine Trauma)

Sadiqi, Said MD, PhDa; Dvorak, Marcel F. MDb; Vaccaro, Alexander R. MD, PhDc; Schroeder, Gregory D. MDc; Post, Marcel W. PhDd,e; Benneker, Lorin M. MD, PhDf; Kandziora, Frank MD, PhDg; Rajasekaran, S. MD, PhDh; Schnake, Klaus J. MDi; Vialle, Emiliano N. MDj; Oner, F. Cumhur MD, PhDa

Author Information
doi: 10.1097/BRS.0000000000003514

Spine trauma remains a global public health concern, in terms of both care and costs. A subset of spine-injured patients may deal with an associated neurologic injury, which can contribute to substantial disability with long-term consequences and considerable associated health care-related costs.1–4 The reported annual incidence of traumatic spine fractures in various articles varies between 19 and 88 per 100,000 population.5,6 The estimated global incidence-rate of traumatic spinal cord injury is reported as 23 per million, with a worldwide variation of 3.6 to 195.4 per million.7–9

In the absence of a validated disease-specific outcome instrument for traumatic spinal column injury patients, a wide range of tools have been used including generic outcome measures and instruments designed for polytrauma patients and patients with chronic degenerative back pain.10–13 This makes it difficult to compare outcomes of different treatments of the spinal column injury within and between studies, and may contribute to treatment variation among these patients.14–16 Therefore, an initiative was undertaken by AOSpine Knowledge Forum (KF) Trauma to develop a disease-specific outcome measure for spine trauma patients: the AOSpine PROST (Patient Reported Outcome Spine Trauma). Based on the systematic approach and Core Set development methodology of the International Classification of Functioning, Disability, and Health (ICF) of the World Health Organization (WHO) the tool was developed.17,18 Four preparatory studies, followed by an international consensus conference, led to the development of a Dutch version of AOSpine PROST.19 In the developmental process of the AOSpine PROST we sought to focus on patients sustaining injuries to their spinal column and excluded completely paralyzed (ASIA impairment grade A or B at discharge from hospital) and polytrauma patients (Injury Severity Score [ISS] >15), to identify specific problems related to spine trauma. A validation study among traumatic spinal column injury patients in the Netherlands showed very good results.20

The aims of this study are to translate and cross-culturally adapt the AOSpine PROST into English, and test its psychometric properties among North-American spine trauma patients.

MATERIALS AND METHODS

Translation and Cross-cultural Adaptation

Established guidelines were used for the translation and cross-cultural adaption of the AOSpine PROST from Dutch into English.21 The translation process consisted of four stages: forward translations, synthesis, back translations, and expert committee review, which was followed by validation.

In the first stage, two native English language speakers translated the AOSpine PROST from Dutch into English. The translators used English as their mother tongue and translated independently of each other. The first translator was a local Masters study program manager and the second translator a professor in Orthopedic biomechanics. During a meeting including the translators and a recording observer the translations were discussed, leading to one synthesis of the two translations. Subsequently, two independent uninformed bilingual translators with Dutch as their mother tongue performed a translation back from English into Dutch. These back-translators were Orthopedic researchers, one with medical and another technical medicine background. They were blinded to the original AOSpine PROST to avoid information bias. In the next step, an expert committee meeting (including forward-translators, back-translators, two recording observers, a professor of spinal surgery, and a professor in spinal cord injury rehabilitation) reviewed all translations and written reports. Equivalence of the prefinal version and original Dutch AOSpine PROST was examined for semantics, idioms, and conceptual meaning. After discussion, the expert committee developed a pre-final English version of the questionnaire. This was sent to the North-American AOSpine KF Trauma Steering Committee members for a final review.

Patients and Study Procedures

Adult traumatic spinal column injury patients (≥18 years of age) within 13 months post-trauma were included. They had to be capable of understanding and adequately filling out the questionnaires (e.g., no cognitive impairments). Completely paralyzed patients (ASIA impairment grade A or B at discharge from hospital) and polytrauma patients (ISS >15) were excluded.

Patients were recruited from two level-1 trauma centers in North-America: Rothman Institute (Philadelphia) and University of British Columbia (Vancouver, Canada). Patients in follow-up at the outpatient clinic within 13 months post-trauma were checked for eligibility. After informed consent, they received an email with an online link or postal mail with a login code to complete the questionnaire. If the questionnaire was not completed within 3 days, a reminder via email or telephone was sent. For the purpose of test–retest, 1 week after completing the questionnaire a part of the included patients were asked to fill out the same questionnaire once more. Ethics approval was obtained from the institutional review board of both participating centers.

Instruments

The questionnaire that was administered to the patients through an online system consisted of different parts. The first part was the AOSpine PROST consisting of 19 questions on a broad range of aspects of functioning (Appendix 1, http://links.lww.com/BRS/B569). Each item has a 0 to 100 numeric rating scale, with 0 indicating no function at all and 100 the pre-injury level of function. The scale is supported by smileys at the ends of the ruler. For the purpose of concurrent validity, the AOSpine PROST should be compared to a validated outcome instrument designed for patients with traumatic spinal column injuries. However, as no such instrument is available, the AOSpine PROST was followed by a generic health-related quality of life measure, the Medical Outcomes Study 36-item Short-Form Health Survey (SF-36) as reference standard.22 The SF-36 is the most frequently used generic instrument to measure outcomes in studies including spine trauma patients.12,23,24 It consists of 36 questions measuring eight health subscales from which the physical component summary (PCS) and mental component summary (MCS) are calculated. Scores range from 0 to 100 with higher scores indicating better outcome. Finally, patients were asked to answer additional questions: any difficulties in filling out the AOSpine PROST, any inapplicable questions, any relevant question missing, and self-reported degree of recovery.

Another part of the online system was completed by health professionals. They were asked to complete background data (sociodemographic characteristics and trauma-related variables), and to make an assessment of patients’ degree of recovery based on clinical and radiological findings.

Statistical Analysis

Descriptive statistics and frequency analysis were used to analyze patient characteristics. Content validity was assessed by evaluating the responses to the additional questions (difficulties in filling out the AOSpine PROST, inapplicable questions, and relevant questions missing). Also floor and ceiling effects were investigated, which could occur if >15% of the patients achieve the lowest or highest possible score, respectively. Furthermore, correlation between the mean total scores with the degree of recovery as reported by patients and assessed by clinicians was analyzed using Welch analysis of variance.

Concurrent validity explored the Spearman correlation coefficient (rs) between AOSpine PROST and SF-36. The rs ranges from + 1 to−1, with + 1 indicating a perfect association, 0 no association, and −1 perfect negative association.25 Concurrent validity is supported if the coefficient is at least 0.70.26

For internal consistency Cronbach α and item-total correlation coefficients were calculated. An α >0.70 is accepted as good results.26,27 Pairwise Spearman correlations between AOSpine PROST items was also investigated.

Intraclass correlation coefficients (ICCs) were calculated to assess test–retest reliability. An ICC of 0.70 to 0.85 and > 0.85 indicate good and excellent reliability, respectively.26

Finally, factor analysis was performed to explore the dimensionality of AOSpine PROST items. Based on the MINEIGEN criterion, an Eigen value >1 would indicate a dimension within the questionnaire. The loading of the items on identified dimensions was also analyzed.

RESULTS

Translation and Cross-cultural Adaptation

The translation and adaptation process proceeded without any difficulties. During the expert committee meeting, there was some discussion on item 14 (concerning emotional functioning). This item was translated as “Emotions and moods” and “Emotions” by the forward-translators. The synthesis was defined as “Emotions and moods.” Discussions during the expert committee meeting led to the definition as “Emotional state.” Ultimately, after a review by the North-American AOSpine KF Trauma members, the item was stated “Emotional function” which brings it in line with the overall aim of AOSpine PROST, that is measuring patients’ functional status. Another discussion point was how to indicate the phrase in the response scale, freely translated as “I do NOT function at all.” Although the forward-translators chose for “I do not function” and “I am completely disabled,” ultimately consensus was reached on “I am NON-FUNCTOINAL.”

Patient Characteristics

A total of 196 patients consented to take part in the study, of which 162 (82.7%) were enrolled: 95 (58.6%) USA and 67 (41.4%) Canadian patients. Nineteen USA and 15 Canadian patients were excluded: 5 were ineligible, 9 did not provide any data at all, 15 did not complete any patient questionnaire, and 5 were >13 months post-trauma. The majority of the included patients were male (67.5%) and the mean age 51.2 years. Table 1 shows the basic patient and clinical characteristics.

TABLE 1
TABLE 1:
Sociodemographic and Clinical Characteristics of the Study Population

Content Validity

The mean time to complete AOSpine PROST as registered by the online system was 7.6 minutes (median = 7.0; Q1 = 5.0, Q2 = 10.0).

Nine (5.6%) patients indicated they experienced some difficulties when filling out AOSpine PROST. In their further explanations, however, they described the limitations that hey experienced in more detail. Thus, no misunderstanding of the questions was mentioned. No patient indicated an item to be inapplicable for measuring their health status and function. Twenty-one (13.0%) patients answered positive to any item missing, but they mainly mentioned items that are already incorporated in AOSpine PROST. Only one patient had a new suggestion: “spiritual practice which supports the inner peace.” Two other patients wondered whether the scores should also apply to other trauma-related injuries.

AOSpine PROST scores relative to the degree of recovery were more strongly related (P < 0.001) to the indication by patients compared to the clinicians’ assessments (Table 2). No floor and ceiling effects were seen.

TABLE 2
TABLE 2:
Mean AOSpine PROST Scores Relative to the Degree of Recovery, Both as Reported by Patients and as Assessed by Clinicians (Mean ± SD [Range])

Concurrent Validity

Table 3 shows AOSpine PROST and SF-36 Spearman correlations. The most strong correlation was seen with the physical components of SF-36 (P < 0.001): role physical (0.82), physical functioning (0.78), and PCS (0.69). A negative correlation was observed with General Health subscale (−0.08; P < 0.001).

TABLE 3
TABLE 3:
Spearman Correlation (r s) Between AOSpine PROST and SF-36, Both for the Subscales and Summary Scales

Internal Consistency

The internal consistency of AOSpine PROST total score was excellent (Cronbach α = 0.97). With a range of 0.50 to 0.90, item-total correlation showed good results (Table 4). “Bowel movement” and “Urinating” had the lowest values (0.50 and 0.52, respectively), whereas the highest values (0.90) were seen for “Energy level” and “Lifting and Carrying.” Cronbach alpha remained excellent and did not change (0.97) when an item was removed. Good results were also obtained with Spearman correlations (Appendix 2, http://links.lww.com/BRS/B569) within AOSpine PROST items (range: 0.29–0.85).

TABLE 4
TABLE 4:
Internal Consistency and Test–Retest Reliability Results. Internal Consistency Is Shown for Each AOSpine PROST Item, Both Item-total Correlation (Rho) and Cronbach α if Item Deleted. Test–retest Reliability Was Assessed by Using ICC Along With Its 95% CI

Test–retest Reliability

Initially, 128 patients were included in the test–retest arm (USA n = 74, Canada n = 54). However, 80 (USA n = 49, Canada n = 31) could not be included in the test–retest analysis (Table 5). The basic patient characteristics were comparable to the overall study population (Appendix 3, http://links.lww.com/BRS/B569). The main reasons were not completing the questionnaire twice (n = 36) or not within an interval of 4 to 14 days (n = 27). A total of 48 patients could be included for test–retest analysis. The interval between the first and second administration was 8.9± 1.9 days (range: 7–14). Excellent test–retest reliability was seen (ICC = 0.97, 95% confidence interval 0.95–0.98). When looking into detail (Table 4), all items had acceptable to excellent reliability results, except for “Urinating” (ICC = 0.66).

TABLE 5
TABLE 5:
Reasons of Exclusion From Test–retest Analysis, n (%)

Factor Analysis

Two factors were found that had an Eigen value >1, indicating two possible dimensions across AOSpine PROST items. These factors had an Eigenvalue of 12.9 and 1.6, and explaining 67.6% and 8.1% of the variance, respectively. As shown in Appendix 4, http://links.lww.com/BRS/B569 and Figure 1, rotated factor analysis revealed that most items load high on Factor 1 and low on Factor 2, that is those items contribute considerably to the dimension represented by Factor 1. “Urinating” and “Bowel movement” seemed to load high on Factor 2 and low on Factor 1. A few items loaded high on both factors, that is, “Walking,” “Personal care,” “Emotional function,” and “Sexual function.” No item showed low loadings on both factors indicating that a possible third factor is not expected.

Figure 1
Figure 1:
Rotated factor pattern of AOSpine PROST factor analysis for the English version.

DISCUSSION

The present study describes the translation and cross-cultural adaptation of AOSpine PROST into English following established guidelines,21 and showed excellent psychometric properties among patients from 2 North-American centers.

In the translation and cross-cultural adaptation process it was chosen to incorporate stage V (prefinal testing) in the validation study. We found this to be a grounded decision as pretesting was already performed for the Dutch version in a previous study.19 As hypothesized, patients had no misunderstanding of the questions and no question was indicated as inapplicable. Patients provided very useful information on the general concept of AOSpine PROST: it was questioned whether they should also take the trauma-related injuries into account when scoring an item. This will be taken into account in the further development and refinement of the instrument.

Slight differences were observed in characteristics between American and Canadian patients (Table 1). The Canadian patients were younger, had more often no comorbidities, cervical spine fractures, neurologically impaired, and surgically treated. These characteristics together with the finding of sports or recreation as the main cause of trauma might indicate a healthier lifestyle among Canadian patients in the present study.28 However, increasing spine injuries have been seen in various thrill-seeking sports.29–32 A fall as cause of injury was frequently seen in our study population, which has shown to be a major public health problem in North-America and many other industrialized countries.29,33 Violence as the cause of injury was only seen in American patients. In previous literature it has been described that act of violence is not as common in Canada.34,35 Nevertheless, the AOSpine PROST showed very good results. This may be related to the unique approach of the tool by asking the patients to compare their current function with the pre-injury level of function (Appendix 1, http://links.lww.com/BRS/B569). Undoubtedly, besides patient-reported outcomes also clinical and radiological parameters are deemed relevant to have a holistic view on patients’ health and functional status. Therefore, the AOSpine KF Trauma is also making efforts on the development of an outcome measure from the perspective of the treating surgeons: AOSpine CROST (Clinician Reported Outcome Spine Trauma).36

With both Cronbach α and ICC being 0.97, very good internal consistency and test–retest reliability was seen. Alpha values remained high when an AOSpine PROST item was removed (Table 4), indicating that the tool might have some item redundancy. In a next phase including a larger patient sample, a stricter test of unidimensionality could be performed to identify redundant items. We did perform a factor analysis with the available data in the current study which showed somewhat contradictory results with the identification of two possible dimensions. All items loaded high to one specific dimension with the exception of “Urinating” and “Bowel movement.” This finding may be related to our patient selection with no or only mild neurological impairment. Bladder and bowel dysfunctions may be major impairments in patients with severe or complete neurological deficit.37,38 A remarkable finding from the concurrent validity analysis was the negative correlation with SF-36 General Health subscale. We could not find an explanation for this result. A positive correlation (rs = 0.58) was found in another study that investigated the psychometric properties of the Dutch version of AOSpine PROST.20

We do recognize this study has several limitations. The patient characteristics of the Canadian and USA patients were slightly different. Nevertheless, we believe this is an adequate reflection of the North-American spine trauma patients, and an adequate intent to include both populations in the present study. Another limitation was a higher than expected number of patients that were excluded from the test–retest analysis; however, even with these exclusions, the results of the study are still convincing. Furthermore, no prospective responsiveness analysis was performed, as it has already been done for the Dutch version and showed excellent results.20

In conclusion, the English version of AOSpine PROST showed to be a valid and reliable tool to measure outcomes in spine trauma patients. In the next phase, its applicability and psychometric properties among completely paralyzed patients will be tested. The AOSpine PROST is considered as a valuable tool for use in the clinical setting and research. It has the potential to contribute to the reduction of ongoing controversies in spine trauma care when used in a global setting, allowing us to compare different treatment options in a valid and reliable fashion.

Key Points

  • The AOSpine PROST (Patient Reported Outcome Spine Trauma) is a new disease-specific outcome measure for spine trauma patients.
  • Very good results were obtained for reliability and validity of the English version of AOSpine PROST.
  • The AOSpine PROST has the potential to compare different treatment options in a valid and reliable fashion, and contribute to the reduction of ongoing controversies in spine trauma care.

Acknowledgments

The authors thank AOSpine International for their support, as well as Vicky Kalampoki and Kathrin Espinoza-Rebmann (from AOCID) for their statistical analysis support. Also thanks to Leilani Reichl and Christie Stawicki for their contribution to patient recruitment, and Keita Ito, Sarah Opitz, Maaike Braham, and Sebastiaan Wijdicks for their help during the translation work.

References

1. Budh CN, Osteraker AL. Life satisfaction in individuals with a spinal cord injury and pain. Clin Rehabil 2007; 21:89–96.
2. Capoor J, Stein AB. Aging with spinal cord injury. Phys Med Rehabil Clin N Am 2005; 16:129–161.
3. Holtslag HR, Post MW, Lindeman E, et al. Long-term functional health status of severely injured patients. Injury 2007; 38:280–289.
4. Leucht P, Fischer K, Muhr G, et al. Epidemiology of traumatic spine fractures. Injury 2009; 40:166–172.
5. Hu R, Mustard CA, Burns C. Epidemiology of incident spinal fracture in a complete population. Spine (Phila Pa 1976) 1996; 21:492–499.
6. Roche SJ, Sloane PA, McCabe JP. Epidemiology of spine trauma in an Irish regional trauma unit: a 4-year study. Injury 2008; 39:436–442.
7. Jazayeri SB, Beygi S, Shokraneh F, et al. Incidence of traumatic spinal cord injury worldwide: a systematic review. Eur Spine J 2015; 24:905–918.
8. Lee BB, Cripps RA, Fitzharris M, et al. The global map for traumatic spinal cord injury epidemiology: update 2011, global incidence rate. Spinal Cord 2014; 52:110–116.
9. Nijendijk JH, Post MW, van Asbeck FW. Epidemiology of traumatic spinal cord injuries in The Netherlands in 2010. Spinal Cord 2014; 52:258–263.
10. Schoenfeld AJ, Bono CM. Measuring spine fracture outcomes: common scales and checklists. Injury 2011; 42:265–270.
11. Stadhouder A, Buckens CF, Holtslag HR, et al. Are existing outcome instruments suitable for assessment of spinal trauma patients? J Neurosurg Spine 2010; 13:638–647.
12. Oner FC, Jacobs WC, Lehr AM, et al. Toward the development of a universal outcome instrument for spine trauma: a systematic review and content comparison of outcome measures used in Spine Trauma Research Using the ICF as reference. Spine (Phila Pa 1976) 2016; 41:358–367.
13. Oner C, Sadiqi S, Lehr AM, et al. The need of validated disease-specific outcome instruments for spine trauma. J Orthop Trauma 2017; 31: (suppl 4): S33–S37.
14. Korovessis P, Baikousis A, Zacharatos S, et al. Combined anterior plus posterior stabilization versus posterior short-segment instrumentation and fusion for mid-lumbar (L2-L4) burst fractures. Spine (Phila Pa 1976) 2006; 31:859–868.
15. Siebenga J, Leferink VJ, Segers MJ, et al. Treatment of traumatic thoracolumbar spine fractures: a multicenter prospective randomized study of operative versus nonsurgical treatment. Spine (Phila Pa 1976) 2006; 31:2881–2890.
16. Wood KB, Li W, Lebl DR, et al. Management of thoracolumbar spine fractures. Spine J 2014; 14:145–164.
17. Selb M, Escorpizo R, Kostanjsek N, et al. A guide on how to develop an International Classification of Functioning, Disability and Health Core Set. Eur J Phys Rehabil Med 2015; 51:105–117.
18. WHO, World Health Organization. International Classification of Functioning, Disability and Health: ICFed. 2001.
19. Sadiqi S, Lehr AM, Post MW, et al. Development of the AOSpine Patient Reported Outcome Spine Trauma (AOSpine PROST): a universal disease-specific outcome instrument for individuals with traumatic spinal column injury. Eur Spine J 2017; 26:1550–1557.
20. Sadiqi S, Post MW, Hosman AJ, et al. Reliability, validity and responsiveness of the Dutch version of the AOSpine PROST (Patient Reported Outcome Spine Trauma). Eur Spine J in press
21. Beaton DE, Bombardier C, Guillemin F, et al. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976) 2000; 25:3186–3191.
22. Ware JE Jr, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection Med Care. Med Care 1992; 30:473–483.
23. Dvorak MF, Johnson MG, Boyd M, et al. Long-term health-related quality of life outcomes following Jefferson-type burst fractures of the atlas. J Neurosurg Spine 2005; 2:411–417.
24. Fisher CG, Noonan VK, Smith DE, et al. Motor recovery, functional status, and health-related quality of life in patients with complete spinal cord injuries. Spine (Phila Pa 1976) 2005; 30:2200–2207.
25. Mukaka MM. Statistics corner: a guide to appropriate use of correlation coefficient in medical research. Malawi Med J 2012; 24:69–71.
26. Terwee CB, Bot SDM, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007; 60:34–42.
27. OUP Oxford, Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Useed. 2008.
28. Qi V, Phillips SP, Hopman WM. Determinants of a healthy lifestyle and use of preventive screening in Canada. BMC Public Health 2006; 6:275.
29. Jackson AB, Dijkers M, Devivo MJ, et al. A demographic profile of new traumatic spinal cord injuries: change and stability over 30 years. Arch Phys Med Rehabil 2004; 85:1740–1748.
30. Tarazi F, Dvorak MF, Wing PC. Spinal injuries in skiers and snowboarders. Am J Sports Med 1999; 27:177–180.
31. Tator CH, Duncan EG, Edmonds VE, et al. Changes in epidemiology of acute spinal cord injury from 1947 to 1981. Surg Neurol 1993; 40:207–215.
32. Tator CH, Edmonds VE, New ML. Diving: a frequent and potentially preventable cause of spinal cord injury. Can Med Assoc J 1981; 124:1323–1324.
33. Kannus P, Niemi S, Palvanen M, et al. Continuously increasing number and incidence of fall-induced, fracture-associated, spinal cord injuries in elderly persons. Arch Intern Med 2000; 160:2145–2149.
34. Dryden DM, Saunders LD, Rowe BH, et al. The epidemiology of traumatic spinal cord injury in Alberta, Canada. Can J Neurol Sci 2003; 30:113–121.
35. Fisher CG, Noonan VK, Dvorak MF. Changing face of spine trauma care in North America. Spine (Phila Pa 1976) 2006; 31:S2–S8. discussion S36.
36. Sadiqi S, Verlaan JJ, Lehr AM, et al. Surgeon reported outcome measure for spine trauma an international expert survey identifying parameters relevant for the outcome of subaxial cervical spine injuries. Spine (Phila Pa 1976) 2016; 41:E1453–E1459.
37. Benevento BT, Sipski ML. Neurogenic bladder, neurogenic bowel, and sexual dysfunction in people with spinal cord injury. Phys Ther 2002; 82:601–612.
38. Anderson KD. Targeting recovery: priorities of the spinal cord-injured population. J Neurotrauma 2004; 21:1371–1383.
Keywords:

AOSpine PROST; function; health; outcome instrument; patient perspective; spine trauma

Supplemental Digital Content

Copyright © 2020 The Author(s). Published by Wolters Kluwer Health, Inc.