The term outcome measure (OM) may be used broadly to refer to any tool used to assess change in health status. This could include patient reported OMs such as questionnaires or standardized clinical assessment tests such as timed walking tests. OMs have different potential uses including informing clinical decision making, assessing quality of care, and facilitating audit and research.1,2 There is a high level of interest in the use of OMs in health care generally,1 and there is a body of literature considering the use of OMs in prosthetics and orthotics.2–13
Use of OMs offers the clinician and service user benefits such as aiding communication and clinical decision making.14 OMs seem particularly useful in prosthetics and orthotics, as in some cases measures may be repeated on the same occasion to detect immediate responses to a change in prosthetic/orthotic prescription. Although OMs may be used with prosthetic/orthotic patients by other professionals during their care pathway, there may be benefits to administration by prosthetists/orthotists. These could include rapid feedback on changes in prosthetic/orthotic prescription, the opportunity to use OMs when working without the support of a wider clinical team, and increased prosthetist/orthotist understanding of OM results when administered by others. It has been argued that prosthetists and orthotists need to take a more active role in contributing to the prosthetic and orthotic evidence base.15
Use of OMs by an individual profession is of interest as it is a specific element of clinical practice, which may have implications for educational planning. There is work published on the use of OMs by specific health professionals such as physiotherapists.16–18 There is some work on the use of OMs by prosthetists in the United States (US)3,12; however, there is no literature considering practice among orthotists. In addition, literature considering practice among prosthetists in the United Kingdom (UK) is limited to one study with a small sample size of prosthetists (n = 17).13 Knowledge of practice concerning OMs among prosthetists and orthotists is therefore limited. This preliminary study was therefore undertaken to explore practice concerning OMs by prosthetists and orthotists in the UK including OMs used, frequency of OM use, and barriers to OM use.
As no existing instrument was identified, a 34-item online questionnaire was devised (Appendix 1, see Supplemental Digital Content 1, http://links.lww.com/JPO/A13) by the authors and administered using online software19 hosted via the Web site of the British Association of Prosthetists and Orthotists (BAPO). Existing work was reviewed, and similar themes were included in the survey.3,16 Themes included types of OMs being used, barriers to use of OMs, and perceived benefits to use of OMs. Various factors that might affect frequency of use were included such as age, geographical location, education, experience, clinical specialism, and employer. Factors such as education have been linked to frequency of OM use in existing work.18 To determine the frequency of OM use, participants were asked to class their use as “never,” “rarely,” “sometimes,” “most of the time,” or “for every episode of care,” similar to the categories used in work that surveyed prosthetists in the US.3 Because the aim was to identify all OMs that prosthetists/orthotists were using, we left a free text box to enter any OM. Questions addressing understanding of OMs were included but are not reported on in this paper. The survey was initially piloted with 4 orthotists who were colleagues of one of the authors to see if the questions were clear and that the online format was fully functional. Minor changes to the wording or response options of 3 questions were made in response to comments from the piloting.
Participants were recruited by sending an email to all BAPO members and by advertising on the BAPO Web site. An advertisement was also placed on an online NHS forum.20 Participants were anonymous and were informed of the purposes of the survey. It was explained that completion of the survey indicated consent to participate in the study and that the study would be published. In the UK, research of this nature does not require approval by an NHS ethics committee,21 and this was confirmed for this study in writing by the lead author's research governance office. The survey ran for 1 month during July 2014 and was open to all prosthetists and orthotists registered with the Health and Care Professions Council (HCPC). The total number of prosthetists/orthotists registered with the HCPC in the UK was 998 as of October 2014.22
Data were analyzed using SPSS 22 software. Chi-square tests were performed, with a confidence interval of 95%. The chi-square tests compared demographic factors and reported barriers to OM use between two groups—those classed as routine users and nonroutine users. Routine users were defined as those reporting use “most of the time” or “every episode of care.” Nonroutine users were defined as those reporting use “never,” “rarely,” or “sometimes.” This aimed to identify factors that may positively or negatively influence frequency of use and to determine whether reported barriers varied between the groups and was based on the categories used in a similar study.3
PARTICIPANTS AND USE Of OMs
A total of 109 complete responses were received. Participants were diverse in terms of age, experience, and geographical location (Table 1). A total of 28.4% reported routine use of OMs. Only 6.4% reported use as “never,” whereas 23.9% reported use as “rarely.” The mean estimated percentage of episodes of care for which OMs are used was 40.0% (SD, 30.4) for all participants, 81.0% (SD, 13.0) for routine users, and 24.0% (SD, 17.6) for nonroutine users. There were no statistically significant differences between routine and nonroutine users in terms of age, experience, preregistration training, or perceived pressure to use OMs. There was no significant difference in routine OM use between those practicing only as orthotists and those practicing only as prosthetists, although slightly more orthotists than prosthetists were classed as routine users (30.6% and 27.0%, respectively). There were significantly more people classified as routine users who reported working in NHS outpatient (P = 0.007) or private clinics (P = 0.002) as compared with nonroutine users. There were significantly more people classified as routine users who reported having a Master’s degree (P = 0.032). There were significantly more people classified as routine users who reported their primary employer was a private company (P = 0.046). There were significantly more people classified as routine users who reported specializing in spinal orthotics, prosthetics-diabetes, prosthetics-vascular, or prosthetics–hip disarticulation (P = 0.023, P = 0.003, P = 0.000, and P = 0.032, respectively). Multiple specialties were reported, with a mean of 3.8 specialities among orthotists (SD, 2.3), 3.2 among prosthetists (SD, 2.1), 3.1 among those in dual practice (SD, 2.4), and 3.6 overall (SD, 2.3). A large number (n = 72) of individual OMs were reported (see Appendix 2, Supplemental Digital Content 2, http://links.lww.com/JPO/A14). Measures reported with a frequency of 5 or more are shown in Figure 1. The TUG, SCS, and NRS were the most commonly used. The TUG, NRS, and 10MWT were all used by prosthetists, orthotists, and those in dual practice.
Only 9.2% of participants reported any training in OMs during their preregistration training, and 59.6% reported that they had not received any training in OMs since qualifying.
BARRIERS TO USE OF OMs
The most frequently reported barriers to use of OMs were time limitations (75.2%), insufficient training in OMs (56.0%), and availability of clinical time for review appointments (46.8%) (Table 2). Most of the reported barriers to use of OMs were not significantly different when comparing clinicians classed as routine users and nonroutine. Availability of appointments and training was significant however (P = 0.006 and P = 0.000, respectively). Only 10.0% of participants in services without routine reviews were routine OM users, whereas the figure for services with routine reviews was 36.8%. This was significant, with clinicians working in services with routine review appointments more likely to be routine users (P = 0.013). Lack of motivation was negatively associated with OM use (P = 0.036).
USE OF OMs
It is encouraging that both prosthetists and orthotists in the UK are using OMs. A small sample (n = 17) of UK prosthetists was included in a recent PhD thesis of which 41% reported they were regular OM users.13 This seems higher than the frequency in this study; however, the definition of regular use was different—defined as at least once per week, which limits comparability. Reported routine use of OMs was lower (28.4% of all participants and 27% of prosthetists) than the 38% reported by prosthetists in the US in a study using a comparable definition of routine use to that used in this study.3 The study had a smaller sample size and included prosthetists who agreed to complete an OMs training program, so it may represent a more highly motivated cohort than in this study. If use of OMs is simply more established as a part of clinical practice among prosthetists in the US, then it implies that further work is required to increase their use in the UK. A survey of physical therapists in the US found that 47.8% use OMs, and of these users 23.8% use them regularly, which is lower than the results in this study.16 A study in Scotland found that 44% of physiotherapists and 35% of occupational therapists use OMs, but did not look at whether measures were used routinely or not.23 It is therefore difficult to make any firm conclusions about how OM use among UK prosthetists/orthotists compares to other health professions.
The reasons for the relationship between employer and work environment and OM use are not clear. Increased OM use in private clinics may be related to reduced time pressures. In the UK, public prosthetic and orthotic services are often provided by subcontracted private companies. The increased OM use among clinicians employed by private companies could reflect company initiatives to implement OMs, although this was not explored by the survey. The increased use of OMs by clinicians specializing in spinal orthotics might be explained by the clinical relevance and long-established use of measures of body structure such as the method for calculating Cobb angle. Increased use among prosthetists who report hip disarticulation as a specialism may be due to a requirement to justify high cost components. Reasons for increased OM use among clinicians reporting they specialize in prosthetics-diabetes and prosthetics-vascular are unclear.
Although 72 individual OMs were reported, only 14 were reported with a frequency of 5 or more. This may reflect a disjointed approach to OMs, with different centers and clinicians selecting different measures. There could be various benefits to a more standardized group of measures being used including increased ease of comparing data between centers.
BARRIERS TO USE OF OMs
Time is commonly cited as the main barrier to using OMs by other allied health professionals (AHPs)16,17 whereas Copeland and colleagues18 suggest that understanding of OMs is a more important factor. Barriers in this study seem to include service level considerations, such as appointment availability and routine review appointments, in addition to clinician factors such as training and motivation. The influence of factors at the individual and organizational level on OM use is recognized in the literature.24 Variation in the quality of UK orthotic services has previously been identified, with routine review appointments a specific area for improvement.25,26 Available time in appointments was the most commonly reported barrier; however, it was not a significant factor when comparing routine with nonroutine users. The fact that availability of review appointments was a significant barrier indicates that the availability of clinical time generally is a factor. This implies that change in practice and increased OM use may require service redesign and support at higher management level in addition to clinician motivation; services need to be consistently commissioned so that the provision of routine review appointments is possible.
The association between Master’s level education and OM use identified in this study has also been reported by other allied health professionals.18 Lack of training is reported in this study as a barrier, meaning that some form of training is needed to increase OM use. This could include undergraduate or postgraduate training. Specific postgraduate training in OMs has been shown to improve confidence in their use among prosthetists in the short and longer term.3,12
STRENGTHS AND LIMITATIONS
The completed responses (n = 109) represented 10.9% of the total population of UK prosthetists/orthotists at the time of completion, which slightly exceeds the sample of a similar survey of UK orthotic clinical practice.27 The sample size also exceeds existing work looking at use of OMs among prosthetists/orthotists.3,12,13 However, the sample by its nature represents clinicians who are interested in responding to a research questionnaire and may therefore be more likely to be interested in research and use of OMs. One of the main limitations was the survey used to gather data, which was devised by the authors and unvalidated. Respondents may have misunderstood some questions, for example, in the list of reported OMs (Appendix 2, see Supplemental Digital Content 2, http://links.lww.com/JPO/A14), some items are included which are not OMs. These included Dr. Foster, which is a health care data analysis service, and BACPAR toolkit, which is a list of OMs. The broad scope of the questionnaire and range of OMs considered may also be a weakness in the study. As a result of the nature of the data obtained, only chi-square tests were performed. If redesigning the survey, changing the response options would allow parametric analysis to be carried out.
Future work could build on this preliminary study by examining a more focused area in detail such as particular types of OMs, or OMs for specific patient groups or clinical specialisms. Repetition of a similar survey in future could attempt to monitor any changes in practice, similar to existing research.12,17,28
This preliminary study presents a picture of current practice concerning OMs among UK prosthetists and orthotists. Although the majority of prosthetists and orthotists use OMs and a wide variety of OMs are in use, most are not routine users and the OMs used are very varied. There may be benefits to increased use of patient-reported OMs particularly, and use of a smaller more standardized group of measures. Use of OMs is affected by factors including education, work environment, and clinical specialism. Lack of training, availability of appointments, and motivation are barriers to the use of OMs.
This study was used to inform the development of educational material by BAPO.29 The authors would like to acknowledge the other members of the BAPO Outcome Measures Working Group, Chris Cody, and Howard Woolley.
1. Black N. Patient reported outcome measures
could help transform healthcare. BMJ
2. Robinson C, Fatone S. You’ve heard about outcome measures
, so how do you use them? Integrating clinically relevant outcome measures
in orthotic management of stroke. Prosthet Orthot Int
3. Gaunaurd I, Spaulding SE, Amtmann D, et al. Use of and confidence in administering outcome measures
among clinical prosthetists: Results from a national survey and mixed-methods training program. Prosthet Orthot Int
4. Resnik L, Borgia M. Reliability of outcome measures
for people with lower-limb amputations: distinguishing true change from statistical error. Phys Ther
5. Xu J, Kohler F, Dickson H. Systematic review of concepts measured in individuals with lower limb amputation using the International Classification of Functioning, Disability and Health as a reference. Prosthet Orthot Int
6. Lindner HY, Nätterlund BS, Hermansson LM. Upper limb prosthetic outcome measures
: review and content comparison based on International Classification of Functioning, Disability and Health. Prosthet Orthot Int
7. Deathe AB, Wolfe DL, Devlin M, et al. Selection of outcome measures
in lower extremity amputation rehabilitation: ICF activities. Disabil Rehabil
8. Hill W, Kyberd P, Norling Hermansson L, et al. Upper Limb Prosthetic Outcome Measures
(ULPOM): a working group and their findings. J Prosthetics Orthot
9. Kohler F, Cieza A, Stucki G, et al. Developing core sets for persons following amputation based on the International Classification of Functioning, Disability and Health as a Way to Specify Functioning. Prosthet Orthot Int
10. Gallagher P, Desmond D. Measuring quality of life in prosthetic practice: benefits and challenges. Prosthet Orthot Int
11. Condie E, Scott H, Treweek S. Lower limb prosthetic outcome measures
: a review of the literature 1995 to 2005. J Prosthet Orthot
12. Hafner BJ, Spaulding SE, Salem R, et al. Prosthetists’ perceptions and use of outcome measures
in clinical practice: Long-term effects of focused continuing education
. Prosthet Orthot Int
13. Scopes J. Outcome measures of physical function in adult unilateral lower limb amputees during prosthetic rehabilitation: use in clinical practice and psychometric properties
[dissertation]. Edinburgh: Queen Margaret University, 2016. Available at: http://etheses.qmu.ac.uk/2604/1/2604.pdf
14. Stokes EK. Rehabilitation Outcome Measures
. London: Churchill Livingston Elsevier; 2011.
15. Ramstrand N, Brodtkorb TH. Considerations for developing an evidenced-based practice in orthotics
. Prosthet Orthot Int
16. Jette DU, Halbert J, Iverson C, et al. Use of standardized outcome measures
in physical therapist practice: perceptions and applications. Phys Ther
17. Stokes EK, O’Neill D. Use of outcome measures
in physiotherapy practice in Ireland from 1998 to 2003 and comparison to Canadian trends. Physiother Can
18. Copeland JM, Taylor WJ, Dean SG. Factors influencing the use of outcome measures
for patients with low back pain: a survey of New Zealand physical therapists. Phys Ther
19. Schmitz C. LimeSurvey: an open source survey tool. Hamburg
2012. Available at: http://www.limesurvey.org
22. Health and Care Professions Council. Prosthetists/orthotists]. 2014 [cited 2014 Oct 6]. Available at: http://www.hpc-uk.org/aboutregistration/professions/index.asp?id=9#profDetails
23. Chesson R, Macleo M, Massie S. Outcome measures
used in therapy departments in Scotland. Physiotherapy
24. Duncan EA, Murray J. The barriers and facilitators to routine outcome measurement by allied health professionals in practice: a systematic review. BMC Health Serv Res
25. Hutton J, Hurry M. Orthotic Service in the NHS: Improving Service Provision. 2009.
26. Boxer P, Flynn T. Orthotic Pathfinder: A patient focused strategy and proven implementation plan to improve and expand access to orthotic care services and transform the quality of care delivered. 2004. http://www.bhta.net/sites/default/files/document-upload/2012/orthotic_pathfinder_report_july_2004.pdf
27. Eddison N, Chockalingam N, Osborne S. Ankle foot orthosis-footwear combination tuning: an investigation into common clinical practice in the United Kingdom. Prosthet Orthot Int
28. Skinner A, Turner-Stokes L. The use of standardized outcome measures
in rehabilitation centres in the UK. Clin Rehabil
29. Young J, Rowley L, Lalor S, et al. Measuring Change: an Introduction to Clinical Outcome Measures in Prosthetics and Orthotics
. 1st ed. Paisley: British Association of Prosthetists and Orthotists; 2015.