Secondary Logo

Assessing Satisfaction With Orthotic Devices and Services: A Systematic Literature Review

Peaco, Anna BS; Halsne, Elizabeth BS; Hafner, Brian J. PhD

JPO Journal of Prosthetics and Orthotics: April 2011 - Volume 23 - Issue 2 - p 95-105
doi: 10.1097/JPO.0b013e318217a0fe
Article
Free

Patient satisfaction is an abstract and multidimensional concept that is recognized to be an important component of evidence-based health care. Although there have been increased attempts to develop and use standardized, patient-reported outcome (PRO) measures for the evaluation of satisfaction with rehabilitation services, analogous efforts in orthotics and prosthetics (O&P) are less common. Measuring satisfaction in O&P may be challenging because patients' satisfaction is derived from their experiences with both the services and the devices provided. Similarly, patients' experiences with orthoses may differ from prostheses, because their motivations for use may be different. The purpose of this literature review is to identify and appraise PRO instruments that have been used to assess satisfaction with orthotic devices and services. A secondary purpose is to advocate for measures that are suitable for an O&P clinical environment. A systematic search of medical databases was conducted using a combination of the terms “satisfaction” and “orthotic” (and related synonyms, such as “orthosis” and “orthoses”). Publications were selected for review if they identified instruments that were administered with a clear and standardized protocol by an orthotist or a pedorthist to patients or participants with a diagnosis indicating orthotic intervention. The reviewed measures were classified as either formal (i.e., had an established history of psychometric development) or ad hoc (i.e., had not undergone formal psychometric testing of any kind). For each reviewed measure, the subdomains (i.e., determining factors of satisfaction) were identified through a review of the measure's items and the body of the associated publication(s). Fourteen publications met the selection criteria and were chosen for review. Publications dates ranged from 1989 to 2009, and studies included a variety of orthotic devices (e.g., ankle-foot orthoses, knee-ankle-foot orthoses, cranial helmets, spinal, etc.). Seven ad hoc and four formal PRO instruments were identified in the review. Formal measures included the Orthotics and Prosthetics Users' Survey (OPUS), the modified SERVQUAL instrument, the Quebec Users' Evaluation of Satisfaction with assistive Technology 2.0, and the Scoliosis Research Society 22/24 questionnaire. Seventeen subdomains of satisfaction were identified within the reviewed outcome measures. Eight subdomains pertained to device satisfaction (e.g., cosmesis or comfort), eight subdomains contributed to satisfaction with services (e.g., competence or communication), and one (i.e., overall satisfaction) was applicable to both device and service domains. The OPUS and SERVQUAL were identified as measures that may be well suited to the assessment of orthotic services (SERQUAL and OPUS) and devices (OPUS) in a clinical environment. Future research is recommended to focus on the use of standardized measures of satisfaction and dissemination of experiences with these instruments.

Patient satisfaction is an abstract and multidimensional concept that is recognized to be an important component of evidence-based health care. Although there have been increased attempts to develop and use standardized, patient-reported outcome (PRO) measures for the evaluation of satisfaction with rehabilitation services, analogous efforts in orthotics and prosthetics (O&P) are less common. The purpose of this literature review is to identify and appraise PRO instruments that have been used to assess satisfaction with orthotic devices and services. A secondary purpose is to advocate for measures that are suitable for an O&P clinical environment.

ANNA PEACO, BS, ELIZABETH HALSNE, BS, AND BRIAN J. HAFNER, PhD, Department of Rehabilitation Medicine, University of Washington, Seattle, Washington.

Disclosure: The authors declare no conflict of interest.

This work was supported by the University of Washington Department of Rehabilitation Medicine.

Correspondence to: Brian J. Hafner, PhD, Department of Rehabilitation Medicine, University of Washington, 1959 NE Pacific Street, Box 356490, Seattle, WA 98195; e-mail:bhafner@u.washington.edu.

The primary goals of a prosthetist and orthotist are to restore the physical functioning and improve the overall welfare of patients through the provision of a device. The American Board for Certification in Orthotics, Prosthetics and Pedorthics (ABC) describes the parameters of patient care for a certified prosthetist and orthotist in the Orthotics, Prosthetics & Pedorthics: Scope of Practice.1 This consensus document describes the clinical responsibilities of ABC- credentialed professionals, which include patient assessment, formulation and implementation of an appropriate treatment plan, and evaluation of the care provided.1 To support this professional paradigm, ongoing efforts must be made to assess and improve the effectiveness of orthotic and prosthetic (O&P) treatments.

Measuring and quantifying the success of patient care require the use of outcome measures, standardized tools, or instruments designed to assess treatment outcomes.2 The timely and judicious use of appropriate outcome measures allows practitioners to monitor O&P treatment plans over time and track patients' outcomes in a valid and reliable manner. Outcomes data collected using outcome measures can be saved, compiled, and analyzed for trends.2,3 Collecting information in this way could assist practitioners and businesses in identifying aspects of patient care that may be reinforced or altered to optimize patient outcomes. Presumably, the most relevant and informative data about treatment success in O&P may be collected using tools specific to orthotic and prosthetic devices and services. As such, research efforts have often targeted the development and use of O&P-specific outcome measures.

Recent reviews of prosthetic outcome measures have identified a number of O&P-specific instruments that are intended to be used in clinical practice and research.4,5 Most of the measures identified in these reviews were developed to assess functional capabilities or performance during functional tasks, such as lower-limb prosthetic mobility or upper-limb prosthetic hand function.6–10 This focus highlights the historic and common emphasis on orthotic or prosthetic devices and the functional outcomes achieved through using them. However, there have also been recent and increased efforts to evaluate patients' satisfaction with rehabilitation services through the use of standardized, patient-reported outcome (PRO) instruments.11–14 The attention given for developing, collecting, and analyzing patient satisfaction stems from a desire to improve the quality of rehabilitative care and, as a result, enhance outcomes achieved through the use of therapeutic interventions. From a business perspective, higher rates of patient satisfaction may also provide a competitive advantage to clinics and contribute to economic prosperity.15,16

Quantifying satisfaction is uniquely challenging because of the many elements of care that may influence a patient's level of satisfaction. Satisfaction in O&P additionally depends on how well patients' experiences met their expectations for both the devices and the services provided by practitioners and facilities.17 The typical provision of professional services in O&P differs from related rehabilitation disciplines (e.g., physical or occupational therapy) in that clinicians frequently provide tangible devices (i.e., orthoses or prostheses) as a part of the standard of care. The success of treatment outcomes and patients' satisfaction with their care are therefore often inherently linked to both the service experience and the use of their device(s). As such, the use of satisfaction outcome measures that are developed to address services may not be appropriate for measuring satisfaction with devices or vice versa. Furthermore, specific aspects of patients' satisfaction may vary based on the type of device they are prescribed and receive.

Fundamentally, prosthetic devices are intended to replace missing anatomical structures. In contrast, orthoses are indicated for a variety of reasons. An orthosis may maintain range of motion, support weakened musculature, or correct deformity. Prosthetic users typically rely on their device to perform desired activities, whereas orthotic users may choose not to use their device and still function, albeit suboptimally. Therefore, orthotic patients may be inclined to abandon a device if they are dissatisfied with its function. In addition, orthotic devices often serve as temporary interventions, whereas prosthetic devices are traditionally worn for a lifetime. This limited period of orthotic use may result in patients being less invested in the treatment and less inclined to pursue an entirely satisfactory outcome, especially if the device does not meet their expectations or needs. Given these differences, it seems appropriate to evaluate orthotic users' satisfaction with outcome measures that incorporate specific issues of importance to this patient population.

Although assessment of patient outcomes is required by ABC for facility accreditation, there are no formal recommendations for measures best suited to these measurements.18 Recently, the American Academy of Orthotists and Prosthetists has undertaken efforts to review and recommend existing outcome measures to assess prosthetic outcomes, but similar efforts for orthotic outcomes have yet to occur.4,5 Before selecting a measure of satisfaction that is appropriate for recipients of orthotic devices and services, it seems prudent to review the body of literature for measures that have been used to assess satisfaction with this patient population. A thorough review of the body of scientific literature is expected to identify measures and/or specific items related to orthotic users and may provide the information needed to make recommendations for those outcome measures most suited to an O&P clinical environment. Therefore, the purpose of this literature review was to identify and appraise existing PRO instruments that have been used to evaluate satisfaction with orthotic devices and services. Further, the intent was to identify those measures that are most conducive to administration in a clinical setting.

Back to Top | Article Outline

METHODS

LITERATURE SEARCH

A systematic search of the scientific literature was conducted using several computerized databases, including PubMed (1950 to January 2010), CINAHL (1982 to January 2010), and RECAL Legacy (1900 to 2007). The aim of the search was to locate publications that reported the use of PRO measures for the assessment of satisfaction with orthotic devices and/or services. A combination of the terms “satisfaction” and “orthotic” and related terms, such as patient satisfaction, orthosis, orthoses, and orthotic devices, was used to search each database. Reference lists in relevant publications were also reviewed for potential additions.

Back to Top | Article Outline

SELECTION CRITERIA

Publications were first considered for inclusion in the review if the abstract described the use of any type of outcome measure to assess satisfaction. This was to ensure that patient satisfaction was a focus of the publication. Articles were then retained for full review if they met the following additional selection criteria:

  • The satisfaction outcome measure used in the study was administered with a clear and standardized methodology. One goal of this review was to identify procedures for measuring patient satisfaction that may be appropriate for an orthotic and prosthetic (O&P) clinic. Therefore, studies were excluded if the protocol described in the publication was unclear or lacked sufficient detail to replicate administration of the instrument(s) in a clinical setting.
  • Participants involved in the study must have had a diagnosis indicating orthotic intervention. To target study population that were representative of patients traditionally seen in O&P clinics, publications were only considered for review if the participants described in the study had a condition that necessitated treatment with an orthotic device. Studies that described the use of orthotic interventions by able-bodied individuals (e.g., healthy, nondisabled individuals with no history of neurological or orthopedic pathology) were therefore not included in this review.
  • Any interventions described in the publication were fit by an orthotist or pedorthist. For the purposes of this review, the professionals involved in a study should be representative of those who typically fit orthotic devices in the O&P clinic (i.e., certified orthotists or pedorthists). Therefore, studies in which surgeons or physical therapists applied the intervention were excluded from this review.
  • The full-text article was written in English or available in an English translation.
  • Publications that met all four inclusion/exclusion criteria were selected for further assessment.
Back to Top | Article Outline

ASSESSMENT OF LITERATURE

The selected publications were first reviewed and classified based on the type (i.e., formalized or ad hoc) of outcome measure that was used to assess satisfaction. Formalized measures were defined as those measures that had undergone psychometric development to establish their reliability and validity. For the purposes of this review, a measure was classified as a formal outcome measure if the publication or cited literature described the development of the measure and/or the results of its psychometric testing. Ad hoc measures were defined as instruments (or questions) that had been developed for the study in which they were used, and no evidence of psychometric development or testing was provided.

For each selected publication, specific components (i.e., subdomains) that contributed to satisfaction were identified, extracted, and assessed for similarities with those described in other reviewed publications. Subdomains were subjectively identified from both the body of the publication and the individual items contained within the satisfaction measure, if they were provided. In the event that the publication did not include the described outcome measure (either within the body of the publication or an appendix), the original source publication was obtained from the relevant citations to identify any satisfaction subdomains.

The types of scale(s) used to measure satisfaction in each instrument were next identified. Scales were classified as open ended, nominal, ordinal, interval, or ratio, based on the provided response options. Open-ended satisfaction questions offered no response options but instead allowed the respondent to verbalize or write out an answer. Scales were classified as nominal if they were categorical scales that had no inherent ranking of response options (e.g., gender or etiology). Ordinal scales demonstrated a clear hierarchy (i.e., order) among response options, although the spacing between options was not equal in magnitude (e.g., mild, moderate, severe). Interval scales were represented by equally spaced response options without a meaningful 0 value (e.g., temperature measured in degrees Fahrenheit/Celsius). Ratio scales were characterized by equally spaced response options on a continuous scale with a meaningful 0 value (e.g., height, weight, walking speed). In the event that a reviewed outcome measure included multiple types of scales, each type of scale was recorded for that measure.2,3

Select publication characteristics such as purpose, research question(s), patient population(s), sample size, analysis methods, and results were extracted and summarized. These specific items were deemed to be those that would best indicate how well the satisfaction outcome measures were used in practical applications. The information collected was compiled into a Microsoft (Redmond, WA) Excel spreadsheet to examine similarities and differences among the reviewed studies and publications and to identify those PRO instruments that have the best potential for evaluating satisfaction with an orthosis in an O&P clinic.

Back to Top | Article Outline

RESULTS

LITERATURE SEARCH

A total of 84 unique publications were identified using the described search strategy. Pubmed produced a greater number of publications (n = 56) in the initial keyword search than did either the RECAL Legacy (n = 39) or CINAHL (n = 36) databases. Thirty-one of these publications were initially selected for review based on the content of their abstracts. After application of the aforementioned selection criteria, 14 publications were chosen for full review. An overview of the described search strategy and results is provided in Figure 1.

Figure 1.

Figure 1.

Dates of publication for articles included in this review ranged from 1989 to 2009. The reviewed publications originated from a variety of peer-reviewed journals, including several orthotic and prosthetic (O&P)-specific journals (e.g., Journal of Prosthetics and Orthotics and Prosthetics and Orthotics International). Others were published in more general rehabilitation journals such as Disability and Rehabilitation. The intervention of interest in the reviewed studies ranged from a single type of device (e.g., knee-ankle-foot orthoses) to a variety of both orthotic and prosthetic devices. Of the publications selected for review, seven described the use of ad hoc outcome measures and the remaining seven described the use of formal outcome measures for the assessment of user satisfaction. Specific demographics of the reviewed studies are included in Table 1.

Table 1

Table 1

Back to Top | Article Outline

SUBDOMAINS OF SATISFACTION

All the reviewed outcome measures included items (i.e., questions) related to satisfaction with devices, whereas only four measures addressed satisfaction with provided services. Subdomains within each of these categories were identified and described for the purposes of this review. For example, six different measures included at least one question about satisfaction with the users' satisfaction with the appearance of the orthotic device. These questions were ascribed to the “cosmesis” subdomain. In total, 17 subdomains (Table 2) were identified across the device and service categories. Eight unique subdomains were identified in the device category, eight subdomains were exclusive to the service category, and one subdomain (i.e., overall satisfaction) spanned both the device and service categories.

Table 2

Table 2

Back to Top | Article Outline

SCALES

The satisfaction measures identified through this review used a variety of measurement scales to document responses to individual items (Table 1). One publication described the use of a nominal scale, nine used ordinal scales, three used interval scales, and two used open-ended scales. Only one outcome measure included multiple types of scales.19

Back to Top | Article Outline

SATISFACTION OUTCOME MEASURES

In the interest of providing a historical context for the development and use of satisfaction outcome measures in O&P, the reviewed publications are presented in chronological order and classified as either “ad hoc” or “formal” based on the descriptions and citations included in the reviewed publication(s). The subdomains identified in each publication are noted in Table 3.

Table 3

Table 3

Back to Top | Article Outline

AD HOC MEASURES

Fisher and McLellan developed an ad hoc satisfaction questionnaire to perform a retrospective audit of lower limb orthotic services at Southampton General Hospital (Southampton, UK). The questionnaire was sent to patients who had received custom footwear, ankle-foot orthoses, or knee braces from the hospital. Items on the measure that pertained to device satisfaction included comfort, fit (i.e., dimensions), cosmesis, frequency of repairs and modifications to the device (i.e., durability), and overall satisfaction. This questionnaire was the only ad hoc measure in this review that incorporated items to measure satisfaction with services, including wait time and overall satisfaction with services. There is no description of the specific type of scale used in this measure, but the study results described two categories of responses: “satisfied” or “dissatisfied,” which implies the use of a nominal scale.20

Eames et al. conducted a cross-sectional review of patients who had presented with ruptured Achilles tendons and had received treatment with orthotic devices. Standardized interviews were conducted to assess patients' satisfaction with a specialized treatment protocol (that included the provision of an orthosis). Interview questions solicited patients' satisfaction with three areas: cast, orthosis, and outcome. The participants were asked to rate their satisfaction in these areas on a 0 to 10 visual analog ordinal scale. Participants were also polled about the overall effectiveness of the treatment in terms of residual pain, limp (i.e., gait deviations), joint stiffness, and their ability to return to previous sporting activities. The scale used to assess the presence of these outcomes was not described, but seemed to be a nominal scale, based on the dichotomous results presented.21

Jaspers et al. reported the outcomes of treatment with advanced reciprocating gait orthoses (ARGOs) fabricated for individuals with paraplegia. A standardized phone interview was conducted to evaluate patients' satisfaction with the ARGO in this study. The interview included questions about the amount of assistance needed to use the device (i.e., ease of use), activities performed with the ARGO (i.e., activity), cosmetic appearance of the device (i.e., cosmesis), and general appreciation of the orthosis (i.e., overall satisfaction). Interview questions were open ended.22

Peethambaran studied patients' experiences with the use of two different knee-ankle-foot orthosis designs (i.e., anterior approach and posterior approach). To determine satisfaction with the designs, a questionnaire was administered to patients diagnosed with unilateral postpolio paralysis at their initial orthotic fitting and at 6 weeks after delivery of their device. The questionnaire sought information about patients' perceived comfort, cosmesis, weight, effort associated with wear (i.e., ease of use), use of device (i.e., activity), function of device and effects on gait (i.e., effectiveness), frequency of repairs and adjustments (i.e., durability), and impact of the device on clothing (i.e., dimensions). This questionnaire included the most satisfaction subdomains of any outcome measure in this review. Many of the items (i.e., 11 of 20) in this measure used a 5-point ordinal scale. Each of these items was scored using item-specific response options (e.g., very light, light, neither light nor heavy, heavy, very heavy). Nine items used an open-ended response format.19

Alsancak examined the satisfaction of patients with traumatic radial nerve injury after the provision of dynamic wrist hand orthoses. Satisfaction was evaluated with a written questionnaire that was mailed to the study participants. The questionnaire included items about the devices' mechanical function (i.e., effectiveness), ease of donning and doffing (i.e., ease of use), durability, comfort, ability of the device to facilitate activities (i.e., activity), appearance (i.e., cosmesis), and price. A 4-point ordinal scale (i.e., poor, fair, good, excellent) was used to quantify satisfaction.23

Elwood et al. reported parents' satisfaction with a novel cranial remolding orthosis designed to treat children with deformational plagiocephaly. A retrospective phone survey was conducted to their measure satisfaction with the provided device. Parents were asked whether they would repeat the program with a subsequent child and whether they would recommend the treatment to a friend whose child had a similar condition (i.e., overall satisfaction). Parents were also asked to rate their satisfaction with the improvement (i.e., effectiveness) in their infant's head shape using an ordinal scale from 1 (least improvement) to 5 (most improvement).24

McGrath et al. evaluated the duration of treatment, compliance, and satisfaction of patients with decreased range of motion (i.e., supination and pronation) using an upper limb static progressive orthosis. To assess satisfaction, patients were asked, “Overall, how would you rate your satisfaction with the device?” The ordinal response scale ranged from 0 to 10 points, with 0 points representing complete dissatisfaction and 10 points representing complete satisfaction.25

Back to Top | Article Outline

FORMAL MEASURES

Geertzen et al.17 examined satisfaction with services administered to patients at four O&P facilities in The Netherlands in 2002. The authors elected to modify and use the SERVQUAL questionnaire to assess satisfaction. SERVQUAL was derived from a model of service quality developed by Parasuraman et al. in 1985. The original model consisted of 97 items that measured 10 different aspects of service: access, communication, competence, courtesy, credibility, reliability, responsiveness, security, tangibles, and understanding/knowing the customer.26 These were later consolidated into 22 SERVQUAL questions that spanned 5 “domains” of service quality including reliability, assurance (i.e., competence, courtesy, credibility, and security), tangibles, empathy (i.e., access, communication, and understanding/knowing the customer), and responsiveness.27 Each item in the SERVQUAL questionnaire was intended to be asked twice: once to ascertain the relative importance of the item to the respondent and then again to assess the respondent's experience with receiving that aspect of service.

Geertzen et al. modified and expanded SERVQUAL to include items of importance to O&P consumers based on feedback obtained from two-user focus groups. The modified questionnaire included 30 items about satisfaction with the device (i.e., overall, cosmesis, and dimensions) and services (i.e., wait time, communication, competence, consumer input, facility, follow-up care, respect, and team approach). Items were assessed using a 5-point ordinal scale (i.e., “important” to “not important” and “very good” to “very bad” for the importance and experience questions, respectively). For the purposes of the study by Geertzen et al.,17 participants were considered satisfied with an aspect of service if their rated experience was equal to or higher than the level of importance they had ascribed to it.

In 2003, Heinemann et al. proposed to develop a set of self-reported outcome measures with sound measurement properties that could be used in O&P clinics for the assessment of functional status, quality of life, and client satisfaction. The Orthotics and Prosthetics Users' Survey (OPUS) was created as a result of these efforts. The developers initially reviewed existing generic and O&P-specific outcome measures with a multidisciplinary advisory team to inform the development of OPUS. The advisory team identified three areas of focus for the instrument (i.e., functional status of the limb, health-related quality of life, and client satisfaction) based on that review and subsequent discussions. Child and adult versions of the self-report measure were originally developed for each of the focus areas. The initial OPUS satisfaction measure consisted of 27 items that addressed both satisfaction with the device and satisfaction with provided services.28 Two OPUS field tests and subsequent rating scale (i.e., Rasch) analyses were conducted to assess the effectiveness of the OPUS instrument.

The first field test was a phone interview with 66 participants (i.e., 52 adults and 14 children by parent proxy) who had received O&P services at the Rehabilitation Institute of Chicago (Chicago, IL). Based on initial results of the Rasch analysis and feedback from the participants, the adult and child versions of the instrument were combined. The authors identified a desirable hierarchy of item difficulty (i.e., some items were easier to endorse than others) through a second Rasch analysis of the combined data. Results of this analysis were used to remove redundant or misfitting items and to define a single ordinal scale for each area of focus. The revised version of OPUS included an 11-item measure for satisfaction with devices and a 10-item measure for satisfaction with services.

A second field test of 164 participants (i.e., 80 adults and 84 children) was conducted to assess the revised version of OPUS. Rasch analysis of these data produced acceptable results, and no further modifications were made to the instrument. Items pertaining to device satisfaction ultimately included weight, comfort, and pain associated with use (i.e., comfort), ease of use, cosmesis, durability, and fit and effect of the device on clothing (i.e., dimensions). To measure satisfaction with services, OPUS included wait time, respect, communication, consumer's input, team approach, and training with the device. Both satisfaction measures employed the same ordinal scale with four “extent of agreement” response options (i.e., strongly agree, agree, disagree, and strongly disagree).28

Wessels and De Witte29 described the reliability and validity testing of the Dutch version of the Quebec Users' Evaluation of Satisfaction with Assistive Technology (D-QUEST) conducted in a population of assistive device users (which included 217 users of orthotic devices). The Quebec Users' Evaluation of Satisfaction with Assistive Technology (QUEST) was originally developed in 1996 by Demers et al. at the Université de Montréal (Quebec, Canada). QUEST was designed to evaluate satisfaction with assistive technology (i.e., devices) and the accompanying provision of service. It contained 27 items that represented the “factors most likely to influence the degree of user satisfaction” (pg. 8).30 QUEST employed a 5-point ordinal scale that ranged from 1 to 5 (i.e., “not satisfied at all” to “very satisfied”). Like the SERVQUAL measure, each item in the original QUEST asked the respondent to indicate the importance of the item and then to rate their associated satisfaction.27,30 Interestingly, the respondent was only asked to indicate satisfaction for those items he or she deemed to be “quite” or “very” important. The QUEST was later revised, through item analysis and reduction, to QUEST 2.0, a 12-item questionnaire that included only satisfaction questions (i.e., the importance questions were removed). QUEST 2.0 contained eight items that pertained to characteristics of the device and four items that addressed service quality.31

D-QUEST was developed as a translated version of QUEST 2.0 but included one additional question about overall satisfaction in both the device and service areas of the measure.32 These questions were added to assess the validity of the D-QUEST instrument. The investigators also included a “not applicable” response option for all items. The resulting instrument addressed several device (i.e., overall satisfaction, dimensions, ease of use, effectiveness, weight, durability, and comfort) and service (i.e., wait time, competence, and follow-up care) satisfaction subdomains.

Weigert et al.33 studied quality of life and satisfaction of patients who underwent surgical and/or bracing treatments for the management of adolescent idiopathic scoliosis (AIS). To quantify outcomes, the authors used the Scoliosis Research Society (SRS)-24, an instrument originally developed by the society to assess surgical outcomes in AIS. Three items on the SRS-24 assessed satisfaction with the treatment, including two items that addressed treatment outcome (i.e., effectiveness) and one that solicited the patients' willingness to choose the treatment again (i.e., overall satisfaction). Items on the SRS-24 related to satisfaction were scored on a 5-point ordinal scale that ranged from 1 to 5 (i.e., “least satisfied” to “most satisfied”).34

In 2007, Cheung et al. used the SRS-22, a modified version of the SRS-24, to evaluate the impact of bracing on the quality of life and satisfaction of patients with AIS.35 The original SRS-24 questionnaire was determined to have low internal consistency, unknown test-retest reliability, and validity concerns after psychometric testing.36 The SRS-22 was developed through several revisions to the SRS-24 and included 10 of the original SRS-24 questions, 6 questions with modified response options, and 6 new mental health items adopted from the short-form 36.36–38 The satisfaction items of the SRS-22 included only one of the original items regarding treatment outcome (i.e., effectiveness) and a modified version of the original overall satisfaction item from the SRS-24. The satisfaction items on the SRS-22 were scored using 5-point ordinal scales that range from 1 to 5 (i.e., “very unsatisfied” to “very satisfied” and “definitely not” to “definitely yes”).37

In 2008, de Boer et al.39 examined the usage of wrist orthoses in patients with rheumatoid arthritis. To evaluate satisfaction with these devices, the authors used the aforementioned D-QUEST outcome measure.32 Two hundred forty participants completed the questionnaire.39 The items and response options remained the same as that originally described by Demers et al.32 and tested by Wessels and De Witte.29

In 2009, Bosmans et al.40 published a follow-up study to that reported by Geertzen et al.17 The authors again evaluated consumer satisfaction with services provided by O&P facilities in The Netherlands using a modified SERVQUAL questionnaire.40 The intent was to compare satisfaction outcomes at 15 facilities with those 4 reported in the previous study.17 For the purposes of this study, Bosmans et al. elected to modify the SERVQUAL outcome measure previously described and used. The questionnaire was shortened from 30 to 16 items, so as to eliminate redundancy that was indicated by a strong interitem correlation in the previous results. Response options remained the same (i.e., 5-point ordinal scale). As in the original study, respondents were considered satisfied if their rated experience was greater than their importance rating.17,40

Back to Top | Article Outline

DISCUSSION

The purpose of this review was to locate PRO measures that had been used to assess satisfaction with orthotic devices and services. Those identified measures were critically evaluated to assess their psychometric development and determine their potential for clinical use. A systematic review of the scientific literature revealed four formal outcome measures and seven ad hoc measures that have been used to evaluate orthotic patient satisfaction. The extent to which measures examined the construct of satisfaction ranged from general assessments (e.g., a single question about overall satisfaction) to more detailed considerations of contributing factors (i.e., items that spanned multiple subdomains of satisfaction). This diversity likely reflects the inherent challenges associated with measuring patient satisfaction.

Patient satisfaction has been recognized as an abstract and complex concept that is influenced by a multitude of factors. In 1983, Ware et al.41 characterized these as “dimensions” and detailed six dimensions that they believed contributed to overall patient satisfaction: interpersonal manner, technical quality, accessibility/convenience, efficacy/outcomes, continuity, physical environment, and availability. Other authors have similarly described the multidimensional nature of satisfaction.42–46 Although the descriptions of these dimensions vary across the body of literature, authors generally agree that a singular assessment of overall satisfaction may be inadequate to ascertain a comprehensive assessment of a patient's perspective.42,43,45 An additional complexity associated with measuring the patient perspective in orthotics and prosthetics (O&P) is that satisfaction is noted to depend on the quality of the device and the quality of the services provided.17 Thus, for the purposes of this review, device and service were identified as “domains” of satisfaction and the contributing determinants, historically denoted as “dimensions,” “domains,” or “factors,” were considered to be “subdomains” of device or service satisfaction.

Some subdomains such as comfort, ease of use, and cosmesis were more frequently identified in the reviewed outcome measures than others (Table 3). One likely explanation is that the interests or priorities of the researchers were reflected in the types of items and associated subdomains that were included in the measures.42,44,46 The number of items included in an outcome measure may initially seem to be representative of its comprehensiveness. However, the inclusion of many items and, by extension, many subdomains may lead to a longer, inefficient measure (i.e., one that includes redundant or misfitting items). This would increase respondent and administrative burden and may degrade the quality of information obtained. Results can also be influenced by the way items are presented. Most notably, the phrasing of an item or its response options could distort a patient's response if the language is biased or unclear.42,45 To develop a measure that is comprehensive, concise, and clear, psychometric testing is required.

Although the details of psychometric testing are beyond the scope of this review, description or citation of this process within a publication reflects the authors' effort to establish a measure's psychometric properties (e.g., scaling, reliability, validity, and responsiveness).2,3,47 A thorough description of these properties may give a reader or user confidence that the measure is well developed and suited for measuring the construct of interest. For the purposes of this review, the authors focused attention on measures that possessed documented histories of psychometric development, as they are likely most suited to accurately and efficiently measure satisfaction in a clinical environment.

This review identified four satisfaction instruments that have been used in O&P applications and have undergone systematic testing to describe and/or improve their psychometric properties. The developers of the OPUS reported a more extensive process of psychometric development (i.e., field testing, item reduction, and revision of scales) than of any of the measures included in this review. OPUS was also the only measure identified in this review that was designed for the explicit purpose of assessing patient satisfaction in O&P.28,48 Although it was originally developed for use in marketing, the SERVQUAL has been adapted for use in O&P and may be used to measure satisfaction with services.17,40 The psychometric properties of the SERVQUAL instrument have also been recently improved through a process of item reduction that may make the measure more efficient.40 The Quebec User Evaluation of Satisfaction with assistive Technology (QUEST) has a history of publications to support its psychometric development. However, neither QUEST nor its derivatives were explicitly designed for the measurement of patient satisfaction in O&P and may, therefore, not address issues of specific importance to orthotic (or prosthetic) users.29,32,39 Finally, the SRS-22 includes several items pertaining to satisfaction, but it is not a focus of this outcome measure. The SRS-22 was also originally developed to measure surgical rather than orthotic outcomes and may not be an ideal solution for all O&P applications.33,35

One of the original goals of this review was to seek the information needed to advocate for specific, existing outcome measures that may be well suited to measure patient satisfaction with orthotic devices and services. This review revealed that OPUS seems to be an appropriate tool for measuring patient satisfaction with orthotic devices and services in a clinical setting because of its extensive development and specific focus on O&P.28,48 SERVQUAL, as modified by Geertzen et al.17 and Bosmans et al.,40 may be an alternative option for the assessment of patient satisfaction with O&P services, with the recognition that it is not intended to measure satisfaction with devices. Although neither of these instruments are orthotic specific, issues of importance to orthotic users (i.e., cosmesis, comfort, durability) are included in both of these measures. As such, these measures may be appropriate for gathering informative data about patient satisfaction for both orthotic and prosthetic users and may, therefore, be especially useful to the many clinics that provide care in both areas.

Back to Top | Article Outline

LIMITATIONS OF STUDY

The keywords and inclusion or exclusion criteria used for this review were carefully targeted to retrieve publications that addressed orthotic satisfaction. Therefore, it is possible that there are other satisfaction outcome measures available in related fields (e.g., assistive technology) that were not returned in the conducted searches. In addition, there are a number of satisfaction measures that may have been used in clinics but have not been published in the literature. A keyword search of the Google (Mountain View, CA) search engine using the terms described in this review revealed numerous patient satisfaction surveys that are currently used by O&P clinics across the country. This is to be expected, given that ABC facility accreditation requires that clinics periodically assess the quality of their care through patient satisfaction surveys.18 However, this review has identified several reasons why the use of a formalized measure may be preferred to use of ad hoc measures in clinical practice.

Back to Top | Article Outline

FUTURE RESEARCH

One of the voids identified through this review is the presence of accessible, quality literature that describes the use of satisfaction outcome measures in clinical settings. Thus, there is little information about the recommended procedure(s) for administration of the identified measures, including the frequency of assessment, questionnaire format (e.g., paper vs. electronic), steps for data analysis, or utilization of outcomes. In addition, there is limited information regarding the associated benefits and challenges to administration of a formalized satisfaction measures (i.e., administrative burden, respondent burden, and return on investment) available to clinicians who may be interested in administering a formalized satisfaction outcome measure. Ideally, the use of formalized outcome measures and dissemination of administration experiences in the future may help to improve adoption of these techniques and serve to enhance the quality of patient care across the orthotic and prosthetic (O&P) profession.

Back to Top | Article Outline

CONCLUSION

Patient values and perspectives are important components of evidence-based practice. The assessment of satisfaction is one method for obtaining patients' perspectives, and such assessments are beneficial for both patients and clinics if the data are used to improve the quality of care. The ABC recognizes the importance of measuring outcomes and thus encourages accredited facilities to regularly evaluate patient satisfaction.18 This review has identified several measures that are available to clinicians interested in measuring patient satisfaction with orthotic devices and/or services. Use of these measures and dissemination of the results will not only help to improve the content and psychometric properties of these (and future) measures but also promote a professional culture that demonstrates the importance of integrating patients' perspectives into clinical care.

Back to Top | Article Outline

ACKNOWLEDGMENTS

The authors thank Daniel Abrahamson, CPO, and Susan Ewers, CPO, for reviewing the draft manuscript.

Back to Top | Article Outline

REFERENCES

1. American Board for Certification in Orthotics, Prosthetics, and Pedorthics. Orthotics, Prosthetics & Pedorthics: Scope of Practice. Alexandria, VA: American Board for Certification in Orthotics, Prosthetics, and Pedorthics; 2010.
2. Finch E, Brooks D, Stratford P, et al. Physical Rehabilitation Outcome Measures. 2nd ed. Hamilton, Ontario/Canada: Lippicott Williams & Wilkins; 2002.
3. McDowell I, Newell C. Measuring Health: A Guide to Rating Scales and Questionnaires. 2nd ed. New York, NY: Oxford University Press; 1996.
4. Wright V. Prosthetic outcome measures for use with upper limb amputees: a systematic review of the peer-reviewed literature, 1970 to 2009. J Prosthet Orthot 2009;9:3–63.
5. Condie E, Scott H, Treweek S. Lower limb prosthetic outcome measures: a review of the literature 1995 to 2005. J Prosthet Orthot 2006;18:13–45.
6. Gailey RS, Roach KE, Applegate EB, et al. The amputee mobility predictor: an instrument to assess determinants of the lower-limb amputee's ability to ambulate. Arch Phys Med Rehabil 2002;83:613–627.
7. Gauthier-Gagnon C, Grisé MC. The locomotor capabilities index: content validity. J Rehab Outcomes Meas 1998;2:40–46.
8. Wright FV, Hubbard S, Naumann S, et al. Evaluation of the validity of the prosthetic upper extremity functional index for children. Arch Phys Med Rehabil 2003;84:518–527.
9. Wright FV, Hubbard S, Jutai J, et al. The prosthetic upper extremity functional index: development and reliability testing of a new functional status questionnaire for children who use upper extremity prostheses. J Hand Ther 2001;14:91–104.
10. Hermansson LM, Fisher AG, Bernspang B, et al. Assessment of capacity for myoelectric control: a new rasch-built measure of prosthetic hand control. J Rehabil Med 2005;37:166–171.
11. Tervo-Heikkinen T, Kvist T, Partanen P, et al. Patient satisfaction as a positive nursing outcome. J Nurs Care Qual 2008;23:58–65.
12. Ervin NE. Does patient satisfaction contribute to nursing care quality? J Nurs Adm 2006;36:126–130.
13. Goldstein M, Elliott S, Guccione A. The development of an instrument to measure satisfaction with physical therapy. Phys Ther 2000;80:853–963.
14. Roush SE, Sonstroem RJ. Development of the physical therapy outpatient satisfaction survey (PTOPS). Phys Ther 1999;79:159–170.
15. Saxton JW. How to increase economic returns and reduce liability exposure: part 1–patient satisfaction as an economic tool. J Med Pract Manage 2001;17:142–144.
16. Drain M, Kaldenberg DO. Building patient loyalty and trust: the role of patient satisfaction. Group Pract J 1999;48:32–35.
17. Geertzen JH, Gankema HG, Groothoff JW, et al. Consumer satisfaction in prosthetics and orthotics facilities. Prosthet Orthot Int 2002;26:64–71.
18. American Board for Certification in Orthotics, Prosthetics, and Pedorthics. Comprehensive Orthotic & Prosthetic Accreditation Standards. Alexandria, VA: American Board for Certification in Orthotics, Prosthetics, and Pedorthics; 2007.
19. Peethambaran A. The relationship between performance, satisfaction, and well being for patients using anterior and posterior design knee-ankle-foot-orthosis. J Prosthet Orthot 2000;12:33–40.
20. Fisher LR, McLellan DL. Questionnaire assessment of patient satisfaction with lower limb orthoses from a district hospital. Prosthet Orthot Int 1989;13:29–35.
21. Eames MH, Eames NW, McCarthy KR, et al. An audit of the combined non-operative and orthotic management of ruptured tendo achillis. Injury 1997;28:289–292.
22. Jaspers P, Peeraer L, Van Petegem W, et al. The use of an advanced reciprocating gait orthosis by paraplegic individuals: a follow-up study. Spinal Cord 1997;35:585–589.
23. Alsancak S. Splint satisfaction in the treatment of traumatic radial nerve injuries. Prosthet Orthot Int 2003;27:139–145.
24. Elwood ET, Petronio J, Wood RJ. Parental satisfaction with the craniocap: a new cranial orthosis for deformational plagiocephaly. Cleft Palate Craniofac J 2005;42:340–343.
25. McGrath MS, Ulrich SD, Bonutti PM, et al. Static progressive splinting for restoration of rotational motion of the forearm. J Hand Ther 2009;22:3–8.
26. Parasuraman A, Zeithami V, Berry L. A conceptual model of service quality and its implications for future research. J Mark 1985;49:41–50.
27. Parasuraman A, Zeithami V, Berry L. Servqual: a multiple item scale for measuring consumer perceptions of service quality. J Retailing 1988;65:12–40.
28. Heinemann AW, Bode RK, O'Reilly C. Development and measurement properties of the orthotics and prosthetics users' survey (OPUS): a comprehensive set of clinical outcome instruments. Prosthet Orthot Int 2003;27:191–206.
29. Wessels RD, De Witte LP. Reliability and validity of the Dutch version of quest 2.0 with users of various types of assistive devices. Disabil Rehabil 2003;25:267–272.
30. Demers L, Weiss-Lambrou R, Ska B. Development of the Quebec user evaluation of satisfaction with assistive technology (quest). Assist Technol 1996;8:3–13.
31. Demers L, Weiss-Lambrou R, Ska B. Quebec User Evaluation of Satisfaction with Assistive Technology (QUEST Version 2.0)—an outcome measure for assistive technology devices. Webster: Institute for Matching Person & Technology; 2000.
32. Demers L, Wessels R, Weiss-Lambrou R, et al. Key dimensions of client satisfaction with assistive technology: a cross-validation of a Canadian measure in the Netherlands. J Rehabil Med 2001;33:187–191.
33. Weigert KP, Nygaard LM, Christensen FB, et al. Outcome in adolescent idiopathic scoliosis after brace treatment and surgery assessed by means of the scoliosis research society instrument 24. Eur Spine J 2006;15:1108–1117.
34. Haher TR, Gorup JM, Shin TM, et al. Results of the scoliosis research society instrument for evaluation of surgical outcome in adolescent idiopathic scoliosis. A multicenter study of 244 patients. Spine 1999;24:1435–1440.
35. Cheung KM, Cheng EY, Chan SC, et al. Outcome assessment of bracing in adolescent idiopathic scoliosis by the use of the srs-22 questionnaire. Int Orthop 2007;31:507–511.
36. Asher MA, Min Lai S, Burton DC. Further development and validation of the scoliosis research society (SRS) outcomes instrument. Spine 2000;25:2381–2386.
37. Asher M, Min Lai S, Burton DC, et al. Discrimination validity of the scoliosis research society-22 patient questionnaire: relationship to idiopathic scoliosis curve pattern and curve size. Spine 2003;28:74–78.
38. Ware JE Jr, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. conceptual framework and item selection. Med Care 1992;30:473–483.
39. de Boer IG, Peeters AJ, Ronday HK, et al. The usage of functional wrist orthoses in patients with rheumatoid arthritis. Disabil Rehabil 2008;30:286–295.
40. Bosmans J, Geertzen J, Dijkstra PU. Consumer satisfaction with the services of prosthetics and orthotics facilities. Prosthet Orthot Int 2009;33:69–77.
41. Ware JE Jr, Snyder MK, Wright WR, et al. Defining and measuring patient satisfaction with medical care. Eval Program Plann 1983;6:247–263.
42. Chow A, Mayer E, Darzi A, et al. Patient-reported outcome measures: the importance of patient satisfaction in surgery. Surgery 2009;146:3:435–443.
43. Walsh M, Walsh A. Measuring patient satisfaction with nursing care: experience of using the Newcastle satisfaction with nursing scale. J Adv Nurs 1999;29:307–315.
44. Keith RA. Patient satisfaction and rehabilitation services. Arch Phys Med Rehabil 1998;79:1122–1128.
45. Carr-Hill RA. The measurement of patient satisfaction. J Public Health Med 1992;14:236–249.
46. Fitzpatrick R. Surveys of patients satisfaction: I—important general considerations. BMJ 1991;302:887–889.
47. Roach KE. Measurement of health outcomes: reliability, validity and responsiveness. J Prosthet Orthot 2006;18:8–12.
48. Heinemann AW, Gershon R, Fischer WP. Development and application of the orthotics and prosthetics user survey: applications and opportunities for health care quality improvement. J Prosthet Orthot 2006;6:80–85.
Figure

Figure

Keywords:

patient satisfaction; orthotic devices; outcome assessment; questionnaires

© 2011 American Academy of Orthotists & Prosthetists