Secondary Logo

Health Technology Assessment as Theoretical Framework to Assess Lower-Limb Prosthetics—Issues and Opportunities from an International Perspective

Cutti, Andrea G., MEng, PhD; Lettieri, Emanuele, MEng, PhD; Verni, Gennaro, MEng

JPO: Journal of Prosthetics and Orthotics: January 2019 - Volume 31 - Issue 1S - p P55–P73
doi: 10.1097/JPO.0000000000000235

ABSTRACT Lower-limb prosthetics is rapidly advancing through innovative technologies. Simultaneously, patients in need of prosthetic rehabilitation treatments are expected to double by 2050. Unfortunately, available economic resources are not growing at a comparable pace. The result is the widening gap between what is technologically possible and what is economically sustainable. In this context, health technology assessment (HTA) stands out as a multidisciplinary process, grounded on scientific evidence, that can support decision makers on how best to allocate limited funds to health interventions and technologies. The aim of this contribution is to introduce the HTA concepts and methods of possible interest for the field of orthotics and prosthetics (O&P) in general and lower-limb prosthetics, in particular, from an international perspective. This will be accomplished by answering six questions:

1. What is HTA and how does it differ from evidence-based practice (EBP)?

2. Why should we care about HTA in O&P?

3. Do policy makers care about HTA reports?

4. Are there structured tools to run HTA appraisals that are suitable for the O&P field?

5. Are there challenges to consider when running an HTA on medical devices, including lower-limb prosthetics?

6. Are there strategies to adopt to improve evidence collection and HTA on lower-limb prosthetics?

Collectively, the answers culminate in the position that HTA is highly desirable, but poses challenges to O&P, because medical devices have unique features (e.g., incremental innovation and operator dependence), and because the level of evidence is generally low. To address these challenges, we proposed seven driving concepts that can improve evidence collection and the HTA of lower-limb prosthetics. Among them, managed entry agreements, registry big data, and Internet-of-things strategies can support value-for-money approaches and can mark a true change in lower-limb prosthetic accessibility.

ANDREA G. CUTTI, MEng, PhD; and GENNARO VERNI, MEng, are affiliated with the INAIL Prosthetic Center, Vigorso di Budrio, Bologna, Italy.

EMANUELE LETTIERI, MEng, PhD, is affiliated with the Department of Management, Economics, and Industrial Engineering, Politecnico di Milano, Milan, Italy.

Invited submission to the AAOP State-of-Science Reports, May 24, 2018.

Disclosure: The authors declare no conflict of interest.

Correspondence to: Andrea Giovanni Cutti, MEng, PhD, Centro Protesi INAIL, Via Rabuina 14, 40054 Vigorso di Budrio, Bologna, Italy; email:

People who had an amputation have the right to return to an active and engaged life.1 The participation of a patient in an individualized prosthetic rehabilitation program, based on the principles of the International Classification of Functioning Disability and Health (ICF), is instrumental in making this right effective.2

The definition of this program is a complex task.3,4 It requires the cooperation of multidisciplinary health professionals who must:

  • coordinate their expertise and services around the patient (patient-centricity);
  • listen to the patient needs, either expressed or unexpressed;
  • agree with the patient on challenging but achievable goals;
  • identify the technical solutions to achieve the goals;
  • keep the patient engaged; and
  • define and evaluate the outcomes and update the program if needed.

To complete the task, health professionals make decisions based on their knowledge about the patient's general conditions, the safety, efficacy, and effectiveness of alterative technical options, and likely expected outcomes.

However, the complexity of this task extends beyond clinical considerations into economic realities. Ultimately, clinical services must be paid, either personally (e.g., out of pocket) by an insurance or by the national health care service. Because financial resources are limited, they should be allocated on sustainable, effective, and efficient products and services and thus can return value for money. Clinical decision making therefore must align with economic constraints and the willingness-to-pay thresholds of the funding entities. Any decision on this regard, either at the single-patient or societal level, has direct ethical implications. Further, clinical thinking may face several organizational issues. The needed medical, administrative, or legal professionals within the health facility might not be available, theoretically and practically trained or motivated; the matrix of responsibility might be poorly defined; and the facility itself might be inadequate.

There is a widening tradeoff between what is technologically possible and what is economically sustainable.5 This consideration is very relevant in lower-limb prosthetics, if we consider these eight factors:

  1. There are established technologies that we can refer to as “standards of care,”6,7 such as mechanically controlled knees and carbon fiber feet.
  2. Over the past 20 years several “bionic” (mechatronic) components, mostly knees and ankles, have appeared on the market. In addition, the concept of direct attachment of the exoprosthesis to the skeleton of the patient through the surgical implantation of a fixture with an external exposure (osseointegration) has become increasingly common.8
  3. More than one “bionic” option is available.
  4. All solutions are technologically sophisticated.
  5. Manufacturers claim, based on different levels of evidence, substantial improvements for patients in one or more ICF domains.
  6. Typically, new solutions are more expensive compared with previous prosthetic solutions.
  7. Implants expose patients to lifelong biological risks.9
  8. Because of the Internet, patients are increasingly aware of these technologies and their associated promise, but often unaware of their limitations, creating false expectations.

In such context, professionals and patients should have access to comprehensive assessments of emerging solutions. Policy makers and payers have the same need as they must care for the general population and because they are responsible for the public/investors' money.

The consolidating discipline of health technology assessment (HTA) serves this purpose, as it offers a structured theoretical framework and methods to aggregate multiple perspectives and inform decision making on drugs, medical devices (MDs), and procedures.

The aim of this contribution is to introduce the background and methods of HTA to the professionals involved in orthotic and prosthetic (O&P) services in general and lower-limb prosthetics, in particular, from an international perspective. The following six questions are addressed:

  1. What is HTA and how does it differ from evidence-based practice (EBP)?
  2. Why should we care about HTA in O&P?
  3. Do policy makers care about HTA reports?
  4. Are there structured tools to run HTA appraisals that are suitable for the O&P field?
  5. What are the challenges to consider when running an HTA on MDs, including lower-limb prosthetics?
  6. Are there strategies to adopt to improve evidence collection and HTA on lower-limb prosthetics?
Back to Top | Article Outline


Prostheses, orthoses, and their provision fall within the general category of heath technologies (HTs), which are defined by the International Network of Agencies for Health Technology Assessment (INAHTA) as “Any intervention that may be used to promote health, prevent, diagnose or treat disease, or for rehabilitation or long-term care. This includes pharmaceuticals, devices, procedures, and organizational systems used in health care.”10

The assessment of HTs has multiple definitions.11 For this article, we will refer to the one provided by the European Network for HTA (EunetHTA), which states that HTA is a “multidisciplinary process that summarizes information about the medical, social, economic, and ethical issues related to the use of a health technology in a systematic, transparent, unbiased, robust manner. Its aim is to inform the formulation of safe, effective health policies that are patient focused and seek to achieve best value. Despite its policy goals, HTA must always be firmly rooted in research and the scientific method.”

Based and this definition and further indications of the EunetHTA,5 HTA reports address questions about a new technology in these nine domains:

  1. health problems and current use of technology;
  2. description and technical characteristics;
  3. safety;
  4. clinical efficacy and/or effectiveness;
  5. costs and economic evaluation;
  6. ethics;
  7. organizational issues;
  8. patients and social; and
  9. legal.

A full HTA report addresses all nine domains and its production is thoroughly articulated, time-consuming, and financially expensive. This is one of the criticisms of HTA. For this reason, rapid HTA has been developed, which typically focus on points from 1 through 4.5

HTA and EBP have topics in common (see Annex 1 for recalling the definition and aims of EBP), especially regarding points 1 and 4; they both address the background of a pathology and try to identify the most appropriate HT to address it by looking at scientifically solid evidence. For this purpose, they look at the traditional evidence pyramid where systematic reviews and meta-analyses are at the top, followed by randomized controlled trials (RCTs), cohort studies, case-control studies, and case-series/reports. However, the “label” of the study is not necessarily indicative of its quality in terms of robustness of the research design, the quality of its conduction, and the strength of its conclusions.12 Therefore, both HTA and EBP look at transparent systems to appraise studies, either through comprehensive approaches, such as GRADE,13 or specific checklists, such as CASP,14 AMSTAR,15 PRISMA,16 AGREE,17 and STROBE.18 As further overlap with HTA, recent views related to EBP have stressed the relevance of considering the patient's situation and values (e.g., social support, financial situation), and practice context (e.g., limited funding), taking into consideration points 5, 6, and 8, and arousing some debate.19–21

However, there are differences between HTA and EBP.22 The actors in EBP are the clinical professionals addressing the needs of a single patient. Thus, although the scientific literature is the foundation for consideration, it must be integrated with individual clinical expertise, as available evidence may not be applicable or be inappropriate for the specific clinical case under treatment. The two main questions in EBP are therefore, “Can it work?” (in an ideal case) and “Does it work?” (in the real world) when the technology is applied to a single given patient.

EBP guidelines can trigger the start of HTA analyses.

HTA reports extend more broadly and have been addressed to policy makers since its infancy in the early 1970s in the United States and Sweden, with emphases on safety, effectiveness, and costs, extending the core questions of EBP with economic analysis.23 As reported by Banta,24 the field has been driven since its early days by policy makers' concerns about costs. The World Health Organization (WHO) stressed this aspect in its definition of HTA,11 when saying that HTA “is used to inform policy and decision making in health care, especially on how best to allocate limited funds to health interventions and technologies […] in support of universal health coverage.” We can state, therefore, that HTA tries to answer the question “Is it worth it?” We can rephrase this question as “How to get best value for patients out of the available money?” With this mindset, HTA tries to close the gap between research, clinical practice, and public policy.25

Back to Top | Article Outline


There are at least five reasons why we should care about HTA in O&P:

  1. The macroeconomic health care data available from public repositories.26–29
  2. Reports about the quality of health care.
  3. The increase in high-impact pathologies.
  4. Political actions around O&P.
  5. Social media.

By looking at these aspects, we can shed light on why HTA can play a major role to ensure fair treatment for patients at present time.

Back to Top | Article Outline


Figure 1 shows the health care costs as percentage of the gross domestic product (GDP).30,31 After 20 years of percentage increase, in 2008 to 2009, the economic crisis imposed a strong revision of health expenditure, such that, for most countries, health expenditure started to grow with the same rate as the GDP. However, since 2015 to 2016, the health expenditure has started to grow again in excess of the GDP. Specific to O&P, these conflicts increased the gap between the most advanced technologies that are available and the resources that can be invested, thus limiting the capability to adopt new sophisticated and more expensive technologies to treat a large audience.

Figure 1

Figure 1

Back to Top | Article Outline


The Unites States consumes more economic resources for health care than any other country, spending 17% of the GDP. However, the United States is not superior to the other countries in terms of health quality.32,33 The life expectancy in the United States is almost 10 years less than Italy, whose health care expenditure per capita is about two thirds less.34 This suggests a need to scrutinize money allocation and health care accessibility, moving from fee-for-service to fee-for-value policies,35,36 starting from prevention to end with treatment, including O&P services.

Back to Top | Article Outline


The O&P market (4.4 billion, with 1.7 billion in lower-limb prosthetics),37 is a small portion of the overall health care technology expenditure: it accounts for 0.13% of the health care costs in the United States38 and, for instance, 1.7% in Italy.39 Nonetheless, it is attracting attention from national authorities that are putting pressure on how services are provided. The cases of the United States and Italy are examples.

In the United States, the report of the Department of Health and Human Services (HHS) Office of Inspector General (OIG) dated 201140 stated that the amount that Medicare had spent per individual with amputation in the recent past had significantly increased, due to the misconduct of certain O&P services in “upcoding” the level of technology available to their patients.41 Shortly after that, the CMS “Dear Physician” letter downgraded O&P service providers to suppliers, excluding their notes from the clinical records needed to justify the application of specific prosthetic products. This position has recently changed after strong debate.42,43 Moreover, in 2015, the CMS anticipated its intention to tie 30% of Medicare payments to quality or value through alternative payment methods, such as accountable care organizations (ACOs) or bundled payment arrangements.36 This target was reached in January 2016.44

In Italy, the new Essential Levels of Care of the National Health Care Service were approved in 2017 after 18 years. Importantly, in light of health care cost containment, it failed to introduce any innovative products for lower-limb prosthetics aside from energy storage and return feet for K3 and K4 patients and interface liners. In addition, legislators45 are strongly promoting the definition of O&P guidelines and best practice methods, bonding their application to legal liability.

Evidence provided by HTAs, possibly in combination with new reimbursement methods (see Question 5), might help ensure the availability of those new products that are found to improve the quality of care of patients. The recent studies by Dobson et al.46 and Liu et al.47 related to lower-limb prosthetics go in this very direction by supporting the effectiveness of lower-limb prosthetics in relation to costs.

Back to Top | Article Outline


Obesity in young children and diabetes, which are relevant health conditions for O&P, is rising globally.48,49 Moreover, the number of people aged older than 60 years is projected to be over 2.1 billion in 2050 from 0.9 billion in 2015.50 Finally, the number of persons with amputation in the United States is projected to double by 2050.51 This evidence suggests that resources will be needed for treating both relatively young adults as well as the frail elderly.52 Conflicts in budget allocation might arise, especially if treatments and political decisions are not grounded on evidence.

Back to Top | Article Outline


Patients use the Internet to learn about their pathologies, to search for treatments, to share their experiences, and to rate their providers.53,54 Patients in need of O&P services want to be educated, to play an active role in their care,55 and to have their voice heard by lawmakers.56 Unfortunately, patients are sometimes informed about new technologies in a polarized way or by uninformed individuals.54 This is, for instance, the case of 3D printing in O&P.57

Therefore, it is essential that patients receive accurate information throughout their treatment experience, from their initial searches to subsequent discussions with their health care provider, to their performance and understanding of outcome measures. The information should be focused and true, provided in a simple and brief manner, and recognize that information is subjected to serial fading from what the clinician says to what the patient hears and understands, to what he or she remembers, and finally, to what they do.

If properly conveyed to patients, HTA information can address these concerns. New regulations for MDs in Europe58 now consider information provided through the Internet as part of the MD documentation. This means that marketing is confined to accepted regulatory documentation as only supported statements shall be released to the public.

Back to Top | Article Outline


Meeting high demands (more patients) and expectations (due to media exposure) with shrinking resources is challenging. One option is just to preserve the status quo or even cut health care expenditure. However, treating more patients with the same budget translates into a reduction of services or the same services for a smaller percentage of the growing target population. This may also lead to increased out-of-pocket expenses or a decline in patients seeking medical treatments.59–65 In some countries, this was the emergency response to the economic crisis, together with the start of a season of reforms (voluntary or imposed). As the economy is improving,66 a more socially sustainable optimization strategy would include the following:

  1. Provides current health technologies to patients through processes (e.g., organization, rehabilitation plans) that ensures best value.
  2. Scrutinizes new health technologies against established standards of care, to determine any advantages with respect to costs. This can help in implementing disinvestments in obsolete technologies to make room to newer and better ones at a fixed budget.67
  3. Addresses ethical, patient, societal, and legal concerns.
  4. Diffuses this high-quality, scientifically grounded information to all relevant stakeholders (including patients), with (multimedia) summaries tailored to the audience.
Back to Top | Article Outline


Making decisions based on HTA reports is critical. This is due to three main reasons: 1) local background, 2) legal binding of the HTA recommendations, and 3) fragmentation.

First, although the scientific evidence for conducting an HTA are global, the decision on providing the technology is strictly local, because it is based on the local policy maker's willingness-to-pay thresholds, which is conditioned by different approval processes,25,68 safety regulations, budget constraints, and legal and ethical frameworks. A comprehensive assessment on this topic is available through the ISPOR Global Health Care System Road Map, to which the reader is referred for insights.69 On an exemplary level, the Italian Constitution establishes the right to health; citizens are entitled for global coverage,70 based on Essential Levels of Care defined by the NHS, which is financed by general taxation. This description does not apply to the United States. However, health care systems in Italy and United States are similar in terms of decentralized decisions, because in Italy, substantial decisional power is assigned to its 21 Regional Health Care Services.

Second, the intricacy of decision making is further complicated by the level of obligation of decision makers to comply with the findings of HTA reports.69,71,72 Specifically, a recent WHO report72 highlighted that of 111 nations responding to an international survey about the use of HTA, 80% have a technology assessment program but the assessment is binding for only half of them.

Third, fragmentation further confounds any obligations to HTA findings:

  • In the United States, multiple parties run HTA programs.73 At the federal level, once the Food and Drug Administration (FDA) establishes licensing based on safety and efficacy, Centers for Medicare and Medicaid Services (CMS) (which addresses coverage and reimbursement) run formal HTA programs through the Agency for Health Research and Quality (AHRQ) and its “Evidence-Based Practice Centers.”74 However, these HTA reports are informative but not binding for CMS. On the private side, insurance companies may use HTA reports available at the national level and sometimes conduct their own technology appraisals, taking results into account.25,69,71,75
  • In Canada (except for Ontario), a single agency is responsible for HTA, namely, Canadian Agency for Drugs and Technologies in Health (CADTH), which was initiated in 1989 by Canada's federal, provincial, and territorial governments to offer a coordinated approach to assessing health technologies. However, CADTH reports are nonbinding.76
  • In Europe, about 50 agencies perform HTA at the national and regional levels, with additional technology assessment conducted at the local level (e.g., hospitals).77,78 The negative implications of this fragmented picture question whether HTAs are truly exploited for decision making, engender duplication of reports on the same technology within the same timeframe, create nonuniform document structures, cause a dispersion of funds, and support decisions based on different criteria.79 In January 2018, the European Commission proposed a novel regulation for HTA80 to strengthen the cooperation among member states with a focus on clinical aspects (comparative effectiveness, possibility for manufacturers to seek advice from HTA authorities, and the identification of emerging technologies with “horizon scanning”), while leaving decisions about economic, social, ethical, and organizational issues, including pricing and reimbursements, to local agencies. The cooperation on joint clinical assessment will be mandatory after 6 years from approval.

It is outside of the scope of this article to provide a comprehensive review for all European countries. However, we can mention two different approaches:

  • The United Kingdom has established a strong system based on evidence to support decision making inside the NHS. The most influential institution is the National Institute for Clinical Excellence (NICE), which was founded in 1999. NICE considers both clinical and cost-effectiveness in formulating its recommendations and runs single technology appraisal (STA), which covers a single technology for a single indication; multiple technology appraisal (MTA), which normally covers more than one technology, or one technology for more than one indication; and fast track appraisal (FTA), a process for technologies that offer exceptional value for money. NICE recommendations are binding, and NHS must provide treatments recommended by NICE.81
  • In Italy, HTA is considered an essential tool for NHS decision making regarding drugs and MDs. However, the actual implementation of the HTA program is fragmented at the regional level, with HTA tackling different dimensions and with indications that are binding only for 1 of 21 Regions (Veneto).82 To improve this fragmented situation, in 2016, the Italian government pushed toward the centralization of HTA reports at regional level only, under the coordination of a Leading Group, in which three national agencies (Agenas, AIFA, and ISS) collaborate with regional institutions.83–85 Furthermore, in 2017 the Italian government established more stringent regulations for the application of HTA on MDs.86

Although positive and negative elements are evident, the international picture points to an improved use of HTA for decision making, with legislators expected to increase the exploitation of HTA reports. Interestingly, HTA reports can be used incrementally, as stressed by the EUnetHTA,87 from creating awareness of problems in policy makers through policy decisions and applications and the eventual evaluation of the outcome. Producing HTA reports is therefore relevant. In this regard, the definition of common standards—in terms of methods and contents—for the production of HTA reports is key: only through the use of common HTA reports produced by other countries can each country have the evidence required to make informed decisions regarding the emergence of new health technologies.

In lower-limb prosthetics, HTA reports (or elements of it), are increasing. Most of them are focused on microprocessor-controlled knees or knees and ankles, for example, in Canada, France, United Kingdom, and United States.88–91 More recently, the RAND Corporation47 built upon existing evidence through simulation methods to account for uncertainties, as recommended by Iglesias,92 improving the strength of recommendations.47 Moreover, a wider focus on the economic value of lower-limb prosthetics has been promoted by AOPA93 through the Dobson et al.46 study. Finally, AHRQ has recently published a draft report about lower-limb prosthetics.94 Unfortunately, AHRQ as well as recent systematic reviews95–97 pointed out that the quality of studies in O&P is lower compared with the standards applied in the pharmaceutical industries. This quality issue is a key concern and is further discussed at Question 5.

Back to Top | Article Outline


The principles to run an HTA exercise are summarized in Table 1, as proposed by Drummond et al.23

Table 1

Table 1

There are multiple ways to translate these principles into a methodology to run an HTA.98 The EUnetHTA developed a comprehensive approach that is internationally recognized, called the HTA Core Model 3,5 which can serve as example. In this section we will focus on this tool. Specific points can be further complemented through the ISPOR methodologies.99,100

The Core Model was developed to enable international collaboration in producing HTA information and efficient sharing of the results to avoid redundant, overlapping effort in different countries and regions. It has three logical parts:

  1. Ontology: provides a list of the questions an HTA should answer.
  2. Methodological guidance: how to answer the questions.
  3. Reporting Structure: how to present the results.
Back to Top | Article Outline


HTA is intended to support policy makers through evidence-based answers. A clear definition of the questions to answer is essential and will determine how an HTA is run. When the Core Model 3 is used, the HTA team is expected to formulate policy questions addressing one or more of nine domains listed in Question 1. The Core Model 3 goes a step further: by proposing a list of questions that should or could be answered within each domain.

Specifically, questions are grouped into “assessment elements” comprising three “labels,” namely, “domain,” “topic,” and “issue.” The issue is the actual question, and the domain and topic clarify the context of the question. For each assessment element, the Core Model contains a specific “card” or table providing a range of information, including an explanation of the meaning of the question, methodological advice on how to answer the question, and the possible source of information for answering the question. To support the team that is running the HTA exercise in establishing the most relevant questions to address, each card specifies the generally recognized importance of the question and if the answer is likely applicable at the international level, that is, in different contexts. If the element is relevant and transferable, then it is classified as a “core element.” Critical elements that are not transferable are also considered “core elements.” Ultimately, the decision about which questions to answer lies with the team.

For each domain, Table 2 reports:

Table 2

Table 2

  1. the content to be addressed;
  2. the topics; and
  3. an exemplary question.

For domains 4 to 9, Table 2 also includes a discussion about key methodological issues, and application notes specific for O&P.

Back to Top | Article Outline


Methodological sections are distributed at three levels: introduction of the Core Model 3 manual, description of each domain, and within each element card. Suggestions are provided on how to complete the critical appraisals, including sources, best study designs to consider, outcome measures, quality assessment checklists for each study design, and the possible need for meta-analysis, that is, data synthesis. Moreover, indications about how to summarize data in tabular form and how to conduct meta-analyses are provided.

Back to Top | Article Outline


The initial part of the document should provide a summary of findings and introduce the context, the scope, and the methodology used in the analysis. Then, for each domain, the report should introduce the domain itself and the specific methodology used, and then it should follow a question-answer structure. Finally, questions and answers should be discussed and references and appendices provided.

Figure 2

Figure 2

Back to Top | Article Outline


HTA methods are most commonly applied to pharmaceuticals, which feature longer development times, stringent regulations for licensing and market entrance, as well as strict postmarket monitoring. Therefore, when an HTA is started, the drug is an established product, with substantial evidence from phase III and possibly phase IV data. HTA agencies that are accustomed to this standard can be disappointed when dealing with MDs, and very few have designed specific appraisal procedure for MDs.77,123–126 Indeed, MDs present several challenges for HTA that have received increasing attention and that were the subject of research projects in Europe.125,127–129 When addressing HTA on lower-limb prosthetics, is it important to be aware of these challenges and realize that they are common to all MDs.

The first reason for the difference between pharmaceuticals and MDs is that the latter have traditionally been less regulated. In Europe, for instance, clinical data are required but can be collected from smaller RCTs (e.g., less than 50 cases120 or even nonrandomized clinical trials), with loose long-term assessment.125,129,130 This affects the quality of the evidence and the strength of HTA recommendations.

Second, to speed up the approval process, manufacturers may claim equivalence to other devices already on the market (similar to the 510(k) process of the US FDA). This is for instance the case of many microprocessor-controlled knees. However, the US Supreme Court has acknowledged that substantial equivalence is no guarantee that an MD is safe and effective.

Also, MDs have five unique features compared with pharmaceuticals131:

  1. Incremental innovation: An MD can change quickly over time, because manufacturers can progressively introduce new features to make their products more appealing and tailored to the customer's needs, but also to cut costs, thanks to improved production processes. Therefore, it is sometimes difficult to understand when the MD is in “steady state,” and from that instant collect enough clinical data before it becomes obsolete.
  2. Training/operator dependence: The outcomes coming from the implementation of an MD depend, in many cases, on the training of the operators using/providing the MD.
  3. Learning curve: The greater the experience with the application of a given technology, the greater the professional competence, with better outcomes for patients and increased efficiency.
  4. Organization: MD exploitation commonly requires the definition of an interdisciplinary team that must be specifically trained; specific responsibilities must be agreed upon, and the staff must be supervised; quality indexes must be set and monitored, and actions for quality improvement must be planned. The use of an MD might also require substantial monetary investments, including infrastructures, and might require adaptations to the logistic chain and activation of specific maintenance programs.
  5. Dynamic pricing: The price of an MD can change rapidly, because manufacturers improve the value chain, because a newer product is set on the market, or because a competitor starts promoting a similar product claiming equivalence.

All these aspects fully apply to O&P and affect the level of uncertainty in formulating decisions.

Back to Top | Article Outline


New strategies have been described by teams of experts to collect evidence and run HTA on MDs, addressing the challenges reported in Question 5. The adoption of these strategies is expected to increase the chances of obtaining a truthful and unbiased picture of the MD in the nine domains described in Question 4. Once applied to lower-limb prosthetics, seven driving concepts can be formulated: 1) HTA does not evaluate a prosthetic device but a process; 2) we do not provide one prosthesis for a lifetime, but a treatment pathway; 3) HTA can be used to improve the provision of lower-limb prostheses; 4)we need advanced modeling; 5) proof of effectiveness can come from well-conducted observational studies; 6)prostheses as “Internet of things”; and 7) coverage with evidence development (instead of K-levels). These drivers are separately presented but are deeply interconnected.

  1. HTA does not evaluate a prosthetic device but a process: It should be made clear that when assessing a lower-limb prosthetic component, researchers are not testing the device in isolation. They are testing the efficacy and effectiveness of the prosthetic device when delivered by a specific group of professionals that are adopting a specific patient-care model. Therefore, both the device and the delivery methods are appraised. The authors producing evidence should explicitly state, and registries should include: a) which is the patient-care model adopted, for example, including the details of the rehabilitation program, patient engagement, psychological and social support provided; b) because training and learning curve are key factors in MD application, the literature should also report the experience and volume of treatments of the clinical staff.
  2. We do not provide one prosthesis for a lifetime, but a treatment pathway: Patients are not provided a single prosthesis for the lifetime. It is essential to clarify which is the treatment pathway, that is, which prostheses are provided and when, to reach which targets. The definition of the pathway is essential to clarify the cost implication of innovative prosthetic components that can incorporate, in a single device, multiple functions, for example, walking, bathing, and leisure activities. Neglecting the overall treatment pathway can result in wrong conclusions regarding the sustainability of a novel technology.
  3. HTA can be used to improve the provision of lower-limb prostheses: The consequence of point 1 is that cost-utility analysis can be used to understand if the patient-care model that is adopted to provide a prosthetic device is appropriate. The Italian Workers' Compensation Authority (INAIL) has adopted this original viewpoint to understand if the C-Leg microprocessor-controlled knee is fully exploited by different groups of patients when compared with a restricted set of nonmicroprocessor-controlled knees.119 It was possible to conclude that C-Leg has a statistically significant impact on “mobility” for first-time prosthetic users over 40 years, but psychosocial barriers limit its full exploitation. Therefore, the implementation of specific low-cost physical/psychosocial interventions is required to return within cost-utility thresholds for this specific group.
  4. We need advanced modeling: To tackle incremental innovations and increased uncertainties, Bayesian RCT and Bayesian modeling have been suggested as attractive future methods to consider.
  5. Proof of effectiveness can come from well-conducted observational studies: Lower-limb prostheses are rarely provided through a single large facility covering the needs of a vast number of patients. This makes the conduct of strictly ruled, large RCTs unlikely, weakening the strength of recommendations.95–97 However, as reported in Tarricone et al.,132 real-world data stored in registries can play a major role to support effectiveness. Following the example of the advanced modeling technique adopted in Dobson et al.,46 real-world data can be analyzed to adjust for biases, for example, through nonparametric techniques based on the propensity score and, in particular, matching techniques.132 To facilitate economic evaluation, registries should contain “treatment variation,” for example patients with similar characteristics receiving different treatments within the registry. This might be achieved by merging registries of different O&P facilities and taking full advantage of “big data” services.
  6. Prostheses as “Internet of things”: If real-world data are recognized as relevant, then one additional problem must be resolved in O&P, which is the outcome of the treatment that should be collected together with administrative and logistic data. As described in Question 4—Clinical Effectiveness Domain, at present no standard set of outcomes has been established. To be in line with HTA guidelines, both generic and disease-specific outcomes should be collected, but it is essential to collect the patients' health-rated quality of life. Outcome collection has become imperative.133,134 However, the collection of outcomes requires organizational adjustment and can be an additional burden to the clinical staff and patients. One possible alternative is to collect part of the outcomes by embedding specific intelligence into the prostheses, which can become objects of the “Internet of things.”135–137 By embedding sensors into the final product delivered to the patients and exploiting Wi-Fi, 4G/5G connection, or long-time storage memories, we can achieve five fundamental aims:
    • a) Collect extremely accurate real-world data, without the need for the patient or the health care professional to physically move to meet each other just for the purpose of outcome collection;
    • b) Understand the patient's usage of the various features of the prosthetic component: if economic resources are invested to ensure patients to take advantage of innovative functions, then it is reasonable to assess if these functions are actually exploited; if they are not, then changes in prescription can be applied;
    • c) Track the learning curve of the patient in using the prosthetic component; prostheses are MDs where the learning curve principle described in Question 5 applies on both the health professionals and the user;
    • d) Optimize prosthetic services based on usage of the prosthetic device instead of time; and
    • e) Track the life cycle of the prostheses for safety and effectiveness for the benefit of the O&P service provider, the patient, and the payer.
    • The adoption of the “Internet-of-things” model can potentially revolutionize the strength of HTA recommendations in prosthetics.
  7. Coverage with evidence development (instead of K-levels): Policy makers might object that small clinical trials or well-structured observational data are still insufficient to recommend the adoption, that is, reimbursement, of an expensive lower-limb prosthetic component. However, this problem might be addressed by adopting more dynamic policies in O&P, as done in the pharmaceutical industry. Three approaches appear particularly appealing:
    • a) The first is the so-called coverage with evidence development, which allows for the reimbursement of the MD in predefined settings (e.g., O&P centers of excellence), provided that patients become part of a cohort that is prospectively evaluated over time or become part of a (pragmatic) RCT.132,138,139 At the end of a predefined period, evidence collected is assessed and a further decision is made. This is a win-win strategy for patients to receive potentially effective prosthetic components, for manufacturers to be paid back for their innovation efforts, and for the payer to receive higher quality evidence and ultimately allocate money on proficient solutions.
    • b) The second is an extension of a) but focused on the renewal of the prosthesis once a first fitting is completed. As anticipated, renewals might be linked to actual usage of the features of the prosthetic component itself. Reassessment might be based on objective data collected through the “Internet-of-things approach” reported in 4.
    • c) The third is the so-called managed-entry agreement between payers and manufacturers. The Italian Pharmaceutical Agency (AIFA) has been recognized as a pioneer in this sector with the adoption of the “payment-by-result” model.140 This model requires manufacturers to pay back the full price, or part of the price (in the case of risk-sharing), for each patient who fails to respond to the new treatment. This approach has the nonsecondary advantage to form a stringent postmarket surveillance. This model appears very reasonable for enlarging the audience of potential users of advanced technologies if results are those expected.

Altogether, these approaches can completely overcome the problem of K-level allocation, which is under scrutiny by AHRQ,94 by looking at real-world results. This strategy can be extremely efficient when combined with the Internet-of-things approach discussed previously.

Back to Top | Article Outline


The aim of this article was to provide a summary of HTA concepts and methods of potential interests for lower-limb prosthetics. A few conclusions stand out. On one side:

  • Patients in need of O&P services continue to require reliable information.
  • O&P is under political pressure to justify the appropriateness of its services.
  • Innovative lower-limb components are constantly emerging and might be beneficial to patients, although at higher costs.
  • The overall level of scientific evidence in O&P is lower than in the pharmaceutical industry and this weakens the strength of recommendations.

On the other side, HTA:

  • is multidisciplinary, addresses nine domains with a transparent, evidence-based approach;
  • receives attention from policy makers and can be used to ensure fair patient treatment in times of budget restrictions;
  • has established methodologies, including the extensive Core Model 3 developed by EUnetHTA and guidelines from ISPOR; and
  • has identified unique features of MDs that do apply to prosthetics and orthotics.

To tackle these unique features, to improve evidence collection, and to make HTA feasible and effective in O&P, seven driving concepts were listed that were intended to:

  • remind that prostheses are not provided in isolation but within specific treatment pathways;
  • stress the importance of collaborative work through large observational studies to prove effectiveness;
  • promote the innovation of prosthetic technologies to make them part of the “Internet of things”;
  • suggest the adoption of innovative reimbursement policy, that is, coverage with evidence development and managed entry agreement.

To reach these aims, four basic requirements must be satisfied.

First, the O&P community needs to agree on a basic core set of clinically relevant outcomes35 to populate databases which are suitable for utility calculation, and track the real-life use of prosthetic technologies.

Second, the capability of single countries to cope with the O&P pace of innovation and cover all the need for information about emerging and mature technologies is very limited, and cooperation among countries is required. As done within the EUnetHTA experience, standards should be defined for guiding the production of HTA reports about O&P, meeting all the informative needs of the decision makers at the different levels (i.e., national/regional, hospitals, and professionals). In this view, the main goal becomes “reusing” what other countries have already produced rather than “redoing” what has been already available. These standards should take into account the peculiarities of O&P and customize the Core Model——or other frameworks—according to them.

Third, the production of HTA reports is the first necessary step for evidence-based decision making in health care, but another one is the “appraisal,” that is, the synthesis of the rich set of multidimensional evidence that is included in HTA reports to support decision making. In this view, multicriteria decision analysis141 might be the right solution. The definition of a rational, fair, and efficient process that allows decision makers to declare the relative relevance of the different domains (weights) as well as the performance of the technology and its comparators against these domains (scores) is of paramount importance for increasing the transparency of the decision making process and its reproducibility.

Finally, the increasing gap between citizens' expectations and financial resources available raises concerns about the long-term sustainability of the health care systems as we know them presently in the most developed countries. Also, in the case of a wide diffusion of HTA reports, the health care system might not be in the condition to satisfy citizens' needs. In fact, when the cost-utility analysis tells that a new technology is cost-utile, this means that the incremental cost is justified by the incremental benefit for the patient in terms of quality-adjusted life years, but this does not imply that health care systems have the budget to pay for the additional cost. In this regard, there is an increasing attention to disinvestments. By disinvesting from technologies that offer limited “value” to patients, it is possible to save the financial resources that might be needed to sustain the adoption of new, more value-adding technologies. This experience has been run in Canada142 and goes under the name of program budgeting and marginal analysis (PBMA). Again, with weights and scores, decision makers can identify those technologies that might be eliminated and save money to sustain the continuous innovations that health care offers to patients. This approach echoes the Choosing Wisely movement in the United States, where scientific associations of clinicians every year decide which technologies to eliminate in order to save money to sustain the adoption of new technologies.

Back to Top | Article Outline


1. United Nations. Convention on the Rights of Persons with Disabilities; 2006.
2. World Health Organization. How to Use the ICF: A Practical Manual for Using the International Classification of Functioning, Disability and Health (ICF). Exposure Draft for Comment. Geneva: WHO; 2013.
3. Broomhead P, Clark K, Dawes D, et al. Guidelines for managing adults with lower limb prostheses. BACPAR. November. 2012.
4. VA/DOD, Rehabilitation of Lower Limb Amputation (2017). Available at:
5. EUnetHTA Joint Action 2, Work Package 8. HTA Core Model ® version 3.0 (Pdf); 2016. Available at:
6. Cochrane H, Orsi K, Reilly P. Lower limb amputation. Part 3: prosthetics—a 10-year literature review. Prosthet Orthot Int 2001;25(1):21–28.
7. Krajbich JI, Pinzur MS, Potter BK, et al. Atlas of Amputations and Limb Deficiencies. 4th ed. Philadelphia, PA: Lippincott Williams & Wilkins; 2016.
8. Hebert JS, Rehani M, Stiegelmar R. Osseointegration for lower-limb amputation: a systematic review of clinical outcomes. JBJS Rev 2017;5(10):e10.
9. Tillander J, Hagberg K, Berlin Ö, et al. Osteomyelitis risk in patients with transfemoral amputations treated with osseointegration prostheses. Clin Orthop Relat Res 2017;475(12):3100–3108.
10. International Network of Agencies for Health Technology Assessment. Available at:
11. World Health Organization, Health technology assessment – Definitions. Available at:
12. Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med 2016;21(4):125–127.
13. Grading of Recommendations Assessment, Development and Evaluation (GRADE). Available at:
14. Critical Appraisal Skills Programme (CASP). Available at:
15. Assessing the Methodological Quality of Systematic Review (AMSTAR). Available at:
16. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). Available at:
17. Appraisal of Guidelines for Research and Evaluation (AGREE). Available at:
18. Strengthening the Reporting of Observational studies in Epidemiology (STROBE). Available at:
19. Christiansen C, Lou JQ. Ethical considerations related to evidence-based practice. Am J Occup Ther 2001;55(3):345–349.
20. Loewy EH. Ethics and evidence-based practice: is there a conflict? Medscape. Available at:
21. Snyder JE, Gauthier CC. Evidence-Based Medical Ethics: Cases for Practice-Based Learning. Totowa, NJ: Humana Press; 2008.
22. Luce BR, Drummond M, Jönsson B, et al. EBM, HTA, and CER: clearing the confusion. Milbank Q 2010;88(2):256–276.
23. Drummond MF, Schwartz JS, Jönsson B, et al. Key principles for the improved conduct of health technology assessments for resource allocation decisions. Int J Technol Assess Health Care 2008;24(3):244–258; discussion 362–328.
24. Banta D. What is technology assessment? Int J Technol Assess Health Care 2009;25(Suppl 1):7–9.
25. Sullivan NJ, Mayo K, Holmstrom S, et al. Health care decision-making around the world: a global roadmap and country-specific decision- making and reimbursement processes, HTA Forum at the ISPOR 14th Annual International Meeting, May 16–19, 2009, Orlando, FL, USA.
26. World Bank. Open Data. Available at:
27. WHO. Global Health Expenditure Database. Available at:
28. OECD. Statistics. Available at:
30. OECD. “Health expenditure in relation to GDP”, in Health at a Glance 2017: OECD Indicators, OECD Publishing, Paris. 2017. doi:
31. OECD. “Health expenditure per capita”, in Health at a Glance 2017: OECD Indicators, OECD Publishing, Paris. 2017. doi:
32. Peter G. Peterson Foundation, U.S. Health Spending. Available at:
33. Schneider EC, Sarnak DO, Squires D, et al. Mirror, Mirror 2017: International Comparison Reflects Flaws and Opportunities for Better U.S. Health Care, The Commonwealth Fund. Available at:
34. OECD. Health at a Glance. OECD Indicators Life expectancy at birth and health spending per capita, 2015 (or nearest year). 2017. Available at:
35. Porter ME. What is value in health care? N Engl J Med 2010;363(26):2477–2481.
36. Porter ME, Kaplan RS. How Should We Pay for Health Care? Boston, MA: Harvard Business School Working Paper; 2014. (Revised February 2015).
37. AOPA. AOPA Fact Sheet. Available at:
38. Centers for Medicare & Medicaid Services, National Health Expenditure Data. Available at:
39. Corte dei Conti based on NHS data, 2012 (Sole 24 Ore, 12/06/2012).
40. Department of Health & Human Services. Office of inspector general questionable billing by suppliers of lower limb prostheses, OEI-02-10-00170. 2011.
41. Thomas PW, Hughes CA. The New Reality: Documenting O&P Medicare Claims, The O&P Edge. 2013.
42. House of Representatives, H.R.1892 — 115th Congress (2017–2018), Bipartisan Budget Act of 2018.
43. AOPA. Update. Orthotist/prosthetist notes will be considered part of the patient's medical record by CMS!, February 9, 2018. Available at:
45. Gazzetta Ufficiale, LEGGE 8 Marzo 2017, n. 24 Disposizioni in materia di sicurezza delle cure e della persona assistita, nonche' in materia di responsabilità professionale degli esercenti le professioni sanitarie. (17G00041).
46. Dobson A, El-Gamil A, Shimer M, DaVanzo JE. Economic value of prosthetic services among medicare beneficiaries: a claims-based retrospective cohort study. Mil Med 2016;181(2 Suppl):18–24.
47. Liu H, Chen C, Hanson M, et al. Economic Value of Advanced Transfemoral Prosthetics. In: Rand Corporation 2017:
49. WHO. Global Survay on Diabetes. 2016. Available at:
50. WHO. World report on ageing and health. 2015. Available at:
51. Pasquina CP, Carvalho AJ, Sheehan TP. Ethics in rehabilitation: access to prosthetics and quality care following amputation. AMA J Ethics 2015;17(6):535–546.
52. Clegg A, Young J, Iliffe S, et al. Frailty in elderly people. Lancet 2013;381(9868):752–762.
53. Rozenblum R, Bates DW. Patient-centred healthcare, social media and the Internet: the perfect storm? BMJ Qual Saf 2013;22(3):183–186.
54. López DM, Blobel B, González C. Architectural approach for quality and safety aware healthcare social networks. Stud Health Technol Inform 2012;180:793–797.
55. Heinemann AW, Fisher WP Jr, Gershon R. Improving health care quality with outcomes management. J Prosthet Orthot 2006;18(6):46–P50.
56. Amputee Coalition. Available at:
57. AOPA. AOPA Statement on 3-D Printing in Prosthetics and Frequent Errant Estimates of the Average Cost of Upper and Lower Limb Custom-Fabricated Prosthetic Devices 2015. Available at:
58. Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC.
59. Yang BM, Prescott N, Bae EY. The impact of economic crisis on health-care consumption in Korea. Health Policy Plan 2001;16(4):372–385.
60. Karanikolos M, Mladovsky P, Cylus J, et al. Financial crisis, austerity, and health in Europe. Lancet 2013;381(9874):1323–1331.
61. Simou E, Koutsogeorgou E. Effects of the economic crisis on health and healthcare in Greece in the literature from 2009 to 2013: a systematic review. Health Policy 2014;115(2–3):111–119.
62. Aloumanis K, Papanas N. Greek financial crisis: consequences in the healthcare of diabetes and its complications. Hippokratia 2014;18(1):4–6.
63. Stuckler D, Reeves A, Loopstra R, et al. Austerity and health: the impact in the UK and Europe. Eur J Public Health 2017;27(suppl 4):18–21.
64. Maresso A, Mladovsky P, Thomson S, et al, eds. Economic crisis, Health Systems and Health in Europe: Country Experience [Internet]. Copenhagen (Denmark): European Observatory on Health Systems and Policies; 2015: Available at:
65. WHO. Country experience economic crisis, health systems and health in Europe. 2015.
66. OECD. Economic Outlook and Interim Economic Outlook 2018. Available at:
67. Kennedy I. Appraising the Value of Innovation and Other Benefits. A Short Study for NICE 2009. Available at: Accessed February 17, 2018.
68. Eldessouki R, Smith MD. Health care system information sharing: a step toward better health globally. Value Health Regional Issues 2012;1:118–129.
69. ISPOR Global Health Care Systems Road Map. Available at:
71. Goodman CS. HTA 101: Introduction to Health Technology Assessment. Bethesda, MD: National Library of Medicine (US); 2014:
72. WHO. 2015 Global Survey on Health Technology Assessment by National Authorities. 2015. Available at:
73. Sullivan SD, Watkins J, Sweet B, Ramsey SD. Health technology assessment in health-care decisions in the United States. Value Health 2009;12(Suppl 2):S39–S44.
74. Agency for Healthcare Research and Quality (AHRQ). Evidence-based Practice Centers (EPCs). Available at:
75. Garrison L. Introduction to Health Technology Assessment in the U.S. and Canada: Possible Lessons and Implications for Taiwan? Workshop on HTA Policy Formation and Implementation, 2011. University of Washington. Available at:
76. Canadian Agency for Drugs and Technologies in Health. Available at:
77. Fuchs S, Olberg B, Panteli D, Busse R. Health technology assessment of medical devices in Europe: processes, practices, and methods. Int J Technol Assess Health Care 2016;32(4):246–255.
78. ISPOR Directory of Health Technology Assessment Organisations Worldwide.
79. European Parliament Legislative Train Schedule. “Deeper and fairer internal market with a strengthened industrial base / Products – Health Technology Assessment.” 2018. Available at:
80. European Parliament Legislative proposal. “Foster cooperation between EU Member States in Health Technology Assessment. ” 2018/0018(COD) - 31/01/2018.
81. National Institute For Health and Care Excellence. “Technology appraisal guidance.” Available at:
82. Agenzia Nazionale per i Servizi Sanitari Regionali (AGENAS), Società Italiana di Health Technology assessment, Indagine conoscitiva sulle attività di health technology assessment in Italia. 2016. Available at:
83. Presidenza del Consiglio dei Ministri. Rep. 82/CSR 10/7/2014, Patto per la salute 2014–2016.
84. Gazzetta Ufficiale, LEGGE 28 dicembre 2015, n. 208, Disposizioni per la formazione del bilancio annuale e pluriennale dello Stato (legge di stabilita' 2016).
85. Gazzetta Ufficiale. LEGGE 23 dicembre 2014, n. 190 Disposizioni per la formazione del bilancio annuale e pluriennale dello Stato (legge di stabilita' 2015).
86. Cabina di Regia HTA, Programma nazionale HTA Dispositivi Medici, 13/09/2017. Available at:
87. Garrido MV, Kristensen FB, Nielsen CP, Busse R. Health Technology Assessment and Health Policy-Making in Europe - Current status, challenges and potential, World Health Organization 2008, on behalf of the European Observatory on Health Systems and Policies. Available at:
88. HAS. Commision Nationale d'Evalutation Des Dispositifs Medicaux et des Technologies de Sante, 3C100 C-leg (version 4). Available at:
89. Canadian Agency for Drugs and Technologies in Health, Microprocessor-Controlled Prosthetic Knees (C-Leg) for Patients with Above-Knee Amputations: A Review of the Clinical- and Cost-Effectiveness, 2009. Available at:
90. NHS England. 16061/P Clinical Commissioning Policy: Microprocessor controlled prosthetic knees, 2016. Available at:
91. Henrikson NB, Hafner BJ, Dettori JR, et al. Washington State Heath Care Authority—Microprocessor-Controlled Lower Limb Prostheses: Health Technology Assessment. Tacoma, WA: Spectrum Research; 2011.
92. Iglesias CP. Does assessing the value for money of therapeutic medical devices require a flexible approach? Expert Rev Pharmacoecon Outcomes Res 2015;15(1):21–32.
93. AOPA. Prosthetics 2020 and Orthotics 2020.
94. Agency for Healthcare Research and Quality, Lower Limb Prosthesis –Draft Systematic Review, 2017. Available at:
95. Hafner BJ, Sawers AB. Issues affecting the level of prosthetics research evidence: Secondary analysis of a systematic review. Prosthet Orthot Int 2016;40(1):31–43.
96. Highsmith MJ, Kahle JT, Miro RM, et al. Prosthetic interventions for people with transtibial amputation: systematic review and meta-analysis of high-quality prospective literature and systematic reviews. J Rehabil Res Dev 2016;53(2):157–184.
97. Healy A, Farmer S, Pandyan A, Chockalingam N. A systematic review of randomised controlled trials assessing effectiveness of prosthetic and orthotic interventions. PLoS One 2018;13(3):e0192094.
98. Stephens JM, Handke B, Doshi JA. International survey of methods used in health technology assessment (HTA): does practice meet the principles proposed for good research? Comparative Effectiveness Research 2012;2:29–44. Available at:
99. Accessed November 19, 2018.
101. Ernst E, Pittler MH. Efficacy of effectiveness. J Intern Med 2006;260:488–449.
102. Möller HJ. Effectiveness studies: advantages and disadvantages. Dialogues Clin Neurosci 2011;13(2):199–.
103. Gartlehner G, Hansen RA, Nissman D, et al. Criteria for Distinguishing Effectiveness From Efficacy Trials in Systematic Reviews. Rockville (MD): Agency for Healthcare Research and Quality (US); 2006.
    104. de Vries M, Emons WH, Plantinga A, et al. Comprehensively measuring health-related subjective well-being: dimensionality analysis for improved outcome assessment in health economics. Value Health 2016;19(2):167–175.
    105. Lloyd A. EQ-5D: Moving from Three Levels to Five. Value Health 2018;21(1):57–58.
    106. GRADE. Software for Summary of Findings tables, Health Technology Assessment and Guidelines. Available at:
      107. Gailey R, Allen K, Castles J, et al. Review of secondary physical conditions associated with lower-limb amputation and long-term prosthesis use. J Rehabil Res Dev 2008;45(1):15–29.
      108. Hafner BJ, Morgan SJ, Askew RL, Salem R. Psychometric evaluation of self-report outcome measures for prosthetic applications. J Rehabil Res Dev 2016;53(6):797–812.
      109. Resnik L, Borgia M. Reliability of outcome measures for people with lower-limb amputations: distinguishing true change from statistical error. Phys Ther 2011;91(4):555–565.
      110. Jakubiak-Lasock J, Jakubczyk M. Cost-effectiveness versus cost-utility analyses: what are the motives behind using each and how do their results differ? A Polish example. Value Health Reg Issues 2014;4:66–74.
      111. NICE. Guide to the methods of technology appraisal 2013. Available at:
        112. Polinder S, Toet H, Panneman M, van Beeck E. Methodological approaches for cost–effectiveness and cost–utility analysis of injury prevention measures. WHO Regional Office for Europe 2011;
          113. Polinder S, Segui-Gomez M, Toet H, et al. Systematic review and quality assessment of economic evaluation studies of injury prevention. Accid Anal Prev 2012;45:211–221.
          114. Dakin H, Devlin N, Feng Y, et al. The influence of cost-effectiveness and other factors on NICE decisions. Health Econ 2014.
            115. TreeAge Software. Available at:
              116. Brodtkorb TH, Henriksson M, Johannesen-Munk K, Thidell F. Cost-effectiveness of C-leg compared with non-microprocessor-controlled knees: a modeling approach. Arch Phys Med Rehabil 2008;89(1):24–30.
              117. Gerzeli S, Torbica A, Fattore G. Cost utility analysis of knee prosthesis with complete microprocessor control (C-leg) compared with mechanical technology in trans-femoral amputees. Eur J Health Econ 2009;10(1):47–55.
              118. Seelen HAM, Hemmen B, Schmeets AJ, et al. Costs and consequences of a prosthesis with an electronically stance and swing phase controlled knee joint. Technol Disabil 2009;21(1–2):25–34.
                119. Cutti AG, Lettieri E, Del Maestro M, et al. Stratified cost-utility analysis of C-Leg versus mechanical knees: Findings from an Italian sample of transfemoral amputees. Prosthet Orthot Int 2017;41(3):227–236.
                120. Amputee Coalition, Medicare Issue Update, “White House Announces Update on Medicare LCD Proposal.” Available at:
                121. Facey K, Ploug Hansen H, Single A. Patient Involvement in. Health Technol Assess. Singapore: Springer Nature 2017.
                  122. Wale J, Scott AM, Hofmann B, et al. Why patients should be involved in health technology assessment. Int J Technol Assess Health Care 2017;33(1):1–4.
                  123. Fuchs S, Olberg B, Panteli D, et al. HTA of medical devices: challenges and ideas for the future from a European perspective. Health Policy 2017;121(3):215–229.
                  124. Taylor RS, Iglesias CP. Assessing the clinical and cost-effectiveness of medical devices and drugs: are they that different? Value Health 2009;12(4):404–406.
                  125. Rummel P, Hawlik K, Wild C. Health technology assessments on medical devices in Europe. Vienna: Ludwig Boltzmann Institut fuer Health Technology Assessment (LBIHTA). LBI-HTA Rapid Assessment; 2016.
                  126. Ciani O, Wilcher B, Blankart CR, et al. Health technology assessment of medical devices: a survey of non-European union agencies. Int J Technol Assess Health Care 2015;31(3):154–165.
                  127. Tarricone R, Torbica A, Drummond M. Challenges in the assessment of medical devices: the MedtecHTA project. Health Econ 2017;26(Suppl 1):5–12.
                  128. Schnell-Inderst P, Mayer J, Lauterberg J, et al. Health technology assessment of medical devices: what is different? An overview of three European projects. Z Evid Fortbild Qual Gesundhwes 2015;109(4–5):309–318.
                  129. Ciani O, Wilcher B, van Giessen A, Taylor RS. Linking the regulatory and reimbursement processes for medical devices: the need for integrated assessments. Health Econ 2017;26(Suppl 1):13–29.
                  130. Olberg B, Fuchs S, Panteli D, et al. Scientific evidence in health technology assessment reports: an in-depth analysis of European assessments on high-risk medical devices. Value Health 2017;20(10):1420–1426.
                  131. Tarricone R, Callea G, Ogorevc M, Prevolnik Rupel V. Improving the methods for the economic evaluation of medical devices. Health Econ 2017;26(Suppl 1):70–92.
                  132. Tarricone R, Torbica A, Drummond M; MedtecHTA Project Group. Key recommendations from the MedtecHTA project. Health Econ 2017;26(Suppl 1):145–152.
                  133. Umbrell C. Major gains in outcome data. O&P Almanac 2017;20–29.
                  134. Umbrell C. Measure for measure. O&P Almanac 2018:32–.
                  135. Atzori L, Iera A, Morabito G. Understanding the Internet of things: definition, potentials, and societal role of a fast evolving paradigm. Ad Hoc Networks 2017;56:122–140.
                  136. Ng ICL, Wakenshaw SJL. The Internet-of-things: review and research directions. Int J Res Market 2017;34(1):3–21.
                  137. Gómeza J, Oviedo B, Zhuma E. Patient Monitoring System Based on Internet of Things. Proc Comp Sci 2016;83:90–97.
                  138. CMS, Coverage with Evidence Development. Available at:
                  139. Rothery C, Claxton K, Palmer S, et al. Characterising uncertainty in the assessment of medical devices and determining future research needs. Health Econ 2017;26(Suppl 1):109–123.
                  140. Kitamura M, Koch J. When Cancer Treatments Fail, Italy Wants Money Back When Cancer Treatments Fail, Italy Wants Money Back. Bloomberg Available at:
                  141. Marsh K, Goetghebeur M, Thokala P, Baltussen R. Multi-Criteria Decision Analysis to Support Healthcare Decisions. Cham, Switzerland: Springer; 2017.
                  142. Mitton C, Dionne F, Damji R, et al. Difficult decisions in times of constraint: criteria based resource allocation in the Vancouver Coastal Health Authority. BMC Health Serv Res 2011;11:169.
                  Back to Top | Article Outline

                  ANNEX 1—Definition of EBP

                  The definition of evidence-based practice (EBP) is typically tracked back to the papers by Sackett et al.,1 which describes it as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.” This means that evidence must be integrated with the clinical expertise, because the scientific evidence might not be applicable or be inappropriate for a specific patient. But on the other hand, practice alone risks being biased and out of date. To put EBP into practice, the professional must be able to formulate the clinical problem as an answerable question, reach for the best evidence, appraise the evidence for validity, impact and applicability, integrate the evidence with the personal clinical judgment regarding the specific patient, not forgetting what is available and the values and situation of the patients. Finally, the professional must be critically reevaluated, using these steps for improving over the next case. This cyclic process is usually referred to as the 6 As principles: assess, ask, acquire, appraise, apply, and audit.2L

                  1. Sackett DL, Rosenberg WM, Gray JA, et al. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312(7023):71–72.

                  2. Straus SE, Glasziou P. Evidence-based practice: how to practice and teach EBP. Elsevier; 2018.


                  health technology assessment; evidence-based practice; amputation; prosthetics; health policy; Core Model 3; big data; outcome measures; Internet of things; managed entry agreement

                  © 2019 by the American Academy of Orthotists and Prosthetists.