Stevens, Phillip M. MEd, CPO, FAAOP
Evidence-based practice, as originally described and suggested, constituted a rather onerous process. When confronted with clinical uncertainty, practitioners were encouraged to carefully formalize an appropriate clinical question, conduct a thorough search of the literature, select the best and most relevant articles, assess their relative validity, and apply their findings to the original clinical case in question.1
In retrospect, the principles encouraged within this framework were idealistically sound. Rather than practicing in isolation and risk the possibility of providing suboptimal care when new knowledge may exist that might improve outcomes, practitioners were encouraged to look into sources outside of their own clinical experience and established rhetoric to sure up their knowledge base. However, in practice, evidence-based medicine, in its purist, original form, was largely untenable. The barriers to its regular use were both numerous and substantial. In addition to the enormous time requirements involved, there were further challenges of an inadequate literature base, limited access to existing literature, and insufficient academic training to fully assess the articles themselves.
Within this context, it is not surprising to note that within several years of the introduction of evidence-based practice, differing theories regarding its application were put forward. Of these, one that offered insights quite relevant to orthotics and prosthetics was the concept of medical information mastery by Slawson et al.2 and Slawson and Shaughnessey.3 According to the authors, medical information can be appraised according to its “usefulness” to the practitioner. This usefulness is a product of its relevance and validity divided by the work required to obtain it as suggested in the equation below:
“USEFUL” INFORMATION IN ORTHOTICS AND PROSTHETICS
Equation (Uncited)Image Tools
When this equation is applied to prosthetics and orthotics, it becomes clear that “useful” information will be infrequently encountered in primary knowledge sources such as original journal articles. Some of the limitations to relevance, validity, and work within the profession are discussed below.
Relevance is largely based on the frequency of a given problem or concern within your practice.3 For many professional practitioners, their clinical case load is characterized by a tremendous variety in age, presentation and diagnosis. Data from the most recent Practice Analysis, conducted by the American Board for Certification confirm this reality.4 Although some practitioners may work in specialty centers catering to a narrowed age rage, as a profession, practitioners spend roughly one third of their time in pediatric, adult, and geriatric care, respectively.4 Within orthotics, practice areas include both lower and upper limbs, accounting for 55% and 8% of clinical case loads, respectively, spinal (21%), scoliosis (10%), and the broad category of “other” (6%) canvassing such modalities as functional electrical stimulation, burn garments, face masks, and cranial remolding orthoses (Table 1). Even the most commonly provided device type, the ankle-foot orthosis (AFO), accounts for only 26% of the reported case loads in orthotic care and is applied in several different design types and with varying characteristics across a diverse range of patient presentations.
Prosthetic case loads are, by their nature, less diverse with transtibial and transfemoral practice areas represented at 54% and 27%, respectively (Table 2). Yet within these diagnoses, there is tremendous variability with respect to socket designs, skin interfacing, suspension mechanisms, and distal components. Because most practitioners are “generalists,” providing care across much of the previously described spectrum, the likelihood of an individual journal article being immediately relevant to clinical practice is low. The usefulness of the information derived from an excellent, informative article on the management of partial foot amputations, for examples, must be considered against the reality that this patient type accounts for no more than 6% of current prosthetic case loads.4
The validity of a piece of information represents the likelihood that it is true3 or the accuracy with which it may apply to a clinical scenario. In the case of prosthetic and orthotic care, these values tend to be low for several reasons. First, patient populations in research are characteristically quite small and diverse. A given article on upper limb prosthetics may include subjects with amputations at the wrist disarticulation, transradial, transhumeral, and shoulder disarticulation levels. Alternately, an article on a given prosthetic knee or foot type may include patients with both traumatic and vascular etiologies to their amputations with varying limb lengths, ages, and time since amputation. The validity or transferability of findings from such diverse patient populations to an individual patient scenario is inherently compromised.
In addition, in contrast to pharmaceutical research where the interventions are homogenous and easily controlled, interventions in orthotics and prosthetics are characterized by more variability. Variations in AFO design characteristics, socket fit qualities, and compliance to daily usage make it more challenging to derive useful information from original research findings.
Work has been described has how long it takes to obtain the information, the associated costs involved, and the amount of mental energy required to fully understand it.3 The work required to access information from original sources in orthotics and prosthetics tends to be quite high for several reasons. Among these are the challenges of access to original articles relevant to orthotics and prosthetics. Although there are currently two peer-reviewed journals that cater specifically to the profession, the Journal of Prosthetics and Orthotics and Prosthetics and Orthotics International, the majority of the high level studies relevant to the profession are found in other journals. Only approximately 20% of prosthetists and orthotists practice in some type of hospital or university setting where their professional affiliations would grant them access to a broad library of published literature.4 For the remaining 80% of the profession, individual articles may only be available through online purchase, with the associated costs acting as a deterrent.
In addition, although the profession's educational standards continue to increase, training continues to be rightly focused on clinical care rather than the academic pursuits of statistics and research methodologies. Thus, the ability of the average practitioner to conduct literature searches and critically read and evaluate original journal articles may be less than ideal. The time requirements associated with literature searches, obtaining articles, and their subsequent appraisals constitute a considerable amount of work and undermine the potential usefulness of the information.
ESTABLISHING THE NEED FOR SECONDARY KNOWLEDGE SOURCES
By virtue of the lower relevance and validity associated with journal articles in orthotics and prosthetics, along with the impressive amount of work required to seek out, obtain, and understand them, the usefulness of information obtained from original journal articles is quite low. However, this challenge is not unique to orthotics and prosthetics. Although validity values may be higher in the original research of other disciplines, lower relevance and higher work values represent a universal challenge in medicine.5
The usefulness equation can be applied to other established medical information sources to determine the comparative usefulness of their contents. For example, although a standard textbook based on historically accepted and largely unconfirmed knowledge may have lower validity than original, current research, the information is often extremely relevant to clinical practice. Similarly, provided you own the book, the work involved in reading and understanding its contents are comparatively low. Accordingly, the usefulness of the knowledge contained in a standard textbook might be considered moderate.5
Systematic literature reviews tend to contain even more useful information. By addressing specific clinical questions, the relevance to clinical practice is often quite high. Higher validity values are ensured by their basis on current literature findings. Finally, provided that they are reasonably accessible, the efforts of the authors to synthesize and condense the different findings reduce the work load of the reader. Thus, a well-written and accessible literature review may provide highly useful information. These examples demonstrate the value of efforts undertaken to translate the valid findings of original research into formats with greater relevance that are easier to access and understand. These summaries and syntheses are often referred to as secondary knowledge sources.
SECONDARY KNOWLEDGE SOURCES OUTSIDE OF ORTHOTICS AND PROSTHETICS
As healthcare professionals have looked for ways to integrate evidence into clinical practice, there has been a general realization that it is the product of evidence-based medicine, rather than the process, that is of greatest importance. Because most practitioners simply do not have the time and academic background to fully engage the literature for every clinical question that arises, there is a recognized need for secondary knowledge sources, based on literature findings, but condensed into useful, actionable content. Several products of this so-called knowledge translation are described below.
PATIENT-ORIENTED EVIDENCE THAT MATTERS
In 2002, the editor of the British Medical Journal announced the decision to publish a weekly Patient-Oriented Evidence that Matters (POEMs). These were described as summaries of a valid piece of research that carries information that is important to patients and, by association, to their physicians.5 More specifically, each POEM must meet the following three criteria: 1) it addresses a question that doctors encounter. 2) It measures outcomes that doctors and their patients care about. 3) They have the potential to change the way that doctors practice. Within these guidelines, the editor went on to explain that POEMs would be selected by searching the current issues of 100 journals. Once an appropriate study was identified, it was to be evaluated for validity. Studies found to be valid would then be summarized as POEMs, with a new POEM published every week. In this manner, the journal ensured regular useful information to its readership by providing easily accessible summaries of highly valid, original research that was relevant to clinical practice.
TIPS FROM OTHER JOURNALS
One of the values of the POEMs concept was the breadth of its search for useful information outside of those articles published within the hosting journal. This idea can also be found in an interesting feature of the peer-reviewed journal, American Family Physician. Every issue begins with a section entitled “Tips from other journals.” Realizing that their reading audience was composed of family practitioners with very broad case loads, each version of the Tips section contained summaries of up to five articles from other publications and pertaining to two to three broader health categories such as adult medicine, children's health, and women's health. Each summary included background information on the research topic, an overview of the specific study and its results, the conclusions that could be drawn, and the original publication source for the study. As with POEMs, the information contained within this tips concept is characterized by high validity, high relevance, and low work.
CRITICALLY APPRAISED TOPICS
Critically Appraised Topics (CATs) represent another form of knowledge translation into more manageable secondary knowledge sources.6 Unlike systematic reviews that seem to focus their content as much on their own methodology as their findings, CATs are intended to be brief, literature-based answers to specific clinical questions. Once a clinical question is identified, the authors, typically clinicians, search only the best and most recent evidence before writing a concise, clinically focused summary of their findings. CAT libraries currently exist for several disciplines including primary care, occupational therapy, emergency medicine, and pediatrics. Although it could be argued that the relaxed standards of these documents might compromise their validity, their focus on specific clinical questions enhances their relevance, and their succinct presentation further decreases the work associated with obtaining knowledge. Thus, they remain a useful information source.
SECONDARY KNOWLEDGE SOURCES WITHIN ORTHOTICS AND PROSTHETICS
Secondary knowledge sources have begun to appear within the orthotics and prosthetics profession as well. Several of these are described below and summarized, along with their locations, in Table 3.
The International Society for Prosthetics and Orthotics has made available the conference reports from two recent international consensus conferences. The topic of the first was the orthotic management of stroke,7 whereas the second addresses recent developments in the medical management of cerebral palsy and the associated implications and opportunities for orthotics.8 These documents are both broad and thorough in their coverage of their respective topics, with lengths of 273 and 300 pages, respectively. Both documents contain several systematic literature reviews of relevant subtopics and constitute detailed secondary knowledge sources.
STATE-OF-THE SCIENCE CONFERENCES
Beginning in 2004 with the topic of “The Orthotic Treatment of Idiopathic Scoliosis and Scheuermann's Kyphosis,” the American Academy of Orthotists and Prosthetists held its first State-of-the-Science Conference (SSC), the proceedings of which now constitute an openly accessible secondary knowledge source for the management of this population.9 The proceedings of this first SSC, authored by respected experts in the field, addresses questions of clinical relevance, drawing information from 205 original articles in a 58-page document. Similar SSCs have been conducted and their proceedings made available for the following topics:
* Post-Operative Management of the Lower Extremity Amputee
* Orthotic Treatment of Deformational Plagiocephaly, Brachycephaly and Scaphocephaly
* Orthotic and Pedorthic Management of the Neuropathic Foot
* Prosthetic Foot/Ankle Mechanisms
* Outcome Measures in Lower Limb Prosthetics
* Knee-Ankle-Foot Orthoses for Ambulation
* The Biomechanics of Ambulation After Partial Foot Amputation
* Upper Limb Prosthetic Outcome Measures
The content of these SSC proceedings meet the requirements of “useful” information by virtue of their targeted relevance to clinical practice, strong validity based on published evidence where possible and clinical experience when evidence is lacking, and relative ease of accessibility, both in terms of obtaining the summary documents and their subsequent reading.
The first Evidence Note in the profession, “The Use of Ankle Foot Orthoses in the Management of Stroke” was developed at the request of the Orthotics and Prosthetics Outcomes Initiative Steering Committee, which was in turn sponsored by the American Orthotic and Prosthetic Association.10 Its focus was on the management of stroke with specific information regarding the use of AFOs. The document, excluding its 66 references, covers only 2 printed pages and canvases such topics as epidemiology, clinical effectiveness of AFOs, safety, economic implications, and ongoing research.
Two additional evidence notes have since been made available on the management of partial foot amputation11 and ambulatory knee-ankle-foot orthoses and hip-knee-ankle-foot orthoses.12 As with their predecessor, these notes are relatively short, condensing a large amount of information into just over two printed pages when references are discounted. The evidence notes could be said to represent a tertiary knowledge source because they are all summaries of more exhaustive secondary knowledge sources.
INCREASING THE AVAILABLE SOURCES OF “USEFUL” INFORMATION IN ORTHOTICS AND PROSTHETICS
The degree to which the knowledge derived from the most recent published evidence will be incorporated into clinical practice will largely depend on the ability of the profession to better generate and disseminate valid and relevant secondary knowledge sources. The published proceedings of various literature-based conferences constitute one type of secondary knowledge source. Although valuable, they have their limitations. Such conferences and their subsequent proceedings require immense outlays of both time and financial resources. The processes involved in their production are such that by the time they are made available, they may not represent the most current information nor are they amenable to frequent updates. Also, with the exception of the evidence notes, these can be daunting publications for the busy practitioner.
In addition to the continued development and dissemination of such traditional knowledge sources, there is a need in the profession for shorter, quicker, more accessible secondary knowledge sources of the type represented by POEMs, tips from other journals, and CATs. These approaches are all similar in that they draw current information from a broad knowledge base, truncate the most salient, actionable information into very brief summaries, and are made available to practitioners very quickly.
The development and dissemination of these smaller, swifter information sources may well require commitment and participation from across the spectrum of orthotic and prosthetic care. Established associations such as International Society for Prosthetics and Orthotics, American Academy of Orthotists and Prosthetists, and American Orthotic and Prosthetic Association may be called on to provide necessary funding to incentivize prospective authors, to house developing data libraries, and to regulate and oversee their developing content. Experienced clinicians may be needed to identify the areas of clinical interest and ensure the relevance of emerging medical information. Younger clinicians and residents may benefit from assisting in the development of these secondary knowledge sources as a part of their entry in the field. Students in the profession, with their broader access to current original knowledge sources, may also play a role under the guidance of educators. In short, to expand the incorporation of current evidence into clinical practice will likely require a considerable amount of collaboration throughout the profession.
Evidence-based practice is no more than the incorporation of useful, literature-derived medical information into clinical practice. This usefulness requires that information be relevant, valid, and accessible. Traditional secondary knowledge sources of useful information already exist in the profession and facilitate the implementation of evidence-based practice within the profession. However, the need exists for additional secondary knowledge sources that are more current and accessible, similar to those already being used in other medical professions. As the orthotic and prosthetic community moves forward with the development and dissemination of such information sources, the findings of the evidence will better inform daily patient care.
1. Evidence-Based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA 1992;268:2420–2425.
2. Slawson DC, Shaughnessy AF, Bennett JH. Becoming a medical information master: feeling good about not knowing everything. J Fam Pract 1994;38:505–513.
3. Slawson DC, Shaughnessey AF. Obtaining useful information from expert based sources. BJM 1997;314:947–949.
5. Smith R. A POEM a week for the BMJ. BMJ 2002;325:963.
6. Fetters L, Figueiredo EM, Keane-Miller D, et al. Critically appraised topics. Pediatr Phys Ther 2004;16:19–21.
7. Condie E, Cambell J, Martina J, eds. Report of a Consensus Conference on the Orthotic Management of Stroke Patients. Borgervaenget, Denmark: International Society for Prosthetics and Orthotics; 2004.
8. Morris C, Condie D, eds. Recent Developments in Healthcare for Cerebral Palsy: Implication and Opportunities for Orthotics. Borgervaenget, Denmark: International Society for Prosthetics and Orthotics; 2009.
9. American Academy of Orthotists and Prosthetists. Orthotic treatment of idiopathic scoliosis and scheuermann's kyphosis. J Prosth Orthot 2003;15:S1–S58.
10. Fatone S, Campbell JH. Evidence Note: The Use of Ankle Foot Orthoses in the Management of Stroke. Alexandria VA: American Orthotic and Prosthetic Association; 2008.
11. Dillon M and Fatone S. Evidence Note: The Biomechanics of Ambulation After Partial Foot Amputation. Washington DC: American Academy of Orthotists and Prosthetists; 2009.
12. Fatone S. Evidence Note: The Use of KAFOs and HKAFOs for Ambulation. Washington DC: American Academy of Orthotists and Prosthetists; 2009.
KEY INDEXING TERMS: evidence-based medicine; literature; evidence-based practice