The results for each recommendation are shown in Table III. When looking at the use of a non-accidental trauma evaluation in children younger than thirty-six months, there was a slight decrease after the clinical practice guideline was introduced, although this was not significant (p = 0.40). The use of a Pavlik harness or spica cast for children younger than six months also did not change significantly (p = 0.36), with one patient in the pre-guideline group being treated with a long leg cast (nine of ten patients according to the guideline) and all patients in the post-guideline group being treated according to the guideline.
A slight, nonsignificant decrease in the use of spica casts was noted for patients who were six months to five years of age (p = 0.07). In the pre-guideline group, two patients were treated with a Pavlik harness and one was treated with a long leg cast, and in the post-guideline group, seven patients were treated with long leg casts, four were treated with flexible nails, one was treated with a Pavlik harness, and two were treated with external fixators. A small, nonsignificant increase in treatment alteration for those children with >2-cm fracture shortening in a spica cast was observed after the clinical practice guideline was introduced (p = 0.09). A significant decrease in the adherence to the clinical practice guideline suggesting use of flexible nails for the patients who were five to eleven years of age was found (p = 0.03). In the pre-guideline group, ten patients were treated with spica casts and four patients were treated with submuscular plates, and in the post-guideline group, there were eleven patients treated with rigid nails, ten patients treated with submuscular plates, four patients treated with spica casts, and two patients treated with an external fixator and percutaneous pinning.
Little difference was observed in the treatment of patients older than eleven years of age between the groups (p = 0.63) and in the use of regional anesthesia between the groups (p = 0.40).
The risk ratios provided in Table III describe the estimated relative effect of the guidelines between the pre-guideline publication period and the post-guideline publication period. For example, looking at the guideline for non-accidental trauma evaluation in children younger than thirty-six months, the risk ratio of 0.86 indicated that the post-guideline period was associated with 86% of the compliance of the pre-guideline period, with a nonsignificant p value (p = 0.40). However, for the guideline for flexible nails for patients who were five to eleven years of age, the risk ratio was 0.65, with a p value of 0.03, indicating that the post-guideline period was associated with only 65% of the compliance of the pre-guideline period. The risk ratios for the treatment of patients older than eleven years, use of regional anesthesia, and treatment alteration for those children with >2-cm fracture shortening in a spica cast could not be calculated because of low compliance or lack of difference between the groups in the two time periods.
Our assessment of the effects of the AAOS clinical practice guideline on the treatment of pediatric diaphyseal femoral fractures demonstrated little significant change of practice. There appeared to be variable influence of this clinical practice guideline, but little overall change from before to after the guideline was published. In general, there appeared to be a divide between those recommendations in which we found a high percentage of pre-guideline publication adherence (Pavlik harness or spica cast for patients who were younger than six months of age and nail or plate for patients who were older than eleven years of age) and those recommendations that were typically not followed prior to the guideline publication (evaluation for non-accidental trauma for patients who were younger than thirty-six months of age, alteration of treatment for >2-cm shortening in a spica cast, and use of flexible nails for patients who were five to eleven years of age). In those guideline recommendations that demonstrated high pre-guideline publication adherence, we generally observed a high degree of guideline adherence after the clinical practice guideline was released as well. Guideline recommendations that demonstrated low pre-publication adherence generally were found to have low post-publication adherence. One exception appeared to be the recommendation for use of a spica cast in patients who were six months to five years of age. This recommendation had a 95% adherence in the pre-guideline publication period, but the adherence dropped to 84% in the post-guideline publication period. The guidelines that were found to have a low or lower percentage of adherence after guideline publication seemed to involve guideline recommendations for broader patient age ranges and more specific recommendations in which alternative treatments used after the guideline publication accounted for the care inconsistencies.
Evaluation after guideline development has been performed for other topics5-14. A review of the clinical practice patterns for nonsurgical treatment of osteoarthritis of the knee was performed to assess the similarities between clinical practice and clinical practice guidelines for this issue. Dhawan et al. found gaps between AAOS-recommended treatment and actual pre-guideline treatment patterns5. Other medical and surgical specialties have performed various analyses of clinical practice guidelines. Many of these reviews used physician surveys to assess adherence to the guidelines10,13, literature reviews to assess validity or robustness of guidelines6,12, assessment of the potential cost-effectiveness of clinical practice guidelines8,14, or comparisons of hospitals with and without clinical practice guidelines to assess clinical impact9. Although these studies generally found improvements in care standardization with clinical practice guidelines9,11, many of these methodologies lacked the ability to assess areas in which the clinical practice guidelines could be improved for greater impact.
Overall, one reason for the apparent modest clinical effect of this clinical practice guideline in the present study could be the lack of higher-level evidence on which this guideline was based. Sanders et al. suggested that “the quality and usefulness of CPGs [clinical practice guidelines] are determined primarily by the strength and clarity of the underlying evidence.”2 Clearly, there is overall weak evidence to support the pediatric diaphyseal fracture guideline, with 43% (six) of the fourteen recommendations lacking enough data for support and only one of the clinical options supported by enough evidence for a firm recommendation. Another possibility is that this guideline is based on outdated evidence, with routine clinical practice already having moved beyond these data. This is possible when evaluating the significant decrease in the percentage of patients treated according to the guideline on the use of flexible nails for children who were five to eleven years of age. After the guideline was published, fewer children were treated with flexible nails and more were treated with rigid nails and submuscular plates than prior to the publication of the guideline. Newer technology (better plates or nails) or different physician training may have led to these alternative treatments, despite the lack of clinical data in the literature to support the inclusion of these techniques in the guideline.
Although clinical practice guidelines address the question of which procedures work successfully and although there is evidence that they improve the process of care, an assessment of the clinical applicability of individual guidelines is appropriate2. One role of these guidelines is to assess the current literature and to highlight fertile areas for future research in a particular area; however, this clinical evaluation of these guidelines may have suggested more focused or clinically important areas of future research2. Combining the clinical practice guidelines and their effects on clinical practice, it may become evident that there are areas of clinical practice that are diverging from the guidelines, such as is seen in the decreasing rate of non-accidental trauma evaluations, the decreasing use of spica casts for patients who are six months to five years of age, and the decreasing rates of use of flexible nails in patients who are five to eleven years of age. These trends may suggest better areas of study to determine how real clinical practice is changing, rather than just where previous research deficiencies lie. Additionally, data documenting the actual clinical impact of clinical practice guidelines may temper the chance of clinical practice guidelines being used as de facto standards of care in legal and reimbursement situations.
Although the poor clinical performance of some of these clinical practice guidelines on the treatment of pediatric femoral fractures may be due to poor data and weak recommendations, the AAOS committees that develop clinical practice guidelines have already realized this issue, and new processes are in place to avoid this situation in the future2. Better selection of topics for clinical practice guidelines (those with data to support recommendations) and required revisions of clinical practice guidelines already published will help in the future.
There were a number of weaknesses of our study. This is a single-center study, which may have introduced bias regarding the treatment of these femoral fractures. Additionally, bias against the clinical practice guideline, the authors of this clinical practice guideline, or the AAOS clinical practice guideline process, or conflicts of interest related to the clinical practice guideline may have determined fracture treatments other than those recommended by the clinical practice guideline.
Overall, we found that the clinical practice guideline on pediatric diaphyseal femoral fractures produced little change in the clinical practice of treating these fractures at our center. In most of the studied recommendations, there was less agreement with the clinical practice guideline after it was introduced, suggesting a change of clinical practice not addressed in the clinical practice guideline. Clinical assessment of clinical practice guidelines after they are published appears to be an important step in the continued evaluation and evolution of clinical practice guidelines, providing important information with regard to treatment patterns and clinically impactful areas of future research and suggesting potential weaknesses of the guidelines. The current clinical assessment of the pediatric femoral fracture clinical practice guideline points to new areas of potential research in this field, including why there was such a low use of non-accidental trauma assessments in children younger than thirty-six months of age and why there was a significantly lower use of flexible nails in patients who were five to eleven years of age after the introduction of this guideline.
Investigation performed at the Department of Orthopaedic Surgery and Sports Medicine, Children’s National Medical Center, Washington, District of Columbia
Disclosure: None of the authors received payments or services, either directly or indirectly (i.e., via his or her institution), from a third party in support of any aspect of this work. One or more of the authors, or his or her institution, has had a financial relationship, in the thirty-six months prior to submission of this work, with an entity in the biomedical arena that could be perceived to influence or have the potential to influence what is written in this work. No author has had any other relationships, or has engaged in any other activities, that could be perceived to influence or have the potential to influence what is written in this work. The complete Disclosures of Potential Conflicts of Interest submitted by authors are always provided with the online version of the article.
1. Gibson TB, Ehrlich ED, Graff J, Dubois R, Farr AM, Chernew M, Fendrick AM. Real-world impact of comparative effectiveness research findings on clinical practice. Am J Manag Care. 2014 ;20(6):e208–20.
2. Sanders JO, Bozic KJ, Glassman SD, Jevsevar DS, Weber KL. Clinical practice guidelines: their use, misuse, and future directions. J Am Acad Orthop Surg. 2014 ;22(3):135–44.
3. Kocher MS, Sink EL, Blasier RD, Luhmann SJ, Mehlman CT, Scher DM, Matheney T, Sanders JO, Watters WC 3rd, Goldberg MJ, Keith MW, Haralson RH 3rd, Turkelson CM, Wies JL, Sluka P, McGowan R; American Academy of Orthopaedic Surgeons. American Academy of Orthopaedic Surgeons clinical practice guideline on treatment of pediatric diaphyseal femur fracture. J Bone Joint Surg Am. 2010 ;92(8):1790–2.
4. Kocher MS, Sink EL, Blasier RD, Luhmann SJ, Mehlman CT, Scher DM, Matheney T, Sanders JO, Watters WC 3rd, Goldberg MJ, Keith MW, Haralson RH 3rd, Turkelson CM, Wies JL, Sluka P, Hitchcock K. Treatment of pediatric diaphyseal femur fractures. J Am Acad Orthop Surg. 2009 ;17(11):718–25.
5. Dhawan A, Mather RC 3rd, Karas V, Ellman MB, Young BB, Bach BR Jr, Cole BJ. An epidemiologic analysis of clinical practice guidelines for non-arthroplasty treatment of osteoarthritis of the knee. Arthroscopy. 2014 ;30(1):65–71. Epub 2013 Nov 28.
6. Gupta M, McCauley J, Farkas A, Gudeloglu A, Neuberger MM, Ho YY, Yeung L, Vieweg J, Dahm P. Clinical practice guidelines on prostate cancer: a critical appraisal. J Urol. 2015 ;193(4):1153–8. Epub 2014 Nov 4.
7. Hathorn C, Alateeqi N, Graham C, O’Hare A. Impact of adherence to best practice guidelines on the diagnostic and assessment services for autism spectrum disorder. J Autism Dev Disord. 2014 ;44(8):1859–66.
8. Mihalopoulos C, Magnus A, Lal A, Dell L, Forbes D, Phelps A. Is implementation of the 2013 Australian treatment guidelines for posttraumatic stress disorder cost-effective compared to current practice? A cost-utility analysis using QALYs and DALYs. Aust N Z J Psychiatry. 2015 ;49(4):360–76. Epub 2014 Oct 27.
9. Mittal V, Hall M, Morse R, Wilson KM, Mussman G, Hain P, Montalbano A, Parikh K, Mahant S, Shah SS. Impact of inpatient bronchiolitis clinical practice guideline implementation on testing and treatment. J Pediatr. 2014 ;165(3):570–6.e3. Epub 2014 Jun 21.
10. Ross RK, Hersh AL, Kronman MP, Newland JG, Metjian TA, Localio AR, Zaoutis TE, Gerber JS. Impact of Infectious Diseases Society of America/Pediatric Infectious Diseases Society guidelines on treatment of community-acquired pneumonia in hospitalized children. Clin Infect Dis. 2014 ;58(6):834–8. Epub 2014 Jan 6.
11. Roussy JP, Bessette L, Bernatsky S, Rahme E, Lachaine J. Rates of non-vertebral osteoporotic fractures in rheumatoid arthritis and postfracture osteoporosis care in a period of evolving clinical practice guidelines. Calcif Tissue Int. 2014 ;95(1):8–18. Epub 2014 Apr 16.
12. Sawatzky B, DiGiovine C, Berner T, Roesler T, Katte L. The need for updated clinical practice guidelines for preservation of upper extremities in manual wheelchair users: a position paper. Am J Phys Med Rehabil. 2015 ;94(4):313–24.
13. Setabutr D, Adil EA, Chaikhoutdinov I, Carr MM. Impact of the pediatric tonsillectomy and polysomnography clinical practice guidelines. Int J Pediatr Otorhinolaryngol. 2014 ;78(3):517–21. Epub 2014 Jan 9.
Copyright 2015 by The Journal of Bone and Joint Surgery, Incorporated
14. Brotons C, Calvo-Bonacho E, Moral I, García-Margallo MT, Cortés-Arcas MV, Puig M, Vázquez-Pirillo G, Ruilope LM. Impact of the new American and British guidelines on the management and treatment of dyslipidemia in a Spanish working population. Rev Esp Cardiol (Engl Ed). 2014 ;67(11):906–11. Epub 2014 Sep 30.