Secondary Logo

Journal Logo

Limb Lengthening and Reconstruction Society AIM Index Reliably Assesses Lower Limb Deformity

McCarthy, James, J., MD1, a; Iobst, Christopher, A., MD2; Rozbruch, Robert, S., MD3; Sabharwal, Sanjeev, MD4; Eismann, Emily, A., MS5

Clinical Orthopaedics and Related Research: February 2013 - Volume 471 - Issue 2 - p 621–627
doi: 10.1007/s11999-012-2609-8
Clinical Research
Free
SDC

Background Although several systems exist for classifying specific limb deformities, there currently are no validated rating scales for evaluating the complexity of general lower limb deformities. Accurate assessment of the complexity of a limb deformity is essential for successful treatment. A committee of the Limb Lengthening and Reconstruction Society (LLRS) therefore developed the LLRS AIM Index to quantify the severity of a broad range of lower extremity deformities in seven domains.

Questions/Purposes We addressed two questions: (1) Does the LLRS AIM Index show construct validity by correlating with rankings of case complexity? (2) Does the LLRS AIM Index show sufficient interrater and intrarater reliabilities?

Methods We had eight surgeons evaluate 10 fictionalized patients with various lower limb deformities. First, they ranked the cases from simplest to most complex, and then they rated the cases using the LLRS AIM Index. Two or more weeks later, they rated the cases again. We assessed reliability using the Kendall’s W test.

Results Raters were consistent in their rankings of case complexity (W = 0.33). Patient rankings also correlated with both sets of LLRS AIM ratings (r2 = 0.25; r2 = 0.23). The LLRS AIM Index showed interrater reliability with an intraclass correlation (ICC) of 0.97 for Trial 1 and 0.98 for Trial 2 and intrarater reliability with an ICC of 0.94. The LLRS AIM Index ratings also were highly consistent between the attending surgeons and surgeons-in-training (ICC = 0.91).

Conclusions Our preliminarily observations suggest that the LLRS AIM Index reliably classifies the complexity of lower limb deformities in and between observers.

1 Division of Orthopaedic Surgery, Cincinnati Children’s Hospital Medical Center, 3333 Burnet Avenue, ML 2017, 45229, Cincinnati, OH, USA

2 Department of Orthopaedics, Miami Children’s Hospital, Miami, FL, USA

3 Department of Orthopaedic Surgery, Hospital for Special Surgery, New York, NY, USA

4 Department of Orthopaedics, New Jersey Medical School, University of Medicine & Dentistry of New Jersey, Newark, NJ, USA

5 Division of Orthopaedic Surgery, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, USA

a e-mail; james.mccarthy@cchmc.org

Received: March 19, 2012 / Accepted: September 7, 2012 / Published online: October 2, 2012

Each author certifies that he or she, or a member of his or her immediate family, has no funding or commercial associations (eg, consultancies, stock ownership, equity interest, patent/licensing arrangements, etc) that might pose a conflict of interest in connection with the submitted article.

All ICMJE Conflict of Interest Forms for authors and Clinical Orthopaedics and Related Research editors and board members are on file with the publication and can be viewed on request.

This work was performed at Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, USA.

Electronic supplementary material

The online version of this article (doi:10.1007/s11999-012-2609-8) contains supplementary material, which is available to authorized users.

Back to Top | Article Outline

Introduction

Limb length inequality and deformity are common in the general population with approximately 23% of people having a 1-cm or greater discrepancy [16] and many others with limb deformities that vary from mild varus to complex congenital deformities. Accurate assessment of the complexity of a limb deformity is essential for successful treatment [4, 7, 15, 20, 22, 23]. During the past two decades, several classification systems have been established and disseminated for assessing and categorizing specific joints, limb segments, and disease states (Table 1) [1-3, 5, 6, 8, 10-14, 17-19, 24, 25]. Paley [17] used a four-category classification system for femoral deficiency and deformity that is based on the factors that influence lengthening reconstruction of the congenitally deformed femur. He classified congenital femoral deficiency (CFD) into four groups, and although his scheme did allow for assessment of CFD, it failed to account for the broader clinical picture. Similarly, Jones et al. [13] proposed a classification scheme consisting of four morphologic and four radiographic groups in patients with congenital aplasia or dysplasia of the tibia with an intact fibula. Radiographic features were used to distinguish between anatomic variants, whereas morphologic classification was used to guide treatment. Before the treatment of fibular hemimelia, Catagni et al. [9] used a modified Dalmonte classification consisting of three types based on the structural features of the fibula. Birch et al. [5] proposed a new classification system for congenital fibular deficiency that was based on the functionality of the foot and leg length inequality. In a similar fashion, Pappas [19] proposed a nine-category classification system to cover the spectrum of femoral deficiency that later was modified by Paley [17]. The only rating system that has been developed evaluates femoral length discrepancies, but it has yet to be validated [18]. Currently, there are no validated rating systems for determining the severity of general lower limb deformities. A universal, valid, and reliable rating system for such assessment based on multiple criteria could be beneficial for comparing deformities within and between studies and for determining appropriate treatments.

Table 1

Table 1

Table 1

Table 1

The LLRS AIM Index was developed by a committee of the Limb Lengthening and Reconstruction Society through review of the literature and integration of concepts from multiple classification systems for disease-specific limb malformations (Table 1). The LLRS AIM Index measures the severity and complexity of a lower limb deformity through seven domains: location of the deformity, the length of the leg inequality, risk factors, soft tissue injury, angular deformity, infection or bone quality, and motion or stability of the joint. This index provides a uniform assessment of all deformities in a single limb and allows for pretreatment assessment of a broad range of lower extremity disorders. However, it is unclear whether this index is valid and reliable.

We therefore determined (1) whether the LLRS AIM Index shows construct validity by correlating with rankings of case complexity and (2) whether the LLRS AIM Index shows sufficient interrater and intrarater reliabilities.

Back to Top | Article Outline

Materials and Methods

First, members of the Limb Length Reconstruction Society (LLRS) performed a literature review outlining previously established classification systems (Table 1) and complications related to lower limb malformations to guide development of an index for rating the complexity of lower limb deformities. This index was greatly influenced by the scale published by Paley et al. [18] for rating the level of difficulty of femoral lengthening procedures, which accounted for angulation of the deformity, tibial lengthening, joint instability, knee flexion and deformity, joint osteoarthrosis, bone quality, soft tissue quality, and associated medical problems. The reliability and validity of their rating scale has not been evaluated, but many of the components have been incorporated into this index. The small study group of experts met on multiple occasions to determine the relevance and importance of specific items and features of this index. After repeated modification, the index was presented to the entire LLRS during a special session of the annual meeting. The final version of LLRS AIM Index contains seven pretreatment domains that are rated on a scale from 0 to 4 with increasing severity.

The domains are assessed through history and physical examination and include the Location and number of deformities, the Length of the leg inequality at maturity, Risk factors, Soft tissue coverage, Angular deformity, Infection and bone quality, and the Motion and/or subluxation of joints above and below the deformity (LLRS AIM). The scores are combined into a single index of complexity ranging from normal to high complexity (Table 2). The minimal LLRS AIM Index is 0 and the maximum is 28. An index of 0 is considered normal, an index 1 to 5 is considered to be of minimal complexity, 6 to 10 moderate complexity, 10 to 15 substantial complexity, and 16 to 28 high complexity. The relative weights of the score and overall level of complexity were based on summation of the literature and vetted through expert consensus opinion from the LLRS.

Table 2

Table 2

In this study, eight physicians (six attending orthopaedic surgeons and two orthopaedic surgeons-in-training) evaluated 10 fictional patients with various lower limb deformities (Table 3) (Appendix 1. Supplemental material is available with the online version of CORR®). The 10 fictional cases shown in the appendix were developed by the senior authors (JJM, SRR, SS) of this manuscript. All cases were based on real patients whose data were de-identified. The goal was to have a variety of diagnoses with varying degrees of complexity. The physicians were first asked to review the 10 cases and rank them from 1 to 10 in order of apparent complexity, as determined by their experience, with 1 being the easiest and 10 being the most complex. Rankings were performed without use of the LLRS AIM Index. After ranking the cases, they were asked to evaluate them using the LLRS AIM Index and to score each case individually. At least 2 weeks later, the same 10 cases were presented to the same eight raters for reassessment with the LLRS AIM Index.

Table 3

Table 3

The interrater reliability, or agreement between raters, of the initial complexity rankings of patients was evaluated with Kendall’s W (coefficient of concordance) as a result of the ranked nature of the data. Linear regression was performed to determine whether patient rankings correlated with LLRS AIM scores, when controlling for rater. The interrater reliability, or agreement between raters of their LLRS AIM indices (for the first and second evaluations separately), was assessed by the intraclass correlation (ICC2,k) from a two-way random effects ANOVA with absolute agreement [21]. The intrarater reliability, or agreement between the first and second evaluations, was assessed with the ICC1,k from a one-way ANOVA [21]. Kendall’s W and ICC range from 0 (no agreement) to 1 (complete agreement). Statistical analyses were performed using SPSS software (Version 19.0; SPSS Inc, Chicago, IL, USA).

Back to Top | Article Outline

Results

Raters agreed on their rankings of the complexity of the patient cases (W = 0.33, X92 = 23.81, p = 0.005). Patient rankings correlated with the LLRS scores for Trial 1 (r2 = 0.25, p < 0.001) and Trial 2 (r2 = 0.23, p < 0.001), when controlling for rater. The reliability between raters on LLRS AIM Index scores was shown by an ICC2,k of 0.97 (95% CI, 0.93-0.99) for Trial 1 and an ICC2,k of 0.98 (95% CI, 0.94-0.99) for Trial 2. The reliability with time of the LLRS AIM Index scores also was shown by a combined ICC1,k = 0.94 (95% CI, 0.91-0.96) for all raters and ICC1,k values ranging from 0.89 to 1.00 for individual raters (Table 4). On average, raters gave the same score for both trials 41% of the time (Table 4).

Table 4

Table 4

When comparing levels of experience, the LLRS AIM Index ratings were highly consistent between the attending surgeons and surgeons-in-training (ICC2,k = 0.91). Additionally, the agreement between the two surgeons-in-training (ICC2,k = 0.96) was better than the average agreement between the six attending surgeons (ICC2,k = 0.86). The surgeons-in-training (ICC1,k = 0.96) also had slightly better intrarater reliability than the attending surgeons (ICC1,k = 0.94).

Back to Top | Article Outline

Discussion

Accurate assessment of the complexity of a limb deformity is essential for successful treatment, and a validated rating scale for evaluating the complexity of general lower limb deformities currently does not exist. The purpose of this study was to describe the development and perform a preliminary assessment of the validity and reliability of a general limb deformity index for systematic pretreatment assessment of a broad range of lower extremity disorders (the LLRS AIM Index). The index accounts for seven domains, including the number of locations of deformity, leg length inequality, risk factors, soft tissue assessment, angular deformity, infection and bone quality, and motion and stability of the joints.

We caution readers of the limitations of our study. First, this study did not thoroughly assess the validity, the predictive value, or the usefulness of the LLRS AIM Index in guiding treatment. Future studies are necessary to determine whether the LLRS AIM ratings correlate with patient outcomes. Second, all patients, although based on true patient encounters, were fictionalized to protect patient identity among a unique group of patients (who may have seen many of the limb deformity experts) and for greater ease of the study. Third, the LLRS AIM Index does not differentiate between acquired and congenital deformities, although it can be used to evaluate both. Surgical correction for congenital deformities often is more difficult than for acquired deformities, with a higher complication rate [15, 22]. Fourth, development of the score was performed primarily through consensus and expert opinion. Relative weights of the score were extrapolated from the literature. Although the score was vetted on several occasions, including an open forum at the LLRS, there is an inherent difficulty in exactly determining the weighted components of each domain. Further analysis of the specific contribution of the seven domains to complexity scores in a larger study is necessary to validate the LLRS AIM Index and to determine whether specific scores can be used to effectively predict and guide treatment decision-making.

The first aim of this study was to perform a simplistic evaluation of the construct validity of the LLRS AIM Index by determining whether LLRS AIM scores correlate with rankings of case complexity. The raters showed statistically significant agreement in their rankings of the complexity of the patient cases. The patient rankings also correlated with their LLRS AIM scores, suggesting that higher LLRS AIM scores are indicative of more complex cases. There are currently no validated rating systems for evaluating the complexity of general lower limb deformities to compare the validity of the LLRS AIM Index. One study by Paley et al. [18] rated 29 patients who had undergone femoral lengthening over an intramedullary nail using a similar rating scale of complexity to guide treatment choice. Unfortunately, the variability of their patient scores was not reported, and the rating scale has not been validated. Although further validation is necessary for the LLRS AIM Index, preliminary evaluation suggests that it is a valid measure of the complexity of nonspecific lower limb deformities.

The second aim of this study was to evaluate the interrater and intrarater reliabilities of the LLRS AIM Index. The LLRS AIM Index showed near perfect reliability between the eight raters and with time for assessing the complexity of the 10 cases. The raters also were more consistent at evaluating the complexity of patients with lower limb deformities when using the LLRS AIM Index than when simply ranking complexity. Although the LLRS AIM Index is a more complex rating scale, it was capable of producing much more repeatable results. Furthermore, this reliability was obtained despite the varying experience of our raters, which suggests that the LLRS AIM Index can provide for a simple, reproducible, common language of limb deformity.

The LLRS AIM Index is capable of assessing the entire clinical picture of a patient with any lower limb deformity through a common language that is simple enough for referring providers to use when discussing the complexity of a case with a specialist. A complete clinical examination is necessary, especially in a patient with a congenital limb deformity [23]. The LLRS AIM Index not only measures the amount of deformity present with uniform methodology in the affected limb, but also considers other factors, such as infection, soft tissue coverage, and chronic medical conditions, that are known to affect treatment. The LLRS AIM index allows for stratification of patients with higher versus lower complexity deformities in a reliable (repeatable) context, and potentially will be the basis for incorporating risk-adjusted complications in future research. To our knowledge, a rating system that can be used to assess the complexity of general lower limb deformities, whether acquired or congenital, does not exist. The LLRS AIM Index was developed to fulfill this need, and this study showed that the LLRS AIM Index is a highly reliable tool, consistent between raters and with time, for assessing the complexity of various lower limb deformities.

Back to Top | Article Outline

Acknowledgments

We thank the members of the Limb Lengthening and Reconstruction Society and John G. Birch, MD, David J. Klimaski MD, Viral Jain MD, Nathan Faulkner MD, and Esther Cheng BS for their guidance and assistance with this research project and article.

Back to Top | Article Outline

References

1. Achterman, C. and Kalamchi, A. Congenital deficiency of the fibula. J Bone Joint Surg Br. 1979; 61: 133-137.
2. Aitken, GT. In: Aitken, GT. (ed.), Proximal femoral focal deficiency: definition, classification, and management. Proximal Femoral Focal Deficiency: A Congenital Anomaly. 1969. Washington, DC: National Academy of Sciences. 1-22.
3. Andersen, KS. Radiological classification of congenital pseudarthrosis of the tibia. Acta Orthop Scand. 1973; 44: 719-727. 10.3109/17453677308989112
4. Bhave, A., Paley, D. and Herzenberg, JE. Improvement in gait parameters after lengthening for the treatment of limb-length discrepancy. J Bone Joint Surg Am. 1999; 81: 529-534.
5. Birch, JG., Lincoln, TL., Mack, PW. and Birch, CM. Congenital fibular deficiency: a review of thirty years’ experience at one institution and a proposed classification system based on clinical deformity. J Bone Joint Surg Am. 2011; 93: 1144-1151. 10.2106/JBJS.J.00683
6. Boyd, HB. Pathology and natural history of congenital pseudarthrosis of the tibia. Clin Orthop Relat Res. 1982; 166: 5-13.
7. Brouwer, GM., Tol, AW., Bergink, AP., Belo, JN., Bernsen, RM., Reijman, M., Pols, HA. and Bierma-Zeinstra, SM. Association between valgus and varus alignment and the development and progression of radiographic osteoarthritis of the knee. Arthritis Rheum. 2007; 56: 1204-1211. 10.1002/art.22515
8. Catagni, MA., Guerreschi, F., Manzotti, A. and Knuth, A. Treatment of foot deformities using the Ilizarov method. Foot Ankle Surg. 2000; 6: 207-237. 10.1046/j.1460-9584.2000.00219.x
9. Catagni, MA., Radwan, M., Lovisetti, L., Guerreschi, F. and Elmoghazy, NA. Limb lengthening and deformity correction by the Ilizarov technique in type III fibular hemimelia: an alternative to amputation. Clin Orthop Relat Res. 2011; 469: 1175-1180. 10.1007/s11999-010-1635-7
10. Crawford, AH. Neurofibromatosis in children. Acta Orthop Scand Suppl. 1986; 218: 1-60.
11. Hamanashi, C. Congenital vertical talus: classification with 69 cases and new measurement system. J Pediatr Orthop. 1984; 4: 318-326. 10.1097/01241398-198405000-00007
12. Hefti, F., Brunner, R., Freuler, F., Hasler, C. and Jundt, G. Pediatric Orthopedics in Practice Berlin, Germany: Springer; 2007.
13. Jones, D., Barnes, J. and Lloyd-Roberts, GC. Congenital aplasia and dysplasia of the tibia with intact fibula: classification and management. J Bone Joint Surg Br. 1978; 60: 31-39.
14. Kalamchi, A. and Dawe, RV. Congenital deficiency of the tibia. J Bone Joint Surg Br. 1985; 67: 581-584.
15. Kaufman, KR., Miller, LS. and Sutherland, DH. Gait asymmetry in patients with limb-length inequality. J Pediatr Orthop. 1996; 16: 144-150. 10.1097/01241398-199603000-00002
16. McCarthy, JJ. and MacEwen, GD. Management of leg length inequality. J South Orthop Assoc. 2001; 10: 73-85.
17. Paley, D. In: Herring, JA. and Birch, JG. (eds.), Lengthening reconstruction surgery for congenital femoral deficiency. The Child with a Limb Deficiency. 1998. Rosemont, IL: American Academy of Orthopaedic Surgeons. 113-132.
18. Paley, D., Bhave, A., Herzenberg, JE. and Bowen, JR. Multiplier method for predicting limb-length discrepancy. J Bone Joint Surg Am. 2000; 82: 1432-1446.
19. Pappas, AM. Congenital abnormalities of the femur and related lower extremity malformations: classification and treatment. J Pediatr Orthop. 1983; 3: 45-60. 10.1097/01241398-198302000-00009
20. Sabharwal, S. and Zhao, C. The hip-knee-ankle angle in children: reference values based on a full-length standing radiograph. J Bone Joint Surg Am. 2009; 91: 2461-2468. 10.2106/JBJS.I.00015
21. Shrout, PE. and Fleiss, JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979; 86: 420-428. 10.1037/0033-2909.86.2.420
22. Song, KM., Halliday, SE. and Little, DG. The effect of limb-length discrepancy on gait. J Bone Joint Surg Am. 1997; 79: 1690-1698. 10.1302/0301-620X.79B5.7615
23. Stanitski, DF. Limb-length inequality: assessment and treatment options. J Am Acad Orthop Surg. 1999; 7: 143-153.
24. Stanitski, DF. and Stanitski, CL. Fibular hemimelia: a new classification system. J Pediatr Orthop. 2003; 23: 30-34.
25. Torode, IP. and Gillespie, R. The classification and treatment of proximal femoral deficiencies. Prosthet Orthot Int. 1991; 15: 117-126.
© 2013 Lippincott Williams & Wilkins LWW