The LLRS AIM Index was developed by a committee of the Limb Lengthening and Reconstruction Society through review of the literature and integration of concepts from multiple classification systems for disease-specific limb malformations (Table 1). The LLRS AIM Index measures the severity and complexity of a lower limb deformity through seven domains: location of the deformity, the length of the leg inequality, risk factors, soft tissue injury, angular deformity, infection or bone quality, and motion or stability of the joint. This index provides a uniform assessment of all deformities in a single limb and allows for pretreatment assessment of a broad range of lower extremity disorders. However, it is unclear whether this index is valid and reliable.
We therefore determined (1) whether the LLRS AIM Index shows construct validity by correlating with rankings of case complexity and (2) whether the LLRS AIM Index shows sufficient interrater and intrarater reliabilities.
Materials and Methods
First, members of the Limb Length Reconstruction Society (LLRS) performed a literature review outlining previously established classification systems (Table 1) and complications related to lower limb malformations to guide development of an index for rating the complexity of lower limb deformities. This index was greatly influenced by the scale published by Paley et al.  for rating the level of difficulty of femoral lengthening procedures, which accounted for angulation of the deformity, tibial lengthening, joint instability, knee flexion and deformity, joint osteoarthrosis, bone quality, soft tissue quality, and associated medical problems. The reliability and validity of their rating scale has not been evaluated, but many of the components have been incorporated into this index. The small study group of experts met on multiple occasions to determine the relevance and importance of specific items and features of this index. After repeated modification, the index was presented to the entire LLRS during a special session of the annual meeting. The final version of LLRS AIM Index contains seven pretreatment domains that are rated on a scale from 0 to 4 with increasing severity.
The domains are assessed through history and physical examination and include the Location and number of deformities, the Length of the leg inequality at maturity, Risk factors, Soft tissue coverage, Angular deformity, Infection and bone quality, and the Motion and/or subluxation of joints above and below the deformity (LLRS AIM). The scores are combined into a single index of complexity ranging from normal to high complexity (Table 2). The minimal LLRS AIM Index is 0 and the maximum is 28. An index of 0 is considered normal, an index 1 to 5 is considered to be of minimal complexity, 6 to 10 moderate complexity, 10 to 15 substantial complexity, and 16 to 28 high complexity. The relative weights of the score and overall level of complexity were based on summation of the literature and vetted through expert consensus opinion from the LLRS.
In this study, eight physicians (six attending orthopaedic surgeons and two orthopaedic surgeons-in-training) evaluated 10 fictional patients with various lower limb deformities (Table 3) (Appendix 1. Supplemental material is available with the online version of CORR®). The 10 fictional cases shown in the appendix were developed by the senior authors (JJM, SRR, SS) of this manuscript. All cases were based on real patients whose data were de-identified. The goal was to have a variety of diagnoses with varying degrees of complexity. The physicians were first asked to review the 10 cases and rank them from 1 to 10 in order of apparent complexity, as determined by their experience, with 1 being the easiest and 10 being the most complex. Rankings were performed without use of the LLRS AIM Index. After ranking the cases, they were asked to evaluate them using the LLRS AIM Index and to score each case individually. At least 2 weeks later, the same 10 cases were presented to the same eight raters for reassessment with the LLRS AIM Index.
The interrater reliability, or agreement between raters, of the initial complexity rankings of patients was evaluated with Kendall’s W (coefficient of concordance) as a result of the ranked nature of the data. Linear regression was performed to determine whether patient rankings correlated with LLRS AIM scores, when controlling for rater. The interrater reliability, or agreement between raters of their LLRS AIM indices (for the first and second evaluations separately), was assessed by the intraclass correlation (ICC2,k) from a two-way random effects ANOVA with absolute agreement . The intrarater reliability, or agreement between the first and second evaluations, was assessed with the ICC1,k from a one-way ANOVA . Kendall’s W and ICC range from 0 (no agreement) to 1 (complete agreement). Statistical analyses were performed using SPSS software (Version 19.0; SPSS Inc, Chicago, IL, USA).
Raters agreed on their rankings of the complexity of the patient cases (W = 0.33, X92 = 23.81, p = 0.005). Patient rankings correlated with the LLRS scores for Trial 1 (r2 = 0.25, p < 0.001) and Trial 2 (r2 = 0.23, p < 0.001), when controlling for rater. The reliability between raters on LLRS AIM Index scores was shown by an ICC2,k of 0.97 (95% CI, 0.93-0.99) for Trial 1 and an ICC2,k of 0.98 (95% CI, 0.94-0.99) for Trial 2. The reliability with time of the LLRS AIM Index scores also was shown by a combined ICC1,k = 0.94 (95% CI, 0.91-0.96) for all raters and ICC1,k values ranging from 0.89 to 1.00 for individual raters (Table 4). On average, raters gave the same score for both trials 41% of the time (Table 4).
When comparing levels of experience, the LLRS AIM Index ratings were highly consistent between the attending surgeons and surgeons-in-training (ICC2,k = 0.91). Additionally, the agreement between the two surgeons-in-training (ICC2,k = 0.96) was better than the average agreement between the six attending surgeons (ICC2,k = 0.86). The surgeons-in-training (ICC1,k = 0.96) also had slightly better intrarater reliability than the attending surgeons (ICC1,k = 0.94).
Accurate assessment of the complexity of a limb deformity is essential for successful treatment, and a validated rating scale for evaluating the complexity of general lower limb deformities currently does not exist. The purpose of this study was to describe the development and perform a preliminary assessment of the validity and reliability of a general limb deformity index for systematic pretreatment assessment of a broad range of lower extremity disorders (the LLRS AIM Index). The index accounts for seven domains, including the number of locations of deformity, leg length inequality, risk factors, soft tissue assessment, angular deformity, infection and bone quality, and motion and stability of the joints.
We caution readers of the limitations of our study. First, this study did not thoroughly assess the validity, the predictive value, or the usefulness of the LLRS AIM Index in guiding treatment. Future studies are necessary to determine whether the LLRS AIM ratings correlate with patient outcomes. Second, all patients, although based on true patient encounters, were fictionalized to protect patient identity among a unique group of patients (who may have seen many of the limb deformity experts) and for greater ease of the study. Third, the LLRS AIM Index does not differentiate between acquired and congenital deformities, although it can be used to evaluate both. Surgical correction for congenital deformities often is more difficult than for acquired deformities, with a higher complication rate [15, 22]. Fourth, development of the score was performed primarily through consensus and expert opinion. Relative weights of the score were extrapolated from the literature. Although the score was vetted on several occasions, including an open forum at the LLRS, there is an inherent difficulty in exactly determining the weighted components of each domain. Further analysis of the specific contribution of the seven domains to complexity scores in a larger study is necessary to validate the LLRS AIM Index and to determine whether specific scores can be used to effectively predict and guide treatment decision-making.
The first aim of this study was to perform a simplistic evaluation of the construct validity of the LLRS AIM Index by determining whether LLRS AIM scores correlate with rankings of case complexity. The raters showed statistically significant agreement in their rankings of the complexity of the patient cases. The patient rankings also correlated with their LLRS AIM scores, suggesting that higher LLRS AIM scores are indicative of more complex cases. There are currently no validated rating systems for evaluating the complexity of general lower limb deformities to compare the validity of the LLRS AIM Index. One study by Paley et al.  rated 29 patients who had undergone femoral lengthening over an intramedullary nail using a similar rating scale of complexity to guide treatment choice. Unfortunately, the variability of their patient scores was not reported, and the rating scale has not been validated. Although further validation is necessary for the LLRS AIM Index, preliminary evaluation suggests that it is a valid measure of the complexity of nonspecific lower limb deformities.
The second aim of this study was to evaluate the interrater and intrarater reliabilities of the LLRS AIM Index. The LLRS AIM Index showed near perfect reliability between the eight raters and with time for assessing the complexity of the 10 cases. The raters also were more consistent at evaluating the complexity of patients with lower limb deformities when using the LLRS AIM Index than when simply ranking complexity. Although the LLRS AIM Index is a more complex rating scale, it was capable of producing much more repeatable results. Furthermore, this reliability was obtained despite the varying experience of our raters, which suggests that the LLRS AIM Index can provide for a simple, reproducible, common language of limb deformity.
The LLRS AIM Index is capable of assessing the entire clinical picture of a patient with any lower limb deformity through a common language that is simple enough for referring providers to use when discussing the complexity of a case with a specialist. A complete clinical examination is necessary, especially in a patient with a congenital limb deformity . The LLRS AIM Index not only measures the amount of deformity present with uniform methodology in the affected limb, but also considers other factors, such as infection, soft tissue coverage, and chronic medical conditions, that are known to affect treatment. The LLRS AIM index allows for stratification of patients with higher versus lower complexity deformities in a reliable (repeatable) context, and potentially will be the basis for incorporating risk-adjusted complications in future research. To our knowledge, a rating system that can be used to assess the complexity of general lower limb deformities, whether acquired or congenital, does not exist. The LLRS AIM Index was developed to fulfill this need, and this study showed that the LLRS AIM Index is a highly reliable tool, consistent between raters and with time, for assessing the complexity of various lower limb deformities.
We thank the members of the Limb Lengthening and Reconstruction Society and John G. Birch, MD, David J. Klimaski MD, Viral Jain MD, Nathan Faulkner MD, and Esther Cheng BS for their guidance and assistance with this research project and article.
1. Achterman, C. and Kalamchi, A. Congenital deficiency of the fibula. J Bone Joint Surg Br.
1979; 61: 133-137.
2. Aitken, GT. In: Aitken, GT. (ed.), Proximal femoral focal deficiency: definition, classification, and management. Proximal Femoral Focal Deficiency: A Congenital Anomaly.
1969. Washington, DC: National Academy of Sciences. 1-22.
3. Andersen, KS. Radiological classification of congenital pseudarthrosis of the tibia. Acta Orthop Scand.
1973; 44: 719-727. 10.3109/17453677308989112
4. Bhave, A., Paley, D. and Herzenberg, JE. Improvement in gait parameters after lengthening for the treatment of limb-length discrepancy. J Bone Joint Surg Am.
1999; 81: 529-534.
5. Birch, JG., Lincoln, TL., Mack, PW. and Birch, CM. Congenital fibular deficiency: a review of thirty years’ experience at one institution and a proposed classification system based on clinical deformity. J Bone Joint Surg Am.
2011; 93: 1144-1151. 10.2106/JBJS.J.00683
6. Boyd, HB. Pathology and natural history of congenital pseudarthrosis of the tibia. Clin Orthop Relat Res.
1982; 166: 5-13.
7. Brouwer, GM., Tol, AW., Bergink, AP., Belo, JN., Bernsen, RM., Reijman, M., Pols, HA. and Bierma-Zeinstra, SM. Association between valgus and varus alignment and the development and progression of radiographic osteoarthritis of the knee. Arthritis Rheum.
2007; 56: 1204-1211. 10.1002/art.22515
8. Catagni, MA., Guerreschi, F., Manzotti, A. and Knuth, A. Treatment of foot deformities using the Ilizarov method. Foot Ankle Surg.
2000; 6: 207-237. 10.1046/j.1460-9584.2000.00219.x
9. Catagni, MA., Radwan, M., Lovisetti, L., Guerreschi, F. and Elmoghazy, NA. Limb lengthening and deformity correction by the Ilizarov technique in type III fibular hemimelia: an alternative to amputation. Clin Orthop Relat Res.
2011; 469: 1175-1180. 10.1007/s11999-010-1635-7
10. Crawford, AH. Neurofibromatosis in children. Acta Orthop Scand Suppl.
1986; 218: 1-60.
11. Hamanashi, C. Congenital vertical talus: classification with 69 cases and new measurement system. J Pediatr Orthop.
1984; 4: 318-326. 10.1097/01241398-198405000-00007
12. Hefti, F., Brunner, R., Freuler, F., Hasler, C. and Jundt, G. Pediatric Orthopedics in Practice
Berlin, Germany: Springer; 2007.
13. Jones, D., Barnes, J. and Lloyd-Roberts, GC. Congenital aplasia and dysplasia of the tibia with intact fibula: classification and management. J Bone Joint Surg Br.
1978; 60: 31-39.
14. Kalamchi, A. and Dawe, RV. Congenital deficiency of the tibia. J Bone Joint Surg Br.
1985; 67: 581-584.
15. Kaufman, KR., Miller, LS. and Sutherland, DH. Gait asymmetry in patients with limb-length inequality. J Pediatr Orthop.
1996; 16: 144-150. 10.1097/01241398-199603000-00002
16. McCarthy, JJ. and MacEwen, GD. Management of leg length inequality. J South Orthop Assoc.
2001; 10: 73-85.
17. Paley, D. In: Herring, JA. and Birch, JG. (eds.), Lengthening reconstruction surgery for congenital femoral deficiency. The Child with a Limb Deficiency.
1998. Rosemont, IL: American Academy of Orthopaedic Surgeons. 113-132.
18. Paley, D., Bhave, A., Herzenberg, JE. and Bowen, JR. Multiplier method for predicting limb-length discrepancy. J Bone Joint Surg Am.
2000; 82: 1432-1446.
19. Pappas, AM. Congenital abnormalities of the femur and related lower extremity malformations: classification and treatment. J Pediatr Orthop.
1983; 3: 45-60. 10.1097/01241398-198302000-00009
20. Sabharwal, S. and Zhao, C. The hip-knee-ankle angle in children: reference values based on a full-length standing radiograph. J Bone Joint Surg Am.
2009; 91: 2461-2468. 10.2106/JBJS.I.00015
21. Shrout, PE. and Fleiss, JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull.
1979; 86: 420-428. 10.1037/0033-2909.86.2.420
22. Song, KM., Halliday, SE. and Little, DG. The effect of limb-length discrepancy on gait. J Bone Joint Surg Am.
1997; 79: 1690-1698. 10.1302/0301-620X.79B5.7615
23. Stanitski, DF. Limb-length inequality: assessment and treatment options. J Am Acad Orthop Surg.
1999; 7: 143-153.
24. Stanitski, DF. and Stanitski, CL. Fibular hemimelia: a new classification system. J Pediatr Orthop.
2003; 23: 30-34.
© 2013 Lippincott Williams & Wilkins LWW
25. Torode, IP. and Gillespie, R. The classification and treatment of proximal femoral deficiencies. Prosthet Orthot Int.
1991; 15: 117-126.