Orthopaedic trauma researchers often recruit patients to prospective long-term longitudinal studies that aim to follow patients to final recovery, often involving several years of follow-up. This involves a large commitment from patient, clinician, and research team. Most studies involve long-term patient reported outcome measure (PROMS) collection, alongside traditional outcomes, and these studies can be expensive to administer and are certainly time consuming for patients to participate in.
Not surprisingly maintaining high rates of follow-up can be challenging in these and other types of research studies. This has been shown in a previous study into traumatic pelvic ring injuries, with 12% of patients not attending a single follow-up and 40% not completing 90 days of follow-up. Similarly, a study into undifferentiated orthopaedic trauma patients showed high rates (>70%) of noncompliance with at least 1 follow-up appointment in the initial 6-month period. However, not all studies have shown such poor compliance with one study showing that only 8.5% of their patients failed to complete the recommended follow-up course despite their patients being similar to others previously studied.
It is important to minimize loss to follow-up in these long-term studies to avoid potential introduction of bias, especially when follow-up rates differ in different patient groups under study. For example, a previous simulation study has suggested that a loss to follow-up of 20% can frequently change the statistical significance of findings.
The orthopaedic trauma population overall does represent a harder patient group to ensure high rates of follow-up from both a clinical and research perspective. This has been theorized to be due to multiple factors that disproportionately affect orthopaedic trauma patients when compared with other surgical groups. These include patient factors such as a higher rate of substance abuse problems, absence of insurance cover, lower socioeconomic status, psychiatric problems, being of no fixed address, and a tendency to move often.[2,5]
Some have advocated simply excluding patients who are unlikely to comply with the requirements of the trial, for example, those who are of no fixed address or who may find follow-up challenging. However, we do not feel this is something we should do in all cases. We have concerns that excluding these “at-risk” groups from our research may mean that the results of any study would not be valid and applicable to a large proportion of our patient group. It has been shown that patients lost to follow-up are a demographically and clinically different patient population from those who remain engaged with long-term prospective trauma studies.
However if we were better informed about what factors influence completion of follow-up, this could allow researchers to design their studies to take these factors into account. For example, more targeted recall mechanisms could be implemented. Measures targeting potential “at-risk’ groups have previously been implemented in the SPRINT trial.
We proposed using patients recruited via 4 prospective longitudinal outcome studies at our Level 1 Trauma Center to assess the potential predictors for completion of follow-up to 12 months. We theorized that demographic factors such as work status and social assistance would be potential predictors for completion of follow-up.
Patients were identified from 4 prospective studies conducted between 2005 and 2015 at our Level 1 Trauma Center to allow the 1-year follow-up window. The studies were into patients with operatively treated pelvic/acetabular, tibial plateau, tibial shaft, and tibial plafond fractures. Each study was approved by the Institutional Ethics Committee of the University of British Columbia. Informed consent was obtained from all patients before participation. All patients included were appropriately covered by their provincial health care plan.
At discharge from hospital, patients rated their preinjury (baseline) status using the Short Form 36 (SF-36) and Short Musculoskeletal Function Assessment (SMFA) questionnaires. Demographic information was collected as part of the SMFA. Previous literature suggests patients can accurately recall their preoperative quality of life, function, and general health up to 6 weeks postsurgery.[9,10] The same questionnaires were then administered again at 6 and 12 months postsurgery. Patients were not paid for participation or parking/travel as questionnaires were administered remotely, although they did need to attend for radiological follow-up.
We defined complete follow-up as completion of all outcome measures at all time points up to and including 12 months. All patient factors collected at the baseline questionnaire including functional outcomes and demographic factors were utilized as putative factors that might influence completion of follow-up.
2.2 Data analysis
Initial assessment was carried out by univariate logistic regression to check to see if the relationship of any predictor was so weak that it should be removed from the analysis. We fitted a null model with only an intercept term, and then sequentially compared this model with the putative predictor. If the model was significantly different we included the predictor in the future analysis. In addition, any predictors that missed significance, but clinically seemed important were included.
A multivariate model was built using the putative predictors identified in the initial analysis and then model reduction and selection was carried out. This involved assessment of the initial multivariate model using the likelihood ratio test. We then selected the model using backward elimination using the Akaike information criterion (AIC). The model with the smallest AIC was selected as the final model.
A total of 1013 patients were previously recruited at our institution in the relevant studies between 2005 and 2015. Some patients did not have all variables recorded in their baseline information and were excluded to ensure that our analysis could assess all variables reliably. This meant 870 patients were included. Seven hundred seven patients (81.2%) completed follow-up to 12 months. Their summary demographics are detailed in Table 1.
The results of the initial screening by univariate logistic regression are in Table 2. The putative factors that were selected as a result of this analysis to go forward in the initial multivariate model were Education, Marital Status, Work Status, SMFA function at baseline, Disability, Social Assistance. Several of the individual questions related to employment and disability were collapsed into single variables (Tables 2 and 3).
The reductive multivariate regression model built as a result of backward elimination resulted in the final model including SF-36 PCS baseline score, Education status, Marital Status, Work Status, and Social assistance status as the final variables. The final multivariate regression model results are shown in Table 3.
The physical component score of SF-36 at baseline was significantly associated (P = .004) with completion of follow-up with a 3.2% increase in the odds of completion of follow-up for every integer increase in SF-36 PCS score. Those who were not on social assistance at the time of injury were significantly more likely to follow up (OR = 2.227) than those who were (P = .002).
Marital status and educational attainment were also found to be statistically significant factors in predicting completion of follow-up. Those who were single or living with a significant other were less likely to follow up than those who were married (P = .005). Increasing levels of educational attainment were associated with increasing odds of completion of follow-up. We did not observe a statistically significant difference in completion of follow-up based on work status at the time of injury.
Long-term prospective studies into orthopaedic trauma patients require high rates of follow-up to avoid introducing bias. The large commitment that this involves from patients, relatives, and staff can mean that certain groups of patient who initially agree to take part subsequently are lost to follow up. In our cohort of over 800 patients treated operatively for a variety of pelvis, acetabulum, and lower limb fractures, we have identified several factors that were identifiable at baseline that were associated with a poorer or better rate of completion of follow-up to 12 months. Our rate of follow-up at 12 months was consistent with other longitudinal studies collecting PROMS.[11,12]
Few studies have looked previously at the factors that influence completion of follow-up. One study looking at factors influencing follow-up of patients in an orthopaedic trauma clinic suggested that tobacco use, distance from the clinic, lack of private insurance, comorbid status, and having sustained a hip/pelvis fracture made patients significantly less likely to follow-up. Another study looking at loss to follow-up in a clinical setting suggested male gender, smokers, lack of commercial health insurance, and illicit drug abuse were all associated with loss of follow-up at 6 months. A further study found that homeless patients were far more likely to fail to follow up in the orthopaedic trauma clinic, especially when treated nonoperatively. In addition, a study from the UK suggested that those who were attending for follow-up rather than an initial attendance, or for an upper limb injury, were more likely to not attend.
All of these studies did not look specifically at patients recruited to research studies, and considering that research studies often follow patients for a significantly longer period of time than would normally be required for clinical purposes alone, it is possible that different or additional factors may be relevant.
We found that patients who had a poorer reported preinjury physical function as defined by a poorer baseline SF-36 PCS had a significantly lower rate of follow-up. Our model suggests that for every integer increase in their baseline SF-36 PCS the odds of completion of follow-up increase by 3.2%. We suggest that this makes sense as those patients who function worse preinjury are less likely to be able to return to clinic for follow-up as even if they recover near to baseline their initial poor physical function may impair their ability to attend for follow-up. This correlates with findings found by studies into ACL reconstruction patients and those who sustained distal radius fractures were where those who had poorer preoperative physical function had poorer follow-up rates.
We also found that being married had a higher rate of completion of follow-up, whereas lower levels of educational attainment had lower rates of follow-up. Not being on social assistance was strongly associated with high rates of follow-up when compared with those that were not. These latter 3 factors could be seen as surrogates for socio-economic status. Certainly, it has been suggested that those who find themselves in a difficult life circumstance such as being homeless are less likely to follow up. Unmarried status and being unemployed or disabled has been shown to be correlated with poorer completion of follow-up in a cohort of patients with metacarpal fractures.
Previously mental health issues were thought to potentially lead to an increased chance of loss to follow-up, but in our study the SF-36 MCS was not found to be a statistically significant influence on completion of follow-up either at univariate or multivariate analysis. This may be because only certain mental health conditions might influence completion of follow-up and these may not be adequately represented in an SF-36 MCS value. For example, we know that depression can play a large role in the outcomes of trauma patients, for example, in those who have low-energy distal radius fractures. However, the use of outcome measures for specific mental health conditions was not done in our studies due to the worry about respondent burden.
Our study has several weaknesses. Our patient groups only cover those with the specific injuries that were being investigated in our prospective studies and all were managed operatively as inpatients. We cannot extrapolate our findings to patients with other injuries, nor to those who were managed either nonoperatively nor those who had surgery on an outpatient basis. We are also limited in the factors we can investigate by the measures that were collected when we recruited patients to our studies. One hundred forty-three patients were excluded from our study due to the absence of 1 or more pieces of baseline data in their initial assessment, their exclusion means that there could be variables that are significant that we missed due to the exclusion of this data.
We were not able to assess the impact of distance of home to the hospital, nor more specific details of comorbidities or drug/alcohol intake. In addition, we may have underestimated the role of mental health issues in completion of follow-up due to the use of the generic SF-36 MCS rather than specific scores such as the Centre of Epidemiologic Studies Depression scale. As our patients were all Canadian and were covered by their provincial health plans we cannot assess the “absence of insurance coverage” as a factor, nor can we be certain of its full applicability to other countries.
We did not look specifically at respondent burden in our studies, and it is known that this can be a major issue determining patient acceptability of outcome measures. Certainly when developing new outcome measures and starting new studies we need to take account of this factor. For example, a previous study from our institution highlighted the limitations of our current pelvic trauma-specific outcome measures due to respondent burden, amongst other issues. We acknowledge that although SF-36 and SMFA were the “gold standard” measures at the time of our study recruitment, they may no longer be universally seen as the best measures for assessment due to their length. As a result of this we are already moving toward the use of computer adaptive testing such as that in the Patient-Reported Outcomes Measurement Information System (PROMIS)[20–22] for future studies.
However our study is strengthened by the large numbers of patients who were part of our studies, and the breadth of lower limb injuries that they sustained. We also used the 2 most common patient-reported outcome measures for orthopaedic trauma at baseline with their associated demographic information which means that our findings are potentially translatable to any study that utilizes these measures.
In conclusion, our study demonstrates in a large research cohort, the factors at baseline which are associated with a failure to complete follow-up. Although the factors that identify patients “at risk” of loss to follow-up are not modifiable themselves, it is vital to try and facilitate continued follow-up from research study patients who might fall into these groups, as differential follow-up rates would most likely invalidate any study findings.
Previously studies[8,23] have suggested setting exclusion criteria to try and avoid recruiting patients who are likely to not complete follow-up. We would advocate caution in setting exclusion criteria too broadly as this may result in skewing potential results. For example, our study has demonstrated that those who are likely to not follow up are often from a lower socio-economic background and are more likely to be a significant part of any patient group in a trauma study.
Instead, we suggest that researchers recruiting patients to studies should screen for potential risk factors for poor follow-up. These include limited preinjury mobility or being on social assistance. Armed with this knowledge, we can now design studies taking account of these risk factors so patients “at-risk” can be targeted for appropriate measures from recruitment through to follow-up. Such measures could include ensuring several alternate contacts are available for patients, searching other resources such as phone directories for those lost to follow-up, allowing flexibility in the follow-up window, and prioritization of certain outcome measures if a patient is finding the full outcome scoring set too burdensome. We should also ensure that we use modern technology for data collection, for example, online data collection portals for PROMS and ensuring that as much outcome data can be collected by phone or online rather than face to face. These and similar measures have been proposed in the SPRINT and Fluid Lavage of Open Wounds (FLOW) trials.
The authors thank Professor John Petkau and Tingting Zhou of the UBC Department of Statistics for help in the statistical analysis of this article.
1. Cronin K, Hayes C, Hockensmith L, et al. The Invisible Patient: Defining the Orthopaedic Trauma
Patient that Does Not Follow Up. New Orleans, Louisiana: Poster presented at AAOS 2018 Annual Meeting; 2018.
2. Zelle BA, Buttacacoli FA, Shroff JB, et al. Loss of follow-up
in orthopaedic trauma
: who is getting lost to follow-up
? J Orthop Trauma
3. Casp AJ, Wells J, Holzgrefe R, et al. Evaulation of orthopedic trauma
and impact of a routine callback program. Orthopedics. 2017;40:e312–e316.
4. Zelle BA, Bhandari M, Sanchez AI, et al. Loss of follow-up
in orthopedic trauma
: is 80% follow-up
still acceptable? J Orthop Trauma
5. Whiting PS, Greenberg SE, Thakore RV, et al. What factors influence follow-up
in orthopedic trauma
surgery? Arch Orthop Trauma
6. Thoma A, Farrokhyar F, McKnight L, et al. How to optimize patient recruitment. Can J Surg. 2010;53:205–210.
7. Murnaghan ML, Buckley RE. Lost but not forgotten: patients lost to follow-up
in a trauma
database. Can J Surg. 2002;45:191–195.
8. Sprague S, Leece P, Bhandari M, et al. Limiting loss to follow-up
in a multicenter randomized trial in orthopedic surgery. Control Clin Trials. 2003;24:719–725.
9. Bryant D, Norman G, Stratford P, et al. Patients undergoing knee surgery provided accurate ratings of preoperative quality of life and function 2 weeks after surgery. J Clin Epidemiol. 2006;59:984–993.
10. Marsh J, Bryant D, MacDonald SJ. Older patients can accurately recall their preoperative health status six weeks following total hip arthroplasty. J Bone Joint Surg Am. 2009;91:2827–2837.
11. Suk M, Daigl M, Buckley RE, et al. Outcomes after orthopedic trauma
. J Orthop Surg (Hong Kong). 2017;25:2309499016684089.
12. Imam MA, Barke S, Stafford GH, et al. Loss to follow-up
after total hip replacement: a source of bias in patient reported outcome measures and registry datasets? Hip Int. 2014;24:465–472.
13. Kay HF, Sathiyakumar V, Archer KR, et al. The homeless orthopedic trauma
, emergency room usage, and complications. J Orthop Trauma
14. Rhind JH, Lovell ME. Failure of fracture patients to attend their outpatient appointments. Ann R Coll Surg Engl (Suppl). 2012;94:1–3.
15. Lian J, Novaretti JV, Patel NK, et al. Younger age and greater preoperative function predict compliance with 2-year follow-up
visits after ACL reconstruction: an analysis of the PIVOT multicentre trial. J ISAKOS Joint Disord Orthopedic Sports Med. 2018;3:251–257.
16. Tejwani NC, Takemoto RC, Nayak G, et al. Who is lost to followup?: A study of patients with distal radius fractures. Clin Orthop Relat Res. 2009;468:599–604.
17. ten Berg PW, Ring D. Patients lost to follow-up
after metacarpal fractures. J Hand Surg Am. 2012;37:42–46.
18. Yeoh JC, Pike JM, Slobogean GP, et al. Role of depression in outcomes of low-energy distal radius fractures in patients older than 55 years. J Orthop Trauma
19. Lefaivre KA, Slobogean GP, Ngai JT, et al. What outcomes are important for patients after pelvic trauma
? Subjective responses and psychometric analysis of three published pelvic-specific outcome instruments. J Orthop Trauma
20. Rose M, Bjorner JB, Becker J, et al. Evaluation of a preliminary physical function item bank supports the expected advantages of the Patient-Reported Outcomes Measurement Information System (PROMIS). J Clin Epidemiol. 2008;61:17–33.
21. Rose M, Bjorner JB, Gandek B, et al. The PROMIS physical function item bank was calibrated to a standardized metric and shown to improve measurement efficiency. J Clin Epidemiol. 2014;67:516–526.
22. Cella D, Gershon R, Lai JS, et al. The future of outcomes measurement: Item banking, tailored short-forms, and computerized adaptive assessment. Qual Life Res. 2007;16 (suppl 1):133–141.
23. Madden K, Scott T, McKay P, et al. Predicting and preventing loss to follow-up
of adult trauma
patients in randomized controlled trials. An example from the FLOW trial. J Bone Joint Surg Am. 2017;99:1086–1092.