Secondary Logo

Journal Logo

Original Research

Bridging the Gap Between Content and Context

Establishing Expert Consensus on the Content of an Exercise Training Program to Prevent Lower-Limb Injuries

Donaldson, Alex DHSc*; Cook, Jill PhD*,†; Gabbe, Belinda PhD; Lloyd, David G. PhD§; Young, Warren PhD; Finch, Caroline F. PhD*

Author Information
Clinical Journal of Sport Medicine: May 2015 - Volume 25 - Issue 3 - p 221-229
doi: 10.1097/JSM.0000000000000124



Australian Football (AF) is a popular community participation sport.1 It is a dynamic sport incorporating running, jumping and landing, changing direction, rapid acceleration and deceleration, full body contact, including tackling and bumping, and kicking and marking (catching) a ball. Australian Football has a relatively high risk and rate of injury compared with other popular community sports.2–4 For example, the injury incidence rate in AF is 20.3 injuries per 1000 hours of sports participation (in competition and training) compared with 15.2/1000 hours in field hockey, 15.1/1000 hours in basketball, and 12.1/1000 hours in netball.2 Studies have consistently shown that lower-limb injuries (knee, ankle, and thigh/hamstring) are common in community AF,5–8 and muscle or tendon strains and joint or ligament sprains are among the most common injury types.6,8 Given the site and nature of these common injuries, it is likely that a high proportion of lower-limb injuries among community-AF players could be prevented,9 and they have been identified as an injury prevention priority.5,9

A recent review of lower-limb injury prevention exercise protocols identified that balance and control exercises might prevent ankle ligament injuries and that a program that includes balance and control, eccentric hamstring, plyometric, and strength exercises could prevent all types of lower-limb injuries in community AF.9 This is consistent with the contents of existing lower-limb injury prevention programs such as the FIFA 11+10 and the Prevent Injury and Enhance Performance program.11 The purpose of this study was to reach expert consensus about what specific exercises and their appropriate progressions should be included in an evidence-informed training program to prevent lower-limb injuries among community-AF players.


This study used a Delphi design where a panel of experts provided the feedback on the contents of an exercise training program to prevent lower-limb injuries among community-AF players.

Exercise Training Program

A draft exercise training program (now known as FootyFirst) was developed specifically to prevent ankle, knee, groin, hip, and hamstring injuries within the context of community AF. FootyFirst was based on an identified need,5 the available evidence from the literature9 and the biomechanics, physiotherapy, and sports science experience of the research team. The program content was developed for delivery by community-AF coaches with no specific strength and conditioning or human movement training or expertise. It was designed to be a replacement for, or accompaniment to, existing warm-ups used in twice weekly community-AF training sessions. It was considered that some of the FootyFirst drills could be integrated into normal team skill training, such as jumping, landing, sidestepping/cutting, and overhead marking. Club resources (personnel, equipment, etc.), available time, community-AF coach knowledge, and the anticipated fitness and strength levels of community-AF players were considered during program development.

Similar to lower-limb injury prevention programs developed for other sports,12,13 FootyFirst was designed to be implemented throughout the preseason and playing season, delivered by club coaches or strength and conditioning staff to teams of players, and performed twice a week by all players at the start of a training session. The draft FootyFirst program began with a general warm-up, including run throughs and dynamic stretches, to prepare players to participate in the injury prevention exercises. This was followed by lower-limb strength and conditioning exercises and exercises/drills to enhance balance, landing, and side-stepping techniques. Five levels of progression were included for each exercise, with players encouraged to start at level 1 and progress with correct technique to subsequent levels as their strength, muscular endurance, balance, and flexibility improved. An outline of the draft FootyFirst program, as used in the 3 rounds of this study, is provided in Table 1. Copies of the full draft programs are available on request.

Exercise Protocols Used in Rounds 1, 2, and 3 of the Delphi Process

Delphi Technique

The Delphi technique, a multistage iterative process to translate individual expert opinion into group consensus14 was used. This research technique has previously been used to achieve expert consensus in a range of sports safety and sports medicine areas.15–17 In this study, the Delphi consisted of 3 rounds of consultation using online surveys and adhered to the fundamental Delphi principles of respondent anonymity and feedback between rounds.14 Monash University Human Research Ethics Committee approved the study protocol.

Although opinion differs as to what should constitute consensus in Delphi studies, 75% agreement has been frequently accepted and is recommended as a minimum level.18 For this study, consensus was considered to be reached when the sum of the number of experts who “agreed” and “strongly agreed,” or who “disagreed” and “strongly disagreed,” that an exercise or its progressions were appropriate for including in FootyFirst was ≥75% of the total number of respondents for the item.

Identification of Experts

One challenge when conducting Delphi studies is identifying appropriate experts for the topic and context of interest.19 The Australian Football League (AFL) is the elite professional AF competition, and all members of the AFL's Medical Officers Association (AFLMOA) (n = 94), Physiotherapists Association (AFLPA) (n = 50), and Sports Science Association (AFLSSA) (n = 19) were considered to have relevant expertise in lower-limb injury prevention in AF and were invited to participate. Five other Australian-based people with more general clinical and research expertise in sports-related lower-limb injury prevention who were known to members of the research team were also invited to participate.

To facilitate the recruitment of the invited experts, the executive of all 3 membership bodies and the AFL endorsed the Delphi study. The online survey included questions about the experts' qualifications, discipline, and experience, and all experts were asked to provide this background information the first time they completed a survey round.

Questionnaire Development and Survey Methods

For all rounds of the Delphi, experts were sent an e-mail mid-football season (June to July 2011) with a link to an online survey and the relevant electronic version of FootyFirst attached. Access to a list of references used to inform the development of FootyFirst was also provided. Experts were given 12 days to return each survey with reminders e-mailed on days 7 and 12.

Round 1

In round 1, experts rated their agreement with including each of the proposed exercises and its progression in FootyFirst. For ease of expression, the term “exercise” will be used throughout the rest of this article to incorporate “an exercise and its progression (from level 1 to level 5)” unless otherwise stated. A 5-point scale of strongly agree, agree, neutral, disagree, and strongly disagree was used. Experts were also invited to comment on how the exercises could be changed, if necessary, and suggest new exercises to include in FootyFirst. If experts recommended changes or new exercises, they were asked to provide details (eg, number of repetitions, how to perform the exercise, etc.) and supportive evidence (eg, scientific, anecdotal, references, experience, etc.). Experts were advised that suggested changes or additions could be included in a revised program developed for round 2 and that comments would be shared anonymously during subsequent rounds.

Any exercises where consensus had been reached in round 1 were removed from further consultation, and experts were not required to rate or comment on them again. However, to ensure the experts could see the full program when considering each individual exercise, the full revised FootyFirst program was sent to the experts in rounds 2 and 3.

Round 2

Exercises or progressions where consensus was not reached in round 1 were revised in response to the comments received, and included in round 2. A summary of the round 1 agreement ratings and comments, and the revised FootyFirst program were sent to all experts (whether or not they had participated in round 1). The experts again rated their agreement (using the same 5-point scale as for round 1) with including the revised exercises (for which consensus was not reached in round 1) in FootyFirst. They could also comment on how the revised exercises should be changed, if necessary and were again informed that a summary of ratings and comments could be circulated during round 3.

Round 3

The processes undertaken after round 1 were repeated after round 2. Therefore, in round 3, experts were only asked to rate the exercises or progressions where consensus had not been reached in previous rounds. In round 3, experts were provided with a summary of the round 2 ratings and comments and the revised FootyFirst program. As the exercises for which consensus had been reached in rounds 1 and 2 were removed from the Delphi process in round 3, the activity that experts participated in during round 3 was fundamentally different to those they had participated in during the previous rounds.

In round 3, experts were only asked to rank the order of progression of the exercises targeting 1 body region, from easiest [least load on the targeted muscles (level 1)] to hardest [greatest load on the targeted muscles (level 5)] for a community-AF player to perform with correct technique. Given that the aim of these specific exercises was to develop muscular strength, the experts were not asked to consider the technical difficulty of completing the exercises. The “randomize answer choices for each respondent” function of the online survey program was activated for this question, so that responses were not systematically influenced by the order in which the progression options seemed in the survey. The experts were encouraged to either perform the exercises themselves or to seek feedback from someone with a similar background and fitness to a community-AF player who had performed the exercises, before ranking the exercises. Figure 1 provides an overview of Delphi process used in this study.

Overview of the Delphi process used in this study.

Data Analysis

Data from all Delphi rounds were downloaded from the online questionnaire software into an Excel database. Descriptive statistics were generated for quantitative data. Qualitative feedback (ie, comments on existing exercises and suggested new exercises) was reviewed and discussed by research team members, and any changes that were considered appropriate were made to the program. Particular attention was paid to comments from experts who disagreed or strongly disagreed with including an exercise or progression of exercises in FootyFirst. In round 3, the final progression of the 1 set of exercises retained in this round was established by ordering the exercises (ie, level 1 through to 5) according to the ranking average. Where the ranking average was identical, exercises were placed at the level at which the largest proportion of experts indicated it should be located.


Fifty-five experts participated in at least 1 Delphi round. Forty-six of a possible 168 experts participated in round 1 (response rate = 27%), whereas 21 (13%) and 24 (14%) participated in rounds 2 and 3, respectively. Not all experts in each round completed all survey questions so the total number of respondents (n) is provided where applicable below.


Fifty-one of the 55 experts (93%) were men. The mean (range) age of experts was 44.4 (23-74) years. The experts' highest formal qualification and related disciplines, the proportion employed by elite-level AFL clubs at the time of the survey, the career-total years of employment at an AFL club, and membership of AFL-related discipline groups are presented in Table 2. The 10 experts not employed by an AFL club all reported holding positions that demonstrated expertise related to lower-limb injury prevention in AF (eg, amateur team medical officer, former AFLMOA member, sports physician, and human performance scientist).

Characteristic of Experts Who Participated in At Least 1 Delphi Round (n = 55)

Delphi Round 1

Consensus was achieved for the run through and the dynamic stretches components of the warm-up, the hamstring strength exercise, the hip/core strength progression, and the balance, landing, and changing direction exercises in round 1 (Table 3). A summary of round 1 comments and feedback, and the changes made to FootyFirst in response, are summarized in Supplemental Digital Content 1 (see Table,, which provides a summary of comments and feedback.

Expert Agreement Ratings From Rounds 1 and 2 of the Delphi for Inclusion of Exercises and Their Progressions in the FootyFirst Program

Delphi Round 2

Consensus was reached for the groin strength and hip strength exercises in round 2. However, consensus was not reached for the progression of the groin strength exercises (Table 3).

Delphi Round 3

Only the 5 exercises that made up the FootyFirst groin strength exercises were included in round 3. Table 4 presents the experts' relative rankings of the groin strength exercises when asked to rank them from easiest to hardest for a community-AF player to perform with correct technique.

Proportion (95% CI) of Experts Ranking the 5 Groin Strength Exercises From Easiest* to Hardest† for a Community AF Player to Perform With Correct Technique (n = 19)

The final draft FootyFirst program informed by the outcomes of the Delphi consensus process outlined above is summarized below (Table 5).

FootyFirst Exercise Protocol After Round 3 of the Delphi Process


To the best of our knowledge, this is the first study to incorporate consensus from a panel of external experts to finalize the content of a lower-limb injury prevention exercise training program in any sport. The importance and value of using a formal consensus development process to progress FootyFirst from an evidence-informed, researcher-developed exercise program to one that is also expert endorsed and context relevant should not be under estimated. Gaining expert consensus on the content of the exercise training program has had 3 potential benefits. First, it helped to refine and shape FootyFirst to ensure it included exercises and progressions that the experts who participated in the study considered appropriate. Second, it was a significant step toward structuring FootyFirst to meet the specific contextual requirements of AF. Finally, it provided an opportunity to leverage uptake of FootyFirst by end users (ie, community-AF coaches) based on industry expert and opinion leader endorsement as emphasized in Diffusion of Innovations theory.20

This study highlights the differences in expert opinion about exercise-based interventions to prevent different types of sport-related lower-limb injuries. The high level of agreement achieved in round 1 to include some of the proposed exercises (ie, the warm-up and the hamstring, and jumping, landing, and changing direction exercises) contrasts with the lack of consensus achieved on the inclusion of other proposed exercises (ie, the groin and hip/core exercises).

For the warm-up, it is possible that agreement and acceptance among the experts in this study was high because the proposed exercises were similar to those used in elite-AF and some included the use of a football. Given that hamstring injuries are the most frequent and prevalent (missed matches through injury per club per season) injury for AFL (elite) clubs,21 it is reasonable to assume that experts who work in elite-AF are familiar with the relevant literature that demonstrates the effectiveness of eccentric hamstring strengthening exercises (hamstring lower/Nordic hamstring) in preventing sport-related hamstring injuries.22–24 This may help explain why the experts in this study agreed in round 1 that the hamstring lower exercise was appropriate to include in FootyFirst.

Since 2000, there has been Australian research focussed on neuromuscular and biomechanical factors in running, sidestepping, and landing from a mark that may be involved in knee and hamstring injuries in AF players.25–34 Furthermore, this research has identified key training programs that can ameliorate the mechanisms of injury.35–38 This knowledge base was informed by studies of lower-limb injuries from other sporting codes.12,13,22,39–43 Therefore, the high level of expert agreement in round 1 for including the proposed jumping, landing, and changing direction exercises in FootyFirst suggests that the experts who participated in this study were aware of this research.

Round 1 results indicate that there was less agreement among the experts on the inclusion of the proposed groin and hip injury prevention exercises in FootyFirst than for the proposed warm-up and other exercises. This result may reflect the lack of consistency in the literature about the diagnosis, pathology, and mechanisms of sport-related groin injuries.44 Also, the current evidence for tendon and groin injury prevention is underpinned by observational research that is not yet sufficient for establishing risk factors for injury or for identifying a specific intervention.9

Challenges and Limitations

One of the challenges in this study, like other Delphi studies,14 was recruiting and retaining experts. This is reflected in response rates of 27%, 13%, and 14% in rounds 1, 2, and 3, respectively, which may indicate selection bias as only those with a serious interest in preventing injuries through exercise training programs chose to participate. Interestingly, only 3 of the 94 invited AFLMOA members (5% of the total number of experts in the study) participated in this study yet 11 experts (20% of total experts) reported holding a sports medicine qualification. In comparison, 29 of 50 invited AFLPA members and 11 of 19 invited AFLSSA members participated in the study. This could be because many more AFLMOA members have part-time or honorary positions with AFL clubs compared with those from the AFLPA or AFLSSA and may be less inclined to participate in time-consuming research projects. Additionally, the topic of interest (lower-limb injury prevention exercise training programs) is more closely aligned with the role of physiotherapists and sports scientists than it is with medical practitioners.

It is also acknowledged that the external validity of the results is affected by the response rates14 and selection bias. As a consequence, a similar study of other representatives of the same expert groups may have produced different results. In addition, the iterative nature of the Delphi process may have resulted in those experts who did not agree with the majority after round 1 dropping out and not participating in the later rounds. This may have led to higher levels of agreement in later rounds based on attrition of those who disagreed in the earlier rounds. Nonetheless, the Delphi technique is a well-respected method for obtaining and processing subjective information compiled by experts,45 and the multidisciplinary sample for this study, although predominately made up of physiotherapists, included experts with a range of educational levels and years of experience working in the AFL.

A key challenge to achieving agreement among those involved in sports medicine and sports science on the content of the FootyFirst lower-limb injury prevention exercise training program is the variation in the type and strength of evidence underpinning the proposed exercises.9 Additionally, the roles that individual experience and personal preferences play in developing exercise protocols, and the fact that many different potential exercises can target the same injury, may also make achieving agreement difficult. Finally, experts involved in elite-AF may have varying degrees of experience with and understanding of community AF, which was the target context for which FootyFirst was developed. This last issue will be addressed by conducting further consultation with community-level AF coaches, strength and conditioning/fitness personnel, and administrators to ensure that FootyFirst is relevant to their specific context and meets their needs.


There was considerable agreement among the experts who participated in this study that the proposed warm-up, hamstring strength exercise, and balance, landing, and changing direction exercises should be included in a lower-limb injury prevention program for community-AF players. However, it required an expert feedback-informed revision of the groin strength and hip strength exercises and their progressions, and 2 further rounds of consultation before consensus was reached that these exercises were appropriate for inclusion in the program. FootyFirst is now an evidence-informed, researcher-developed lower-limb sports injury prevention exercise program that is also endorsed by the experts who participated in this study and is specific to the context of AF.


1. Australian Sports Commission. Participation in exercise, recreation and sport: annual report 2010. Accessed October 17, 2013.
2. Stevenson MR, Hamer P, Finch CF, et al.. Sport, age, and sex specific incidence of sports injuries in Western Australia. Br J Sports Med. 2000;34:188–194.
3. Cassell EP, Finch CF, Stathakis VZ. Epidemiology of medically treated sport and active recreation injuries in the Latrobe Valley, Victoria, Australia. Br J Sports Med. 2003;37:405–409.
4. Flood L, Harrison J. Hospitalised Sports Injury, Australia 2002-03. Flinders University, Adelaide: Australian Institute of Health and Welfare, 2006. Accessed April 3, 2014.
5. Finch CF, Gabbe B, White P, et al.. Priorities for investment in injury prevention in community Australian football. Clin J Sport Med. 2013;23:430–438.
6. Gabbe B, Finch C, Wajswelner H, et al.. Australian football: injury profile at the community level. J Sci Med Sport. 2002;5:149–160.
7. McManus A, Stevenson M, Finch CF, et al.. Incidence and risk factors for injury in non-elite Australian football. J Sci Med Sport. 2004;7:384–391.
8. Braham R, Finch CF, McIntosh A, et al.. Community level Australian football: a profile of injuries. J Sci Med Sport. 2004;7:96–105.
9. Andrew N, Gabbe B, Cook J, et al.. Could targeted exercise programmes prevent lower limb injury in community Australian football? Sports Med. 2013;43:751–763.
10. Fédération Internationale de Football Association. FIFA 11+. Accessed March 20, 2014.
11. Santa Monica Orthapeadic and Sports Medicinec Group. Prevent injuries enhance performance (PEP). Accessed March 20, 2014.
12. Soligard T, Myklebust G, Steffen K, et al.. Comprehensive warm-up programme to prevent injuries in young female footballers: cluster randomised controlled trial. BMJ. 2008;337:a2469.
13. Myklebust G, Engebretsen L, Brækken IH, et al.. Prevention of anterior cruciate ligament injuries in female team handball players: a prospective intervention study over three seasons. Clin J Sport Med. 2003;13:71–78.
14. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32:1008–1015.
15. Donaldson A, Finch CF. Identifying context-specific competencies required by community Australian football sports trainers. Br J Sports Med. 2012;46:759–765.
16. Donaldson A, Borys D, Finch CF. Understanding safety management system applicability in community sport. Saf Sci. 2013;60:95–104.
17. Abernethy L, McNally O, MacAuley D, et al.. Sports medicine and the accident and emergency medicine specialist. Emerg Med J. 2002;19:239–241.
18. Keeney S, Hasson F, McKenna H. Consulting the oracle: ten lessons form using the Delphi technique in nursing research. J Adv Nurs. 2006;53:205–212.
19. Baker J, Lovell K, Harris N. How expert are the experts? An exploration of the concept of “expert” within Delphi panel techniques. Nurse Res. 2006;14:59–70.
20. Rogers E. Diffusion of Innovations. 5th ed. New York, NY: Free Press; 2003.
21. Orchard JW, Seward H, Orchard JJ. Results of 2 decades of injury surveillance and public release of data in the Australian Football League. Am J Sports Med. 2013;41:734–741.
22. Mjølsnes R, Arnason A, Østhagen T, et al.. A 10-week randomized trial comparing eccentric vs. concentric hamstring strength training in well-trained soccer players. Scand J Med Sci Sports. 2004;14:311–317.
23. Petersen J, Thorborg K, Nielsen MB, et al.. Preventive effect of eccentric training on acute hamstring injuries in men's soccer. Am J Sports Med. 2011;39:2296–2303.
24. Thorborg K. Why hamstring eccentrics are hamstring essentials. Br J Sports Med. 2012;46:463–465.
25. Besier TF, Lloyd DG, Ackland TR, et al.. Anticipatory effects on knee joint loading during running and cutting maneuvers. Med Sci Sports Exerc. 2001;33:1176–1181.
26. Besier TF, Lloyd DG, Cochrane JL, et al.. External loading of the knee joint during running and cutting maneuvers. Med Sci Sports Exerc. 2001;33:1168–1175.
27. Cochrane JL, Lloyd DG, Buttfield A, et al.. Characteristics of anterior cruciate ligament injuries in Australian football. J Sci Med Sport. 2007;10:96–104.
28. Dempsey AR, Elliott BC, Munro BJ, et al.. Whole body kinematics and knee moments that occur during an overhead catch and landing task in sport. Clin Biomech (Bristol, Avon). 2012;27:466–474.
29. Dempsey AR, Lloyd DG, Elliott BC, et al.. The effect of technique change on knee loads during sidestep cutting. Med Sci Sports Exerc. 2007;39:1765–1773.
30. Gabbe BJ, Bennell KL, Finch CF, et al.. Predictors of hamstring injury at the elite level of Australian football. Scand J Med Sci Sports. 2006;16:7–13.
31. Gabbe BJ, Finch CF, Bennell KL, et al.. Risk factors for hamstring injuries in community level Australian football. Br J Sports Med. 2005;39:106–110.
32. Besier TF, Lloyd DG, Ackland TR. Muscle activation strategies at the knee during running and cutting maneuvers. Med Sci Sports Exerc. 2003;35:119–127.
33. Lee MJ, Lloyd DG, Lay BS, et al.. Effects of different visual stimuli on postures and knee moments during sidestepping. Med Sci Sports Exerc. 2013;45:1740–1748.
34. Lloyd DG. Rationale for training programs to reduce anterior cruciate ligament injuries in Australian football. J Orthop Sports Phys Ther. 2001;31:645–654.
35. Cochrane JL, Lloyd DG, Besier TF, et al.. Training affects knee kinematics and kinetics in cutting maneuvers in sport. Med Sci Sports Exerc. 2010;42:1535–1544.
36. Dempsey AR, Lloyd DG, Elliott BC, et al.. Changing sidestep cutting technique reduces knee valgus loading. Am J Sports Med. 2009;37:2194–2200.
37. Donnelly CJ, Elliott BC, Doyle TL, et al.. Changes in knee joint biomechanics following balance and technique training and a season of Australian football. Br J Sports Med. 2012;46:917–922.
38. Gabbe BJ, Branson R, Bennell KL. A pilot randomised controlled trial of eccentric exercise to prevent hamstring injuries in community-level Australian football. J Sci Med Sport. 2006;9:103–109.
39. Arnason A, Andersen TE, Holme I, et al.. Prevention of hamstring strains in elite soccer: an intervention study. Scand J Med Sci Sports. 2008;18:40–48.
40. Askling C, Karlsson J, Thorstensson A. Hamstring injury occurrence in elite soccer players after preseason strength training with eccentric overload. Scand J Med Sci Sports. 2003;13:244–250.
41. Hewett TE, Lindenfeld TN, Riccobene JV, et al.. The effect of neuromuscular training on the incidence of knee injury in female athletes. A prospective study. Am J Sports Med. 1999;27:699–706.
42. Hewett TE, Stroupe AL, Nance TA, et al.. Plyometric training in female athletes: decreased impact forces and increased hamstring torques. Am J Sports Med. 1996;24:765–773.
43. Verrall GM, Slavotinek JP, Barnes PG, et al.. Hip joint range of motion restriction precedes athletic chronic groin injury. J Sci Med Sport. 2007;10:463–466.
44. Jansen JA, Mens JM, Backx FJ, et al.. Diagnostics in athletes with long-standing groin pain. Scand J Med Sci Sports. 2008;18:679–690.
45. Landetaa J, Mateya J, Ruíza V, et al.. Results of a Delphi survey in drawing up the input–output tables for Catalonia. Technol Forecast Soc Change. 2008;75:32–56.

sports injury prevention; Delphi; Australian football; exercise training program

Supplemental Digital Content

Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.