In phase II, regression coefficients from the tryout sample was used to estimate logistic scores and estimated probabilities of ED with levels of the predictors found in the external validation data set. These were then used to calculate ROC areas in the validation data. To correct for overoptimism in the regression model fit in phase I, we adjusted the coefficients according to a method by Van Houwelingen and Le Cessie (equation 77) in the external data set (41).
Internal reliability was assessed with Cronbach α coefficient. The significance level was set to 0.05.
All participants from the Elite Sport High Schools were born in 1992, and there were no differences in age, training background, or BMI between those with and without an ED at pretest (phase I) (Table 1). Among the athletes from the external data set (phase II), a higher percentage of athletes with an ED compared with non-ED athletes competed in weight-sensitive sports (Table 1). Moreover, a higher percentage of the athletes from the external data set compared with the Elite Sport High School athletes at pretest (phase I) competed in weight-sensitive sports and were selected for national teams (Table 1). There were no differences in age or BMI between the athletes from the external data set compared with the athletes attending the Elite Sport High Schools.
Developing a new screening questionnaire
Our starting point in developing a new and briefer screening questionnaire was to examine how well our questions previously used (symptoms associated with ED) discriminated between the Elite Sport High School athletes with and without an ED at pretest (Table 2).
In our search for potential predictors among the symptoms associated with ED, we included the three at-risk criteria with the highest sensitivity and specificity that significantly differed between athletes with and without an ED (“trying to lose weight now”, “tried to lose weight before ≥3 times”, and EDI-BD ≥14) in a logistic regression model as the independent variables and clinical ED (yes or no) as the dependent variable. The significant predictors proved to be “trying to lose weight now” [odds ratio (OR) = 4.0; 95% confidence interval (CI), 1.47–11.2; P = 0.007] and “tried to lose weight before ≥3 times” (OR = 3.1; 95% CI, 1.05–8.9; P = 0.041), whereas EDI-BD score ≥14 was borderline significant (OR = 2.8; 95% CI, 0.97–8.3; P = 0.056). Because frequent weight fluctuations have been suggested as an important trigger factor for the development of an ED in athletes (35), a natural next step in our search for the strongest potential predictors was to combine the variables “trying to lose weight now” and/or “tried to lose weight before ≥3 times” into one variable (dieting) (Table 2). With logistic regression analysis, dieting proved to be a strong significant predictor for ED (OR = 17.4; 95% CI, 5.7–53.2; P < 0.001).
Furthermore, because neither the EDI-DT nor the EDI-BD is sport specific or developed for the purpose of screening athletes, we decided to also examine the different items in the subscales independently. In addition, given perfectionism’s perceived role in the etiology of ED (15), we also examined the items from the EDI-P. Based on a review of the current evidence and the collective expertise of the authors, items that were not able to discriminate between athletes with ED and no ED at pretest, and items focusing on concerns most likely not being relevant for athletes, were eliminated (Table 3).
The next step was to determine the key items that may predict possible ED among athletes. Through group discussions, we ended up with nine items: two items from each of the EDI-DT, EDI-BD, and EDI-P subscales from the EDI-2, and the questions “Have you tried to lose weight?”, “If yes, how many times have you tried to lose weight?”, and “Are you trying to lose weight now?” from the symptoms associated with ED at risk criteria. This resulted in the Brief ED in Athletes Questionnaire (BEDA-Q) that we wanted to test further (from version 1 with seven items and version 2 with nine items) (Table 4).
As seen in Table 4, items from the EDI-DT and EDI-BD completed the variable EDI_4 in BEDA-Q version 1, whereas items from the EDI-DT, EDI-BD, and EDI-P completed the variable EDI_6 included in BEDA-Q version 2.
Predictive Ability of Versions 1 and 2 for a Diagnosis of ED
BEDA-Q versions 1 and 2 showed good ability in distinguishing between the female elite athletes with and without an ED at phase I with ROC areas of 0.83 (95% CI, 0.74–0.92) and 0.86 (95% CI, 0.78–0.93), respectively (Table 5). Version 2 improved with approximately 0.03 area units compared with version 1.
For version 1, we calculated the optimal cutoff point by using the probability score from the variables dieting and subscale 1, which maximized the product of sensitivity and specificity. The cutoff was 0.26 with a sensitivity of 85.7% (95% CI, 80.6–90.8) and specificity of 78.8% (95% CI, 73.0–84.7). This gave a positive likelihood ratio of 4.0 and a negative likelihood ratio of 0.2.
The optimal cutoff point for version II was 0.27 with a sensitivity of 82.1% (95% CI, 76.6–87.6) and specificity of 84.6% (95% CI, 79.4–89.8). For version 2, this gave a positive likelihood ratio of 5.3 and a negative likelihood ratio of 0.2.
Finally, we constructed individual predictive scores using the coefficients from the logistic models for versions 1 and 2 to classify athletes at risk for an ED if the score was greater than the optimal cutoff value and not at risk otherwise. The estimated probabilities of ED for versions 1 and 2 were calculated by the following formulas:
Validating BEDA-Q version 1 versus the external data set
Because no previous studies examining the prevalence of ED among adolescent female elite athletes have included items from the EDI-P subscale, we were only able to carry out an external validation for version 1. We used the regression coefficients from the derivation data set adjusted for overoptimism to estimate logistic scores and estimated probabilities of ED in the validation data by using the adjusted formula above for version 1. The estimated probabilities were then used in the ROC analysis calculation for ED in version 1. The accuracy of version 1 was measured by the area under the ROC curve of 0.77 (95% CI, 0.63–0.91).
The ability of BEDA-Q versions 1 and 2 to predict new cases of ED (posttest assessments)
In this phase, we wanted to test the ability of BEDA-Q versions 1 and 2 to predict new cases of ED among the 53 athletes (100%) attending the posttest classified with no ED diagnosis at pretest attending the control schools. Seven of the 53 athletes (13.2%) had developed an ED during these 2 yr and were classified as new cases of ED at posttest. As shown in Table 5, version 2 showed slightly better diagnostic accuracy than version 1 with an area under the ROC curve of 0.73 (95% CI, 0.52–0.93) compared with 0.70 (95% CI, 0.48–0.92), respectively.
The main finding in this study was the ability of BEDA-Q versions 1 and 2 to distinguish between adolescent female elite athletes with and without an ED. Even though both versions appear well suited for screening purposes in this population with ROC areas above 80%, it is worth noticing that by adding the two items measuring the socially prescribed perfectionism from the EDI-P, the discriminative accuracy increased with approximately 0.03 area units for version 2 compared with version 1. It is difficult to interpret absolute differences in ROC area, but an improvement above 0.02 area units (more than 4%–5%) is regarded to be clinically important (22). Thus, version 2 consisting of nine items seems to be an even better suited version than version 1 in distinguishing between adolescent female elite athletes with and without an ED. However, both versions are inexpensive, are easy to understand, and showed valid results.
The relation between ED and perfectionism has been well established, and it may influence in an indirect manner (24). Among athletes, it is suggested that perfectionism as a personality trait combined with environmental and other factors may increase the risk of developing an ED (15). Both self-oriented perfectionism and socially prescribed perfectionism have been independently and positively related to ED among nonathletes (30), and it has been suggested that women high on socially prescribed and self-oriented perfectionism are especially vulnerable (30). However, few studies have explored this proposed relation in depth (15), and traditionally, EDI-DT and EDI-BD have been included in relation to self-developed questions when screening for symptoms associated with ED among elite athletes (28,37,39).
The increased accuracy found when including the socially prescribed perfectionism items measuring parent’s expectations and avoiding disappointing parents and teachers are in line with previous research. Stoeber and Otto (32) reviewed the consequences of perfectionism among athletes and found that dimensions assessing evaluative concerns (e.g., concern over mistakes, perceived parental and coach pressure) are associated with negative consequences, whereas dimensions assessing a commitment to exceptionally high standards are associated with positive consequences. Furthermore, a recent longitudinal study following a large sample of adolescents age 15–19 yr over a period of 7–9 months showed that perceived parental expectations predicted longitudinal increases in socially prescribed perfectionism. In contrast, no such effect was found for self-oriented perfectionism or for parental criticism (9).
On the basis of the importance athletes tend to ascribe to coaches (29), and that the athletes in our study are at the early stages of their athletic career, it seems liable to suggest that those perceiving parental expectations may transfer these perceptions that also coaches have high expectations of them. If this is the case, these athletes will believe that other people’s (in this case, coaches) acceptance will be contingent upon meeting these expectations being key characteristics of socially prescribed perfectionism (9).
Furthermore, in our study, most of the first year students attending the Elite Sport High Schools with an ED were diagnosed with EDNOS (n = 20, 71.4%) (28). Additionally, Hewitt et al. (20) found that social dimensions of perfectionism were broadly related to ED as well as self-esteem, whereas self-oriented perfectionism was related only to anorexic tendencies among female university students. In addition to the association between perfectionism and ED, a high level of socially prescribed perfectionism has shown strong and consistent positive correlations with negative affect, anxiety, suicidal ideation (10), and athlete burnout among adolescent elite athletes (21). Thus, the socially prescribed perfectionisms association to negative psychological outcomes (13) and its particular importance during adolescence (14) may explain version 2’s higher discriminative accuracy than version 1 in distinguishing athletes with and without an ED. However, due to the cross-sectional nature of this part of the study, it is not possible to interpret causality. Whether the athletes with an ED diagnosis compared with the athletes without an ED diagnosis were more socially prescribed perfectionistic before they developed an ED, or whether this is a consequence or antecedent to the athletic participation itself, needs further investigation.
Concerning the external validity, an important next step in our study was to determine the BEDA-Q efficacy in discriminating between age-matched female elite athletes with and without an ED in the external data set. Unfortunately, there are no previous studies available including items from the EDI-P with a two-tiered approach (questionnaire screening and clinical interview) among female elite athletes. Therefore, we were only able to measure the external validity by using the estimated probabilities from the derivation data set in the ROC analysis calculations for ED in version 1. Version 1 showed high discriminating accuracy with an area under the ROC curve of 77%. Even though we were not able to test the external validity of version 2, there is reason to believe that it would have shown an even better discriminating ability than version 1 as shown in the derivation data set (phase I). This is further supported in phase III where the ability to predict new cases of ED (posttest assessments) increased with 0.03 area units by using version 2 instead of version 1.
The most effective way to reduce the incidence of ED among athletes is to prevent them from occurring in the first place. Thus, a valid screening instrument with the ability to predict new cases of ED among young athletes may be an important step in preventing ED, because treatment and recovery may not occur without identification (38). In the third phase of this study, we therefore wanted to determine the BEDA-Q’s ability to predict new cases of ED. Because this is the first study among adolescent elite athletes with a prospective design aiming to determine BEDA-Q’s ability to predict new cases of ED by posttest assessments, comparisons with other similar studies are not possible. In accordance with what we found in phase I, version 2 revealed a higher diagnostic accuracy than version 1. However, the number of athletes with an ED diagnosis at posttest was low (n = 7, 13.2%); thus, the CI is wide ranging between excellent and poor distinguishing ability. Our finding that version 2 also showed better diagnostic accuracy than version 1 is an important contribution to our understanding of the role social (parental) expectations play in the development of socially prescribed perfectionism as well as ED. Because the only difference between versions 1 and 2 is the two items measuring the athlete’s perception that their parents expect them to be perfect, it implies that these items are probably essential to include in screening questionnaires for adolescent elite athletes.
An important question to answer when developing a new screening questionnaire is how accurate the test should be to be clinically useful. This is related to the prevalence of the disease in the subjects being tested, and in our case, the prevalence of ED among adolescent female elite athletes. For screening tests, negative results are not desirable, whereas a moderate number of false-positive results are usually accepted. However, when it comes to diseases with high morbidity and mortality, the sensitivity of the test (detection of ED) is more important than the specificity (detection of healthy cases). In our study, we calculated the optimal cutoff value for BEDA-Q at which optimal balance between sensitivity and specificity is achieved. In phase I, BEDA-Q showed a high ability in both detecting athletes with an ED as well as athletes without an ED with sensitivity and specificity of 82.1% and 84.6%. In addition, the sensitivity and specificity of the symptoms associated with an ED previously used to classify the athletes in this article “at risk” and not “at risk” for an ED in our previous study (28) were 85.7% and 53.8%. This gave a positive likelihood ratio of 1.9 and a negative likelihood ratio of 0.3. Thus, an athlete with a positive score on the symptoms associated with an ED actually having an ED increases approximately 1.9 times, whereas the likelihood of having an ED with a score at or above the BEDA-Q cutoff is more than fivefold.
The main strengths of this study are (a) recruitment of a large, nationally representative sample of female adolescent elite athletes representing a wide range of sport events, (b) that the clinical interview EDE considered to be the “gold standard” for diagnosing ED was used, and (c) that the tests’ external predictive validity was measured to distinguish adolescent female elite athletes with and without an ED. This study does, however, also have some limitations that should be considered when interpreting the results, such as (a) the athletic groups included consist of adolescent female elite athletes exclusively and we do not know if the results can be generalized to male athletes or other age groups, (b) we were not able to carry out an external validation of BEDA-Q version 2, and (c) due to few new cases of athletes diagnosed with an ED, the test’s prognostic ability need to be tested in a larger sample. Finally, referring to the purpose of this study (to design an accurate yet less comprehensive screening questionnaire with the ability to discriminate between adolescent female elite athletes with an ED from those without an ED), we carefully evaluated BEDA-Q against the 10 questions suggested by Greenhalgh (19) to evaluate the validation of different diagnostic and screening tests. The BEDA-Q fulfilled a total of nine out of these 10 questions. The only question we were not able to fulfill was the following: “Was the test shown to be reproducible?” Because the aim of our study was to develop and validate BEDA-Q as a possible new screening questionnaire among adolescent elite athletes, its reproducibility has not yet been assessed between observers. However, BEDA-Q had high internal consistency, with a Cronbach α of 0.81.
Implications and Applicability
It is well known that even though coaches are in a prime position to monitor their athletes’ behavior and reactions, it may be challenging to determine whether the athletes’ DE and dieting behaviors are transient, safely managed behaviors associated with the specific demands of the sport, or if the symptoms are more stable and signify a clinical ED. To facilitate early identification and treatment, we present in this study a brief and easy administrated screening questionnaire appearing well suited as a first step to identify adolescent elite athletes that may have an ED and are in need of further medical and psychological examination. For professionals working with athletes, BEDA-Q may be an important contribution in making it easier to identify those athletes in need for further examination.
In our study, most of the athletes diagnosed with an ED fulfilled the criteria for EDNOS, which is the most common ED encountered among athletes (28,39). This indicates that the athletes included are representative of the athletic population in which BEDA-Q is meant being used. However, it should be noted that the diagnostic criteria used in this study is based on the DSM-IV (1). The recent revision of the DSM-V has changed the distribution because it entails a lowering of thresholds for AN and BN, making binge eating (BED) a formal ED diagnosis and renaming EDNOS “feeding and eating conditions not elsewhere classified” along with some specifications for subtypes (4). Because we avoided evaluating our screening questionnaire in a sample in which the proportion of cases is artificially high, and the items included from the EDI in our screening questionnaire ask for psychological concepts rather than ED symptoms, we expect BEDA-Q to work equally good in distinguishing athletes with and without an ED in the DSM-V as in the DSM-IV.
BEDA-Q has revealed very promising psychometric and predictive features when it comes to distinguishing adolescent elite athletes with and without ED. However, more studies are needed including larger samples, athletes with different competitive levels and both gender represented, to further confirm these results and also to test the predictive ability of BEDA-Q.
This study shows that BEDA-Q containing nine items is a well-suited screening questionnaire to distinguish between adolescent female elite athletes with and without an ED. This new screening questionnaire (BEDA-Q) may also be a useful instrument for predicting new cases of ED. Socially prescribed perfectionism among athletes and its relation to ED should be further investigated.
The research reported in this article was supported by Oslo Sports Trauma Research Center and through a grant from the Norwegian Olympic Sports Center (Olympiatoppen). The Oslo Sports Trauma Research Center has been established through generous grants from the Eastern Norway Regional Health Authority, the International Olympic Committee, The Royal Norwegian Ministry of Culture, Norsk Tipping AS, and the Norwegian Olympic Committee and Confederation of Sport. There are no potential conflicts of interest.
The research reported in this article was supported by Oslo Sports Trauma Research Center and through a grant from the Norwegian Olympic Sports Center (Olympiatoppen). The Oslo Sports Trauma Research Center has been established through generous grants from the Eastern Norway Regional Health Authority, the International Olympic Committee, The Royal Norwegian Ministry of Culture, Norsk Tipping AS, and the Norwegian Olympic Committee and Confederation of Sport.
There are no potential conflicts of interest.
The results of the study do not constitute endorsement by the American College of Sports Medicine.
1. American Psychiatric Association. Eating Disorders. Diagnostic and Statistical Manual of Mental Disorders: DSM-IV
. Washington (DC): American Psychiatric Association; 1994. p. 539–50.
2. Beals KA. Disordered eating among athletes: a comprehensive guide for health professionals
. Champaign (IL): Human Kinetics; 2004. p. 255.
3. Beals KA, Manore MM. Disorders of the female athlete triad among collegiate athletes. Int J Sport Nutr Exerc Metab
. 2002; 12 (3): 281–93.
4. Birgegard A, Norring C, Clinton D. DSM-IV versus DSM-5: implementation of proposed DSM-5 criteria in a large naturalistic database. Int J Eat Disord
. 2012; 45 (3): 353–61.
5. Bonci CM, Bonci LJ, Granger LR, et al. National athletic trainers’ association position statement: preventing, detecting, and managing disordered eating
in athletes. J Athl Train
. 2008; 43 (1): 80–108.
6. Byrne S, McLean N. Eating disorders in athletes: a review of the literature. J Sci Med Sport
. 2001; 4 (2): 145–59.
7. Byrne S, McLean N. Elite athletes: effects of the pressure to be thin. J Sci Med Sport
. 2002; 5 (2): 80–94.
8. Cole TJ, Flegal KM, Nicholls D, Jackson AA. Body mass index cut offs to define thinness in children and adolescents: international survey. Br Med J
. 2007; 335 (7612): 194–7.
9. Damian LE, Stoeber J, Negru O, B-âban A. On the development of perfectionism in adolescence: perceived parental expectations predict longitudinal increases in socially prescribed perfectionism. Pers Individ Dif
. 2013; 55 (6): 688–93.
10. Enns MW, Cox BJ. The nature and assessment of perfectionism: a critical analysis. In: Flett GL, Hewitt PL, editors. Perfectionism
. Washington (DC): American Psychological Association; 2002. p. 33–62.
11. Fairburn CG, Bohn K. Eating disorder NOS (EDNOS): an example of the troublesome “not otherwise specified” (NOS) category in DSM-IV. Behav Res Ther
. 2005; 43 (6): 691–701.
12. Fairburn CG, Cooper Z, O’Connor ME. Eating disorder examination (Edition 16.0D). In: Fairburn CG, editors. Cognitive Behavior Therapy and Eating Disorders
. New York: The Guilford Press; 2008. p. 265–308.
13. Flett GL, Hewitt PL. The perils of perfectionism in sports
and exercise. Curr Dir Psychol Sci
. 2005; 14 (1): 14–8.
14. Flett GL, Hewitt PL, Oliver JM, Macdonald S. Perfectionism in children and their parents: a developmental analysis. In: Flett GL, Hewitt PL, editors. Perfectionism: Theory, Research, and Treatment
. Washington (DC): American Psychological Association; 2002. p. 88–132.
15. Forsberg S, Lock J. The relationship between perfectionism, eating disorders and athletes: a review. Minerva Pediatr
. 2006; 58 (6): 525–36.
16. Friborg O, Clausen L, Rosenvinge JH. A five-item screening
version of the Eating Disorder Inventory (EDI-3). Compr Psychiatry
. 2013; 54 (8): 1222–8.
17. Garner DM. Eating Disorder Inventory-2: Professional Manual
. Odessa (FL): Psychological Assessment Resources; 1991. IV, 70 p.
18. Gould D, Dieffenbach K, Moffett A. Psychological characteristics and their development in Olympic champions. J Appl Sport Psychol
. 2002; 14 (3): 172–204.
19. Greenhalgh T. How to read a paper—papers that report diagnostic or screening
tests. Br Med J
. 1997; 315 (7107): 540–3.
20. Hewitt PL, Flett GL, Ediger E. Perfectionism traits and perfectionistic self-presentation in eating disorder attitudes, characteristics, and symptoms. Int J Eat Disord
. 1995; 18: 317–26.
21. Hill AP, Hall HK, Appleton PR. Perfectionism and athlete burnout in junior elite athletes: the mediating role of coping tendencies. Anxiety Stress Coping
. 2010; 23 (4): 415–30.
22. Holme I, Pedersen TR, Boman K, et al. A risk score for predicting mortality in patients with asymptomatic mild to moderate aortic stenosis. Heart
. 2012; 98 (5): 377–83.
23. Hopkinson R, Lock J. Athletics, perfectionism, and disordered eating
. Eat Weight Disord
. 2004; (9): 99–106.
24. Keel PK, Forney KJ. Psychosocial risk factors for eating disorders. Int J Eat Disord
. 2013; 46 (5): 433–9.
25. Lampard AM, Byrne SM, McLean N, Fursland A. The Eating Disorder Inventory-2 Perfectionism scale: factor structure and associations with dietary restraint and weight and shape concern in eating disorders. Eat Behav
. 2012; 13 (1): 49–53.
26. Martinsen M, Bahr R, Børresen R, Holme I, Pensgaard AM, Sundgot-Borgen J. Preventing eating disorders among young athletes: a randomized controlled trial. Med Sci Sports Exerc
. 2014; 46 (3): 435–47.
27. Martinsen M, Bratland-Sanda S, Eriksson AK, Sundgot-Borgen J. Dieting to win or to be thin? A study of dieting and disordered eating
among adolescent elite athletes and non-athlete controls. Br J Sports Med
. 2010; 44 (1): 70–6.
28. Martinsen M, Sundgot-Borgen J. Higher prevalence of eating disorders among adolescent elite athletes than controls. Med Sci Sports Exerc
. 2013; 45 (6): 1188–97.
29. Pensgaard AM, Roberts GC. Elite athletes’ experiences of the motivational climate: the coach matters. Scand J Med Sci Sports
. 2002; 12 (1): 54–9.
30. Sherry SB, Hewitt PL, Besser A, Mcgee BJ, Flett GL. Self-oriented and socially prescribed perfectionism in the Eating Disorder Inventory Perfectionism Subscale. Int J Eat Disord
. 2004; 35 (1): 69–79.
31. Stice E. Risk and maintenance factors for eating pathology: a meta-analytic review. Psychol Bull
. 2002; 128 (5): 825–48.
32. Stoeber J, Otto K. Positive conceptions of perfectionism: approaches, evidence, challenges. Pers Soc Psychol Rev
. 2006; 10 (4): 295–319.
33. Sundgot-Borgen J, Larsen S. Pathogenic weight-control methods and self-reported eating disorders in female elite athletes and controls. Scand J Med Sci Sports
. 1993; 3: 150–5.
34. Sundgot-Borgen J. Prevalence of eating disorders in elite female athletes. Int J Sport Nutr
. 1993; 3 (1): 29–40.
35. Sundgot-Borgen J. Risk and trigger factors for the development of eating disorders in female elite athletes. Med Sci Sports Exerc
. 1994; 26 (4): 414–9.
36. Sundgot-Borgen J, Meyer NL, Lohman TG, Ackland TR, Maughan RJ, Muller W. How to minimize the health risks to athletes who compete in weight-sensitive sports
. Review and position statement on behalf of the Ad Hoc Research Working Group on Body Composition Health and Performance, under the auspices of the I.O.C. Medical Commission. Br J Sports Med
. 2013; 47: 1012–22.
37. Sundgot-Borgen J, Torstveit MK. Prevalence of eating disorders in elite athletes is higher than in the general population. Clin J Sport Med
. 2004; 14 (1): 25–32.
38. Thompson RA, Trattner Sherman R. Eating Disorders in Sport
. New York: Routledge; 2010. p. 284.
39. Torstveit MK, Rosenvinge JH, Sundgot-Borgen J. Prevalence of eating disorders and the predictive power of risk models in female elite athletes: a controlled study. Scand J Med Sci Sports
. 2008; 18 (1): 108–18.
40. Torstveit MK, Sundgot-Borgen J. The female athlete triad exists in both elite athletes and controls. Med Sci Sports Exerc
. 2005; 37 (9): 1449–59.
41. Van Houwelingen JC, Le Cessie S. Predictive value of statistical models. Stat Med
. 1990; 9 (11): 1303–25.
Keywords:© 2014 American College of Sports Medicine
SCREENING; VALIDATION; INSTRUMENT; SPORTS; DISORDERED EATING