Reibnegger, Gilbert DSc; Caluba, Hans-Christian; Ithaler, Daniel; Manhal, Simone; Neges, Heide Maria; Smolle, Josef MD
In academic year 2002–2003, medical education in Austria changed in a fundamental way. The traditional, discipline-oriented study program was transformed into a modern, theme-based, diploma-granting curriculum with a timely, module-track structure. Although all three public medical universities in Austria (Medical University of Vienna, Innsbruck Medical University, and Medical University of Graz) adopted this reform in general, each university was free in establishing the details of its curriculum according to its specific strengths and preferences.
The Medical University of Graz curriculum
The reformed curriculum at the Medical University of Graz1 integrates preclinical and clinical topics from the beginning. Early patient contact strongly enhances training in physical examination skills as well as social and communication skills. In addition, better education in scientific research matters and a newly designed “clinical year” are the hallmarks of the new program. The curriculum is designed to be completed in six years.
The initial two study semesters, the “first part of study,” are dominated by the basics of natural sciences in a medical context. The “second part of study,” years 2 through 5, is devoted to the fundamentals of medical knowledge, including normal as well as pathological function and morphology and the various medical and clinical disciplines. The first and second parts of study are organized in theme-centered modules lasting five weeks each. The modules are accompanied by vertical “tracks.” In tracks, specific knowledge and skills are taught during consecutive study years. Students choose 5 out of the required 30 modules from a broad offering of elective modules; 25 modules are obligatory for all students.
In year 6, students participate in the daily clinical routine at different training sites and are constantly guided and supervised by expert clinical teachers. Additionally, during the course of year 6, students also spend five weeks in a general practitioner's office.
Medical school admissions in Austria
In Austria, open admission to university studies has been the rule: Everyone successfully finishing secondary school education is generally entitled to be admitted to whatever university study she or he wants. In medicine, open admission led to particularly unsatisfactory consequences. For example, at the Medical University of Graz, the average number of new medical students varied between 600 and 800 per year, substantially exceeding capacities in terms of staff as well as infrastructure. Thus, study conditions were poor. Frustrated students and faculty made do with little or no small-group lecturing, a predominance of mass lectures, and little bedside teaching, among other limitations. On average, students exceeded the scheduled study time of six years by 50% or more, and approximately half of the students dropped out before reaching graduation.
Austrian medical universities also admitted students from outside Austria. Historically, students from other countries—including member states of the European Union (EU)—were admitted to an Austrian university only after they proved they had also been admitted to the same course of study in their country of origin. According to European law, however, citizens from all EU member states must be treated in the same way as Austrians when applying to Austrian universities. In July 2005, the European Court ruled that Austria's policy of foreign student admission to university studies violated European law.2 This decision was particularly important for medical universities because of circumstances in Austria's neighboring country, Germany, which shares the same language as Austria. In Germany, only 8,000 to 10,000 of the approximately 30,000 applicants for the study of medicine are admitted each year. Therefore, after the court's decision, it was feared that the three Austrian medical universities would be overwhelmed by German students. To avoid this, Austrian law was changed immediately: While admission to most university study programs remained open for all applicants having completed secondary education, admission tests were introduced to regulate access for selected studies. Among the regulated studies were the diploma programs in human medicine and dentistry. Additionally, the European Commission issued a five-year moratorium in 2007,3 entitling Austria to regulate quotas of students until 2012 to ensure that the majority of openings are reserved for Austrian citizens. Seventy-five percent of openings are reserved for applicants who completed their secondary education at an Austrian school, 20% for citizens from other EU states, and 5% for applicants of other nationalities.
Medical University of Graz admissions
In 2005, the Medical University of Graz faced an unfortunate state of affairs. Because of the open admission policy of previous years, there was an inordinate number of students enrolled in the diploma of human medicine program. Further, the new curriculum implemented in 2002–2003 required significantly more resources than the previous program. Under these circumstances, students who had successfully completed the first part of study could not immediately proceed with the second part because of a lack of resources.
To resolve this situation, the university used the new legal situation to manage the numbers of new students entering the university very efficiently. Thus, in academic year 2005–2006, only 107 new students were admitted. In the two following years, the numbers were raised incrementally (154 in 2006–2007, and 282 in 2007–2008). By this measure, we successfully eliminated the backlog of students waiting to continue their studies. Since 2008–2009, 340 to 350 students have been admitted per year, representing the upper limit of capacity. This upper limit was consensually defined with the Federal Ministry of Science and Research on the basis of previous experience.
Two different procedures were applied in our efforts to reform the admission process. First, in academic year 2005–2006, all applicants (more than 1,000) were preliminarily accepted for an initial semester, which entailed exclusively distance learning via the Internet. The contents of the three modules of the first study semester were transformed into electronic documents and were offered to students online by means of the Virtual Medical Campus Graz. This is a comprehensive, Web-based learning platform which had been developed previously at the Medical University of Graz4–6 to support teaching and learning. In January 2006, all preliminarily accepted students had to pass a two-day selection procedure. On day 1, there was a written assessment in multiple-choice (MC) format based on the students' knowledge of the three modules. On day 2, the students took an additional MC test further assessing their knowledge of biology, chemistry, physics, and mathematics on the secondary school level. The available admission openings were awarded to the 107 applicants ranking highest after both assessments. These applicants then were fully admitted to further study. All other applicants were excluded from continuing their study.
The second process was implemented for academic year 2006–2007, and it continues today. The Medical University of Graz employs a selection procedure based on an applicant's performance on a required MC test prior to admission. This test was built on the basis of the test used on day 2 of the previous admission test. It is based mainly on secondary-school-level knowledge of biology, chemistry, physics, and mathematics and further includes assessment of the applicant's comprehension of scientific texts. A major rationale for using an admission test focusing mainly on the natural sciences was the long-standing observation that, because of strong heterogeneities in Austrian secondary school education, many medical students faced massive difficulties—and hence, the largest risk to fail and to drop out of study—during the initial study semesters, which are dominated by these scientific disciplines.
Experienced university faculty produce the test items. The admission test takes place in July each year during the holiday season of schools and universities. Those applicants who rank best on the admission test are admitted to study. Presently, the admission test is used only at the Medical University of Graz, but there are similar admission procedures at some German medical faculties (e.g., University Medical Center Hamburg–Eppendorf), and we are considering cooperating more closely with these faculties in the future.
Studying the effects
In summary, starting with academic year 2005–2006, a fundamental change in Austria's admission practice for medical studies caused leaders at the Medical University of Graz to implement sweeping reforms to their own admissions practices. Not only was the threat of becoming overwhelmed by German students removed, the university was for the first time able to adjust the number of fresh medical students according to the capacities available. Two major research hypotheses—and indeed hopes—accompanied the introduction of selective admission procedures: We expected that students' overlong study times (approximately nine years instead of six years as scheduled) as well as the absurdly high study dropout rates (50% or more) would be efficiently reduced.
We addressed the first of these research questions, namely, the effect of the change in admission practice on study progress rates, in a previous analysis.7 In the present investigation, we investigate the second important question mentioned above: Is there a measurable effect on dropout rate of the change in admission practice from open admission to active selection of students? How large is the putative effect? Do demographic variables such as students' nationality, age, and sex significantly modulate the putative effect?
We included in the study all new students routinely enrolled in the new diploma human medicine program during the academic years 2002–2003 to 2008–2009. We excluded from the investigation students being admitted by any other route (e.g., students with prior credits from medical studies at the Medical University of Graz or elsewhere).
In total, we included 2,860 students for statistical analyses. Of these, 1,971 (68.9%) were openly admitted during academic years 2002–2003 to 2004–2005; 889 (31.1%) were admitted after passing an admission procedure during years 2005–2006 to 2008–2009.
Data on study progress were accumulated from academic year 2002–2003 until the end of the winter semester in academic year 2009–2010 (February 28, 2010). Thus, the observation period varies among cohorts from the investigated academic years. Whereas students who were enrolled in 2002 and 2003 were observed for more than six years and thus were able to reach graduation during the observation time, the observation period for students who were enrolled in 2004 and later was shorter than the scheduled six years of the curriculum.
The study included 1,230 men (43.0%) and 1,630 (57.0%) women. Age range was from 17.51 to 50.03 years (median: 19.69 years; first quartile: 18.92 years; third quartile: 20.89 years). As in our previous investigation,7 for subsequent analysis we arbitrarily dichotomized the variable “age at study entry” at the third quartile of 20.89 years. There was no other motivation for the dichotomization just at this age other than to compare younger and older participants; because the first, second, and third quartile are very close, the third was taken to ensure a reasonable number of participants in the “older” group. Finally, 2,481 of the students (86.7%) were Austrians, 226 (7.9%) were Germans, and 153 (5.4%) came from other nations.
We gathered the deidentified data from information that is routinely collected about medical students' admission, dropout, and graduation dates and examination history, as required by the Austrian Federal Ministry of Science and Research. Because the data were anonymous and no data beyond those required by law were collected for this study, the Medical University of Graz's ethical approval committee did not require approval for this study.
Phenomena such as students prematurely dropping out of a program are intrinsically time-dependent: Besides the question of whether or not a student drops out, it also matters when in the course of study this event occurs. Proper analysis of dropout, therefore, must include the time elapsing between a defined starting event (in our analysis, this is the date of enrollment) and the terminating event under consideration (the date of dropout) as a central variable. Application of ordinary statistical methods, such as analyses of variance or regression techniques, frequently are not suitable in investigations of this type. First, study progress of participants may vary considerably, and one might be interested in drawing sound conclusions without waiting until all participants have either dropped out or reached graduation. Under reasonable circumstances, only a fraction of participants will experience the terminating event “dropout” within a given observation time, and—at least in principle—other participants may get lost from the observation for reasons other than dropout (e.g., graduation). This latter phenomenon is called censoring. Participants experiencing the defined termination event during the observation period carry full information for statistical analysis (“they have experienced the terminating event after a well-defined time interval”). Participants who do not drop out of study during the observation period nevertheless contribute important information, at least for the time period under observation (“they have not experienced the terminating event during a well-defined time interval”) but not thereafter.
In medicine, we meet situations of this type very commonly in survival studies. In these cases, the starting point very frequently is the date of diagnosis of, for example, a malignant tumor, and the terminating event might be the date of detection of tumor recurrence or metastasis or even death. Consequently, we analyzed the effects of open admission versus active admission procedure as well as of some selected demographic variables on dropout rates by statistical methods from the field of survival analysis.8
Here, we distinguish between nonparametric, semiparametric, and parametric methods. The product-limit approach by Kaplan and Meier9 does not make any assumption concerning the underlying hazard function (“baseline hazard”) for the terminating event under scrutiny but estimates the cumulative probabilities of “survival” (for our purpose, this corresponds to “retention in study”) merely from the empirical data at hand. Thus, it is a nonparametric method. The proportional hazards method by Cox10 also does not make any assumption about the baseline hazard; the effect of covariates, however, is modeled by a parameterized analytic expression. The model parameters are estimated from the data and allow, in a multivariate fashion, quantification of the relative predictive strengths of the variables included with regard to the terminating event. The Cox method is thus a semiempiric one. Finally, there are a host of parametric models which provide explicit mathematical models for the baseline hazard as well as covariate effects. These models assume one of several possible distribution models for the baseline hazard (e.g., exponential distribution, Weibull distribution, Gompertz distribution, and others) with adjustable parameters. If appropriate, such models allow the estimation of cumulative probabilities as a function of time by means of an explicit analytic expression.
We used the nonparametric product limit technique by Kaplan and Meier to compute the cumulative probabilities for retention in the course of study for student categories defined on the basis of several variables: mode of admission (open admission versus selection), sex, age, and nationality. Such cumulative probabilities are usually represented graphically by typical step functions decreasing from 1.0 to smaller values, as observation time progresses. We tested differences of cumulative retention probabilities among different categories by the generalized likelihood ratio method (Breslow χ2 statistic).11 To visualize the time-dependent risk of experiencing dropout for students in defined categories, we computed smoothed hazard functions for dropout according to Muller and Wang.12 These smoothed hazard functions give the instantaneous probabilities that a participant will experience a terminating event at time “t.” Roughly, they represent the negative first derivative with respect to time of the cumulative retention probabilities. We employed the semiparametric proportional hazards model by Cox in order to study the combined effects of potential predictor variables in a multivariate manner and to identify the relative strength of each individual predictor variable in the context of all other variables.
All statistical evaluations, including basic statistics for comparison of mean values and frequencies among different groups of students, were done using commercially available software (Stata Statistical Software: Release 11; StataCorp, 2009, College Station, Texas).
As Table 1 shows, the composition of the students' population before and after the introduction of selection procedures in academic year 2005–2006 differed significantly. Of the 2,860 students included in this study, 1,971 (68.9%) were admitted before the academic year 2005–2006. Although markedly more women (1,197/1,971 = 60.7%) than men (774/1,971 = 39.3%) were admitted before introduction of selection procedures, the composition switched to a slight predominance of men afterwards (456/889 = 51.3% men; 433/889 = 48.7% women). This change is highly significant (χ2 = 36.1; 1 df; P < .0001). The composition of students according to nationality also changed strongly (χ2 = 311.4; 2 df; P < .0001). Before 2005–2006, few German students (38/1,971 = 1.9%) were admitted, their percentage being well below the percentage of other foreign students (116/1,971 = 5.9%; the remaining 1,817/1,971 = 92.2% were Austrians), but since 2005–2006, 889 students have been admitted in all, and on the basis of the student quotas employed, German students (188/889 = 21.1%) were by far the strongest group of non-Austrian students in this group (37/889 = 4.2% other countries; the remaining 664/889 = 74.7% were Austrians). Finally, whereas 78.5% (1,548/1,971) of students admitted before 2005–2006 were younger than the common upper quartile of 20.89 years, only 67.5% (600/889) of those admitted after admission tests fell into this younger age group (χ2 = 40.0; 1 df; P < .0001).
Additionally, Table 1 shows dropout rates by sex, age, and nationality of the students before and after introduction of admission tests. Notably, this table summarizes the numbers and percentages of dropout events in the various student groups without investigating the detailed temporal patterns of the dropout occurrences. Additionally, one has to bear in mind that the observation time—and thus the chance to observe dropout events—for students admitted openly was considerably longer than for students admitted after admission tests. Nevertheless, important overall observations can be drawn from this initial analysis. Firstly, whereas 764 of 1,971 (38.8%) openly admitted students dropped out from study, the respective fraction in selected students is only 41 of 889 (4.6%). This difference in dropout rates between both groups is striking and highly significant (χ2 = 353.3; 1 df; P < .0001).
Given the very small frequency of dropout events in students admitted after the implementation of active admission procedures, it is not surprising that analyses of the effects of sex, age, and nationality in these students did not yield any significant results. On the contrary, in students openly admitted, we observed significant influences of these variables. Probability of dropout in these students was significantly higher in female (522 of 1,197; 43.6%) than in male students (242 of 774; 31.3%). The odds ratio between female and male students was 1.70 (95% CI, 1.41–2.06). Similarly, students older than 20.89 years showed a significantly higher dropout rate (231 of 423; 54.6%) than did younger students (533 of 1,548; 34.4%). The odds ratio between older and younger students was 2.29 (95% CI, 1.84–2.85). Austrian students had lower dropout frequency (702 of 1,817; 38.6%) than did Germans (21 of 38; 55.3%) but were similar to students from other nations (41 of 116; 35.34%).
Survival analysis techniques allow taking the variable of time (i.e., different durations of observation as well as different temporal patterns of occurrence of dropout events) properly into account. Figures 1 to 3 show cumulative retention probabilities for different groups and subgroups of students as a function of study time. Students who were openly admitted exhibited significantly higher dropout hazard than did those admitted after active admission procedures; consequently, the cumulative retention probabilities of openly admitted students reached significantly lower values with progress of time than did those of selected students (Figure 1; Breslow test: χ2 = 200.2; 1 df; P < .0001). When adding sex as a second variable (Figure 1), overall significance remains high (χ2 = 248.8; 3 df; P < .0001). Apparently, women exhibited worse prognosis for dropout than did men. Notably, this difference between women and men was significant only in openly admitted students (χ2 = 35.1; 1 df; P < .0001) but not in students admitted after admission tests (χ2 = 0.43; 1 df; P = .51). Similarly, although the combination of mode of admission and age (Figure 2) also yielded globally highly significant cumulative probabilities of retention (χ2 = 273.0; 3 df; P < .0001) with older students showing worse results, the effect of age alone was significant only in openly admitted students (χ2 = 58.1; 1 df; P < .0001). In students selected after admission testing, the age effect disappeared (χ2 = 0.04; 1 df; P = .83). Finally, the combination of the variables of mode of admission and nationality of students (Figure 2) showed strong global significance (χ2 = 208.1; 5 df; P < .0001). For the openly admitted subgroup, nationality was borderline significant (χ2 = 5.11; 2 df; P = .078), with German students showing the worst prognosis for dropout. In students selected by admission procedures, students from foreign countries other than Germany showed the strongest dropout risk (χ2 = 9.75; 2 df; P = .0076). Thus, the initial results drawn from Table 1 are confirmed by taking time into account as a central variable of interest.
Figure 3 demonstrates the combined effects of sex and age on cumulative retention probabilities for the category of openly admitted students only. The curves are significantly different (χ2 = 113.4; 3 df; P < .0001) and demonstrate a striking distinction between the extreme groups: Younger men showed a long-term cumulative retention probability of 70% to 75%, whereas older women approached a retention probability of only about 35%. For these four student groups, Figure 3 depicts the respective smoothed hazard estimates, showing maximum hazards between one and two years after enrollment. Thus, the risk for dropout is highest within this time interval.
Sex, age, and nationality did not exert significant effects on dropout rates in students who passed active selection procedures. Therefore, we restricted multivariate statistical analyses investigating the simultaneous effects of these predictor variables to the category of the 1,971 openly admitted students. Furthermore, given the small number of Germans (38) and students from other countries (116) in contrast to Austrians (1,817) in this group, we did not analyze nationality as another variable in Cox regression analyses. Rather, we treated nationality as a stratifying variable and performed the analyses using sex and age as independent variables separately in the three groups of different nationalities.
Table 2 shows the results of these Cox regression analyses. Although the results for German students and students with other nationalities were not very conclusive because of the relatively high standard errors of the respective regression coefficients, the analysis for the group of Austrian students clearly showed that both age and sex are strong and mutually independent predictors for dropout: Women show a 1.74-fold higher dropout hazard than men; students older than 20.89 years exhibit a dropout hazard 2.08-fold higher than that of younger students.
According to the rules of the proportional hazards model, we can combine these hazard ratios in a multiplicative way. Thus, the average dropout hazard of an Austrian woman aged above 20.89 years at study entry is 1.74 × 2.08 = 3.6 times higher than that of a younger male Austrian colleague.
The systematic alteration from a state of open access to medical studies in Austria toward active selection of students provides a unique possibility to study the impact of a shift which, in most other countries, happened decades earlier. We have already published an initial analysis showing dramatic academic improvement for students admitted after admission testing7; although only about 20% of students admitted openly were able to finish the first part of study within the scheduled first two study semesters, this success rate rose to 80% or more for students selected by admission testing.
In the present study, we focused on the important question of dropout rates, which, for openly admitted students, used to be 50% or more. As expected, implementation of active student selection strongly and very significantly lowered dropout rates compared with open admission. We can, for example, express the effect using the result of a Cox regression which employs presence versus absence of active admission procedures as single independent variable: The dropout hazard for selected students is only 14.5% (95% CI, 10.6%–19.9%; P < .0001) of the dropout hazard for openly admitted students. Similarly, a Dutch study reported that the relative risk for dropping out of medical school was 2.6 times lower for actively selected students as it was for lottery-admitted controls,13 lowering the risk of dropout from 100% to 38.5%. Our study is quite different from this one, but the strong advantage of active selection procedures is obvious in both settings.
Our investigation yields interesting additional information. Students' sex and age at enrollment were found to be strong and independent predictors for dropout under an open admission policy. The strongest univariate predictor of higher dropout rates in this category of students was age at study entry: Students aged above the global third quartile of 20.89 years showed a 1.84-fold higher hazard for dropout than did younger students, and women had a 1.55-fold higher dropout risk than did men.
The observation of higher dropout risks for older students in the group of openly admitted students is not surprising, at least in the environment of Austrian universities. A recent study analyzing the reasons for dropout across all Austrian universities and across all studies arrived at the overall conclusion that higher age at study entry predicts a higher probability of dropout.14 Specifically, in more than 28,000 new students admitted in academic year 2006–2007 in Austria, students dropping out within the first three semesters were, on average, 1.5 years older at study entry compared with their more successful colleagues. Interestingly, an earlier study of factors affecting the probability of early medical student dropout in the United Kingdom15 arrived at a contrary conclusion: Students aged 21 years at study entry were less likely to drop out than were those aged 19 years or younger at study entry. However, the situation in the United Kingdom is very different from the Austrian one; the overall dropout rates in the United Kingdom are dramatically smaller than those in Austria, which probably is a direct consequence of the Austrian peculiarity of general open access to university studies.
The finding that women show a significantly higher hazard of dropout under open admission deserves particular attention. Until admission standards were put in place, the fraction of women among enrollees was consistently 60% and higher. However, as our data show, the fraction of women dropping out was also significantly higher than the fraction of men dropping out. Since the introduction of active admission procedures in Austria in 2005, however, a specific problem consistently occurs and attracts strong public attention in Austria, irrespective of the admission tests actually used. Admission tests like the one discussed in this paper, as well as the Swiss “Eignungstest für Medizinische Studien” (EMS; a general intelligence test used as an admission test by Swiss medical faculties and also by the Medical Universities of Vienna and Innsbruck), regularly are a disadvantage to women. Although women still dominate in the group of test candidates (55%–60% female versus 40%–45% male applicants), the proportion of the sexes within the successful candidates is reversed (more than 50% of those accepted are men). This phenomenon obviously cannot be contributed to the type of admission test, because it is observed practically identically with both the Graz admission test and the EMS, and the reasons for its occurrence could not be satisfactorily explained so far. Moreover, the phenomenon seems to be much more pronounced in Austrian applicants as compared with, for example, German candidates. Public discussion about this displeasing finding is vivid, and most experts of educational psychology in Austria believe that there are deep-seated general social and educational prejudices about gender roles with respect to science, technology, and related topics which may explain the phenomenon.
In contrast to the significantly higher dropout rates among women openly admitted compared with their male peers, we found no difference between the sexes in students admitted after admission testing. Our previous study also showed a lack of differences between sexes in study progress rates among selected students.7 Because rigorously selected women and men seem to perform on par with one another, one might look to the subject matter (natural sciences) and/or format (MC questions) of the admission tests to examine the reasons for underrepresentation of women among selected students. Of note, MC questions are used for the admission test as well as for the overwhelming part of the examinations during the study program of human medicine. Thus, it is possible that women in the era of open admission were at higher risk of failing during the course of study if the MC format posed a particular problem, whereas now they encounter the MC format during the admission test. We stress that, at present, this is not more than just one of several possibilities. However, some support for this speculation seems to come from an earlier study of the study success rates at the Medical University of Vienna.16 In a prospective study on 674 (50.8%) of all 1,327 new students enrolled openly in academic year 2002–2003, factors potentially associated with study success were collected. These data were then correlated with the results of a compulsory assessment after the first study year. Importantly, male students turned out to be significantly more academically successful than females. Notably, the compulsory assessment was also based on MC questions.
There is a lot of literature available dealing with the problem of gender fairness of MC tests, but the picture is not completely clear. Notably, the advantages for males observed in several large-scale test situations focusing on science and mathematics may result from a complicated mixture of gender effects and effects caused by the test format itself. DeMars17 reports of a modification of the Michigan High School Proficiency Test where, in order to discriminate between gender and format effects, free-response (FR) questions were used in addition to MC questions. In mathematics and natural sciences, male participants did better on MC questions, whereas females were superior in FR questions in mathematics. In FR questions on natural sciences, there was no difference observed between female and male participants. Willingham and Cole18 have collected 12 studies in which tests based on discrete, open-ended questions requiring brief answers are compared with tests based on stem-equivalent questions in MC format. Nine of these studies addressed the problem of construct difference (open-ended questions versus MC questions). In 7 out of these 9 studies, no effect of test format was detected. Five of the 12 studies addressed the gender fairness problem; none of these investigations found a difference between females and males.
Generally, there is no clear picture about success rates of women and men at Austrian universities. One study19 reports that, across Austria, only 49% of female students, but 55% of male students, complete a degree program; but there are also studies reporting contrary results.20 On an international scale, a systematic review of the literature also showed marginally higher success rates for women.21 More studies are needed to resolve these discrepancies.
As a consequence of the consistent observation of the disadvantage that the Austrian medical school admission procedure poses for women, we are strongly engaged to pay particular attention to the development of good learning materials helping prospective applicants to prepare for the admission test. We are in close contact with all Austrian secondary schools to try to motivate interested pupils and teachers to start preparing for medical studies as early as possible, and we hope that these measures will help to eliminate the observed gender unfairness. Clearly, additional research work must be done to explain the specific gender unfairness problem of admission tests for medical studies in Austria.
In this study, we employ statistical techniques stemming from the field of survival analysis.8–12 Although such methods seem to be rarely used in educational research, their application in analyzing longitudinal observations in medical education has been suggested previously.22 Because these techniques are generally applicable in all fields of analysis of waiting times, we strongly recommend their application for the analysis of all aspects of students' careers involving time as a principal variable.
The present investigation suffers from several shortcomings. First, it lacks an experimental design. Rather, it is a retrospective analysis of the consequences of a unique political decision in a local setting—namely, the first introduction of active admission testing after a long history of open access to medical studies.
Another weakness is the fact that the Medical University of Graz admission test is not an internationally accepted standard test but, rather, has been developed very rapidly as a consequence of an important decision of the European Court and the following adaptation of Austrian law. Within weeks of the new law in 2005, medical universities in Austria had to establish active admission procedures. As a consequence, we are fully aware of the necessity to steadily monitor and improve our admission procedure. Because our admission test is strongly focused on knowledge of natural sciences, one important desire is the broadening of the personal competencies that are relevant to secure a population of future medical professionals with a broad spectrum of distinct abilities. As important milestones on this road we have added a situational judgment test to the admission procedure in 2010, to include at least to some extent the psychosocial competencies of the applicants. In 2011, for candidates applying for the dentistry program, we shall introduce another new test component dealing with manual skills. Notably, these additional parts of the admission procedure are very recent and by no means have any influence on the results reported in this investigation.
Our study reports an analysis of student career data generated under the impact of a fundamental change in admission policy. The results strongly support the application of admission testing for medical studies. There is no doubt that selecting students by admission tests leads to significantly better study progress as well as to significantly lower rates of premature dropout. It thus ensures a much more efficient, effective, and responsible use of financial and infrastructural resources as well as of valuable lifetime resources of students, teachers, and other staff. The study also shows interesting new conclusions regarding demographic characteristics of students which stimulate further research work on issues such as gender fairness in selection of students for medical studies.
The authors gathered the deidentified data from information that is routinely collected about medical students' admission, dropout, and graduation dates and examination history, as required by the Austrian Federal Ministry of Science and Research. Because the data were anonymous and no data beyond those required by law were collected for this study, the Medical University of Graz's ethical approval committee did not require approval for this study.
1 Reibnegger G, Haas J, Neges H, Smolle J. Die Reform des Medizinstudiums an der Medizinischen Fakultät/Universität Graz. Zeitschrift für Hochschulentwicklung. 2008;3:48–61.
4 Smolle J, Neges H, Staber R, Macher S, Reibnegger G. Virtuelles Eingangssemester im Studium der Humanmedizin. Kontext, Nutzung, Ergebnisse. In: Seiler Schiedt E, Kälin S, Sengstag C, eds. E-Learning—Alltagstaugliche Innovation? Münster, Germany: Waxmann; 2006:287–295.
5 Staber R, Smolle J, Bauer P, et al. Virtual Medical Campus Graz. An e-learning environment has its 5 year-anniversary. GMS Med Bibl Inf. 2007;7;Doc43.
7 Reibnegger G, Caluba HC, Ithaler D, Manhal S, Neges HM, Smolle J. Progress of medical students after open admission or admission based on knowledge tests. Med Educ. 2010;44:205–214.
8 Kleinbaum DG, Klein M. Survival Analysis. A Self-Learning Text. 2nd ed. New York, NY: Springer Science + Business Media, LLC; 2005.
9 Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. J Am Stat Assoc. 1958;53:457–481.
10 Cox DR. Regression models and life tables (with discussion). J R Stat Soc. 1972;34:187–220.
11 Breslow N. A generalized Kruskal–Wallis test for comparing k samples subject to unequal patterns of censorship. Biometrika. 1970;57:579–594.
12 Muller HG, Wang JL. Hazard rate estimation under random censoring with varying kernels and bandwidths. Biometrics. 1994;50:61–76.
13 Urlings-Strop LC, Stijnen T, Themmen APN, Splinter TAW. Selection of medical students: A controlled experiment. Med Educ. 2009;43:175–183.
15 Arulampalam W, Naylor R, Smith J. Factors affecting the probability of first year medical student dropout in the UK: A logistic analysis for the intake cohorts of 1980–92. Med Educ. 2004;38:492–503.
16 Frischenschlager O, Haidinger G, Mitterauer L. Factors associated with academic success at Vienna Medical School: Prospective survey. Croat Med J. 2005;46:58–65.
17 DeMars CE. Gender differences in mathematics and science on a high school proficiency exam: The role of response format. Appl Meas Educ. 1998;11:279–299.
18 Willingham WW, Cole NS. Gender and fair assessment. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.; 1997.
19 Dell'mour R. Landler F. Quantitative developments at Austrian universities 1973–2000. In: Publications of the Institute of Demography at the Academy of Sciences. Vienna, Austria: Austrian Academy of Sciences; 2002.
20 Hackl P, Sedlacek G. Analysis of study duration at the Vienna University of Economics and Business Administration. In: Dutter F, ed. Festschrift 50 Years. Vienna, Austria: Austrian Statistical Society; 2002:41–59.
21 Ferguson E, James D, Madeley L. Factors associated with success in medical school: Systematic review of the literature. BMJ. 2002;324:952–957.