In spite of considerable progress over the past decade, training investigators for inter-rater reliability for clinical trials remains a major problem. The aim of the present study was to promote a new tool to increase data homogeneity by introducing conformity into groups of clinical investigators. The investigators' scoring grid we are proposing–the Mean Deviation of Inter-rater Scoring (MDIS)–involves the calculation of the score deviation for each investigator relative to the median score of an expert group who had evaluated the same videotape-recorded clinical case. Whatever the scale, the score deviation is calculated as the absolute deviation value from the median score obtained by the experts for each item. The MDIS value is then evaluated from all the scores given by an investigator by dividing the total sum of the previously defined values by the number of items of the scale. Some examples from practice are given using several rating scales: (i) Hamilton Anxiety Rating Scale; (ii) Hamilton Depression Rating Scale; (iii) Montgomery Åsberg Depression Rating Scale; and (iv) Positive And Negative Symptoms Scale. Finally, such a method could also be employed by experts to evaluate the quality of videotape-recorded clinical cases used in clinical trials, as well as by teachers to evaluate initial or continuous medical training.
aSociété nantaise de Psychiatrie biologique, Nantes
bEA Neurobiologie de l'anxiété et de la dépression, Université de Nantes, Nantes
cDépartement de Pharmacologie, Université de Lille II, France
Correspondence and requests for reprints to Professor M. Bourin, EA 3256 – Neurobiologie de l'Anxiété et de la Dépression, Laboratoire de Pharmacologie, Faculté de Médecine, BP 53508, 44035 Nantes Cedex, France
Tel: +33 2 40 41 28 53; fax: +33 2 40 41 28 56;
Received 15 April 2003 Accepted 13 January 2004