Secondary Logo

Journal Logo

O0074 INTER-RATER RELIABILITY OF THE ROME CRITERIA IN CHILDREN

Saps, M.1; Di Lorenzo, C.1

Journal of Pediatric Gastroenterology and Nutrition: June 2004 - Volume 39 - Issue - p S36
ABSTRACTS: Oral Presentation Abstracts
Free

1Division of Pediatric Gastroenterology, Children’s Hospital of Pittsburgh, Pittsburgh, United States

Submitted by: miguel.saps@chp.edu

Introduction: Functional gastrointestinal disorders (FGID) are common in children. It has been suggested that the diagnosis of these conditions should be based on symptoms and should follow the pediatric Rome II criteria(1). The inter-rater reliability (agreement between different observers) for the DSM-IV, another symptom based criteria has been found to be near perfect in multiple studies. There are no studies assessing the reliability of the Rome II criteria in children.

Methods: Ten pediatric gastroenterologists were provided with 20 clinical vignettes, a sample of the Rome II criteria, and a list of 15 possible diagnoses that included all the pediatric categories of the Rome criteria plus “none of the above” or “not enough information”. Each of the raters was instructed to select one or more diagnosis for each vignette.

Results: 1) There was total diagnostic agreement between all the raters in only one of the 20 vignettes. The inter-rater percentage of agreement coefficient per clinical case was greater than 50% in only 5 of the 20 cases and less than 25% in 7 of the vignettes. The average percentage of agreement coefficient among the raters was 41% with a range of 13% to 100%. The interpretation of percentage of agreement is considered inadequate when less than 70%. 2) To correct for the possibility of agreement by chance, we calculated kappa coefficient, a measure of pair-wise agreement. The k was 0.3 (z= 21.3; p<0.0001), indicating that the agreement was 30% better than would be expected on the basis of chance alone. (Kappa values are considered as slight agreement between 0–0.2, fair 0.2–0.4, moderate 0.4–0.6, substantial 0.6–0.8 and almost perfect between 0.8–1). 3) When we analyzed the data in 2 separate subgroups of pain and constipation related disorders, the inter-rater percentage of agreement coefficient per each clinical case ranged between 13% to 80% with a mean of 35% in the constipation subgroup and 13% to 80% with a mean of 42% in the pain subgroup. The kappa value for both subgroups was lower than the overall kappa, but remained fair (constipation: kappa=0.2; z=7.0; p=< 0.0001) (pain: kappa=0.3; z= 11.8; p=< 0.0001).

Conclusion: Conclusion: The inter-rater reliability of the Rome II criteria among pediatric gastroenterologists is low. Further validation of the criteria is necessary. More user friendly criteria need to be developed in order to enhance their diagnostic accuracy. Alternatively, further physicians’ education in the use of these tools should be sought.

Back to Top | Article Outline

Reference(S):

1.Rasquin-Weber A, Hyman PE, Cucchiara S, et al. Childhood functional gastrointestinal disorders. Gut 1999; 45:II60–II68.
© 2004 Lippincott Williams & Wilkins, Inc.