Adapting traditional paper–pencil instruments to computer-based environments has received considerable attention from the research community due to the possible administration mode effects on obtained measures. When differences due to mode of completion (i.e., paper–pencil, computer-based) are present, threats to measurement validity are posed. In this research, administration mode effects of web-based and conventional paper–pencil versions of a parent-completed developmental questionnaire, the Ages & Stages Questionnaires (ASQ), were estimated. Setting this study apart from similar studies reported in the literature, the ASQ requires parents to observe and rate their children's behavior. Most measures adapted for web-based administration are not observational. Using item response model invariance testing procedures, analyses tested whether ASQ items administered via the Internet function differently from corresponding traditional paper–pencil items. Analyzing the 4-, 12-, and 24-month ASQ intervals, statistically significant differences (i.e., DIF, or differential item functioning) were obtained on 10 of the 90 items examined in this study. Although DIF was observed for some items, the overall DIF model was rejected for all domains. On the basis of these results, the paper–pencil and web-based measures can be considered equivalent and the mode effect is not present; ASQ measures obtained from either mode are therefore interchangeable. Noteworthy remedies are considered for web-based administration of the 10 specific items lacking invariance.
Department of Teaching and Learning, Southern Methodist University, Dallas, Texas (Dr Yovanoff); Center on Human Development, University of Oregon, Eugene (Dr Squires); and Department of Special Education, University of Guam, Mangilao (Dr McManus).
Correspondence: Jane Squires, PhD, Center on Human Development, University of Oregon, 1585 E 13th Ave, Eugene, OR 97403 (firstname.lastname@example.org).
The Ages & Stages Questionnaires is a registered trademark of the Paul H. Brookes Publishing Company. Dr. Squires discloses a conflict of interest as an author of the Ages & Stages Questionnaires who receives royalties for its publication. No conflicts for the other authors.