Background: As traditional methods have become increasingly difficult, the Internet offers a mechanism for conducting survey research quickly and efficiently. However, the validity of this research depends on the ability of respondents to accurately report health status. We used a large Internet-based inflammatory bowel disease (IBD) cohort to validate self-reported IBD against physician reports.
Methods: Between June 22, 2012, and April 01, 2013, all participants of CCFA Partners (n = 6681) were invited to participate, and 450 were selected by random stratified sampling. We sent physicians a survey to confirm IBD diagnosis and characteristics. We used descriptive statistics to compare data.
Results: A total of 4423 participants (66%) indicated interest. Of 450 selected, 261 (58%) consented, and physician reports were obtained for 184 (71%). Physicians confirmed IBD status in 178 (97%) and type in 171 (97% of confirmed). The matching between patient and physician reports for Crohn's disease (CD) was 82% for disease location, 89% for the presence of perianal disease, and 46% for disease behavior. For ulcerative colitis (UC), disease location matched 54% of the time. Physician reports confirmed the status of ever having bowel surgery for 97% of CD and 94% for UC and confirmed current pouch or ostomy in 84% of CD and 81% of UC.
Conclusions: Self-reported IBD in CCFA Partners is highly accurate, and participants are willing to release medical records for research. Self-reported phenotypic characteristics were less valid. The validity of IBD diagnoses among the participants of CCFA Partners supports the use of this cohort for patient-centered outcome research.
Article first published online 21 January 2014
*University of North Carolina School of Medicine, Chapel Hill, North Carolina;
†Center for Gastrointestinal Biology and Disease, and
‡Division of Gastroenterology and Hepatology, Department of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina;
§Worldwide Epidemiology GlaxoSmithKline, Research Triangle Park, North Carolina;
‖Section of Biostatistics and Epidemiology, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire; and
¶Division of Gastroenterology and Hepatology, Department of Pediatrics, University of North Carolina, Chapel Hill, North Carolina.
Reprints: Rachel L. Randell, BA, University of North Carolina at Chapel Hill, Campus Box 7080, Chapel Hill, NC 27599-7080 (e-mail: email@example.com).
Supported by the Crohn's and Colitis Foundation of America with additional support from GlaxoSmithKline and the National Institutes of Health (P30 D34987).
The authors have no conflicts of interest to disclose.
Received November 07, 2013
Accepted December 10, 2013
Health-related Internet use has become increasingly prevalent. Sixty percent of Internet users report searching for health information,1 patient Web portals can improve patient outcomes,2 online interventions can promote health behaviors,3 and health care information technology can enhance health care quality and efficiency.4 The Internet could also be a valuable tool for medical research.
As traditional methods for survey research such as household and telephone interviews have become increasingly difficult, the Internet offers a mechanism for conducting survey research quickly and efficiently.5,6 In fact, the Internet has become increasingly popular for symptom reporting for medical research, especially in the past 2 decades.7,8 Yet, the validity of Internet surveys for health research crucially depends on the ability of patients to accurately report their disease status.
In June 2011, we launched CCFA Partners, a novel, Internet-based cohort of inflammatory bowel disease (IBD) to facilitate patient-centered outcome research and translational research.9 IBD includes Crohn's disease (CD) and ulcerative colitis (UC), which are chronic illnesses affecting as many as 1.5 million individuals in the United States10 with significant medical costs, impact on the quality of life,11 and persisting gaps in the quality of care.12 Unlike traditional observational studies of IBD, which rely on administrative or medical record data or of patients at tertiary care centers, CCFA Partners collect data on exposures and outcomes directly from the patient. Use of the Internet also allows for a very large, diverse cohort that can prospectively be studied without the high costs typically associated with prospective cohort studies.
Although CCFA Partners has the potential to serve as a powerful platform for IBD research, little is known about the validity of self-reported health information collected over the Internet. In this study, we sought to determine the feasibility of obtaining clinical data from outside treating physicians of CCFA Partners participants and assess the validity of self-reported IBD status by comparing it with physician reports.
The CCFA Partners cohort was established in June 2011 by recruiting through the Crohn's and Colitis Foundation of America (CCFA) e-mail rosters, chapter events, and other promotional activities. Inclusion criteria are ≥18 years of age and Internet access. Participants complete a baseline survey upon registration, which includes questions on demographics, disease type and activity, medications, health behaviors, and other topics. Every 6 months, participants receive e-mail reminders to complete follow-up surveys. Details of the CCFA Partners design are described by Long et al.9
All CCFA Partners participants completing a survey between June 22, 2012, and April 1, 2013 (n = 6681) were invited to participate in a study to validate IBD diagnosis. Of those who indicated interest and provided a mailing address, 450 were selected based on random sampling stratified by disease type, age younger than or ≥50 years and use of IBD medications. Each selected participant was mailed a consent form, Health Insurance Portability and Accountability Act waiver, and request for physician contact information. Selected participants received up to 4 e-mail reminders as needed. We mailed physicians a 10-question survey to confirm the participant’s disease diagnosis, type, location/extent, behavior, surgery, and current pouch or ostomy status. Criteria were developed using the NIDDK IBD Genetics Consortium Phenotype Operating Manual13 and the Montreal classification.14 The survey could be completed on paper or online using a unique patient identification number. Physicians were compensated $100 for their effort. Up to 2 contacts via telephone and fax to physician offices were made if needed.
Data and Statistical Analysis
Comparisons between the groups were made using t tests for normally distributed continuous variables, or equivalent nonparametric tests. All statistics were computed using SAS Version 9.3 (SAS Inc., Cary, NC). The study protocol was approved by the Institutional Review Board at the University of North Carolina at Chapel Hill.
A total of 6681 participants of CCFA Partners completed any survey between June 22, 2012, and April 1, 2013, and 4423 (66%) indicated interest in sharing medical records. Of 450 selected, 314 (70%) returned documents and 261 (83%) consented to participate in the study. Physician medical record information was obtained for 184 (71% of consented) (Fig. 1). The response rate of physicians was 69% for university or academic physicians and 72% for physicians in private practice. Overall, 66% of the cohort considered participating in a study involving medical records, 38% consented, and physician reports were obtained for 27%.
The median age of the study population was 46 years, 74% were women, and 72% reported taking IBD medications (Table 1). Median disease duration was 16 years. No significant differences were found across sex, disease type, activity, or median duration across those who indicated interest in study participation and those who did not indicate interest. Those who indicated interest were slightly younger (43.6 versus 45.0 years) and more likely to report taking IBD medications (86.7% versus 82.1%). Of the participants selected for the study, no significant differences were found across age, sex, medication use, disease type, disease activity, or median duration between responders (those from whom physician reports were obtained) and nonresponders (those for whom physician reports were not obtained) (Table 1).
Of the 184 responders, physicians confirmed IBD status in 178 (97%) and IBD type in 171 (97% of confirmed IBD). The matching between patient and physician reports for CD was 82% for disease location, 89% for the presence of perianal disease, and 46% for disease behavior (stricturing or penetrating). For UC, disease location matched 54% of the time. Physician reports confirmed the status of ever having bowel surgery for 97% of CD and 94% for UC and confirmed current pouch or ostomy in 84% of CD and 81% of UC (Table 2).
CCFA Partners has used novel methods to assemble a large cohort of patients with IBD. Self-reported diagnoses of CD and UC appear to be highly valid within the cohort, although self-reported phenotypic characteristics are less valid. The validity of IBD diagnoses in CCFA Partners support the use of this novel, Internet-based cohort to conduct patient-centered outcome research.
A majority of participants were willing to release medical records for research, and obtaining reports from community physicians was feasible. Up to 66% of the cohort considered participation in a study involving medical records, and consent was obtained for 58% of the random sample of those who considered participating.
Daniels et al15 conducted a study to validate the diagnosis in a longitudinal Internet-based autism cohort, in which 63% consented and 61% provided documentation of the child’s illness. The results from the autism study cannot be directly compared with the CCFA Partners sample because Daniels et al15 recruited newly registered participants who were more likely to participate in the study based on previous findings.16 Furthermore, they retrieved disease documentation and compensated the participant. Nevertheless, a participation rate of approximately 60% seems to be consistent for Internet survey validation studies. Physician response rates were not significantly different between practitioners in a university or academic center and private practice. These findings demonstrate the feasibility of future studies of CCFA Partners that require detailed medical history and physician documentation from a diverse community.
Self-reported IBD, type, surgery, and pouch/ostomy status was highly valid among those for whom medical records were obtained; IBD diagnosis was confirmed 97% of the time. Similarly, Daniels et al15 found that 98% of families who provided documentation verified a physician diagnosis. Self-reported phenotypes of IBD, including disease location and behavior, are less valid, which may reflect poor patient understanding of IBD, limited education, or study timing.
The generalizability of these findings may be limited by self-selection of participants; however, the similar characteristics across the participants of CCFA Partners who indicated interest and those who did not support the generalizability of these findings. Although those who indicated interest were slightly younger and more likely to repot taking IBD medications, the magnitude of these differences was quite small. Additionally, similar characteristics of responders and nonresponders within the study sample support the validity of these findings. A limitation of all Internet-based research is access and literacy. Barriers to Internet use include low income, low education, and minority ethnic groups, but these are also the fastest growing groups of Internet users.5,6 Participants of CCFA Partners were enrolled primarily from rosters of an organization dedicated to IBD education (CCFA), so participants may be more educated and motivated than the larger universe of Internet users. In addition, other non-IBD patient populations may differ. Similar findings from an Internet-based autism study,15 however, may support widespread trends. It should also be recognized that in this study, physicians reported disease characteristics using a standardized case report form but not source data such as reports from colonoscopic or pathologic assessments. As we were unable to validate physician reports through source data validation, the survey used may not be a true gold standard, and the possibility of incorrect information provided by physicians remains.
In summary, self-reported IBD in CCFA Partners is valid, whereas self-reported IBD phenotypes are less valid. Participants are willing to release their medical records for research, and obtaining records from a diverse group of outside physicians is feasible. Although published data on accuracy of self-reported health information in Internet cohorts remains limited, our findings are similar to an Internet-based autism cohort. In all, the validity of IBD diagnoses in CCFA Partners support the use of this cohort in patient-centered outcome research and show that the Internet can offer a practical, cost-effective strategy to enroll large numbers of subjects for patient-reported outcome research.
Author contributions: Study concept and design, data acquisition, analysis and interpretation of data, drafting, and critical revision of the manuscript: R. L. Randell. Study concept and design, analysis and interpretation of the data, and critical revision of the manuscript: M. D. Long. Study concept and design, and critical revision of the manuscript: S. F. Cook. Data acquisition and review: C. E. D. Wrennall. Computer programming and data acquisition and analysis: W. Chen. Analysis and interpretation of data, statistical analysis, and critical revision of the manuscript: C. F. Martin. Study design, data analysis, and critical revision of the manuscript: K. Anton. Study concept, critical revision of the manuscript, and study supervision and the principal investigator of CCFA Partners: R. S. Sandler. Study concept and design, analysis and interpretation of data, critical revision of the manuscript, and study supervision: M. D. Kappelman.
1. Atkinson NL, Saperstein SL, Pleis J. Using the Internet for health-related activities: findings from a national probability sample. J Med Internet Res. 2009;11:e4.
2. Osborn CY, Mayberry LS, Mulvaney SA, et al.. Patient web portals to improve diabetes outcomes: a systematic review. Curr Diab Rep. 2010;10:422–435.
3. Portnoy DB, Scott-Sheldon LA, Johnson BT, et al.. Computer-delivered interventions for health promotion and behavioral risk reduction: a meta-analysis of 75 randomized controlled trials, 1988-2007. Prev Med. 2008;47:3–16.
4. Chaudhry B, Wang J, Wu S, et al.. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144:742–752.
5. Walker DM. The Internet as a medium for health service research. Part 1. Nurse Res. 2013;20:18–21.
6. Walker DM. The Internet as a medium for health services research. Part 2. Nurse Res. 2013;20:33–37.
7. Johansen MA, Henriksen E, Horsch A, et al.. Electronic symptom reporting between patient and provider for improved health care service quality: a systematic review of randomized controlled trials. Part 1: state of the art. J Med Internet Res. 2012;14:e118.
8. Johansen MA, Berntsen GK, Schuster T, et al.. Electronic symptom reporting between patient and provider for improved health care service quality: a systematic review of randomized controlled trials. Part 2: methodological quality and effects. J Med Internet Res. 2012;14:e126.
9. Long MD, Kappelman MD, Martin CF, et al.. Development of an Internet-based cohort of patients with inflammatory bowel diseases (CCFA partners): methodology and initial results. Inflamm Bowel Dis. 2012;18:2099–2106.
10. Kappelman MD, Rifas-Shiman SL, Kleinman K, et al.. The prevalence and geographic distribution of Crohn's disease and ulcerative colitis in the United States. Clin Gastroenterol Hepatol. 2007;5:1424–1429.
11. Rocchi A, Benchimol EI, Bernstein CN, et al.. Inflammatory bowel disease: a Canadian burden of illness review. Can J Gastroenterol. 2012;26:811–817.
12. Kappelman MD, Palmer L, Boyle BM, et al.. Quality of care in inflammatory bowel disease: a review and discussion. Inflamm Bowel Dis. 2010;16:125–133.
13. Nguyen GC, Torres EA, Regueiro M, et al.. Inflammatory bowel disease characteristics among African Americans, Hispanics, and non-Hispanic Whites: characterization of a large North American cohort. Am J Gastroenterol. 2006;101:1012–1023.
14. Silverberg MS, Satsangi J, Ahmad T, et al.. Toward an integrated clinical, molecular and serological classification of inflammatory bowel disease: report of a working party of the 2005 Montreal World Congress of Gastroenterology. Can J Gastroenterol. 2005;19:5–36.
15. Daniels AM, Rosenberg RE, Anderson C, et al.. Verification of parent-report of child autism spectrum disorder diagnosis to a web-based autism registry. J Autism Dev Disord. 2012;42:257–265.
16. Kalb LG, Cohen C, Lehmann H, et al.. Survey non-response in an Internet-mediated, longitudinal autism research study. J Am Med Inform Assoc. 2012;19:668–673.