Secondary Logo

Journal Logo


Patterns in On-time, Daily Submission of a Short Web-Based Personal Behavior Survey in a Longitudinal Women's Health Study

Crowder, Hannah R. MPH*; Brown, Sarah E. BS*†; Stennett, Christina A. MPH*†; Johnston, Elizabeth CRNP; Wnorowski, Amelia M. MD§; Mark, Katrina S. MD; Brotman, Rebecca M. PhD, MPH*†

Author Information
Sexually Transmitted Diseases: August 2019 - Volume 46 - Issue 8 - p e80-e82
doi: 10.1097/OLQ.0000000000001001

Timely measurement of self-reported sexual and feminine hygiene behaviors is important in longitudinal studies of sexually transmitted infections.1,2 Biomarkers for sexual exposure, including Y-chromosome and prostate-specific antigen detection in women, continue to emerge1,3,4; however, they are not in widespread use. Obtaining self-reported data, specifically sexual practices, can be challenging due to underreporting and overreporting of socially undesirable and desirable behaviors, respectively.2,5 In addition, error in recall increases as more time passes between the events of interest and their reporting.6,7 One study found recall error was between 37% and 51% when reporting 3 to 5 days after an event.8 Daily surveys are often used to collect the most accurate and reliable prospective behavioral data.9,10

Electronic data collection is a solution for some study populations because it allows anonymity; reduces social desirability bias, and in longitudinal studies, may facilitate increased frequency of reporting.9,11–16 Similarly, Ghanem et al.17 demonstrated that audiocomputer assisted self-interview results in a more reliable assessment of sexual risk than face to face interviews. Web-based surveys have been shown to increase survey completeness and maintain participant satisfaction compared with paper formats; however, there is little data on data accuracy and response rates in longitudinal studies using Web surveys.18

In an observational longitudinal study of adult women, we sought to evaluate patterns of daily online survey submissions, as well as factors associated with timely submission.


This analysis reports on 52 participants recruited between May 2017 and December 2018 who completed the Gynecology and Lubricant Effects (GALE) study, a 10-week observational cohort study in Baltimore, MD, that seeks to investigate changes in the vaginal microenvironment among women exposed to vaginal lubricants ( Women older than 18 years, who were referred for transvaginal ultrasound (TVUS), were eligible for the study. Possible reasons for referral included abnormal periods, uterine fibroids, ovarian cysts, and pelvic pain. Exclusion criteria included diagnosis of diabetes, human immunodeficiency virus, or other immunosuppressive conditions, current pregnancy, or antibiotic, or antifungal use in the month before TVUS.

Every evening, participants were emailed a link to a 30-question survey on REDCap,19 a Web-based application for building surveys, which included questions on sexual behaviors, feminine hygiene, and partners for the prior 24 hours. Questions were formatted as yes/no and there were 3 comment fields for text (Supplemental Fig. 1, Completion of the diary takes less than 5 minutes and could be completed on any device with an Internet connection. No reminders were sent following incomplete diaries; however, study coordinators sent reminders to participants with consecutive incomplete diary days. Diaries were submitted by ID number and were therefore deidentified. The REDCap portal supplied a timestamp for submission. The GALE study included a US $125 bonus at the final visit for 80% compliance with all study requirements including 3 clinical visits, weekly sample drop-offs, and diary submissions within 3 days of the initial email prompt. Diaries submitted over 3 days late would be considered “overdue.” This analysis was limited to the 52 women (3,419 submitted diaries) who used the Web-based format to complete daily diaries. Six women who chose to submit paper copies of the diaries are excluded from this analysis.

Because there were some variations in study observation follow-up for each participant, we calculated the proportion of diaries that were either missing or overdue. We defined timeliness across the study as 70% or greater of diaries submitted within 3 days of the initial email prompt. Altering this cutoff to 60% or 80% did not significantly alter our findings (Supplemental Fig. 2, Diaries completed in batches were determined by subtracting the number of unique logins to REDCap from the total number of completed daily diaries per woman, leaving a proportion of those diaries which were submitted in batches of more than 1 diary. We sought to describe patterns of daily submission and baseline participant factors associated with timeliness. We conducted univariate Fisher exact tests to evaluate baseline variables that might be associated with timeliness. All statistical analyses were completed using StataCorp Software 14. In defining overdue diaries, we chose the cutoff of 3 days late because women were also tasked with self-collecting vaginal samples twice-weekly for the parent study. This 3-day interval allowed for relatively timely submission of diaries.

The GALE study was approved by the Institutional Review Board at the University of Maryland Baltimore (HP-#00061371).


Participants included in this analysis ranged in age from 20 to 63 years, and the majority (71.15%) identified as African American, followed by white (15.38%), multiracial (9.62%) and Asian (3.85%). Most (36.00%) reported an annual income between US $20,000 and US $40,000, and 47.06% of participants did not report education beyond a high school diploma. The median number of lifetime sexual partners was 6.5 (range, 1–60). The majority of women reported never being married (48.08%), and most reported having children (57.69%).

Figure 1 displays the frequency of daily diary submissions per participant. As an example, participant ID 227 completed diaries a few times per week in the first 3 weeks, followed by batched diaries submitted less than once a week. The time intervals between her submissions also tended to grow toward the end of study follow-up. Timely submission also appeared to decline at the end of the study for a number of participants (ie, IDs 235, 238, 245 and 248).

Figure 1
Figure 1:
Patterns of daily diary submissions in a longitudinal women's health study. The size of the diamond icons indicates the number of days late a diary was submitted, and a blank space indicates no diary was submitted that day. The smallest dots indicate a single diary submission within 24 hours of the initial email prompt. Sporadic gaps represent intervals in which no diaries were submitted, and multiple large diamonds reflect multiple late submissions. Participants are stratified as having greater or less than 70% of their diaries submitted within 3 days (on-time).

Daily survey data were submitted for over 91% (3419 of 3747) of observation days. Submitted diaries were highly complete, with responses recorded for over 98% of variable fields. Fifty percent of the diaries were completed within 24 hours of the email prompt. No women submitted 100% of diaries within 24 hours of the email prompt, but half of participants (n = 26) had no diaries more than 3 days late (overdue). Nine percent of diaries were missing, and 10% were submitted more than 3 days late. A median of 27.4% of diaries per participant were submitted in batches (range, 1–79%).

Participants who had 30% or greater of their diaries missing/overdue were categorized as “less than 70% timely” in submissions (n = 12, 23%). This subset of participants was responsible for the majority of large diary batching. Among the less than 70% timely participants, an average of 48.1% of diaries were missing/overdue, whereas among the 70% or greater timely participants (n = 40, 77%), an average of only 8% of diaries were missing/overdue. Among 70% or greater timely participants, 20.7% of diaries were completed in batches, but only 1.7% of diaries were overdue, suggesting that the overwhelming majority of these batches were done in small 1- to 3-day increments. Among less than 70% timely participants, 64.5% of diaries were completed in batches, and 32.6% of diaries were overdue, suggesting larger batching.

Women who reported greater number of lifetime sex partners were more likely to have less timely submissions (P = 0.009). Sixty-three percent of the women with less than 70% timely submission reported 11 or more lifetime partners compared with 23% of respondents with timely diary submission. Women who smoked also tended to be less timely with submission, although the finding was not statistically significant (P = 0.2). Other factors, including children in the household, participant age, number of sexual partners in the 2 months before enrollment, race/ethnicity, education, income, marital status, TVUS indication/findings, and other behavioral factors were not significantly different between groups.


In this cohort study of women referred for TVUS, participants had both high compliance with submission (91%), as well as completion of all survey fields, in a longitudinal study with daily online behavioral diaries; however, diaries were often submitted later than the study requested and observation days were often submitted in batches. These findings are consistent with those of previous studies and suggest access to electronic timestamped forms may allow for easier participation and may enable researchers to better discern timeliness and reliability of data.16,20

We evaluated our baseline survey to determine if there were any factors that were associated with timely submission. Such information could inform study staff to be more aware of challenges that participants face in cohort studies and make arrangements so that diary submission is less onerous. We found that only 1 factor (greater number of reported lifetime sex partners) was associated diary submission delay. Other factors were not statistically significant. The strength of this study is that the volume of diary time points (n = 3419) in Figure 1 allowed us to visualize a descriptive presentation of submission patterns. The study was not designed to formally evaluate the Web-based instrument.

Late or batched diary data may be more problematic for behaviors which are not usually practiced, such as feminine hygiene practices or specific sexual behaviors, or for self-report of transient urogenital symptoms. The batching of surveys may also reflect that self-collected biological samples, in our case vaginal specimens, may also have been delayed in collection or collected in batches. Based on these findings, future studies should consider utilization of timestamped forms either by Web-based surveys or mobile devices. Weekly meetings between study coordinators and participants during sample drop-offs could benefit from enhanced focus on submission patterns. Studies may also consider using different monetary bonus schemes, such as weekly bonuses or scaled bonuses based on submission participation, to encourage on time submission of diaries and samples.


1. Zenilman JM, Yuenger J, Galai N, et al. Polymerase chain reaction detection of Y chromosome sequences in vaginal fluid: Preliminary studies of a potential biomarker for sexual behavior. Sex Transm Dis 2005; 32:90–94.
2. Mark KP, Smith RV, Young AM, et al. Comparing 3-month recall to daily reporting of sexual behaviours. Sex Transm Infect 2017; 93:196–201.
3. Brotman RM, Melendez JH, Smith TD, et al. Effect of menses on clearance of Y-chromosome in vaginal fluid: Implications for a biomarker of recent sexual activity. Sex Transm Dis 2010; 37:1–4.
4. Gallo MF, Warner L, Hobbs MM, et al. Differences in misreporting of sexual behavior over time: Implications for HIV trials. Sex Transm Dis 2015; 42:160–161.
5. Schroder KE, Carey MP, Vanable PA. Methodological challenges in research on sexual risk behavior: II. Accuracy of self-reports. Ann Behav Med 2003; 26:104–123.
6. Stalgaitis C, Glick SN. The use of web-based diaries in sexual risk behaviour research: A systematic review. Sex Transm Infect 2014; 90:374–381.
7. Garry M, Sharman SJ, Feldman J, et al. Examining memory for heterosexual college students' sexual experiences using an electronic mail diary. Health Psychol 2002; 21:629–634.
8. Zafar SN, Luby SP, Mendoza C. Recall errors in a weekly survey of diarrhoea in Guatemala: Determining the optimal length of recall. Epidemiol Infect 2010; 138:264–269.
9. Nyholm D, Kowalski J, Aquilonius SM. Wireless real-time electronic data capture for self-assessment of motor function and quality of life in Parkinson's disease. Mov Disord 2004; 19:446–451.
10. Johannes C, Woods J, Crawford S. Electronic versus paper instruments for daily data collection. Ann Epidemiol 2000; 10:457.
11. Turner CF, Ku L, Rogers SM, et al. Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science 1998; 280:867–873.
12. Kissinger P, Rice J, Farley T, et al. Application of computer-assisted interviews to sexual behavior research. Am J Epidemiol 1999; 149:950–954.
13. Gaertner J, Elsner F, Pollmann-Dahmen K, et al. Electronic pain diary: A randomized crossover study. J Pain Symptom Manag 2004; 28:259–267.
14. Weber BA, Yarandi H, Rowe MA, et al. A comparison study: Paper-based versus web-based data collection and management. Appl Nurs Res 2005; 18:182–185.
15. Quinn P, Goka J, Richardson H. Assessment of an electronic daily diary in patients with overactive bladder. BJU Int 2003; 91:647–652.
16. Johannes CB, Crawford SL, Woods J, et al. An electronic menstrual cycle calendar: Comparison of data quality with a paper version. Menopause 2000; 7:200–208.
17. Ghanem KG, Hutton HE, Zenilman JM, et al. Audio computer assisted self interview and face to face interview modes in assessing response bias among STD clinic patients. Sex Transm Infect 2005; 81:421–425.
18. Marcano Belisario JS, Jamsek J, Huckvale K, et al. Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods. Cochrane Database Syst Rev 2015; 7:MR000042.
19. Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42:377–381.
20. Njuguna HN, Caselton DL, Arunga GO, et al. A comparison of smartphones to paper-based questionnaires for routine influenza sentinel surveillance, Kenya, 2011–2012. BMC Med Inform Decis Mak 2014; 14:107.

Supplemental Digital Content

Copyright © 2019 American Sexually Transmitted Diseases Association. All rights reserved.