Secondary Logo

Caught in a Web of Confusion

Assessing the Readability of University Webpages for Victims of Sexual Assault

Duncan, Shannon1; Yeatts, Paul, PhD2; Kapusta, Ann, MA3; Allen, Denise, MEd3; Wilson, Jennifer, DNP, RN, CPN1; Tilley, Donna Scott, PhD, RN, CA SANE3

doi: 10.1097/JFN.0000000000000228
Original Articles
Free

ABSTRACT Assuring the effectiveness of Web sites in communicating critical information to a college student who has experienced sexual assault is important and complex. Recognizing that the average American reads at the eighth grade level, the National Institutes of Health and the American Medical Association recommend that information be written at a sixth-grade reading level. A sample of 10 U.S. institutions of higher education (IHE) Web sites made up the sample of the current study. The IHEs included were all participants in a project called “Cultivating Safe College Campuses,” funded by the Department of Health and Human Services. This study aimed to gain more insight into the readability of college Web sites about sexual assault resources for the intended consumer—college students. The reading level of IHE Web sites with information about sexual assault should align with the reading level of their intended audience. The average readability of sexual assault Webpages for this study's sample of IHEs was over 13, well above the reading level of the average “first time in college” student. All IHEs should review the readability of their Web sites and revise them to use consistent and defined terms and present the material in a way that is clear and accessible for the student who has experienced trauma.

Author Affiliations:1College of Nursing, Texas Woman's University;

2Center for Research Design & Analysis, Texas Woman's University;

3Texas Woman's University.

The authors declare no conflict of interest.

Correspondence: Donna Scott Tilley, PhD, RN, CA SANE, Texas Woman's University, P.O. Box 425619 Denton, TX 76204. E-mail: DTilley@twu.edu.

Received August 31, 2018; accepted for publication December 5, 2018.

Institutions of higher education (IHEs) across the country are struggling with changing expectations and responsibilities surrounding the problem of campus sexual assault (CSA). Sexual assault is a serious problem on U.S. university campuses. Nearly one in four female undergraduate students reports being sexually assaulted (AAU, 2015), along with 11.2% of male undergraduate students. The physical and emotional sequelae of sexual assault, which include chronic health problems and persistent mental health issues, can be devastating (Santaularia et al., 2014; Zinzow et al., 2011). The American College Health Association (ACHA) also identified academic consequences of sexual assault. These include alienation, barriers to academic success, lower graduation rates, and fear of retaliation (ACHA, 2016). While facing the personal consequences, CSA victims must also consider their options for reporting the assault, getting counseling, or finding other forms of assistance from the IHE, frequently turning to their college Web sites for quickly accessible information (Hayes-Smith & Levett, 2007).

The number of students who potentially need sexual assault information for handling their own experiences is significant. A recent 15-year review of the literature on CSA prevalence research (Fedina, Holmes, & Backes, 2018) yielded some startling findings. The prevalence of completed rape was reported to be as high as 8.4% for college women and almost 1% for college men. The prevalence of unwanted sexual contact without penetration ranged as high as 34% for college women, with most researchers reporting rates over 20%. Unwanted sexual contact for college men was as high as 31%. Incapacitated (drug and/or alcohol facilitated) sexual assault prevalence rates ranged as high as 14.2% for women and 1.9% for men. About one third of college women reported experiencing some form of sexual coercion. When even broader definitions of sexual assault are utilized, prevalence rates are reported to be as high as 44.2% (Fedina et al., 2018). Students who identify as lesbian, gay, bisexual, transgender, or queer experience sexual violence at even higher rates—as high as 43% for bisexual women (Rothman, Exner, & Baughman, 2011).

The experience of sexual violence can leave survivors with life-changing trauma, which can impair their ability to receive information they may need about sources of help and support. Processing even simple information for a person who has experienced recent trauma can be difficult. One of many distressing symptoms of acute stress disorder is difficulty concentrating (American Psychiatric Association, 2013). Survivors of trauma may have difficulty with organizing information and making decisions (Campbell, 2012).

Efforts to both curb and respond to CSA almost always include a university Web site with information about sexual assault (Lund & Thomas, 2015). These Web sites often provide information about the meaning of consent, ways to reduce the likelihood of becoming a victim, and information about how to get help for students who have been sexually assaulted, including reporting options for the victims. Although information on Web sites should provide clear and easily accessible information (Dameron, DeTardo-Bora, & Bora, 2009), this is not always the case. Lund and Thomas (2015) reported that information about sexual assault was often located in multiple locations on university Web sites, most often in two or more places. In addition, many college students, particularly those who are in crisis because of their own sexual assault or that of a friend, would not search through complex or policy-laden content to find needed information (Lund & Thomas, 2015).

Back to Top | Article Outline

Readability Measures

Readability measures have been created over time to measure written documents and have subsequently been applied to online materials. When conducting a readability study, a comprehensive understanding of the formula components and development is critical. It is also important to consider how the reading grade level may differ for the whole Web site as opposed to a sample of the material, such as the type of information on a CSA Web site. Readability measures use a combination of words, syllables, and sentences to a reading level number or grade level. The five readability measures used in this study were the Simple Measure of Gobbledygook (SMOG) Index, the Flesch–Kincaid Grade Level, the Gunning Fog Index, the Coleman–Liau Index, and the Automated Readability Index (ARI).

Wang, Miller, Schmitt, and Wen (2013) found that, when applying readability measures to health-related materials, the most commonly used measure was the Flesch–Kincaid Grade Level. However, the SMOG Index had the most consistent results. The Gunning Fog Index (Walsh & Volsko, 2008), the Coleman–Liau Index, and the ARI also appear in health and patient education Web site readability studies (Hansberry et al., 2014).

The SMOG Index counts 10 consecutive sentences near the beginning, middle, and end of the article and adds up the number of words with three or more syllables (or the whole article if 30 sentences or less). A formula is used on that information to determine the reading grade level (Walsh & Volsko, 2008). The Flesch–Kincaid Grade Level calculates grade levels using the average number of words per sentence as well as the number of syllables per word. Anything higher than Grade 12 is measured as Grade 12 (Walsh & Volsko, 2008). The Gunning Fog Index uses a formula that includes the percentage of polysyllabic words along with the average number of words per sentence (Walsh & Volsko, 2008). The Coleman–Liau Index applies sentence length and character count in its formula (Svider et al., 2013). The ARI applies sentence length and character count in its formula (Hansberry et al., 2014).

Readability formulas, even when applied to the same materials, produce varied grade level estimates because of differing formulas and validation methods (Wang et al., 2013). In the Wang et al. (2013) study, the researchers found that the SMOG Index was most suited for healthcare use because of its basis on more recent criteria than other tools and because it has been validated against 100% comprehension.

Back to Top | Article Outline

Research Problem

Examining the effectiveness of CSA Web sites in communicating critical information to a college student who has experienced a traumatic event is an important and complex topic. Recognizing that the average American reads at the eighth grade level, the National Institutes of Health and the American Medical Association recommend that information be written at a sixth-grade reading level (Kugar, Cohen, Wooden, Tholpady, & Chu, 2017). Sixty-three percent of high school graduates read below a 12th-grade level (National Assessment Governing Board, 2016). A 2016 report indicated that over one million entering college students do not read at the college level (Bureau of Labor Statistics, 2016). When developing material on a CSA Web site, reading level and readability are important. In addition, a trauma-informed approach to CSA Web site development is essential because students who have experienced trauma such as sexual assault may be confused, distracted, and forgetful (ACHA, 2016).

The possible lack of alignment between existing Web sites about CSA, recommendations about readability for healthcare information, and trauma-informed approaches led to the following research questions:

  1. What is the difference in the readability of IHE sexual assault reporting page and the average American adults' reading level?
  2. Is there a statistical difference between the three readability testing platforms (Readable, Webpagefx, and Readability Formulas)?
  3. What is the difference in the consistency of terminology of addressing sexual assault in 10 different IHE sexual assault reporting pages?
  4. What is the difference in the readability of IHE sexual assault reporting page and the readability of IHE application page, health/medical services page, counseling services page, and financial aid page?
Back to Top | Article Outline

Design/Methods

A sample of 10 IHE Web sites made up the sample of the current study. The IHEs included were all participants in a project called “Cultivating Safe College Campuses,” funded by the Department of Health and Human Services. Each of the participating IHEs was tasked with establishing bystander intervention training for incoming students, establishing a CSA task force and Sexual Assault Response Team, conducting a campus climate survey, and establishing or reviewing current campus policies regarding sexual assault. The 10 I.E. in the project are diverse, representing small and large, rural and urban, and public and private institutions. A summary of the universities that were utilized for the current study is provided in Table 1.

TABLE 1

TABLE 1

First, a one-sample t test was used to compare the reading level of the IHE sexual assault reporting pages with the average American reading level. Next, a one-way analysis of variance (ANOVA) was conducted to compare three different readability scoring Web sites. An Alexa ranking, which is a measurement of Web site popularity, was utilized to identify the popularity of each readability scoring Web site. In addition, a content analysis was conducted to investigate the differences in the consistency of terminology across the IHE pages. To do so, the prevalence of common terms was tallied and recorded for each sexual assault Webpage. Finally, a repeated-measures ANOVA was used to compare readability scores of five different IHE Webpages (i.e., sexual assault, application, health and medical, counseling, and financial aid). The repeated-measures (i.e., within-subjects) design was advantageous in this case because it allowed for the partitioning of variance that was because of individual Web site differences.

Back to Top | Article Outline

Results

Research Question 1

A one-sample t test was used to compare the mean readability of sexual assault reporting pages with the average American adult reading level. Results indicated that the mean reading level of the sexual assault pages was significantly higher than the average adult reading level, t(9) = 6.26, p < 0.001.

Back to Top | Article Outline

Research Question 2

A one-way ANOVA was conducted to compare the reading level across three different platforms to see if there was a difference in how they rated the reading level of the sexual assault pages. Results indicated that there was not a significant difference between the three platforms (p = 0.367). Because there was not a significant difference between the three platforms, the Alexa ranking (a tool designed to record Web site traffic statistics) was used to determine which platform was the most popular. On the basis of Alexa rankings (see Table 2), Webpagefx.com was the most popular of the three; thus, it was used as the ranking for the remaining analyses.

TABLE 2

TABLE 2

Back to Top | Article Outline

Research Question 3

Each sexual assault Webpage was inspected to determine the usage of common terminology related to sexual assault. Terms were tallied, and percentages were calculated to identify the prevalence of each term's use. The most common term used was “sexual assault” (90%). The least common term was “molestation” (0%). Table 3 presents frequency of use for each term.

TABLE 3

TABLE 3

Back to Top | Article Outline

Research Question 4

A repeated-measures ANOVA was used to compare the mean reading level of five different Webpages within each university's Web site (Webpages included sexual assault, university application, health and medical, university counseling, and financial aid). As displayed in Table 4, there were significant differences in the reading level of the different pages, F(4, 36) = 3.23, p = 0.02, η2 = 0.26. The effect size of 0.26 indicated that the magnitude of differences between the pages was large (Cohen, 1988). Post hoc pairwise comparisons were consulted to further investigate differences among the Webpages. The mean reading level of health and medical pages was significantly higher than those of the financial aid and application pages (p < 0.05). There was not a significant difference between the mean reading level of the sexual assault page and the other four pages.

TABLE 4

TABLE 4

Back to Top | Article Outline

Discussion

This study aimed to gain more insight into the readability of college Web sites about sexual assault resources for the intended consumer—college students. The reading level of IHE Web sites with information about sexual assault should align with the reading level of their intended audience (Dameron et al., 2009). The average readability of sexual assault Webpages for this study's sample of IHEs was over 13, well above the reading level of the average “first time in college” student. The results of this study were consistent with those of Taylor (2018) who reported an average readability of over 14 for most IHE sexual assault Webpages. Students who have been sexually assaulted are likely to experience difficulty concentrating and organizing information (American Psychiatric Association, 2013; Campbell, 2012), further exacerbating the Web site readability issues. IHEs should consider recruiting students, perhaps using student organizations, to provide feedback about the information on their Web sites related to sexual assault, specifically whether the information is useful, clear, and reasonably easy to find.

In addition to readability problems, the Web sites used a variety of terms to describe various acts of sexual misconduct. The most commonly used terms were “sexual assault,” “sexual harassment,” “sexual misconduct,” “rape,” and “sexual violence.” Although all of these terms are not interchangeable, many are considered to be interchangeable (Rape, Abuse, & Incest National Network, n.d.). Clearly defining terms on Web sites using widely accepted definitions, such as those provided by the Rape, Abuse, & Incest National Network, can enhance student understanding of content. IHEs should consider including clear definitions for each term, perhaps with examples of each term.

The primary limitation of this study is the small sample size using a convenience sample. Future studies should include a larger sample size using random selection. A follow-up study to assess the accessibility of Web sites, or the number of clicks required to get to specific pages starting from the university home page, would also be beneficial. Finally, researchers can assess the readability and accessibility of Web sites with special populations including English as a Second Language learners and those with learning disabilities. Because this study did not involve use of human participants, it did not require institutional review board approval.

Back to Top | Article Outline

Implications for Clinical Forensic Nursing Practice

Sexual Assault Nurse Examiners (SANEs) and other forensic nurses are on the frontlines of sexual violence prevention and response and are in a position to contribute to the readability of IHE Web sites regarding sexual assault. Through partnership with CSA response teams and task forces, SANEs can provide feedback about making Web sites more readable and accessible, particularly through the lens of trauma-informed care.

Back to Top | Article Outline

Conclusions

All of the IHEs in this study met the basic expectation of a Web site providing information for their students who needed information about sexual assault. Unfortunately, this important information is not as accessible to students who may be in crisis as it could be. All IHEs should review the readability of their Web sites and revise them to use consistent and defined terms and present the material in a way that is clear and accessible for the student who has experienced trauma.

Back to Top | Article Outline

References

AAU. (2015). Climate survey on sexual assault and sexual misconduct. Retrieved from http://www.aau.edu/key-issues/aau-climate-survey-sexual-assault-and-sexual-misconduct-2015?id=16525
American College Health Association. (2016). Addressing sexual and relationship violence: A trauma informed approach. Retrieved from https://www.acha.org/ACHA/Resources/Addressing_Sexual_and_Relationship_Violence_A_Trauma_Informed_Approach.aspx
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.
Campbell R. (2012). What it means to be trauma-informed: Best practices in Title IX investigations and hearings. Presentation to the Consortium for Title IX and Equality in Higher Education, Dallas, TX.
Cohen J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
Dameron S. L., DeTardo-Bora K. A., & Bora D. J. (2009). An assessment of campus security and police information on college/university websites. Security Journal, 22, 251–268. http://psycnet.apa.org/doi/10.1057/palgrave.sj.8350082
Fedina L., Holmes J. L., & Backes B. L. (2018). Campus sexual assault: A systematic review of prevalence research from 2000 to 2015. Trauma, Violence, & Abuse, 19(1), 76–93. https://doi.org/10.1177%2F1524838016631129
Hansberry D. R., Agarwal N., Shah R., Schmitt P. J., Baredes S., Setzen M., Carmel P. W., Prestigiacomo C. J., Liu J. K., Eloy J. A. (2014). Analysis of the readability of patient education materials from surgical subspecialties. The Laryngoscope, 124, 405–412. https://doi.org/10.1002/lary.24261
Hayes-Smith R. M., & Levett L. M. (2007). Sexual assault resources on campus: Availability and adequacy. Paper presented at the 59th annual meeting of the American Society of Criminology, Atlanta, GA.
Kugar M., Cohen A., Wooden W., Tholpady S., & Chu M. (2017). The readability of psychosocial wellness patient resources: Improving surgical outcomes. Journal of Surgical Research, 218, 43–48. https://doi.org/10.1016/j.jss.2017.05.033
Lund E., & Thomas K. (2015). Necessary but not sufficient: Sexual assault information on college and university websites. Psychology of Women Quarterly, 39(4), 530–538. https://doi.org/10.1177%2F0361684315598286
National Assessment Governing Board. (2016). The nation's report card: 12th-Grade scores lower in Math, steady in reading since 2013. Retrieved from https://www.nagb.gov/naep-results/reading/2015-reading-math-grade-12.html
Rape, Abuse, & Incest National Network. (n.d.). The realities of sexual assault on campus. Retrieved from https://www.rainn.org/types-sexual-violence
Rothman E., Exner D., & Baughman A. (2011). The prevalence of sexual assault against people who identify as gay, lesbian or bisexual in the United States: A systematic review. Trauma, Violence & Abuse, 12(2), 55–66. doi:10.1177/1524838010390707
Santaularia J., Johnson M., Hart L., Haskett L., Welsh E., & Faseru B. (2014). Relationships between sexual violence and chronic disease: A cross-sectional study. BMC Public Health, 14(1), 868–882. https://doi-org.ezp.twu.edu/10.1186/1471-2458-14-1286
Svider P. F., Afarwal N., Choudhry O. J., Hajart A. F., Baredes S., Liu J. K., & Eloy J. A. (2013). Readability assessment of online patient education materials from academic otolaryngology-head and neck surgery departments. American Journal of Otolaryngology, 34(1), 31–35. https://doi.org/10.1016/j.amjoto.2012.08.001
Taylor Z. W. (2018). Unreadable and underreported: Can college students comprehend how to report sexual assault? Journal of College Student Development, 59(2), 248–253. https://doi.org/10.1353/csd.2018.0023
Walsh T. M., & Volsko T. A. (2008). Readability assessment of internet-based consumer health information. Respiratory Care, 53(10), 1310–1315.
Wang L., Miller M. J., Schmitt M. R., & Wen F. K. (2013). Assessing readability formula differences with written health information materials: Application, results, and recommendations. Research in Social and Administrative Pharmacy, 9(5), 503–516. doi:10.1016/j.sapharm.2012.05.009
Zinzow H. M., Amstadter A. B., McCauley J. L., Ruggiero K. J., Resnick H. S., & Kilpatrick D. G. (2011). Self-rated health in relation to rape and mental health disorders in a national sample of college women. Journal of American College Health, 59(7), 588–594. https://doi-org.ezp.twu.edu/10.1080/07448481.2010.520175
Keywords:

Higher education; readability; sexual assault; web pages

© 2019 by the International Association of Forensic Nurses. All rights reserved.