The evolution of digital technology in medical education has reshaped the residency application process for medical students worldwide. The Internet has dramatically enhanced the ability of applicants to compare residency programs and has extended the educational bandwidth of resident recruitment. Studies have continued to demonstrate the increasing importance and utilization of residency program websites across specialties in helping students find the program that best fits their personal preferences and career goals. Given the competitiveness of urology residencies and the limited number of match slots nationwide, these program websites play an even more critical role in this field. A recent survey conducted on 346 urology applicants demonstrated that over 75% applied to more than 50 programs. The information on urology residency websites can help applicants evaluate these programs while reducing the time and finances required in the match process.
Despite the increasing significance and utilization of residency program websites, however, research in various fields also demonstrates the inadequacy of residency websites in providing the information that applicants find most important when choosing a program. A recent study showed a similar level of variability in the content accessible on residency program websites in the field of urology. By increasing the number and diversity of variables analyzed, our study builds on the findings of these prior studies and provides the most comprehensive evaluation of urology program websites. The objective of this study was to determine the accessibility and content of residency websites specifically in the field of urology.
MATERIALS AND METHODS
The American Urological Association (AUA) website was accessed for a list of accredited U.S. urology residency programs as of August 2020. Institutional review board approval was not required as this study did not involve human subjects. A total of 53 data points were selected and subsequently divided into five categories for the analysis. The first category was personnel information, which comprised 16 of the 53 data points largely centered around faculty, residents, and fellows. The second category was applicant information, which contained seven data points regarding the residency application and match process. The third category was program information. This group comprised 10 of the 53 data points including content such as program description and clinical sites. The fourth category was training/research which contained 13 data points focused on life as a resident. The fifth and final category was resident benefits. This group comprised the remaining seven of 53 data points including content such as resident salary, insurance options, and vacation time. All 53 data points are listed and categorized in Table 1.
The data points were selected based on previously published literature from a variety of medical fields that evaluated the content of program websites that were important to applicants. While no research has evaluated the content most important to urology applicants specifically, we analyzed studies from several specialties to more accurately generalize the data. Data points regarding personnel information, fellowship programs, and interview dates were evaluated both qualitatively (faculty list, resident list, fellow list, fellowship program description, interview dates) and quantitatively (number of faculty, number of interns, number of residents, number of fellows, number of fellowship programs, number of interview dates). This was to distinguish programs that provide an itemized list of this information from those that exclusively provide a numerical value for these data points. Web search was performed on the 155 urology residency program websites based on the AUA listing of accredited programs. Of these programs, 14 were excluded from the study for the following reasons: Ten programs lacked a residency website, three programs lacked functioning hyperlinks, and one program had a website with Spanish-only text. Programs that had multiple tracks into which an applicant can match (i.e., clinical vs. research track) were only counted once in their inclusion of the total. Accessible program websites were accessed and analyzed for the presence or absence of the 53 aforementioned data points. To ensure consistency in the data collection process, each data point was individually defined and discussed by the team before website analysis. The findings were recorded and the percentage of programs possessing each data point was calculated. The presence of variables that were specific to fellowships was only analyzed on the websites of programs with accredited fellowships offered.
Of the 155 programs identified on the AUA listing of accredited urology residency programs, 141 were included in the final analysis following the exclusion of 14 programs due to inaccessible residency websites. None of the 141 programs had all 53 data points available. Of the 53 data points examined, 24 were found on at least 50% of the applicable program websites. The only data point found on 100% of the websites was program description. The data points that were most commonly found on the websites included: Program description (100%), coordinator contact information (88.65%), clinical sites (87.94%), faculty list (87.23%), number of faculty (87.23%), and curriculum (87.23%). Furthermore, fellowship program description was found on 95.24% of program websites that offer fellowships (n = 63). The data points that were least commonly found on the websites included: Alumni contact information (2.84%), resident contact information (5.67%), description of night float (5.67%), board pass rates (5.67%), and frequently asked questions (9.22%). Fellow contact information was only found on 4.76% of program websites that offer fellowships (n = 63). Complete results for all 53 criteria are listed in Table 2.
In addition to evaluating the prevalence of each data point among all of the residency programs, the sum of data points on each individual program website was also calculated and analyzed. Of the 141 programs included in the analysis, 84 programs (59.57%) had between 21 and 30 data points present. Furthermore, 25 (17.73%) programs had between 31 and 40 data points preset while zero programs had over 40 data points present on their website. Complete results demonstrating the prevalence of data points on each program website are illustrated in Figure 1.
Residency program websites have modernized the dissemination of program information to applicants. A survey performed by Gaeta and associates on residency applicants revealed that 78% of respondents reported a positive correlation between the information on residency websites and their decision to apply to particular programs. The increasing dependence on program websites provides endless opportunities for residency directors to highlight the prestige and benefits of their programs. This opportunity has become even more critical in light of the novelty facing this year’s application cycle due to the COVID-19 pandemic. With the impact of this pandemic on visiting rotations, application deadlines, and the interview process, residency websites may serve as the only portal through which students can experience programs during application season. Our aim was to perform a comprehensive analysis on urology residency program websites to identify potential deficiencies in accessibility and content.
Although 155 separate urology programs were listed on the AUA website, 14 had to be excluded due to lack of program websites, disabled hyperlinks, or, in one case, content written in a foreign language. Without the availability of program websites, applicants must rely on other, less informative sources of program details such as opinions from other students and colleagues. The lack of functioning hyperlinks on three program websites undermines the reliability of programs to provide accurate, up-to-date information to prospective applicants while limiting the ability of programs to advertise their strengths and offered opportunities.
The survey performed by Gaeta and associates demonstrated that information on program websites regarding the curriculum, the hospital and its affiliates, faculty and residents, and research activities were most influential to applicants. According to our analysis, information regarding program curriculum was present on 87.23% of websites, which was the fifth-highest percentage of the 53 factors analyzed. While the majority of programs presented information on their curriculum, only 58.87% provided a rotation schedule on their website. Applicants may analyze rotation schedules when envisioning their future life as potential resident at a program. Furthermore, programs demonstrated an even larger variability in the presence of other factors that pertained particularly to resident training such as electives, call schedule, teaching responsibility, work hours, night float, and board pass rates, all of which were featured on <20% of program websites. The paucity of these factors on program websites makes it difficult for applicants to accurately gauge the temporal responsibilities of the programs’ residents. While Gaeta’s survey demonstrated the importance of research activities to residency applicants, our analysis showed that only 67.38% of programs mentioned the research opportunities available to students. Furthermore, only 41.13% of programs listed active or previous research on their websites. Programs should strive to illuminate the opportunities and advancements in research on their websites given the importance of research in the pursuit of urology fellowship opportunities and future career promotions.
Perhaps the information most relevant for prospective applicants is the eligibility requirements and selection criteria. While 80.14% of programs highlighted eligibility requirements, such as graduation from an accredited medical school or up-to-date immunization status, only 35.46% of programs provided information regarding selection criteria. These criteria could include information such as desired board scores, extracurricular activities, or level of research involvement. The lack of this information undermines the transparency of these programs and makes it difficult for applicants to gauge their likelihood of matching at a particular institution. An Electronic Residency Application Service hyperlink was available on 73.05% of websites, creating a convenient access point for applicants to select programs for application submission. However, only 26.24% of programs provided their specific interview dates. Applicants rely on this information when planning their interview trail, and its absence could potentially preclude applicants from accepting interviews from certain programs if multiple interview dates overlap.
The only data point that was included on all 141 program websites was program description. While websites varied greatly on the level of detail in their descriptions, it is reassuring to see programs provide a general glimpse into what they have to offer applicants. Another data point that was widely available was clinical sites, being mentioned on 87.94% of websites. Descriptions of these facilities and hospital size were less represented, being mentioned on 60.28% and 41.13% of websites, respectively.
The third-most influential data point according to the survey by Gaeta and associates was information of program faculty and residents. While coordinator contact information was the third-most available data point at 88.65%, this still illuminates a huge deficit in the programs that lack this information. Residency coordinators often serve as the liaison between applicants and the programs, helping to address student questions and concerns. A listing of the faculty members was found on 87.23% of websites whereas faculty personal contact information was found on only 47.52% of websites. While <50% of programs listed faculty contact information, there is often little need for applicants to contact faculty directly during the match process, especially with the availability of residency coordinator contact information on most websites. A listing of residents was similarly found on 80.85% of websites, whereas resident contact info was found on only 5.67% of websites. While students may wish to reach out to residents to gain firsthand information on a program, that lack of resident contact info is likely an intentional gesture by the programs to limit distractions and additional workload for their trainees.
Our last set of analyzed data points pertained to residency benefits. The resident salary was featured on 68.79% of program websites. However, this information was often found under the “Graduate Medical Education” link accessible from the residency page. While our results show a substantial deficit in the number of programs reporting resident salaries, this deficit may be in part due to this information being listed elsewhere on a program’s website. Likewise, information regarding insurance options and vacation time was found on approximately 73% of program websites whereas information regarding parking, meals, and education funds was found on <45% of websites. The lack of transparency regarding resident benefits may pose consequences on an applicant’s decision to apply to certain programs. Without this information available, applicants are unable to assess the financial implications of their future training.
While this study included significantly more variables than the studies in other specialties that were analyzed, comparison was made between the content of urology residency websites and the content available on other specialty websites, specifically the surgical fields of plastics and orthopedics. In their evaluation of content availability on plastic surgery websites, Hashmi and associates found that 15 of the 31 variables analyzed were found on at least 50% of program websites. This is comparable to our study which found 24 of the 53 criteria on at least 50% of program websites. Furthermore, Hashmi and associates found that the variables most mentioned on program websites included coordinator information (92%), number of residents (92%), and clinical sites (91%). Information regarding research requirements, active and previous research, didactics, and resident benefits were comparable between plastic surgery and urology residency websites. Davidson and associates similarly evaluated the content on orthopedic residency websites and found comparable information regarding didactics, call schedule, program description, eligibility requirements, and current residents. Overall, these findings suggest that urology residency websites contain comparable content to websites in plastic surgery, orthopedic surgery, and potentially other surgical subspecialties. Our comparison did not delineate a consistent deficit in urology websites when compared to the other surgical specialties analyzed.
This study has a few notable limitations. First, the data points analyzed were not collected from a formal study regarding the interests of urology applicants when reviewing a program’s website. Rather, our list of variables was delineated from several prior studies in other fields. Second, our study analyzed the presence or absence of the data points listed rather than the quantity of information related to those data points. Third, the residency websites were independently reviewed by two individuals. Interpretation of which pieces of information were counted as “present” and which were counted as “absent” from websites may differ based on the individuals collecting the data. To minimize this potential subjectivity, the findings by both individuals were discussed, and any discrepancies in data collection were identified and corrected. Data points were also discussed and specified by the team before analysis to remain consistent during the data collection process.
While applicants may differ on the content they find most important when evaluating a program, we feel that the ideal program website should include personnel information, applicant information, program information, training/research information, and information regarding resident benefits. This study provides a comprehensive list of data points that should be considered for inclusion on residency program websites. In addition, the findings in this study largely reinforce and expand upon those found by Patel and associates in their evaluation of 44 criteria on 135 program websites. This information will enhance both the applicant’s impression of the program as well as the program’s ability to recruit a stronger and more diverse applicant pool.
Urology residency websites serve as an important source of information for both applicants and programs during the match process. However, our data support and expand upon the findings of prior studies indicating deficits in the content provided on residency websites in both urology and other specialties. The inconsistent availability of information limits the applicant’s ability to fully evaluate the programs and thus could potentially impact an institution’s ability to attract competitive candidates who lack additional sources of information regarding that program. The ability to recruit top medical students from around the country through residency websites should prompt programs to market themselves to the best of their abilities. This study’s demonstration of online informational deficiencies provides an opportunity for programs to improve the content available on their websites.
Financial support and sponsorship
Research reported in this publication was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health under award number UL1TR003096. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Conflicts of interest
There are no conflicts of interest.
1. Embi PJ, Desai S, Cooney TG Use and utility of Web-based residency program information:A survey of residency applicants J Med Internet Res 2003 5 e22
2. Winters RC, Hendey GW Do web sites catch residency applicants? Acad Emerg Med 1999 6 968 72
3. Chen VW, Hoang D, Garner W Do websites provide what applicants need?Plastic surgery residency program websites versus applicant self-reported needs Plast Reconstr Surg Glob Open 2018 6 e1900
4. Lebastchi AH, Khouri RK Jr, McLaren ID, Faerber GJ, Kraft KH, Hafez KS, et al. The urology applicant:An analysis of contemporary urology residency candidates Urology 2018 115 51 8
5. Hashmi A, Policherla R, Campbell H, Khan FA, Schumaier A, Al-Mufarrej F How informative are the plastic surgery residency websites to prospective applicants?J Surg Educ 2017 74 74 8
6. Davidson AR, Loftis CM, Throckmorton TW, Kelly DM Accessibility and availability of online information for orthopedic surgery residency programs Iowa Orthop J 2016 36 31 6
7. Wakefield DV, Manole BA, Jethanandani A, May ME Jr, Marcrom SR, Farmer MR, et al. Accessibility, availability, and quality of online information for US radiation oncology residencies Pract Radiat Oncol 2016 6 160 5
8. Patel BG, Gallo K, Cherullo EE, Chow AK Content analysis of ACGME accredited urology residency program webpages Urology 2020 138 11 5
9. Accredited Listing of U S Urology Residency Programs. Accredited Listing of U. S. Urology Residency Programs-American Urological Association Available from: https://www.auanet.org/education/auauniversity/for-residents/urology-residency-and-fellowship-programs/accredited-us-urology-programs/accredited-listing-of-us-urology-residency-programs
[Last accessed on 2020 Aug 24]
10. Gaeta TJ, Birkhahn RH, Lamont D, Banga N, Bove JJ Aspects of residency programs'web sites important to student applicants Acad Emerg Med 2005 12 89 92
11. Sinno S, Mehta K, Squitieri L, Ranganathan K, Koeckert MS, Patel A, et al. Residency characteristics that matter most to plastic surgery applicants:A multi-institutional analysis and review of the literature Ann Plast Surg 2015 74 713 7