The New York Times has praised digital technology for “letting in new voices, creating new formats for exploration, and allowing fans and other creators to participate in a glorious remixing of the work.”1 This underscores the broad societal implications of digital technology. Within medical education, the internet serves as a major portal through which medical students gain more insight into different residency programs. Medical schools have created websites providing students with information on possible specialty choices, residency preparation, and career assessment and planning.2 A previous study demonstrated that almost 80% of applicants believe that the online presence of a residency program influenced their decision to apply to that program.3 Hashmi et al.3 and Silvestre et al.4 evaluated various components of plastic surgery residency program websites (PSRWs) to determine website quality, and concluded that websites are lacking in content and accessibility. But what would applicants themselves like to see on websites, and, are websites adequately fulfilling this need? Ultimately, there is a paucity of evidence to guide plastic surgery departments in designing quality websites that provide information in line with what applicants desire in their application and program selection process.
The objectives of this study were to (1) use a web survey to investigate information that plastic and reconstructive surgery residency applicants desire to see on PSRWs; and (2) review all PSRWs in the United States to determine whether websites meet the information needs of those applicants.
Self-reported Needs of Plastic Surgery Residency Program Applicants
Upon USC institutional review board exemption from human subjects research under 45 CFR 46.101(b)2, a request to participate in a web-based survey (SquareSpace, New York, N.Y.) was e-mailed to all 189 plastic and reconstructive surgery residency program applicants to USC for the 2015 to 2016 application cycle (February 23, 2016). Name identifiers were removed, and completed entries were automatically linked to an Excel sheet. Duplicate submissions were manually filtered out. The survey was constructed based on a combination of elements similar to those utilized in previous literature.3 , 4 It included 15 questions, consisting of applicants’ use of residency program websites in their research on programs, any impact on decision-making in applications and interviews, and components of a websites that applicants believed to be most important. Answers incorporated Likert scales, multiple choice, yes-no, or open-ended systems (see appendix, Supplemental Digital Content 1, which displays a list and format of survey questions for applicants, http://links.lww.com/PRSGO/A865).
Measuring PSRW Information Quality of US Plastic Surgery Residency Programs
A scoring system was created to assess the quality of PSRWs in meeting the information needs of plastic surgery residency program applicants (Table 1). Criteria were selected based on previously published literature from various medical specialties, on contents of residency program websites that were important to applicants.3–7
The American Medical Association Fellowship and Residency Electronic Interactive Database was accessed for a listing of plastic and reconstructive surgery residency programs in February 2016. All included websites were each searched by the authors (D.H. and V.W.C.) for the presence or absence of criteria listed in Table 1, with the addition of components such as broken links, nonfunctional pages, donation links, and website feedback links. Forty-nine components were assessed, and a website was given a score of 1 if the information was present and a score of 0 if the information was unavailable or inaccessible. Information was only considered to be addressed on program sites if it was directly available—links to outside pages were not evaluated.
Data analysis was largely descriptive; counts and percentages were used for all categorical variable outcomes. If necessary for comparison, responses to Likert scale questions were converted to binary outcomes by grouping responses like “agree” and “strongly agree” together. A value of P < 0.05 was considered statistically significant.
USC Plastic Surgery Residency Program Applicant Needs
Out of 369 total applicants to plastic surgery in the nation,8 the survey was sent to the 189 applicants to USC, and 87 responded (46% response rate).
Almost all respondents (98%) used PSRWs during the application process. Most applicants (59%) visited websites after deciding on their specialty but before the application process (Fig. 1). In addition to websites, other resources utilized are detailed in Figure 2. Few applicants found the websites “very useful” (8%; Fig. 3). Over a third of students reported that the quality of the websites influenced their decision to interview (34%), and some students reported that the quality of websites influenced their decision to apply (22%). Of note, many applicants (41%) reported a discrepancy between the website and the interview encounter of the respective program.
Applicants most valued information on: faculty (95%), residency curriculum (91%), current residents (89%), career and fellowship (89%), and resident research (84%; Table 1). Design aspects that applicants most valued included video content (95%), consistent page layouts (87%), aesthetic quality (71%), and programmatic content (63%; Table 2).
PSRW Information Quality
Of the 93 residency programs, 92 programs had websites available (98.9%). Average score (SD) among all 92 websites was 18.7 (5.3) out of 49 points (36.9%).
Among content categories that applicants rated as important (Table 1), 70% of websites included faculty information, 70% included residency curriculum, 68% included resident information, 39% included career and fellowship information, and 72% included resident research. Out of the 5 website content categories stated above as most valued by applicants, only 1 website included information in all 5. Out of 22 common components between the applicant survey and scoring assessment of websites, there was a significant difference in 17 components (77%) between those rated as “important” by applicants and availability of the corresponding component on websites (P < 0.05; Table 1).
Few websites displayed some form of video content (9.8%), significantly different from the number of applicants who found video content important (P < 0.001). Over 30% of websites displayed broken links or missing or outdated information. Few websites displayed a link to provide website feedback (19.6%), whereas most websites (62.0%) displayed links to donate to the institution or affiliated hospital.
With the continued impact of technological advances on medical education,9 the behaviors and preferences of students are evolving as well. For example, applicants now reportedly find multimedia applications useful for learning about intangible aspects of a residency program that can help them determine fit and make an informed decision about a potential program, such as “how happy people are.”10
Of the residency applicants surveyed, almost all utilize websites in their research of residency programs. Correspondingly, almost all plastic surgery residency programs provide a website for their program. However, our study found that most applicants did not find these websites to be optimally useful, and for many applicants, these websites influence their decision to interview. This finding is important considering the competitive nature of applying to this specialty, in which applicants are often recommended to diversify the types of programs at which they interview.10 Of note, an important factor of the interview process is that it is resource intensive11 for both the residency program and the applicant. Although this does not necessarily imply that applicants prioritize website experience over their training, it demonstrates that websites provide a virtual window into a program and serve as an important preliminary factor in many applicants’ decisions to interview. It is also important to note that the website also does not necessarily take precedence over other factors in deciding where to interview, such as mentor suggestions, geography, or program fit, but it may serve as an initial access for applicants with no preliminary knowledge of a particular program. Interestingly, we found that 41% of applicants reported a discrepancy between the website and their interview experience. It is unclear whether this is due to overall aesthetic appeal of the website or simply inaccurate information. While applicants to plastic surgery traditionally apply broadly, in 2016 there was a decrease in applications per applicant, at 33.4 down from 50.9 the previous year.8 Applications per applicant in 2017 remained decreased at 35.5. It is therefore important to consider that plastic surgery applicant trends continue to change.
Furthermore, we found incongruity between applicant needs and website content. Based on the survey, applicants most valued academic and career characteristics, such as career and fellowship placement, faculty profiles, and residency research and curriculum. Most websites adequately provided faculty profiles, curriculum, and resident profiles, but our data identify unique program features, career and fellowship placement, and residency electives as major areas for improvement. Additionally, some applicants valued categories that almost no websites included (< 5%), such as alumni information, board pass rates, and resident finances and debt management. This represents a unique opportunity for websites to stand out if they choose to provide this information. For example, information on finance and debt management, while rarely found on websites, could potentially provide residents with resources for budgeting, estate planning, investment strategies, and retirement planning—all elements that can offer significant long-term benefits. Indeed, when surveyed, residents and fellows demonstrate low financial literacy and investment-risk tolerance, high debt, and deficits in financial preparedness.12 Overall, we found misalignment between websites and applicants’ surveyed needs. It is unclear whether this misalignment is because programs are unaware of what applicants value when visiting websites or because website design represent lower priority within a program’s agenda. However, without adequately fitting applicant needs, websites fail to serve their fundamental purpose—that is, as efficient communication tools for visitors.
Another major discrepancy lies in website design and interaction. Applicants value aspects such as video content, page layout, and aesthetic quality, but only a minority of websites provided video content and many websites exhibited functional issues such as broken links, missing or outdated information, or a nonfunctioning page. The use of more interactive and collaborative content continues to increase13–15 and websites are progressing from simple text-based applications. Many medical journals, from The New England Journal of Medicine to Journal of American Medical Association, now share updates on Facebook, postinformational videos on YouTube, and tweet new and free content on Twitter. Therefore, in addition to providing text-based content, programs should consider including more interactive interface on their websites, such as utilizing videos to uniquely present information.
Our study has its limitations. First, our study surveyed 189 applicants to USC out of a total of 369 applicants to plastic and reconstructive surgery overall.8 This is similar to the reported average applications per program, which was reported to be 178.5 in 2016. Out of those who responded, it is possible that there is a bias toward those who utilize more technology. It is also likely that there is a geographical bias within this sample, which does not necessarily represent opinions of the entire application pool. Second, while concrete factors such as broken links and interactive content may provide some insight into the aesthetic component of a website, an appropriate qualitative assessment of the website would provide a more accurate picture in line with applicants’ experiences with these websites. The categories we chose, while attempting to be as comprehensive as possible, were based on those assessed in previous studies. It is still possible that more minor categories were missed that may be important to applicants. It is also important to consider that while these content categories offer a more objective way of viewing websites, plastic surgery residencies do not exist in a vacuum. Multiple factors play a role in delivery of information, and often involve not only information specific to a plastic surgery residency program but also information on the institution as a whole and its relationship to other services, such as general surgery.
Furthermore, while we collected data on social media use, we did not include it in our analysis, as the data required further evaluation. However, we recognize social media as an important force in the evolution of online content, and it is especially relevant in its potential role in recruiting applicants.16 This is also a cross-sectional study based solely on website presentation at the time that the search was conducted. We recognize that websites are not static, that most websites are conventionally updated annually, but that many programs are also limited by their technology departments in implementing changes. It is our hope that websites will move toward incorporating live content and updated data that is not only user-friendly but also easier for programs to update.
Overall, we found a discrepancy between the assessed websites and applicants’ self-reported needs, despite the importance of online resources10 , 17 to residency program applicants.5 , 7 , 18 , 19 “User-friendly” now represents more than availability of necessary information, but also encompasses the ability to engage, collaborate, and view information through various modalities. As the field of plastic surgery—and medicine as a whole—evolves, delivery of information and content must also evolve to fit ever-changing needs and behaviors,13 whether it be the use of social media or integration and automation of content. Our results suggest that even small changes with adding minor content such as unique program features can significantly improve applicants’ experiences with websites. Better website depiction of a program allows applicants to make more informed decisions on the most suitable program, which ultimately strengthens the quality of the residency match program. Further studies can qualitatively examine the aesthetic aspects and interactive interface of plastic surgery websites to better guide residency programs in designing their websites. They can also further explore the temporal relationship in the application process in progressing from away rotations to interviews to ranking, and the use of websites, especially in relation to other factors such as mentor suggestions, cost, or location. It is important to note that most applicants visit websites before the application process, which is approximately the same time as application to away rotations. In the end, quality residency program websites benefit both students and residency programs. For the students, websites hold potential for impacting students’ decision in choosing the right specialty and in choosing the right program. For residency programs, websites serve as vital communication tools in attracting quality applicants who best fit each individual program.
1. Manjoo F. How the internet is saving culture, not killing it. The New York Times. March 15, 2017. Available at http://www.nytimes.com
. Accessed March 18, 2017.
2. Zink BJ, Hammoud MM, Middleton E, et al. A comprehensive medical student career development program improves medical student satisfaction with career planning. Teach Learn Med. 2007;19:55–60.
3. Hashmi A, Policherla R, Campbell H, et al. How informative are the plastic surgery residency websites to prospective applicants? J Surg Educ. 2017;74:74–78.
4. Silvestre J, Tomlinson-Hansen S, Fosnot J, et al. Plastic surgery residency websites: a critical analysis of accessibility and content. Ann Plast Surg. 2014;72:265–269.
5. Gaeta TJ, Birkhahn RH, Lamont D, et al. Aspects of residency programs’ web sites important to student applicants. Acad Emerg Med. 2005;12:89–92.
6. Mahler SA, Wagner MJ, Church A, et al. Importance of residency program web sites to emergency medicine applicants. J Emerg Med. 2009;36:83Y88.
7. Chu LF, Young CA, Zamora AK, et al. Self-reported information needs of anesthesia residency applicants and analysis of applicant-related web sites resources at 131 United States training programs. Anesth Analg. 2011;112:430–439.
9. Chu LF, Chan BK. Evolution of web site design: implications for medical education on the Internet. Comput Biol Med. 1998;28:459–472.
10. Tadisina KK, Gharb BB, Orra S, et al. Reply: applying to integrated plastic surgery residency programs: trends in the past 5 years of the match. Plast Reconstr Surg. 2017;139:330e–331e.
11. Susarla SM, Swanson EW, Slezak S, et al. The perception and costs of the interview process for plastic surgery residency programs: can the process be streamlined? Plast Reconstr Surg. 2017;139:302e–309e.
12. Ahmad FA, White AJ, Hiller KM, et al. An assessment of residents’ and fellows’ personal finance literacy: an unmet medical education need. Int J Med Educ. 2017;8:192–204.
13. Boulos MN, Maramba I, Wheeler S. Wikis, blogs and podcasts: a new generation of Web-based tools for virtual collaborative clinical practice and education. BMC Med Educ. 2006;6:41.
14. McLean R, Richards BH, Wardman JI. The effect of Web 2.0 on the future of medical practice and education: Darwikinian evolution or folksonomic revolution? Med J Aust. 2007;187:174–177.
15. Kamel Boulos MN, Wheeler S. The emerging Web 2.0 social software: an enabling suite of sociable technologies in health and health care education. Health Info Libr J. 2007;24:2–23.
16. Chu LF, Zamora AK, Young CA, et al. The role of social networking applications in the medical academic environment. Int Anesthesiol Clin. 2010;48:61–82.
17. Rabinowitz C, Prabhakar H, Amorosa J. An Internet-based approach to advising radiology residency applicants. Acad Radiol. 2002;9:551Y556.
18. Kutikov A, Morgan TM, Resnick MJ. The impact of residency match information disseminated by a third-party website. J Surg Educ. 2009;66:212Y215.
19. Embi PJ, Desai S, Cooney TG. Use and utility of Web-based residency program information: a survey of residency applicants. J Med Internet Res. 2003;5:e22.