As is typical of implementation of new systems, frustration was noted during the learning phase. Comments from department users indicated that they found initial data entry, necessary to create the MIV record, to be time consuming and tedious, particularly for senior faculty members with many publications and accomplishments. There were also unanticipated bugs and glitches related to migration of the system to the campus server, and these also frustrated the users. Despite these growing pains, all participating departments were supportive. Department users commented that the consistency and standardization of format for both dossiers and CVs were positive benefits. Departments found it particularly advantageous to have dossier information saved in a central repository that only required updating for the next action. Departments also commented that they eagerly anticipated a truly paperless process for academic reviews in the near future.
Surveys were also completed by the associate dean for academic affairs and the FPC members (Table 3 and 4). Reviews of hard-copy dossiers prepared using MIV were viewed chiefly as equivalent to those prepared using conventional methods. The largest improvements and benefits related to the online capacity for review. FPC members commented that electronic reviews facilitated group discussion. Previously, only the primary and secondary reviewers had hard copies of the dossier, but with MIV, all members of the committee could simultaneously view dossiers via laptops during their meetings. The availability of the dossier online also made access to the packet more convenient for review at home or in distant offices by both FPC members and the associate dean. Large, bulky, paper dossiers no longer had to be transported. Locating sections of interest in the dossier was also considerably easier because of the electronic table of contents. Thus, both FPC members and the associate dean felt that they spent more efficient and effective time in review. The office of academic affairs found that managing dossiers was considerably easier, because fewer department preparation errors were identified, there was less paper for analysts to handle, and less physical space was required for storage in the office.
All users agreed that MIV was a favorable advance with much future promise, though all thought that additional functional enhancements were necessary, particularly regarding increasing the speed of the system. Departments reported that few faculty members have yet used the CvOnline or NIHForm products during this initial stage of merit and promotion implementation.
Analysis and Lessons Learned
We have found that creating an electronic faculty database such as MIV has been a worthwhile effort for academic departments and for the school in general, particularly when it is used in the academic review process. Despite the considerable time and effort required for several years to develop such a system, the expected frustrations in the learning phase of bringing up a new system, and the time spent populating the new database, MIV is perceived quite favorably by all who are involved and is considered to be worth the investment of human and financial resources. All users anticipate significant long-term benefits that include less time and effort spent recreating packets for each subsequent review, increased convenience and ease in abstracting information for those reviewing the packets, and efficiencies in storage and retrieving completed academic records. The uniformity of the packet presentation, which includes improvement in formatting and consistency in including accomplishments and documentation, should also ensure easy readability by the reviewers—an important feature to reviewers who may review a hundred or more packets a year.
One of MIV's key virtues is its flexibility to accommodate scientific and academic products from multiple disciplines, making it applicable to a variety of schools and departments. For example, images of works of art can be uploaded to facilitate review. MIV's current ability to upload publications directly from PubMed illustrates the possibility of arranging similar uploads from other electronic libraries and databases. Other schools and colleges on our campus have also piloted MIV and have had a positive experience. The MIV system has been demonstrated to the leadership at other University of California campuses, and they have recognized its many advantages. This has lead to considerable interest in extending MIV beyond our own campus and across the 11-campus University of California system.
We encourage other schools to pursue the development of similar systems, because the benefits can be significant. Our experience provides several lessons learned, described below, which may benefit others in this task:
A multidisciplinary team is necessary to create and implement such a system and should include all major stakeholders to ensure success. The multidisciplinary team should include computer programmers and systems experts, faculty, and personnel from both the dean's office and departments who oversee and are intimately familiar with the academic review process and its requirements. We found that it was important in the development process to include users of all levels to achieve the necessary perspective and buy-in.
It was our hope that individual faculty members would become more familiar with MIV. Faculty members could be considered to be the primary stakeholders in the development of an electronic system for academic reviews, because their merit advancements and promotions are dependent on the presentation of their accomplishments through the packets created by the new system. We had hoped to motivate the active participation of faculty by using MIV as a personal tool to maintain their own CVs or to create biosketches for NIH grant applications. These personal documents were intended to be appealing features that would motivate individual faculty members to regularly update their own MIV data repositories, decreasing the burden on administrative staff. In general, however, faculty members have not expressed much interest in learning to use MIV and have preferred to have department staff maintain their databases for them. This is, in part, attributable to time constraints. Many faculty are not interested in taking time to learn a new system, given their heavy clinical loads, teaching obligations, and research commitments. Clinical faculty members have recently been required to learn the health system's new electronic medical record system. Requiring them to learn another new system is perceived as too much. In addition, success in the merit and promotion process is not dependent on presentation via MIV versus the conventional paper packet, so this is not a major motivator for faculty to use MIV themselves. Merit and promotion denials are infrequent in our school, usually averaging fewer than 10% of all submitted actions per year. This denial rate did not change during the pilot period, indicating that the merit and promotion results are not affected by the method of preparation. Before MIV, department staff invested considerable time and effort to correctly prepare each faculty member's packet so that it met the university's required format, ensuring that the faculty member's academic accomplishments were presented well in the review process. The real stakeholders in the success of MIV are, therefore, the department staff who bear the brunt of the administrative burden in preparing the packets. An electronic system such as MIV can introduce efficiencies and alleviate some of this heavy administrative burden, potentially giving department staff members more time to accomplish other tasks.
Interestingly, department staff are not enthusiastic about having faculty trained to use MIV, because they feel that the integrity of the database and product formatting will be better maintained and will have fewer errors if department staff are solely responsible for the database. Many of the younger generation of faculty members are more accustomed to using technology, however, and have expressed interest in learning to use MIV as a personal tool.
Key design features should include security, the ability to upload information from other databases, and the ability to make changes easily. Early in the development phase, faculty expressed concern about who would have access to their data. Individual security was therefore deemed to be of high priority, to ensure faculty acceptance. The system was therefore created so that a faculty member must designate the individuals to whom he or she allows access. A faculty member who wishes to opt out of the electronic merit/promotion process for privacy reasons may lock his or her account to all other users and continue to submit hard-copy paper packets. With time, the faculty members have become less concerned with access and security. This may be because they have gained trust in MIV, but also because most faculty members do not want to be bothered with maintaining their own records and preparing their own packets.
The ability to upload information from other databases is also a feature that is important to the success of this type of system. Populating the MIV database has been one of the most time-consuming and tedious aspects of implementation. Eliminating the necessity to individually enter all data elements speeds implementation and helps to maintain a positive outlook regarding the system. As an example, the ability to upload publications from PubMed was not a feature of the early versions of MIV, contributing to less-than-enthusiastic reactions from department staff, particularly when entering information about senior faculty members with long lists of publications and other accomplishments.
It is important that any system allow for easy changes and updates. Faculty members often provide incomplete or inaccurate data, which need to be corrected or updated. One of the most problematic aspects that we noted was the ability to resequence entries, such as publications. Resequencing in our initial system could only be done by moving an entry up or down one place on the list at a time. This was found to be too slow and was frustrating for data-entry staff, who would have preferred the ability to cut and paste or quickly skip to the correct place on the list.
Training must be frequent and ongoing. As mentioned earlier, hands-on training classes for MIV users were held in a computer classroom several times per month. Trainees were instructed to bring a packet to work on at the session. A staff expert was also available in the office of academic affairs to assist departments as needed. Application and performance questions could also be sent directly to the pilot administrative team via a specially designated e-mail link on the MIV home page. An MIV pilot program listserve was created and used for announcements and information regarding system problems and upgrades. The MIV home page also contained regularly updated information on enhancements, fixes, processes, and work in progress.6 We had hoped that training an individual in each department would result in a train-the-trainer-type situation, and that heavy reliance on central training would not be necessary. We noted, however, that there was frequent turnover in department staff, so trained individuals were usually not available for long within departments. This was chiefly attributable to the fact that entry-level staff members are usually assigned to the packet-preparation task, and these individuals tend to seek advancement opportunities quickly. Strong, ongoing training from a central office within the school or campus is therefore necessary.
Users should be rewarded and encouraged. Initial entry of faculty data is a large and time-consuming task, but it is necessary for implementing this type of program. Staff time to accomplish this may be scarce. To facilitate the kick-off of MIV and to motivate departmental participation, the UCD Health System generously provided funds to pay for either staff overtime or temporary staff for a limited time so that a large number of faculty members could be entered into the database. This offer was met with enthusiasm and was essential to implementing MIV. Support for the unanticipated bugs and glitches, which are inevitable in the development of any new system, is also very important. We provided users with many opportunities to suggest improvements and provide feedback. In addition to the pilot program listserve and home page e-mail link mentioned above, MIV issues are a regular discussion item at the monthly meeting of department assistants with the academic affairs manager and staff, ensuring that departments feel engaged and supported as members of the development team.
The pilot program within the school of medicine has been successful. As a result, MIV preparation and electronic submission is required as the standard preparation and review method for all school of medicine merit and promotion actions for the 2006–2007 academic year. MIV is expanding to more schools and colleges on the UCD general campus as well.
Use of MIV has led to opportunities for fine tuning system performance tasks and utilities, and we anticipate that this will continue as its use expands and as new bugs are discovered. Enhancements will continue to be developed. Some of the enhancements planned for the immediate future include electronic document archiving, the ability to upload scanned letters from external referees, improved sorting and resequencing abilities, consolidation of some screens, and reduction of font size and print space in some sections.6 Long-term enhancements include incorporation of electronic voting by department faculty for merit and promotion reviews, search capabilities, input of letters from external referees directly into the system, uploading electronic teaching evaluations, and the development of new applications, such as public relations profiles of individual faculty members, for use by the school's and medical center's public affairs campaigns. MIV's performance and its downstream effects will continue to be monitored by the school of medicine, the campus pilot-development team, and the users group. We encourage other schools in their efforts to develop similar electronic databases and systems, and we recommend that they consider similar models for design, piloting, and monitoring.
The authors thank and acknowledge their partners in the UCD School of Veterinary Medicine and the College of Biological Sciences for their excellent contributions in the development of MIV, and the Campus Council for Information Technology for their support of MIV and their efforts in implementing MIV campus-wide.
2 Howell LP, Hogarth M, Anders TF. Creating a mission-based reporting system for an academic health center. Acad Med. 2002;77:130–138.
3 Howell LP, Hogarth M, Anders TF. Implementing a mission-based reporting system at an academic health center: a method for mission enhancement. Acad Med. 2003;78:645–651.
4 Howell LP, Green R, Anders TF. A mission-based reporting system applied to an academic pathology department. Hum Pathol. 2003;34:437–443.
© 2007 Association of American Medical Colleges
5 Anders TF, Hales R, Shahrokh N, Howell LP. Mission-based reporting in academic psychiatry. Acad Psychiatry. 2004;28:129–135.