Secondary Logo

Journal Logo

Special Theme: Cultural Competence: RESEARCH REPORTS

The Cost of Operating Institutional Review Boards (IRBs)

Wagner, Todd H. PhD; Bhandari, Aman MPH; Chadwick, Gary L. PharmD, MPH, CIP; Nelson, Daniel K. MS, CIP

Author Information
  • Free

Abstract

Recent reports claim that institutional review boards (IRBs) are underfunded,1,2 yet little is known about the costs of operating IRBs.3 With IRB-related problems and the desire to increase support on the rise, a better understanding of their costs is warranted.

The last study to estimate the costs of operating an IRB was published in 1979. Brown et al.,4 using administrative data from their medical center, estimated that operating their IRB cost approximately $100 per protocol for a total of 850 protocols. Adjusting this estimate for inflation yields a current per protocol cost of approximately $270. However, inflating these estimates may not be completely appropriate because the complexity of study protocols has changed dramatically with the advancement of genetic and biomedical research.1,2,5 Also, Brown et al.'s estimate was based on a single medical center, and they could not account for variation across medical centers.

Although this paper focuses on the costs of operating IRBs, there is growing recognition that the IRB is only one part of an effective human research protection program (HRPP). The emergence of HRPPs reflects a dramatic change in organizational culture. HRPPs are meant to be more encompassing than IRBs, and they usually involve additional activities to support research, such as investigator education, monitoring, and audit programs. HRPPs are outside the scope of this paper, but future research should consider their costs and benefits.

The primary goal of this study was to estimate the costs of high-volume and low-volume IRBs. After a sensitivity analysis, we ran three subsequent analyses with the cost estimates. First, we evaluated whether IRB size was related to economic efficiency (also known as economies of scale). Second, on the premise that overworking staff may lead to decreases in quality, we evaluated the effect of staffs' workload limits on costs. Like setting a benchmark, this involved limiting the number of actions a staff member could handle per year and re-estimating IRB costs. Throughout the report, we refer to these as the benchmark estimates. Third, because most IRBs receive institutional support, often from grant facility and administration (F&A) revenues, we compared IRB costs with direct grant revenues to identify the degree to which F&A funds cover IRB costs.

METHOD

Cost-estimation Framework

Under the federal regulations, IRBs provide services that include reviewing study protocols and consent forms. They also monitor ongoing studies, review adverse-event reports, and handle a variety of other research-related materials. Individually, these services are often referred to as actions. IRBs vary widely in the numbers of actions they process per year. High-volume IRBs handle thousands of actions per year, while low-volume IRBs typically handle fewer than 125. Although we expect high-volume IRBs to be more expensive than low-volume IRBs because they use more resources, their costs remain largely a mystery.

To measure the cost of operating an IRB, we used information from four inputs: (1) personnel; (2) space; (3) computers, databases, equipment and supplies; and (4) training and education. Personnel included the IRB administrator, support staff, the chair of the review committee, and review committee members. We excluded any institutional officials who did not have operational responsibilities for the IRB (e.g., deans and presidents). Our study also excluded the costs of operating biohazard or animal-subjects compliance programs, which are related functions at some institutions.

All cost data were for the year 2001. As described below, we used the salaries for biomedical professionals for our estimates. Given the focus of the Bell Report data (see below) and the biomedical salary estimates, the results probably do not generalize to non-biomedical IRBs, such as behavioral IRBs at liberal arts colleges, or to independent, centralized IRBs. Further methodologic details are available on the Internet.6

Background of the Bell Report

Our estimation of the costs of operating IRBs draws largely from a report prepared by James Bell and Associates for the Office for Protection from Research Risks (now known as the Office for Human Research Protections).7 The report, hereafter referred to as the Bell Report, targeted all IRBs operating under a Multiple Project Assurance (MPA). The Bell Report compared high-volume IRBs, defined as the top decile, with low-volume IRBs, defined as the lowest decile. The report provided insufficient information to assess the costs of medium-volume IRBs.

The main goal of the Bell Report, was “… to greatly improve the systematic knowledge of the universe of active IRBs operating under MPAs.”7 The Bell Report was based on surveys sent to IRB administrators, institutional officials, IRB committee chairs, IRB committee members, and investigators. The sample excluded IRBs that reviewed fewer than ten initial protocols per year and IRBs that were regulated solely by the FDA. According to the Bell Report, response rates for committee chairs and IRB administrators were 80% or higher.7 The report also acknowledges that the “… validity of the findings contained herein rests on the truthfulness of the respondents, and on the accuracy of their estimates and factual reports.”7

Personnel Costs

A distinction between the IRB committee and the institutional office that oversees the IRB committee is rarely made because, except for very small operations, all IRB committees have some sort of support. Federal regulations technically refer to IRBs as the protocol review committee (i.e., chair and members). Our use of the term IRB in this paper includes the office that supports and oversees the committee's work, a perspective that is consistent with the Bell Report.

IRBs have a number of methods of coping with increasing workload so that protocols are reviewed in a timely manner. An IRB can either add committees or have the committees meet more frequently or for longer periods. According to the Bell Report,7 low-volume IRBs in 1995 had approximately seven committee meetings per year, with an average meeting time of 105 minutes, and high-volume IRBs had an average of 12 committee meetings per year and an average meeting time of 145 minutes. The Bell Report also noted the average times for reviewing protocols and adverse-event reports at high-volume and low-volume IRBs.7 From this information, we estimated the numbers of actions (i.e., initial reviews, continuing reviews, amendments, and adverse-event reports) conducted per committee meeting and calculated that a low-volume institution would need one IRB committee to process the actions that arise in a typical year, whereas a high-volume institution would need four IRB committees.

In the Bell Report, the typical high-volume IRB had a doctoral-level administrator. Although 85% of the high-volume institutions had two full-time equivalents (FTEs) for support staff (i.e., an IRB coordinator), 15% of the high-volume IRBs had no support staff.7 The Bell Report did not mention the use of database analysts or administrative assistants. Given current trends to have large computerized databases, we assumed that high-volume IRBs had a full-time database analyst and a full-time administrative assistant. In the model, we assumed that the IRB covered 50% of the IRB committee chair's salary. IRB committee members averaged 108 hours per year (5% FTE), based on the Bell Report. We assumed that the wage rates were the best indicator of opportunity costs, or the value of a resource in its next best use.8 Therefore, if a committee chair volunteered two hours, we treated it as the equivalent of two hours of paid work.

According to the Bell Report,7 low-volume IRBs had considerably more variability in staffing. All had an administrator; 28% of the administrators had a bachelor's degree and the remainder had more training. Sixty percent of the low-volume IRBs had no support staff,7 and we assumed that none had a database analyst or an administrative assistant. According to the Bell Report, committee members worked 19 hours per year (0.9% FTE), on average.

Salary estimates for administrative and research personnel were obtained from a human resources company that tracks salaries9; benefits were included at a rate of 28%. In keeping with the Bell Report, we used national salary averages for biomedical professionals.

Having multiple IRB committees has advantages, namely more options and flexibility when trying to staff the IRBs. Although not reported in the Bell Report, many high-volume institutions now use teams of committee chairs and/or vice chairs to spread the workload and to save costs. For the base calculations on high-volume IRBs, we assumed that each of the four IRB committees used a separate chair. This is the most costly arrangement, so we also calculated the potential savings from using vice chairs with lesser time commitments in the sensitivity analysis.

Space Costs

Although many IRBs are not required to pay for space, there are implicit space costs. Space not only requires upkeep, but it is an asset that could be rented to someone else, generating revenues.8 We used rental rates to approximate space costs.10 Office space was apportioned to FTE employees, with slightly more space for the IRB administrator than to the other office employees. Our estimate of office space also included rooms for filing protocols and for having IRB committee meetings. We assumed that the IRB meeting room was shared with other departments, as is typically done to defray some of the cost. The annual rental cost per square foot was calculated at $34.71, which was the weighted average rental rate for metropolitan areas in June 2001.11,12 Given that space is often not itemized on an IRB's budget, we also present IRB costs minus space to facilitate interpretation.

Supplies and Training and Education Costs

To estimate the costs of supplies and training and education, we used year-2000 data from the University of North Carolina at Chapel Hill, inflated to 2001 dollars using the general Consumer Price Index.

Supplies

Supplies, including faxes, phones, paper, and copying, cost approximately $8.33 per action. To obtain the total cost of supplies, we multiplied this average cost estimate by the number of actions. This method assumes that supply costs vary proportionately with the number of actions. We also assumed that there was one computer per FTE and that large IRBs also have a local area network. Each computer was valued at $2,000 with a five-year life expectancy and the network was estimated at $5,000 per year for support and maintenance.

Education and training

We estimated that education and training for each committee chair cost $1,000. Education and training for staff members was estimated at $1,000 per staff FTE per year. These cost estimates were based on the experiences of two of the authors' (GC and DN). Training costs included sending IRB members and staff to conferences, purchasing journals and newsletters, and having in-house educational sessions.

Sensitivity Analysis

After estimating IRB costs, a sensitivity analysis was performed to test the robustness of the results.8 For the sensitivity analysis, we systematically varied the four inputs (personnel costs, space costs, supplies, and training and education costs). For personnel costs, we varied the salary rates ±20%, reflecting regional extremes in wages. In our main analysis, we used $34.71 as the rental cost per square foot per year. In addition, our main analysis also presented the total costs excluding space because many administrators do not pay for it. Although that provides some sensitivity information, we further varied space costs ±$15. We adjusted the cost of supplies per action ±$5. Finally, we adjusted the training and education costs by −$500 and +$4,000. Each input was varied across these ranges while holding the other inputs constant.

Economies of Scale

As we describe above, IRBs use personnel and other inputs in the process of producing a service (an output). The relationship between inputs and outputs depends on economic efficiency. A more efficient IRB could produce more services with the same inputs or it could produce the same services with less inputs. One way of comparing the economic efficiency of high-volume and low-volume IRBs is to compare their unit costs of production. To do this, we used the average cost per action (total costs/total actions). Actions, as mentioned earlier, reflect the amount of initial reviews (full, expedited, and exempt), continuing reviews, amendments, and adverse-event reports. According to the Bell Report, in 1995 the typical high-volume IRB processed 2,780 actions (794 initial reviews, 879 continuations, 472 amendments, and 635 adverse-event reports), and the average low-volume IRB processed 96 actions (32 initial reviews, 36 continuations, 19 amendments, and nine adverse-event reports).7

Benchmark Estimates

Government reports and many experts have indicated that IRBs are understaffed.1,2 The current conventional wisdom is that a full-time professional staff can handle approximately 300–350 actions per year. Although this benchmark often surfaces at national meetings, and has been mentioned by Sugarman,13 we have not been able to find any supporting empirical data. Keeping in mind this limitation, we used this staffing benchmark and recalculated IRB costs. We refer to these as the benchmark estimates.

For the benchmark estimates, we included a 5% reimbursement for each IRB committee member because IRB service goes above and beyond the commitment required for a typical committee, and some institutions are beginning to recognize that service with salary support. Because staffing benchmarks necessitated more personnel, we increased the space and training costs proportionately.

IRB Costs and Direct Grant Revenues

Although some IRBs charge initial review fees ranging from $1,000 to $2,500, most academic and research institutions use internal funds to support their IRBs. When an institution receives federal grant support, it also receives F&A funds that support many administrative activities, including the IRB. We compared IRB costs with grant revenues, which yielded the percentage of F&A funds required to support the IRB.

Unfortunately, the Bell Report7 has no information about grant revenues at the different institutions. For this comparison, we used a Department of Veterans Affairs (VA) database that tracks research studies involving human subjects as well as all grant revenues (VA- and non–VA-funded) at each medical center. We re-estimated IRB costs for the VA medical centers. IRB committees at high-volume VA medical centers reviewed approximately 1,792 actions per year. We did not calculate the cost at low-volume VA medical centers because we could not differentiate between the low-volume sites that did and did not have an IRB. Instead, we calculated the costs for medium-volume VA medical centers (439 actions per year).

RESULTS

Following staffing patterns reported in the Bell Report, the estimated cost for operating a high-volume IRB was $770,674 (see Table 1). The estimated cost for a low-volume IRB was $76,626. When space costs were subtracted, the costs of high-volume and low-volume IRBs were $733,673 and $63,644, respectively.

T1-19
Table 1:
Cost and Benchmark Estimates for a High-volume and a Low-volume IRB Review Based on Bell Report7 and Expenditure Data

Sensitivity Analysis

The sensitivity analysis showed more variability in low-volume IRBs than in high-volume IRBs. Small changes in the costs of inputs had a larger effect on low-volume IRBs (see Figure 1).

F1-19
Figure 1:
Average costs per action for high- and low-volume IRB reviews based on Bell Report7 and expenditure data; note: bars represent the base case, whiskers represent the extreme bounds from the sensitivity analysis.

In the baseline model, we assumed that high-volume IRBs had four committees and used multiple chairs for each committee. If high-volume IRBs used chairs and vice chairs, which is often done, this would yield greater savings and would create a larger discrepancy between the high-volume and low-volume IRBs. For instance, if a high-volume IRB used two chairs (0.5 FTE) and two vice chairs (0.3 FTE) instead of four chairs, the total savings would be approximately $59,280 (8%). These savings cannot be realized in low-volume IRBs where there is only one chair.

Economies of Scale

The average cost per action at a high-volume IRB was $277 ($264 per action without space costs). At low-volume IRBs, the average cost per action was $799 ($663 without space costs). This suggests that high-volume IRBs are more economically efficient than are low-volume IRBs. Therefore, a high-volume IRB could do the work of a low-volume IRB with fewer resources than are used by the low-volume IRB. We could not tell whether this difference was statistically significant because we have only two cost estimates and no measure of variance. However, the sensitivity analysis suggests that even if we vary the input parameters, the estimated costs for high-volume and low-volume IRBs do not overlap. This can be seen in Figure 1, where the bars represent the baseline costs and the whiskers are the extreme bounds from the sensitivity analysis.

Benchmark Estimates

Current thinking is that a full-time professional staff can handle approximately 300–350 actions per year, although we cannot find any supporting empirical data. If IRBs had to maintain this benchmark, a high-volume IRB would require eight FTE committee support staff to handle 2,780 actions. In addition to these eight committee support staff, we assumed the IRB would also need a full-time administrator, a full-time administrative assistant, and a database analyst. Assuming that an IRB committee uses a chair and nine members, we estimated that it would take four IRB committees meeting monthly to process all of the protocol reviews. Each IRB committee chair, assumed to be a physician, was allotted a 0.5 FTE and each committee member was assigned a 0.05 FTE. In this scenario, the estimated cost of operating a high-volume IRB was $1,227,802, or a 59% increase from the baseline estimate.

With the staff benchmark, a low-volume IRB would need a half-time administrator and a half-time committee support staff member to process 96 actions per year. An administrative assistant and a database analyst are probably not needed for the workload, so we did not include these people. We recognize, however, that this may slow the IRB's work and affect quality. The single committee in this model uses a chair and nine members. The IRB committee chair, assumed to be a physician, was allotted a 0.25 FTE and committee members were calculated at a 0.02 FTE. The total estimated operating cost for the low-volume IRB was $117,653, or a 54% increase from the baseline estimate.

It is interesting to note that the economies of scale persisted among benchmark costs. The average costs per action for high-volume and low-volume IRBs were $442 and $1,226, respectively.

IRB Costs and Direct Grant Revenues

We compared IRB costs with direct grant funds. Because our grant data were for the VA, we estimated these costs for VA medical centers only. At high-volume VA medical centers, which reviewed approximately 1,792 actions per year, IRB costs represented 5.3% of all VA and non-VA grant revenues. At medium-volume VA medical centers (439 actions per year), IRB costs represented approximately 16.3% of all grant revenues. As we noted above, we did not calculate the IRB cost at low-volume VA medical centers because we could not differentiate between the low-volume sites that did and did not have an IRB.

DISCUSSION

In absolute dollars, high-volume IRBs cost substantially more than low-volume IRBs. However, the average cost per action at high-volume IRBs was approximately one third that of the low-volume IRBs ($277 versus $799, respectively). The data suggest that high-volume IRBs are more economically efficient than low-volume IRBs, indicating that a high-volume IRB could do the work of a low-volume IRB with fewer resources. There are two factors that underlie this relative efficiency. First, according to the Bell Report,7 high-volume IRBs handle more actions per FTE than do low-volume IRBs. Second, our cost model also shows that high-volume IRBs are better able to spread fixed costs across the workload.

With the staffing benchmark, the cost of operating IRBs increased substantially (54–59%). Although more research on staff workload benchmarks is needed, this discrepancy raises concerns about whether IRBs are underfunded.

Perhaps it is not surprising to report that IRBs are underfunded because these results were based on data collected in 1995. Over the past few years, many institutions have reorganized their IRB and/or compliance programs. Many of these changes have been motivated by federal scrutiny and by increased sanctions. In cases where the institution was cited and human subjects research was halted, dramatic changes have been made, often involving large commitments of financial resources. These changes are not reflected in the Bell Report, so the model here may underestimate the true costs of operation.

To obtain additional funds, there are different mechanisms available to IRBs. The first, and perhaps easiest, way is for the IRB to request more funds from its parent institution. In turn, the institution will try to share these costs with granting agencies. Our simulation with VA medical centers suggests that running a high-volume VA IRB would require 5.3% of the F&A funds. Operating a medium-volume IRB would require 16.3% of the F&A funds. Although we were not able to calculate the percentage of grant revenues needed to operate a low-volume IRB, we expect that the percentage would be even higher. Given that many of the medium-volume and small IRBs are located at hospitals, appropriating more funds to run the IRB could affect the hospital's ability to provide patient care, unless it can recoup those costs from funding agencies, which is not easy. F&A rates are negotiated, many types of grants provide very limited support, and educational institutions face a 26% maximum for administrative expenses. Therefore, one solution would be to have IRBs charge a review fee and for grants with human subjects to treat IRB review fees as a direct cost (i.e., a line item on the budget). This issue was raised in 197914 and again recently by the National Bioethics Advisory Commission.15 The potential costs and benefits of such a system warrant further exploration.

As IRBs look for ways to save money, these data suggest that cooperation between institutions or merging small IRBs may lead to savings through greater economies of scale. Even at the extreme bounds of the sensitivity analysis, we found that high-volume IRBs were more efficient than were low-volume IRBs. Another interpretation of our data is that sites conducting only small numbers of human-subjects studies per year may wish to contract with an IRB at another organization or use an independent centralized IRB. However, our study is limited in that we were not able to statistically account for differences in protocol complexity. Some IRBs handle a wide range of technically complex protocols. These IRBs may incur more costs because their reviewing protocols may take more time and staff may need additional training. In addition, we were not able to control for differences in IRB review quality, which may also be associated with IRB costs. One concern, echoed by popular opinion, is that large IRBs are less expensive because they cut corners. This issue was raised in a 1998 Office of the Inspector General report2 that stated, “They [IRBs] review too much, too quickly, with too little expertise. This is especially apparent in many of the larger institutions” [emphasis added]. However, it is not clear that high-volume institutions save money by sacrificing quality. IRB reviews at larger institutions may be of equal quality but more efficient. It could be argued that being too small reduces efficiency because the IRB fails to develop expertise and does not get enough practice in all types of reviews. To address these limitations, we have begun a study that involves surveying all IRBs affiliated with the VA.

The sensitivity analysis confirmed that large changes in the cost of education and training had little effect on the costs of IRBs. This is very important because such investments may greatly improve IRB review quality, yet these costs are often the first to be cut in budget negotiations. Salaries may increase as personnel become trained and specialized. This will have a slightly larger impact on high-volume IRBs because 90% of their budgets cover personnel costs. Although low-volume IRBs may be more insulated from salary effects, their budgets are sensitive to the costs of space. A 75% increase in rental rates would increase the cost per action 13% among low-volume IRBs, versus a 3% increase for high-volume IRBs.

Although adequate resources are an important component in enhancing protections for human subjects, more resources do not necessarily guarantee a better IRB. That will require, in addition to resources, better education and training for IRB staff and committee chairs and members, ongoing quality improvement efforts, and a cultural change that emphasizes research ethics at the institutional level. These efforts are now beginning, and many organizations, including the Institute of Medicine, the Association of American Medical Colleges, and the National Bioethics Advisory Commission, are formulating recommendations for change. But, more research is needed to guide these changes and to ensure that they will be beneficial.

REFERENCES

1. Woodward B. Challenges to human subjects protections in U.S. medical research. JAMA. 1999;282:1947–52.
2. Institutional Review Boards: A Time for Reform. Rockville, MD: Office of the Inspector General, Department of Health and Human Services, 1998. OEI-01-97-00193.
3. Sugarman J. The role of institutional support in protecting human research subjects. Acad Med. 2000;75:687–92.
4. Brown JH, Schoenfeld LS, Allan PW. The costs of an institutional review board. J Med Educ. 1979;54:294–9.
5. Grob G. Scientific research: continued vigilance critical to protecting human subjects. Washington, DC: General Accounting Office; 1996. GAO/HEHS-96-72.
6. Wagner TH, Bhandari A, Chadwick GL, Nelson DK. Technical Appendix for the Cost of Operating Institutional Review Boards. 〈http://www.herc.research.med.va.gov/IRB%20technical%20appendix.pdf〉. Accessed March 3/3/03. VA Health Economics Resource Center, Menlo Park, CA, 2002.
7. James Bell & Associates. Evaluation of NIH implementation of Section 491 of the Public Health Service Act, mandating a program of protection for research subjects. 〈http://ohrp.osophs.dhhs.gov/hsp_report/hsp_final_rpt.pdf〉. Accessed 3/3/03. Office of Human Research Protections, Arlington, VA, 1998.
8. Gold MR, Siegel JE, Russell LB, Weinstein MC (eds). Cost–effectiveness in health and medicine. Oxford, U.K.: Oxford University Press, 1996.
9. 〈http://www.salary.com/〉. Accessed 3/3/03. Salary.com, Inc. 2003.
10. Rosenheck R, Frisman L, Neale M. Estimating the capital component of mental health care costs in the public sector. Administration and Policy in Mental Health. 1994;21:493–509.
11. ONCOR International. 〈http://www.oncorintl.com/framset_5.html〉. Last accessed October 11, 2000. ONCOR International, Chicago, IL, 2000.
12. 〈http://www.reis.com/〉. Accessed 3/3/03. Reis, Inc., New York, 2003.
13. Sugarman J. The role of institutional support in protecting human research subjects. Acad Med. 2000;75:687–92.
14. National Commission for the Protection of Human Subjects of Biomedical Research. Report and Recommendations: Institutional Review Boards. Washington, DC: DHEW; 1978.
15. National Bioethics Advisory Commission. Ethical and policy issues in research involving human participants. Bethesda, MD, December 19, 2000.
© 2003 Association of American Medical Colleges