An audit of institutional ethics committee queries raised after initial project submission by a single research department at a tertiary referral center in India

Abstract

Introduction: 

The institutional ethics committees (IECs) raise queries following protocol reviews. The quality of these queries would be a useful metric to assess how well the IEC executes its fundamental role of protecting participants.

Methods: 

Queries received after the initial review and replies sent by a single research department were evaluated. A content analysis was done to identify the domains and categories of queries. We categorized these queries as administrative, ethics related, and scientific. The impact of each query in improving the science or safeguarding the rights and safety of research participants (ethics) was evaluated by two authors of this manuscript: one affiliated and the other nonaffiliated to the institute. Kappa statistics were used to evaluate for agreement between the two.

Results: 

A total of 13 studies (investigator-initiated studies [IISs]: 7 and pharmaceutical industry-sponsored studies [PSSs]: 6) formed the final sample size for analysis. The total number of queries was 364 (IIS: 106 and PSS: 258; P < 0.001). With regard to the categories, we found n = 42 (11.54%) to be irrelevant at that stage of the review process; n = 51 (14.01%) were about information already available which the IEC had missed; n = 67 (18.41%) queries where the IEC needed paraphrasing; n = 50 (13.74%) were entirely relevant with the need for further clarification; and n = 154 (42.31%) had been missed by the investigator during the initial submission. The overall agreement between the affiliated and unaffiliated investigators was just 12.9% (P < 0.001).

Conclusions: 

We found that approximately 25% of the queries raised by the IEC were redundant. It is our opinion that this redundancy could have been channeled into greater focus on scientific and ethical aspects of the protocol. Ongoing dialog between investigators and ethics committees may help address this. Perspectives between the affiliated and the unaffiliated investigators with regard to the relevance of queries were grossly different.

INTRODUCTION

An institutional ethics committee (IEC) is a body that functions independently despite being associated with an institution.^[1] The main goal of an IEC is to safeguard the rights, safety, well-being, and dignity of the research participants.^[1,2] To do so, the fundamental responsibility of an IEC is a well-rounded risk–benefit analysis of each research proposal.^[2,3] This is ensured by the IEC by reviewing proposals submitted to them for the scientific merit and ethical considerations before the project can be initiated.^[4] The IECs also have the mandate of monitoring ongoing trials to ensure compliance with the approved protocol.^[4]

In India, the New Drugs and Clinical Trials rules of 2019 has made it mandatory that IECs reviewing pharmaceutical industry-sponsored studies or bioavailability/bioequivalence studies be registered with the Central Licensing Authority (CLA) while those IECs that review biomedical and health research, involving human participants be registered with the Department of Health Research^[1] Some IECs, in addition, also get themselves accredited with agencies such as the National Accreditation Board for Hospitals and Healthcare Providers, Strategic Initiative for Developing Capacity in Ethical Review, Association for the Accreditation of Human Research Protection Programs and similarly to strengthen their systems and capabilities.^[3]

The registration and accreditation processes primarily ensure that the systems and processes are in place but do not necessarily provide assurance on the quality, accuracy, and adequacy of the review process itself. The present study was, thus, envisaged with the primary objective of auditing queries raised by the IEC during the initial review of research proposals submitted by us as a metric to assess the quality of the review process. Our secondary objectives were to assess the extent to which the queries/comments were perceived by us to improve the science of the study, quality of the study as also safeguard the rights and safety of research participants. A tertiary objective was to assess the differences between the investigator-initiated studies (IISs) and pharmaceutical industry-sponsored studies (PSSs).

METHODS

Ethics

Our institution has three IECs (IEC-1 for regulatory studies and IEC-2 and IEC-3 for academic and other biomedical research) and the present study received an exemption from the review of IEC-3.

Selection criteria and time frame of the sampling

We included all studies (whether PSS or IIS and whether interventional or otherwise) submitted by us to any of the three IECs during a 2-year period (July 2017–June 2019). We excluded studies that were exempt from review such as meta-analyses, audits, nonbiomedical studies, and research on data available in the public domain. The documents that were audited include letters sent by the IEC to the investigator containing queries, the response letter sent by the investigators back to the IEC, and all subsequent communications until a final approval is obtained to conduct the study.

Processes followed by our institutional ethics committees

The procedure followed by all three committees is that following initial review, the secretariat conveys its comments and queries in writing to the principal investigator. These are sent in a structured format under the following heads: general queries, protocol queries, informed consent document queries, indemnity insurance policy document queries, clinical trial agreement queries, case record format queries, and miscellaneous queries.

Our classification of institutional ethics committees queries and comments into various domains

We classified all queries into three main domains: administrative, scientific, and ethics-related as described below. These domains were based on a previously published study.^[5]

Administrative

These included queries and comments on the clinical trial agreement, indemnity insurance policy, other site IEC approvals, and administrative and regulatory approvals.

Scientific

These included queries and comments related to methodological aspects of the study (e.g., related to the appropriateness of study design, comparator, eligibility criteria of participants, study site and outcome measures, sample size estimation, and statistical methods).

Ethics related

These included queries and comments related to participant safety and well-being, participant rights, maintenance of confidentiality, participant remuneration, research-related injury, risk-mitigation aspects, and informed consent process.

All the study investigators independently classified the queries into domains and categories. In case of discrepancy, a decision was taken based on consensus.

Qualitative synthesis of data

A content analysis^[6] was performed by two investigators (JPR and US) independently to confirm the domains and subclassify them into various subdomains. Discrepancies were resolved by the senior investigators (NJG, SBB, and UMT). Two investigators (NJG – author affiliated to the department [AAD] investigator and SBB – author unaffiliated to the department [AUD]) independently assessed each query for relevance under a three-category Likert scale (unlikely, neutral/cannot say, and likely). This Likert scale was also arrived at based on consensus. Both AAD and AUD were of the cadre of professors in the respective medical specialty (clinical pharmacology and pediatrics, respectively) with approximately 200 publications in national and international peer-reviewed journals. All five authors have more than 5 years of experience in clinical research.

In the case of scientific or ethics-related queries, relevance was assessed by answering the question, how likely is that specific query going to improve the science of the study or safeguard the rights and safety of participants respectively, from the perspective of the evaluator. In the case of administrative queries, the relevance was assessed by answering the question, how likely is the query going to impact the quality of the conduct of the study from the perspective of the evaluator.

Outcome measures

Our primary outcome measure was to classify the queries/comments under various domains, namely, administrative, scientific, or ethics-related (see below for definitions),^[5] and to assess the different categories of these queries/comments. The five categories included (a) not relevant at that stage of review process, (b) already available in the initial submission but was missed by IEC, (c) IEC requested for a simple paraphrasing of text, (d) not present in the initial submission submitted by the investigator, and (e) IEC required further clarification for better understanding and clarity. Our secondary objectives were to assess the extent to which the queries/comments were perceived to improve the science, safeguard the rights and safety of research participants, or have an impact on the quality of the conduct of the study, and to assess for differences between the IIS and PSS with regard to the primary objective.

Data management and statistical analysis plan

Data entry was done using Microsoft Excel (Publisher: Microsoft Corporation, Redmond, Washington, USA, 2016), and Statistical Package for the Social Sciences (SPSS) for Windows, version 20.0 (Publisher: IBM, Armonk, New York, USA, 2011) was used for data analysis. Descriptive statistics and inferential statistics were used. Demographic data, the domains, and the categories of queries were summarized using frequency and percentages. Comparisons between IIS and PSS studies were done using the Chi-squared test and post hoc Beasley’s technique with Bonferroni’s correction for discrete data or Mann–Whitney U-test for continuous data. Cohen’s kappa was used to assess for agreement between AAD and AUD’s perspective. Statistical significance was set at P < 0.05.

RESULTS

Demographics

Our department submitted 19 research projects to the IEC for review over the 2-year period. Six studies were exempt from review and hence were excluded from further analysis. Of the 13 eligible studies, 6/13 (46.2%) were PSS and the rest (7/13 [53.8%]) were IIS. The total number of queries/comments in the selected studies was 364 with 258 (70.9%) of them from PSS and 106 (29.1%) from IIS. The median (interquartile range) of queries per a study in the PSS and IIS was 43.00 (29.25, 55.25) and 15.00 (11.00, 19.00), respectively, and this difference was statistically significant (P < 0.001). All the PSS were regulatory studies and none of the IIS was regulatory in nature.

Analysis of domains

Of the 364 queries, 126 (34.62%) were of administrative domain, 138 (37.91%) were ethics-related, and 100 (27.47%) were of scientific domain. The coding frame of the content analysis is given in Table 1. There was a significant difference in the number of comments per domain between IIS and PSS (Pearson’s χ² = 35.331; P = 0.000000021275), the details of which along with the results of the post hoc analysis are tabulated in Table 2. The ethics-related queries (P = 0.0007) were significantly more in the PSS, whereas scientific queries (P < 0.0001) were more in the IIS.

Table 1: Domains and subdomains of queries and comments - Coding frame from the content analysis

Table 2: Proportion of queries in different domains of investigator-initiated studies and pharmaceutical industry-sponsored studies

Categories

With regard to the categories of queries, 42 (11.54%) were not relevant at that stage of submission; 51 (14.01%) were about information already available (submitted at the time of initial submission) but IEC had missed; 67 (18.41%) IEC requested paraphrasing; 154 (42.31%) not submitted in the initial submission but missed by the investigator; and 50 (13.74%) were relevant with the need for further clarification. There was a significant difference in the number of various categories of queries between IIS and PSS (Pearson’s χ² = 56.001; P < 0.00000001), the details of which along with the post hoc analysis are tabulated in Table 3. Queries that were not relevant (P = 0.0030) and those due to IEC missing information (P = 0.0003) were higher in PSS, while queries due to investigator missing to provide information to the IEC during initial submission (P < 0.0001) were higher in IIS.

Table 3: Proportion of queries of different categories in investigator-initiated studies and pharmaceutical industry-sponsored studies

Investigators’ perspectives

The AUD evaluated 221 out of 364 (60.71%) queries to be unlikely to have any relevance, whereas the AAD classified 85 out of 364 (23.35%) in the same category. Favorable and neutral stands were taken for 129 (35.44%) and 19 (5.22%) queries/comments by the AAD, whereas the AUD classified 145 (39.84%) and 134 (36.81) queries/comments as favorable and neutral, respectively. The details of domain-wise perspective evaluation and Cohen’s kappa are depicted in Table 4. The overall agreement was 0.129 (P < 0.001).

Table 4: Investigators’ perspectives on queries’ impact on the study conduct

DISCUSSION

This audit was conducted in a tertiary care referral center to ascertain the domains of the comments/queries raised by the IEC and how many of these comments/queries actually translate to safeguard the rights and safety of research participants. We report that there were significantly greater numbers of queries in the PSS when compared to the IIS. Scientific queries were more in the IIS, whereas ethics-related queries were more in the PSS. The overall agreement about the nature of queries, between the AAD and AUD, was significantly low (0.029, P < 0.001).

The quality assurance process of the IECs in the USA is very well established through frequent inspections conducted by the United States Food and Drug Administration since 1980.^[7] Over 1000 IECs have been registered with the CLA since it was mandated in 2013. Given the sheer magnitude of work, the inspections by the CLA have not been quite common.^[8] In general, the purpose of auditing IECs is to appraise their members on how their functioning translates into an assurance that the ethical review of research proposals is carried out as per the established standards, thereby protecting the rights and safety of the research participants.^[9] Thus, the authors believed that this audit would provide our IECs with an opportunity to identify shortcomings and improve their functioning that translates into better participant protection and safety.

The mean number of queries in the PSS was significantly more when compared to the IIS, and comparatively, there were more ethics-related queries and not much of scientific queries in the PSS. This is probably because the PSS, unlike the IIS, are usually prepared for regulatory submissions and are often approved by the CLA and/or other sites’ IECs at the point of our IEC submission. Further, there are dedicated and experienced protocol development teams in the industry that oversee protocol development and writing. Therefore, the IECs do not find errors in science. However, the IECs are vested with the responsibility to review for specific local issues that are unique to the site and are predominantly ethics-related,^[9] thus explaining the reason for more ethics-related queries in PSS as seen in our study. This is also the reason for a greater number of queries in the PSS as protocols are prepared in a generalized way that suits most centers and do not cover all local site-specific issues. Second, PSS have more complex issues such as posttrial access, arrangements of compensations, transport of biological samples outside the institution and/or outside the country, and status of regulatory approvals,^[10] which is usually not the case in IIS. On the other hand, IIS are largely single-center studies usually nonregulatory in nature, with comparably less complexities and site-specific issues already addressed.

We found that many queries in PSS were related to information/documents that had already been submitted to the IEC. This is probably because most PSS submissions are voluminous as they are usually regulatory in nature. Another possible reason could be the lack of dedicated full-time staff at most IECs in India,^[11] who review these proposals, thereby potentially missing out on already available information. Many times, IEC staff are contractual with higher attrition rates who may not be vested in the review process. On the contrary IIS, submissions had more queries related to mandatory information that were not available in the initial submission. One possible reason could be that the IEC dossiers of IIS submissions are mostly prepared by young researchers or postgraduate students (in the case of a thesis) unlike PSS submissions which are usually prepared by dedicated professionals who work in teams. New researchers or students who get admitted to academic institutions like ours usually do not undergo formal training on the prerequisites of an IEC submission and they usually learn “on the job.” Another reason could be that research often takes a back seat to clinical practice in a hospital like ours and there is no dedicated research secretariat.

An audit conducted at our institution previously had reported that there was a total of N = 1387 studies during the time period January 2011 to August 2014 of which 818 (59%) were dissertations of postgraduate students and 406 (29%) were IIS contributing to the major bulk of the proposals being reviewed by IEC.^[12] Thus, if formal training is conducted periodically, a lot of effort and time may be saved by both the researchers and the IEC as the number of queries about mandatory information/documents to be submitted would drastically fall. The IEC could also provide templates for protocols for different types of studies and consent and assent forms at its website for the guidance of young researchers. In addition, better supervision of the dissertation protocols by guides may also help in minimizing the queries raised by the IEC. This would, in turn, decrease the resource crunch faced by the IECs and would also comparatively increase the efficiency of their working with the available manpower.

With regard to the two investigator’s perspectives on the relevance of the queries, we report that there was more of a disagreement rather than agreement. Both investigators perceived that many ethics-related queries would have had a positive impact. However, the science and administrative queries saw a major difference in opinions. These differences in perceptions could potentially be attributed to the differences in their roles they play in clinical research as a stake holder and their differential training (one investigator was a clinical pharmacologist and the other a pediatrician).^[13] This finding indeed validates the need for the presence of IEC members from various backgrounds – both scientific and nonscientific, the importance of quorum and consensus-based decisions.

The strength of our study was that the perspectives of an in-house investigator and an independent investigator were compared. Our study does have a few limitations. It was an audit of IECs within one institution which follows the same SOPs. Thus, findings from this study may not reflect the status of other IECs in the country. Second, the time period chosen was of 2-year which corresponds to just one term of the IEC members. There may be differences in the findings with changes in members from time to time. Furthermore, the three senior investigators of this study were members of IECs either within or outside the institution which was a potential conflict of interest.

CONCLUSIONS

The IECs serve the fundamental purpose of safeguarding the rights, safety, and well-being of the research participants. This is borne out by the greater number of ethics-related queries seen in the PSS and higher number of scientific queries in IIS. However, approximately 25% of the queries raised by the IEC were redundant and this could have been channeled into greater focus on scientific and ethical aspects of the protocol. The authors believe that the appointment of full-time staff in IECs would go a long way in improving the quality of review. Furthermore, formal training of young academic investigators or those newly employed on IECs either as members or in an administrative capacity will minimize investigator side errors in submitting all mandatory documents as an investigator vested in the study has a totally different perspective from that of the IEC members. Ongoing dialog between Investigators and Ethics Committees is necessary to serve the final goal of participant protection.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.