Abstract
This study analyzes the Institutional Review Board (IRB) quality and efficiency at a leading hospital in Central Southern China, under the first three years of a Human Research Protection Program (HRPP). We conducted a descriptive, retrospective analysis from 2015 through 2017. We extracted characteristics from the protocol archive in duplicate. Of 396 protocols submitted, 98% were approved. Mean protocol review time decreased from 23 to 15 calendar days, 344 serious adverse events SAEs were reviewed, and 93 conflicts of interest (COIs) were disclosed. IRB quality and efficiency mechanisms improved. Remaining needs include increased monitoring of COIs and unanticipated problem involving risks to subjects or others, distinctions between research types, and cooperation with specialized committees.
Keywords
Introduction
Clinical research activity involving human subjects is increasing in developing countries (Atallah et al., 2018; Normile, 2008). In today's research environment, Institutional Review Boards (IRBs) are fundamental to the protection of human research participants (Rwabihama et al., 2010). IRBs maintain the ethical integrity of research by reviewing research protocols to ensure that research participants receive safe and ethical treatment, that participants are provided with informed consent, and that the potential for conflicts of interest is minimized (Coleman & Bouësseau, 2008).
Standard guidelines for research ethics involving human subjects were largely determined by international consensus following World War II, including the Nuremberg Code in 1947, the Declaration of Helsinki in 1964, and more recently, the Good Clinical Practice (GCP) guidelines developed by the International Conference on Harmonisation (ICH). In the decades since, many countries have used international guidelines as a starting point for constructing national research ethics systems, including IRBs to enforce research ethics review (Famenka, 2016). Despite an increasing number of countries following international research ethics guidelines, IRBs globally still face numerous obstacles in achieving their goal of improving the protection of research participants (Famenka, 2016).
In China, where biomedical research capacity and activity is growing rapidly, university-affiliated hospitals began to establish IRBs in the 1990s (Zhang et al., 2014). Since then, an increasing number of regulations governing the research ethics review for various types of research have been developed by regulatory structures, including the National Health Commission (NHC) and National Medical Products Administration (NMPA). Although not enforceable by law, such regulations and regulatory structures have moved research ethics review standards in China toward increasing consistency with international guidelines—with the China Food and Drug Administration (CFDA) (now NMPA) officially joining International Conference on Harmonisation-Good Clinical Practice (ICH-GCP) in 2017 (ICH Official Website, 2017). NMPA and NHC also emphasize the responsibility of research institutions to ensure that IRBs operate independently from external pressures (National Health Commission of the People’s Republic of China, 2016; National Medical Products Administration & National Health Commission of the People's Republic of China 2020a). However, some research suggests that IRBs in China are generally not yet operating at a sufficiently consistent, independent level of review, facing issues that include limited legislation, insufficient supervision or quality assessment, and lack of relevant training (Wang et al., 2019).
Revisions to national law in 2020 marked a significant stride toward standardization of ethics review in China, establishing at the legal level for the first time that all research must be reviewed and approved by IRBs (Common Law, 2020; Law of the People’s Republic of China on Basic Medical and Health Care and Health Promotion, 2019; Measures for the Administration of Drug Registration, 2020). In addition to policy change, another recent move toward more deeply rooted, enforced research ethics review standards has come in the form of Human Research Protection Programs (HRPPs), with eight institutions in mainland China obtaining accreditation from the Association for the Accreditation of HRPPs (AAHRPP) between 2011 and 2017 (Association for the Accreditation of Human Research Protection Programs, 2020a; Applied Clinical Trials, 2016; PR Newswire, 2013; PR Newswire, 2016).
Research indicates that establishing systematic HRPPs can improve the quality of IRBs (Kennedy et al., 2006). HRPPs have a broader range of responsibilities for IRBs that go beyond protocol review, including quality and efficiency assessments, continuous improvement, training, and handling unanticipated problems involving risks to participants or others (UPIRSOs) and conflicts of interest (COIs) (Association for the Accreditation of Human Research Protection Programs, Inc., 2018). In the late 1990s and early 2000s in the United States, after several federally supported research programs at academic institutions were suspended due to noncompliance with research on human subjects, many institutions applied for voluntary accreditation of HRPPs, among other reforms, to assess and improve the quality of institutional IRBs and overall HRPPs (Tsan et al., 2010).
In 2015, as the third institution in China to receive AAHRPP accreditation, a hospital in Central Southern China established an HRPP following a 3-year accreditation process. The accreditation process involved familiarization with international regulations on the protection of research subjects, identification of gaps between the hospital's practices and international regulations, integration of international regulations with Chinese regulations, and strengthening collaboration between departments for ethical review (Burt et al., 2014). During its first three years, the IRB also launched an online review system and developed new practices for handling UPIRSOs and COIs. The objective of this review is to describe this IRB's first three years of experience as the core of the new HRPP, including protocols submitted for approval and the review process itself.
Method
Data Collection
Performance evaluation can be an effective tool in improving the quality of HRPPs, as demonstrated by evaluations published by institutions, such as the U.S. Department of Veterans Affairs (VA) (Tsan & Nguyen, 2017; Tsan & Nguyen 2018). But the HRPP Quality Indicators published by the VA include several VA-specific requirements, which are not applicable to non-VA institutions’ HRPPs (Tsan & Tsan, 2015). Since there are not yet uniform international and national standards for external evaluation of IRBs, self-assessment has been put forward as a suitable approach for intermediary analysis of IRB quality and efficiency, particularly in low- and middle-income countries (Silverman et al., 2015).
Our study was conducted as a self-assessment by IRB members at the study hospital, employing a descriptive, retrospective, and quantitative approach. The analysis focused on data found in the IRB's paper files and electronic database between January 1, 2015 and December 31, 2017. Specifically, we examined the types of research protocols submitted and the departments from which protocols were submitted, to describe the general activity of the IRB and define areas in which future IRB training is most needed. We also examined the mean protocol review time to evaluate the IRB's efficiency. Lastly, we examined serious adverse events (SAEs), UPIRSOs, COIs, and member evaluation scores to evaluate the IRB's quality. These criteria were determined in light of a combination of AAHRPP standards and standards determined by national regulations—as, for example, AAHRPP recommends reporting UPIRSOs to IRBs, while Chinese guidelines, in alignment with ICH-GCP, require reporting SAEs but not UPIRSOs to IRBs (Association for the Accreditation of Human Research Protection Program, 2020b; World Medical Association, 2013).
Data analysis
Data analysis was conducted using SPSS 18.0 (IBM, Chicago, IL, USA). Chi-square tests were used to analyze differences in the characteristics of protocols reviewed, the distribution of the protocols, and characteristics of SAEs and COIs from 2015 to 2017. A difference was considered to be statistically significant at a p-value of <.05.
Results
Profile of Protocols Submitted
Two types of protocol review are made by the IRB at the study hospital: convened review and expedited review. A convened review is an IRB review with all board members present to evaluate a new protocol or protocols with large amendments; an expedited review is reserved for research processes with minor amendments and document updates. The data included in this analysis is derived from convened reviews during the period from January 1, 2015 to December 31, 2017.
Table 1 shows the data collected from 2015 through 2017 on the numbers of protocols submitted for convened review. A total of 396 human subjects research protocols were submitted and qualified for inclusion in our analysis. The number of protocols evaluated varied by year: the IRB reviewed 156 (39%) protocols in 2015, 113 (29%) in 2016, and 127 (32%) in 2017. The study showed statistically significant differences (p < .05) in the source of funding, type of study, and type of intervention. The majority of protocols were sponsored by pharmaceutical and medical device companies (311, 79%), followed by those sponsored by investigators (85, 21%). All protocols were reviewed and carried out internally at the hospital.
Characteristics of Protocols Submitted to the Institutional Review Board (IRB) for Convened Review: 2015–2017.
Clinical trial
Observational study;
Other;
Chemical;
Biological;
Chinese herbal medicine;
Drug;
Medical device;
IIT.
Chi-square test.
Fisher exact probability.
*p < .05 was regarded as statistically significant when comparing the overall differences among 2015–2017 and adjusted p < .016 was regarded as statistically significant when performing a further subgroup comparison between each of the 3 years.
The most common types of research protocols submitted were clinical trials (333, 84%) and observational studies (37, 9%), followed by studies with other designs (26, 7%). Most protocols proposed prospective studies (389, 98%), while only a small number of protocols entailed retrospective research (7, 2%). Protocols were grouped into four categories based on intervention type, including drug trials (265, 67%), medical device trials (30, 7%), and investigator-initiated trials (IITs) (101, 26%). Among drug studies, interventions were further categorized into chemical (197, 50%), biological (62, 16%), and Chinese herbal medicine (6, 1%). Upon completion of its analysis and evaluation, the IRB approved 389 (98%) of the 396 protocols submitted, rejected 6 (2%) protocols, and terminated or suspended 1 (0%) protocol.
Distribution of Protocols by Departments
This study examined the IRB review of research protocols from the 30 departments of the study hospital, 18 of which are approved by the NMPA (formerly the CFDA) to do clinical trials research. For the purposes of our study, we divided the hospital's departments into four general categories: internal medicine; surgery; phase I clinical research; and gynecology, pediatrics, and others. The “phase I clinical research” category solely included the hospital's Center for Phase 1 Clinical Research.
Table 2 shows a statistically significant difference (p < .05) between internal medicine; surgery; phase I clinical research center; and gynecology, pediatric and other protocols. The majority of protocols were submitted by the internal medicine departments (206, 52%), followed by phase I clinical research center (85, 21%), the surgery departments (77, 20%), and the gynecology, pediatric, and other departments (28, 7%).
Distribution of Protocols Submitted from Different Departments to Institutional Review Board (IRB) for Convened Review.
Internal medicine;
Surgery;
Phase I clinical research;
Gynecology, Pediatric, and others.
Chi-square test.
*p < .05 was regarded as statistically significant when comparing the overall differences among 2015–2017 and adjusted p < .008 was regarded as statistically significant when performing a further subgroup comparison between each of the 3 years.
Mean Time from Submission to Notification
Lengthy review times for IRBs are a well-known barrier to research (Spellecy et al., 2018). One of the main concerns of the IRB process is the timeliness of protocol review. Considerable time and resources are invested in the ethics review process (Adams et al., 2014).
An online platform was implemented at the beginning of 2015 to increase the IRB's efficacy in reviewing protocols. This platform is accessed through the hospital's website and serves as an electronic system for training researchers and for submitting and maintaining protocols. It provides researchers with the instructions and templates needed to complete an application for protocol review, including standard protocol templates, informed consent documents, regulations and guidance, and other necessary administrative documentation.
After the implementation of the online platform, we observed a decrease in the mean time in calendar days from protocol submission to notification of convened review. In 2015, the time from submission to notification had a mean of 23 days and a range of 15–30 days. In 2016, the mean was 15 days and the range was 11–22 days. In 2017, the mean was 15 days and the range was 12–20 days.
Review of SAEs and UPIRSOs
ICH-GCP guidelines state that SAEs must be reported to the IRB promptly. SAEs in clinical trials include new or prolonged hospitalizations, disabilities, negative effects on the ability to work, endangerment of life, deaths, and birth defects (ICH Harmonised Tripartite Guideline for Good Clinical Practice E6(R1), 2016). Two Chinese regulations, The Drug Registration Management Measures (2007) and the Medical Device Clinical Trial Quality Management Regulations (2016), stipulate that all SAEs should be reported to the IRB within 24 h during clinical trials.
There are no such requirements in Chinese regulations for UPIRSOs, which include events that are unexpected, are related or possibly related to the research, and involve new or increased risks to participants or others (OHRP, 2007). In order to facilitate researchers' reporting of UPIRSOs in accordance with AAHRPP requirements, the hospital produced a separate UPIRSOs report form.
SAEs and UPIRSOs were discussed and reviewed by members at IRB meetings. The IRB established a continuing review mechanism for all protocols. Protocols with a high number of SAEs or UPIRSOs were adjusted for higher follow-up review frequency, such as 3 months or 6 months instead of 1 year, or protocol suspension. In addition, considerable effort was made to assess outcomes, including results that may indicate that research subjects have been harmed or their rights have been violated, and factors that may lead to harm to human subjects.
Table 3 provides a comparison of the types of SAE reporting departments. At p = .268, there was no statistically significant difference. The total number of SAE cases submitted to the IRB was 344, the majority of which were submitted by the internal medicine departments (320, 93%).
Characteristics of Serious Adverse Event (SAE) Cases Submitted to the Institutional Review Board (IRB): 2015–2017.
Fisher exact probability.
*p < .05 was regarded as statistically significant when comparing the overall differences among 2015–2017.
Management of COI
Table 4 shows a comparison of COI among researchers and IRB members. At p = .773, there was no statistically significant difference. A total of 93 COIs were disclosed, of which 90 (97%) were declared by IRB members, and 3 (3%) were disclosed by researchers. In all cases, when COIs were reported by principal investigators who were IRB members (90, 97%), members involved in the research recused themselves from IRB review. The policy of the IRB allowed for other members of the IRB to ask protocol-related questions of the members with COIs before the members recused themselves from both protocol discussion and voting—a policy in line with IRBs at many research institutions in the United States and other countries (Office of Human Subjects Research—Institutional Review Board, 2020).
Conflicts of Interest Disclosed in 2015–2017.
Chi-square test.
*p < .05 was regarded as statistically significant when comparing the overall differences among 2015–2017.
Education and Training
Education is a very important aspect of HRPP activity aimed at IRB quality improvement. The hospital developed different types of personnel and different content of both online and offline education and training programs. Specific education and training activities include an e-IRB online HRPP training and examination system for researchers, a 10-min HRPP training for IRB members before monthly IRB meetings, an HRPP training course for researchers every spring semester, HRPP training before the start of each clinical trial for researchers and research staff, and an annual clinical research ethics seminar held by the IRB for a National Continuing Medical Education program. Besides, participants and people in the community can obtain research knowledge and HRPP information from the HRPP website.
IRB member evaluations are another common method for improving IRB function and effectiveness (Jaoko et al., 2016). The performance of IRB members was evaluated by an HRPP Institutional Official annually, to assess that IRB members had knowledge, skills, and abilities appropriate to their respective roles (Association for the Accreditation of Human Research Protection Programs, Inc., 2018). The evaluation was based on questions asked of IRB members regarding each of the following criteria: (1) ability to apply regulations, ethical principles, and IRB policies and procedures to research; (2) completion of reviewer requirements for assigned protocols; (3) willingness to review IRB minutes, provide consult to staff/investigators, and attend off-cycle meetings; and (4) working relationship with IRB members and staff. In each of these areas evaluated by the Institutional Official, members received a score of 5 (on a scale of 1–5, with 1 being “extremely unsatisfied,” and 5 being “extremely satisfied”) for each year that the evaluation was conducted. The investigators conducting this analysis also noted that as the training was enhanced over the course of 3 years, IRB members' suggestions for protocol modification became more specific and more evidence based.
Discussion
Within 3 years of the establishment of an IRB at the hospital in this study, several significant improvements have already been made to the quality and efficiency of the IRB's review. The IRB began using an online platform, shortened review times, established new mechanisms for reporting COIs and UPIRSOs, and enhanced differentiation between different categories of research. However, many improvements still need to be made, including the distinction between faculty and student research, and more cooperation with Scientific Committees, Biosafety Committees, and Data Safety Monitoring Boards (DSMBs). This discussion will put forward several suggestions for accomplishing such improvements.
During the study period, the IRB's mean time from submission to review decreased from 23 to 15 calendar days, largely as a result of the introduction of an online protocol review process. Additionally, the IRB shortened review times by establishing a routine, monthly meeting schedule and implementing a new policy to provide notification of review results within 15 days of protocol submission. The shortening of the review cycle was a significant gain for the IRB's efficiency.
During the years under study, the IRB developed new methods for reporting COIs and UPIRSOs that may serve both as a starting point for further development of these aspects of review both for this IRB and as a model for other IRBs in China. The concept of COIs was first introduced into the field of clinical research in China in 2010, leading to improvements in the protection of human subjects (China Food and Drug Administration, 2010).
As the influence of the pharmaceutical industry on medical research increases, the need for transparency in these relationships increases. Therefore, the process of monitoring COIs established by this hospital's IRB is a significant improvement to ethical review. This or similar processes should be adopted by other IRBs throughout China. As there are no specific regulations for COIs in clinical research in China, the hospital adopted AAHRPP's COI standard, developed a written policy and standard operating procedure, and designed declarations, training, and avoidance measures for IRB members' COIs. However, further enhancement of the COI committee, a key component of HRPP, is still needed to address COIs. In the process of monitoring human subjects’ research, IRBs should work with HRPP COI committees to ensure compliance with national regulations. Under their separate mandates, IRBs and COI committees may take various actions as needed, such as limiting protocols that implicate significant financial interests of the institution, or its decision makers (Freedman & McKinney, 2013).
As China does not have specific requirements for reporting UPIRSOs, the study hospital also took a significant step to enhance the ethical review by establishing a form for reporting UPIRSOs in addition to SAEs. It should be noted that there are some types of incidents, experiences, and outcomes that occur during the conduct of human subjects research that would be reported as SAEs, but are unrelated to research and thus would not be classified as UPIRSOs. For example, if a study subject came to the hospital for a research study and on his way home, was involved in a car accident and hospitalized in another hospital, this would be an SAE even though unrelated to the research. Although in the past, China's regulations required reporting of all SAEs to IRBs, a new revised version of China Good Clinical Practice (GCP) 2020 was implemented on July 1, 2020 (National Medical Products Administration, 2020b). One significant change as of 2020 is that SAEs will no longer be required to submit to IRB for review, except for death reports. IRBs will focus more on reviewing suspected unexpected serious adverse reactions (SUSARs), which are unexpected events in clinical trials whose nature and severity exceed the available data, such as the investigator's manual of the experimental drug, the specification of the marketed drug, or the product characteristic summary (National Medical Products Administration & National Health Commission of the People's Republic of China, 2020a). In light of this change, further training of investigators regarding how to define a SUSAR as distinguished from an SAE will be needed.
The IRB made a further improvement by categorizing different types of research upon creating different forms for different research categories, including drugs, devices, and IIT. However, another important distinction that the IRB can make more clear in the future is that between student and faculty research. Many student projects are submitted under the name of their faculty mentor, making it harder to distinguish between student and faculty research proposals. We found that the most commonly reviewed protocols by this IRB were clinical trials sponsored by pharmaceutical and medical device companies. Although protocols submitted by faculty and students were less common, they still pose challenges to the IRB due to the relative inexperience of the hospital's faculty and students in developing research protocols, adhering to ethical guidelines, and following IRB application procedures.
For such IIT research, the obligation of a sponsor–investigator includes both those of a sponsor and those of an investigator (ICH, 2016). It should be the responsibility of the IRB to examine whether the qualifications, experience, and technical ability of the researchers are sufficient, and whether qualified researchers are responsible for obtaining informed consent and are ready to receive consultation on safety issues. As new researchers, students especially need to receive adequate training. If the research is initiated by a student principal investigator (PI), additional information should also be provided, such as whether the project will be completed before graduation, the nature of guidance from the advisor to help ensure the quality of the research. For these reasons, a mechanism to distinguish between student and faculty research would improve the IRB's capacity to identify and train inexperienced researchers in ethical standards. This would allow for more consistent application of specific standards to different types of protocols.
Another area of needed improvement is to address higher risk studies by establishing specialized committees to advise the IRB, another core requirement of AAHRPP. This review found that most of the studies at the study hospital were prospective studies. The most common type of intervention among the protocols reviewed by the IRB was chemical drug interventions. As government policies have increasingly encouraged innovation in recent years, there has been a shift in these protocols from being largely studies of generic drugs to more so being studies of innovative drugs. As a result, protocol research design is becoming more complicated and the risks to subjects are higher. Protocols for research on biological and Chinese herbal medicines are also increasing. Studies involving these types of research often have higher levels of potential adverse effects and other risks to participants, posing a challenge for IRBs to evaluate the scientific nature as well as risks and benefits of the studies. Therefore, to maximize efficiency and quality, IRBs including that at the study hospital need the support of other components of HRPP, such as scientific or biosafety committees, to review the studies collaboratively.
Furthermore, there is still a need for the IRB at the study hospital to take additional protective measures regarding risk/benefit evaluation and sufficient informed consent for early phase clinical trials, which are particularly high risk (Suntharalingam et al., 2006). According to the results of this study, the internal medicine departments submitted the most clinical trial protocols, followed by phase I clinical research center. Most early clinical trials carried out in the phase I clinical research center are at the essential transitional stage, in which testing moves from animals to humans. Due to the especially high-risk nature of these early phase human studies, the subjects’ potential benefit in participating is often far outweighed by the risk, and the IRB must closely review the criteria in the evaluation of risks and benefits. Continued training is needed to address this heightened risk. In addition, CFDA emphasizes the importance of DSMB or the regular monitoring of the safety of human subjects (National Medical Products Administration & National Health Commission of the People's Republic of China, 2020a). IRBs including that at the study hospital can cooperate more with DSMBs to facilitate the evaluation of safety issues. The IRB and DSMB also need to develop good communication and collaboration mechanisms to enhance the analysis and management of subject safety data during studies.
It is important to note that the success and capacity of the study hospital is not generalizable to all IRBs that have been accredited by other kinds of external human subject protection programs. In relation to rising standards for ethical review in China, the establishment and review of an IRB at the hospital examined in this study may serve as a model for ongoing improvement of ethical review standards in China. In order to establish more consistent practices by the IRB at this hospital and others, improvements in management, compliance evaluation, and training of new researchers are urgently needed. As scientific research in China is expanding in scope and influence, the results of this study can inform the future development of standards for evaluating IRB quality and efficiency in China.
Conclusion
This is a novel assessment of a hospital IRB as a component of HRPP in China. An effective human research protection system must have appropriate mechanisms for the oversight of IRBs. In some ways, the IRB examined in this study has improved to establish such mechanisms. Areas where further improvements are needed include distinctions between different types of research and interactions with specialized committees and DSMBs. IRB assessment tools also need to be further modified and verified.
As the number of IRBs expands rapidly in China, there is a national-level need for the development of standards for evaluating IRB quality and efficiency. This study is the first to report on the review process and the overall quality of the protocols submitted in a hospital that has established an HRPP. The results of this study can therefore be used to improve both the quality and efficiency of the IRB described, and to further the development of evaluation standards for quality and efficiency of IRBs in China.
Educational Implications
There is a strong need to build HRPP-based IRBs in China, which should comply with ICH-GCP and Chinese subject protection regulations, be evaluated with reference to external standards, include review of unanticipated problems and COI management systems, and adopt diversified education and training to continuously improve the ability to protect human subjects. This study may serve as a model for evaluating and enhancing HRPP-based IRBs in China.
Best Practice
When establishing and evaluating elements of HRPPs within Chinese institutions, international HRPP guidelines should be contextualized to the existing structure and resources of the institution, and the HRPP's mechanisms for operation should be suitable for the institution.
Research Agenda
This study highlights that IRBs are a key component of HRPP. The results of the study can be applied to improve both the quality and efficiency of the study hospital IRB, and to further the development of evaluation standards for quality and efficiency of IRBs in China, thereby meeting an urgent need.
Footnotes
Acknowledgments
We are grateful to the members and staff of the HRPP of the Third Xiangya Hospital of Central South University.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
This work was supported by the National Institutes of Health Fogarty International Center Masters Level Bioethics Program at Central South University in Changsha, China under award number R25 TW007700 and National Major New Drug Creation Project of China (NO: 2020ZX09201010).
Disclaimer
This publication represents the views of the authors and does not reflect the official position or policy of Central South University, Dalian University of Technology, or Yale University.
