Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals

Abstract

We hypothesized that typical tissue and clinical chemistry (ClinChem) end points measured in rat toxicity studies exhibit chemical-independent biological thresholds beyond which cancer occurs. Using the rat in vivo TG-GATES study, 75 chemicals were examined across chemical-dose-time comparisons that could be linked to liver tumor outcomes. Thresholds for liver weight to body weight (LW/BW) and 21 serum ClinChem end points were defined as the maximum and minimum values for those exposures that did not lead to liver tumors in rats. Upper thresholds were identified for LW/BW (117%), aspartate aminotransferase (195%), alanine aminotransferase (141%), alkaline phosphatase (152%), and total bilirubin (115%), and lower thresholds were identified for phospholipids (82%), relative albumin (93%), total cholesterol (82%), and total protein (94%). Thresholds derived from the TG-GATES data set were consistent across other acute and subchronic rat studies. A training set of ClinChem and LW/BW thresholds derived from a 38 chemical training set from TG-GATES was predictive of liver tumor outcomes for a test set of 37 independent TG-GATES chemicals (91%). The thresholds were most predictive when applied to 7d treatments (98%). These findings provide support that biological thresholds for common end points in rodent studies can be used to predict chemical tumorigenic potential.

Keywords

liver weight clinical chemistry adverse outcome pathway constitutive activated receptor transcript profiling liver cancer peroxisome proliferator-activated receptor α aryl hydrocarbon receptor genotoxicity estrogen receptor key events molecular initiating events cytotoxicity

Introduction

There is very little known about the potential for most chemicals currently in use to cause cancer in humans. These include over 140,000 substances registered by Registration, Evaluation, Authorization and Restriction of Chemicals,¹ ∼ 30,000 chemicals in widespread commercial use in the United States and Canada,² and ∼41,000 chemicals on the US EPA’s Toxic Substances Control Act Inventory (https://www.epa.gov/tsca-inventory; accessed March 19, 2020). Determining the human carcinogenic potential of these chemicals remains a formidable challenge that currently relies primarily upon the 2-year rodent bioassay as the “gold-standard.” However, only ∼1,500 chemicals in commercial use have been evaluated by the 2-year bioassay, due to the considerable resource expenditures required to assess a chemical in this manner (>800 rodents, histopathological analysis of more than 40 tissues, ∼US$2-4 M).^3
-5 Further, the use of animal bioassay data for prediction of human cancer potential remains complex due to interspecies differences as well as the reproducibility of animal study results even in the most controlled conditions (Gold et al 1989⁶, Haseman 2000⁷). This situation demands new, resource-efficient methods that identify the carcinogenic potential of environmental chemicals and pharmaceuticals in short-term exposures as well as methods that determine dose and exposure boundaries of human-relevant risk.

Pathway information organized by mode of action (MOA)^8,9 or more recently, by an adverse outcome pathway (AOP) construct,^10,11 plays a central role in providing frameworks for compiling evidence of effects and determining human relevance for chemical carcinogens. An AOP, which is chemical-agnostic, starts with the molecular initiating event (MIE) in which a chemical interacts with a receptor followed by a series of downstream key events (KEs) that ultimately lead to an adverse outcome (AO). The overlay of chemical-specific information including absorption, disposition, metabolism, and excretion (ADME), prediction of chemical concentrations at the site of the MIE, and quantification of MIE perturbation onto a discrete AOP construct then permits the derivation of the corresponding MOA for risk assessment.¹⁰ Use of AOPs in risk assessment strategies will enable the utilization of MIE and KE level information from new approach methodologies (NAMs) that measure perturbations in human models.

Toxicity assessments in animals routinely measure organ to body weight (BW) changes and alterations in clinical chemistry (ClinChem) parameters. Increases in liver weight to body weight ratios (LW/BW) may suggest treatment-related effects including hepatocellular hypertrophy (eg, enzyme induction or peroxisome proliferation) as well as hepatocellular hyperplasia.¹² In our earlier study evaluating 75 TG-GATES chemicals, we showed that increases in LW/BW occur as early as 3 days after initial chemical exposure, and the increases were often associated with liver tumorigenic risk.¹³ However, LW/BW increases by themselves were not very accurate at predicting liver tumors for chemicals examined as part of ≤ 1-year National Toxicology Program (NTP) studies.¹⁴

Blood chemistry tests provide important information about the effects of chemical exposure on the function of the kidneys, liver, and other organs, and an abnormal amount of a substance in the blood can be a sign of disease. These tests are routinely performed on serum and include commonly measured ions and metabolites (sodium [NA], potassium [K], chloride [CL], bicarbonate, blood urea nitrogen [BUN], creatinine, glucose, calcium [CA]) and additional tests (total protein, albumin [RALB], alkaline phosphatase [ALP], alanine aminotransferase [ALT], aspartate aminotransferase [AST], bilirubin). The levels of cholesterol and triglycerides are also routinely measured. A number of these tests (eg, ALT and AST) are used to determine whether a chemical is cytotoxic in the liver after chemical exposure. A comprehensive assessment of ALT and AST levels as measures of chemical-induced hepatocyte toxicity identified chemicals that likely cause liver tumors through a cytotoxic MOA.¹³ While routine measurements of organ weight and ClinChem changes in short-term rat studies are found in large databases that include genomic and histopathological changes such as TG-GATES¹⁵ and DrugMatrix,¹⁶ they have not been comprehensively examined to determine their quantitative relationships with liver tumorigenic risk.

A central premise of the AOP framework is that KEs are required but not sufficient at a qualitative level to produce an AO. Progression beyond an MIE or KE depends on the level of molecular or cellular perturbation that can then propagate the chemical effects toward the AO.¹⁷ This idea has led to increased interest in quantitative effect thresholds, or “molecular tipping points,” as a basis for adversity determinations using short-term data.^18,19 A biological or effect threshold is one that is chemical-independent within a particular model system. It defines, for example, the level of receptor activation that may be needed for cancer to occur. This concept is critical for risk assessment because thresholds would prescribe chemical-agnostic hazard levels that enable predictive evaluations using an AOP construct. While there are numerous examples of KE relationships within individual AOPs,^10,20 neither the concept of chemical-independent thresholds or application to AOPs have been widely investigated within toxicological or pharmacological sciences.

In a previous study, we identified chemical-independent thresholds for gene expression biomarkers that predict activation of 6 of the major MIEs that lead to liver cancer in rodents. We found that these thresholds can be used to accurately predict liver cancer.²¹ In the present study, we hypothesized that many of the routine measures of a typical short-term rat exposure study also have chemical-independent thresholds beyond which tumors occur. To test this hypothesis, we examined LW/BW and ClinChem end points in a number of publicly available databases including TG-GATEs, DrugMatrix, ToxRefDB, and NTP findings in Chemical Effects in Biological Systems (CEBS) at chemical doses with known liver tumor outcomes in 2-year bioassays. We found that, based on the data set compiled from rat studies, many of the ClinChem markers as well as LW/BW changes have measurable thresholds that are chemical- and study-independent. Furthermore, we found that these thresholds can be used to identify doses of chemicals predicted to cause liver tumors in long-term studies.

Methods

Description of the Sources of Information

The sources of information used in the present study are described below.

The TG-GATES study

The standard protocol in the TG-GATES study²² used 5 Sprague-Dawley rats per sampling point treated with compounds at 3 different dose levels (low, middle, high). The maximum tolerated dose of each compound was estimated from a preliminary 7-day repeated dosing study and was employed as the highest dose level. In general, the ratio of the concentrations for the low-, middle-, and high-dose levels was set as 1:3:10, respectively.¹⁵ For acute exposure studies, rats were euthanized 3, 6, 9, and 24 hours after dosing. For continuous exposure studies, the animals were treated daily for 3, 7, 14, and 28 days and euthanized 24 hours after the last dose (4, 8, 15, and 29 days). Euthanasia was performed using ether anesthesia and exsanguination from the abdominal aorta, followed by immediate collection of tissue samples from the left lateral lobe of the liver. Microarray analysis was conducted on 3 of 5 samples from each group using Affymetrix Rat Genome 230 2.0 arrays (Affymetrix).²² In addition to microarray data, the database contains absolute LW and BW, ClinChem results, and the results from pathological examination of tissues. Brain weights were not available to normalize LWs. All data are found in the TG-GATES database (http://toxico.nibiohn.go.jp/english/).

The DrugMatrix study

DrugMatrix is a publicly available toxicogenomic reference database that contains microarray and ClinChem data from the tissues of rats administered pharmaceutical agents, environmental chemicals, or other substances. Most chemicals were examined at 2 doses; the rats were sacrificed usually at 6 hours, 1, 3, 5, and 7 days. The DrugMatrix database originally developed by Incyte Genomics, Inc, and Iconix Pharmaceuticals, Inc was acquired by NTP in 2010. All data are available at https://ntp.niehs.nih.gov/drugmatrix (accessed March 28, 2019).

The Toxicity Reference Database (ToxRefDB)

ToxRefDB version 2.0 is a publicly available database (https://doi.org/10.23645/epacomptox.6062545.v3) constructed by the US EPA Office of Research and Development National Center for Computational Toxicology.²³ ToxRefDB includes information curated mainly from studies conducted in accordance with or by specifications similar to the EPA Office of Chemical Safety and Pollution Prevention (OCSPP) 870 series Health Effects Test Guidelines, with records associated with 1142 chemicals and 5960 studies (including guideline and guideline-like studies). The single largest source of these data were reviews of registrant-submitted toxicity studies, known as data evaluation records (DERs), from the US EPA’s Office of Pesticide Programs, with additional data obtained from studies available from the NTP, the pharmaceutical industry, and the publicly available literature. The database includes information regarding the study design, the 870 series guideline if applicable, chemical identity, treatment group parameters, and treatment-related effect levels, that is, the dose at which an effect was significantly different from control.

Findings from the NTP database

The NTP conducts toxicology and carcinogenicity testing in rodent models, including Fischer 344 and Harlan Sprague Dawley rats. Subjects are treated with compounds daily for the duration of the study, generally 14 days to 13 weeks for short-term studies or 2 years for long-term studies. Depending on the compound tested, 3 to 5 dose levels are typically used. The standard protocol calls for retro-orbital sinus blood collection for ClinChem end points. Organs are collected immediately during necropsy and weighed. A full list of specifications for the conduct of studies including general protocol definitions is available at https://ntp.niehs.nih.gov/ntp/test_info/finalntp_toxcarspecsjan2011.pdf. All data are available in the NTP CEBS database (http://cebs.niehs.nih.gov).

The MARCAR study

The aim of the MARCAR project was to identify early biological indicators (biomarkers) that can be used to predict the effects of nongenotoxic carcinogens. A complete description of the protocols followed for the exposure studies can be found in.^24,25 Briefly, male Wistar Hanover rats (Crl: WI[Gl/BRL/Han]IGS BR) 8 to 10 weeks old were assigned to dose groups (5 rats/group). Chemicals were administered by gastric gavage for up to 14 days (in a volume of 5 mL/kg BW/d) based on the group mean weekly BW for each dose group. The fifteen test substances examined were suspended (wt/wt) in either corn oil (CO), or a 0.5% (wt/vol) carboxymethylcellulose (CMC)/deionized water preparation (5-g CMC/1-L deionized water). Diethylstilbestrol and piperonyl butoxide were administered for 1, 3, and 7 days. All other compounds were dosed for 1, 3, 7, and 14 days. The rationale for dose selection was based on those reported to induce liver tumors in the 2-year rat bioassay.²⁴ From each treatment group, 3 animals that showed at least some changes in the liver as observed by histopathological examination were selected for microarray analysis. Time-matched control groups of equal size treated with the corresponding vehicles methylcellulose or CO, served as a reference to determine the changes upon treatment. Microarray analysis was carried out on Affymetrix GeneChip RAE230A arrays. Liver weight, ALT, and AST were available for analysis from this study.

Classification of hepatotumorigenicity of chemical-dose pairs

In our earlier study,¹³ we evaluated 134 chemicals that were examined in the TG-GATES study for doses that have known effects in the rat liver for ability to induce hepatocellular adenomas or hepatocellular carcinomas (liver tumorigens) using information in the Carcinogenicity Potency Data Base⁴ (https://toxnet.nlm.nih.gov/cpdb/), Physicians’ Desk Reference,²⁶ or in PharmaPendium (https://www-elsevier-com-s.web.bisu.edu.cn/solutions/pharmapendium-clinical-data). The Carcinogenicity Potency Database is now found at ftp://anonftp.niehs.nih.gov/ntp-cebs/datatype/Carcinogenic_Potency_Database_CPDB/ (accessed March 9, 2020). In the present study, additional annotations for the chemicals in ToxRefDB, DrugMatrix, and NTP databases were made using the Carcinogenicity Potency Database. Annotations were made only when there was a clear response (positive or negative) at a given dose level in male or female rats of any strain. In contrast to other studies (eg, Fielden et al¹⁶), we did not make any assumptions about the ability of the chemicals to induce liver tumors, and thus our analysis resulted in fewer chemical-dose-time comparisons to which we could ascribe tumor induction (evidence is summarized in Supplemental File 2). For each tumorigenic chemical, we identified the lowest dose at which there was a clear induction of hepatocellular adenomas or carcinomas based on liver tumor incidence from the CPDB or other sources. The doses identified had ≥ ∼20% increase in liver tumors over controls. For the nontumorigenic chemicals, we identified the highest dose used in the bioassay(s) that did not induce liver tumors. All data were based on rat studies and no formal statistical analysis was carried out. The doses used in the short-term studies were then classified for liver tumorigenic potential. They were classified tumorigenic if the dose used in the short-term study was equal to or exceeded the lowest tumorigenic dose. They were classified as nontumorigenic if the dose used was equal to or below below the highest nontumorigenic dose used in the bioassay. All others were put into a separate category as unknown tumorigenic outcome and could include chemicals that were examined at doses that exceeded the highest nontumorigenic used in the bioassay. Our inclusive approach (all sexes, all strains) then might lead to a greater number of false positives (FPs) but in fact that does not seem to be the case based on our recently published work using the same annotations.^21,27

Analysis of LW/BW and ClinChem measurements

The LW/BW and ClinChem values were extracted and processed as described below for the specific data sets. Comparisons were selected for analysis if the dose used could be ascribed a tumorigenic/nontumorigenic classification as described above.

The TG-GATES data were used to calculate relative changes for all chemical-dose-time comparisons for LW/BW and ClinChem end points. Liver weights and BWs were also analyzed independently to distinguish possible changes in LW/BW driven only by decreases in BW. There were a total of 3127 comparisons examined for 133 chemicals. Out of these, a total of 1066 comparisons for 75 chemicals could be classified for doses that were either tumorigenic or nontumorigenic. For the 3127 comparisons, significant changes at each end point were identified by analysis of variance (ANOVA) for each chemical-time group across 4 doses (control and 3 doses). A Benjamini-Hochberg multiple test correction was applied to the ANOVA P values (corrected P value < .05) followed by a Tukey’s honestly significant difference (HSD) post hoc comparison between individual dose levels and the control group, at each time point for each chemical. Chemical-time groups were included if any dose group was significantly different from the control group. Subsequent analysis was carried out with the significant changes. All end points were assumed to be normally distributed. All of the significant changes from the analysis can be found in Supplemental File 2.

ToxRefDB version 2.0 is a publicly available database (https://doi.org/10.23645/epacomptox.6062545.v3) constructed by the US EPA Office of Research and Development National Center for Computational Toxicology.²³ For the ToxRefDB, the R code (version 3.6.1) used to extract the information for end point targets including “alanine aminotransferase (alt/sgpt),” “aspartate aminotransferase (ast/sgot),” “liver” with a change in LW “relative to body weight,” is available as Supplemental File 1. ToxRefDB version 2.0 contains dose-response information. Effects reported related to changes in LW relative to BW and ClinChem measures including changes in ALT and AST were extracted specifically. Ratio was calculated as the effect value minus the control value, divided by the control value. Comparisons examined came from studies carried out with CD(SD)BR, Fischer 344, Sprague Dawley, and Wistar rats in subchronic and chronic exposures from mostly feed studies. For a small subset of nontumorigenic chemical-dose pairs that exceeded the LW/BW threshold, we performed targeted searches to determine whether the increases were due to steatosis.

To calculate fold changes from the DrugMatrix data set, we normalized each measurement according to each study’s controls. For each chemical within a study, the replicate measurements (including control measurements) were averaged for every time point/dosage combination. We then normalized each measurement by dividing by the control to obtain a fold change value and setting the control measurement to a fold change of zero. Fold change and standard error were calculated, followed by Kendall trend test and pairwise comparison using Shirley or Dunn test (Shirley test was used when trend P value ≤ .01, otherwise Dunn test was used) between a given experimental group and corresponding control group. There were 19 ClinChem end points evaluated for 559 comparisons with 109 chemicals. Of these, a total of 285 comparisons for 89 chemicals could be classified for doses that were either tumorigenic or nontumorigenic. All of the significant changes from the analysis can be found in Supplemental File 2.

National Toxicology Program LW/BW ratios were reported for ∼200 chemical-dose-time comparisons, and ClinChem end points were available for 900 chemical-dose-time comparisons. Fold change was calculated for each compound-time point. Studies examined included mostly 13-week exposures of male and female rats. A minor number of exposures were for 2, 3, and 14 days. Like the ToxRefDB analysis above, we performed targeted searches to determine whether the increases for nontumorigenic chemical-dose pairs that exceeded the LW/BW threshold were due to steatosis.

Identification of differentially expressed genes in BaseSpace Correlation Engine (BSCE) microarray data sets

All of the statistically filtered gene lists that were used in the present analysis were generated using BSCE standardized microarray analysis pipelines and are available in a searchable annotated format in the BSCE database (https://www.illumina.com/informatics/research/biological-data-interpretation/nextbio.html). As part of this pipeline, raw microarray data (.cel files) available in Open TG-GATEs or available in Gene Expression Omnibus (eg, for DrugMatrix study and additional smaller studies) were imported and analyzed by the BSCE analysis pipeline. Differentially expressed genes were identified as described in a previous publication.²⁸

Annotation of a rat liver gene expression compendium

The annotation of the rat liver compendium was carried out as described earlier.¹³ All individual comparisons with gene expression in rat liver were annotated for study characteristics allowing a systematic assessment of the effect of chemicals and other factors on a number of transcription factors important in hepatocarcinogenesis. The list of descriptors provided for each of the individual comparisons included study ID (ie, source), name, classification of factor (eg, chemical, diet, genotype, etc), name of chemical or treatment, sex, source of material (eg, liver or hepatocyte), and microarray type. This information is provided in Supplemental File 2.

Comparison of gene expression biomarkers to microarray profiles to assess MIEs

Gene expression biomarkers for the receptors aryl hydrocarbon receptor (AhR), constitutive activated receptor (CAR), estrogen receptor (ER), and peroxisome proliferator-activated receptor α (PPARα) and that predict genotoxicity were built and characterized in our previous study.¹³ A gene expression biomarker for cytotoxicity was built and characterized in Corton et al.²¹ The biomarkers consisting of lists of genes and associated ratio values were built using a weight of evidence strategy similar to our previous studies in which we built and characterized biomarkers for AhR, CAR, PPARα, and STAT5b to predict effects in chemically treated mouse livers.^29

-32 The balanced accuracies of the biomarkers for activation of the MIEs in rat liver were 92%, 91%, 92%, 96%, 97%, and 96% for genotoxicity, AhR, CAR, ER, PPARα, and cytotoxicity, respectively. To assess activity of the MIEs, each list of significantly altered genes from a chemical versus control comparison was compared to each of the biomarkers using the Running Fisher test,²⁸ as in our previous studies.^29

-32 These studies demonstrated that a cutoff P value ≤ .0001 after a Benjamini-Hochberg correction of α = .001 produced a balanced accuracy of 95%, 97%, 98%, and 98% for AhR, CAR, PPARα, and STAT5b activation in the mouse liver, respectively.^29

-32

Determination of thresholds

Thresholds were derived by identifying the maximum and minimum ratio values in the nontumorigenic chemical-dose-time comparisons from the TG-GATES data. Values were derived from 3 sets of chemicals: a TG-GATES training set consisting of 38 chemicals including 9 that were tumorigenic, a TG-GATES test set consisting of 37 chemicals that did not overlap with those in the training set and included 9 that were tumorigenic, and the entire TG-GATES data set. The training sets and test sets were chosen randomly. We ensured that there were equal numbers of tumorigenic chemicals in training and test sets which were randomly distributed. We also randomly distributed the remaining chemicals that were nontumorigenic into the training and test sets. In deriving the thresholds, we considered if the data set contained outliers. This point is especially important; if the threshold is set too high for upper thresholds or too low for lower thresholds, true positive (TP) chemicals may be missed in subsequent screens. In our previous study, we identified outliers that were clearly separate from the rest of the values for that end point.²⁷ In the present study, there was only 1 value removed out of the 17 upper and 17 lower thresholds identified (BUN-upper in the training set and the entire TG-GATES data set). Support for removing this 1 outlier was that the value was clearly separated from the rest of the values and the value exceeded ≥ 1-fold from the next smaller value (indicated in box and whisker plots, Supplemental Figure 1). No values were removed in the test set.

The following thresholds could not be determined because there were no values in the nontumorigenic group: upper or lower thresholds from the training set for LDH, CRE, AGRATIO, GTP; upper thresholds in the training set for GLC, phospholipid (PL) and TG; upper thresholds in the test set for ALP; lower thresholds in the training set for total bilirubin (TBIL); lower thresholds in the test set for ALP, AST, CA, and CL. The following thresholds were based on only 1 value: upper thresholds in the training set for TBIL, NA, K, and CA; upper thresholds in the test set for AST, ALT, TG, TBIL, NA, K, CL, CA, and RALB; lower thresholds in the training set for total cholesterol (TC) and PL; lower thresholds in the test set for TC, NA, and RALB.

Comparison of thresholds to upper and lower bounds of historical ranges

Historical ranges of control values for the ClinChem values came from Charles River (https://www.crj.co.jp/cms/pdf/info_common/50/8250933/rm_rm_r_clinical_parameters_cd_rat_06.pdf; accessed Aug 12, 2019). Upper and lower bounds of historical ranges were converted to ratios by dividing by the provided means.

Determination of predictive accuracy

Predictive accuracy was determined either on an individual treatment level or by aggregating individual comparisons that examine the same chemical. The tumorigenic prediction by chemical aggregate for each threshold derivation method was scored as TP, true negative (TN), FP, or false negative (FN) based on whether the ratio exceeded the thresholds for 1 or more of the end points. To be diagnosed as a FN, all treatments for a tumorigenic chemical had to be below the thresholds. If even one of the individual comparisons for a nontumorigenic chemical exceeded 1 or more thresholds, it was diagnosed as an FP. The calculations for accuracy were the following: sensitivity (TP rate) = TP/(TP + FN); specificity (TN rate) = TN/(FP + TN); positive predictive value (PPV) = TP/(TP + FP); negative predictive value (NPV) = TN/(TN + FN); balanced accuracy = (sensitivity + specificity)/2.

Additional methods

Hierarchical clustering and visualization were carried out using the Cluster and TreeView programs from the Eisen laboratory (http://bonsai.hgc.jp/∼mdehoon/software/cluster/software.htm). Clustering was carried out using uncentered correlation with complete linkage.

Results and Discussion

Strategy to Derive and Characterize Thresholds

Our study focused on identifying chemical-independent thresholds for LW/BW and ClinChem end points beyond which rat liver tumors occur. Figure 1 outlines the steps to identify the thresholds and use them for prediction of tumorigenicity. The end points examined in our study are listed in Table 1. Liver weight, BW, and ClinChem marker raw values were derived from the TG-GATES data set in which 562 chemical-dose pairs representing 75 chemicals could be ascribed a liver tumor response (yes or no). Global gene expression of the livers was carried out using microarrays allowing an indirect assessment of the activation of the 6 MIEs using previously characterized biomarkers.^13,21 Ratios for LW/BW and ClinChem markers between treated and control samples were rank-ordered and compared between tumorigenic and nontumorigenic treatment conditions to identify upper and lower thresholds, defined as the maximum or minimum values in nontumorigenic conditions. The thresholds were examined to determine (1) the extent to which the end points exceed their thresholds, (2) their time- and dose-dependence, (3) if the thresholds derived from the TG-GATES study approximate those found in other studies, and (4) if the thresholds when used together can identify liver tumorigens in short-term studies.

Figure 1.

Strategy to identify thresholds and use them to predict liver cancer. Identification and characterization of thresholds. The TG-GATES study was used to identify thresholds for LW/BW and ClinChem end points. Additionally, liver gene expression profiles and gene expression biomarkers were used to indirectly measure MIEs. Thresholds were defined as the highest or lowest value from the nontumorigenic conditions. ClinChem indicates clinical chemistry; MIE, molecular initiating event.

Table 1.

End Points Examined.

End point	Abbreviation	Evaluated in TG-GATES study	Evaluated in DrugMatrix study	Evaluated from ToxRefDB	Evaluated from NTP studies	Evaluated in MARCAR study
Alanine aminotransferase	ALT	Yes	Yes	Yes	Yes	Yes
Aspartate aminotransferase	AST	Yes	Yes	Yes	Yes	Yes
Liver weight to body weight	LW/BW	Yes		Yes	Yes	Yes
Alkaline phosphatase	ALP	Yes	Yes
Blood urea nitrogen	BUN	Yes	Yes
Chloride	CL	Yes	Yes
Creatinine	CRE	Yes	Yes
Glucose	GLC	Yes	Yes
Inorganic phosphate	IP	Yes	Yes
Potassium	K	Yes	Yes
Lactate dehydrogenase	LDH	Yes	Yes
Sodium	NA	Yes	Yes
Albumin	RALB	Yes	Yes
Total bilirubin	TBIL	Yes	Yes
Total cholesterol	TC	Yes	Yes
Total protein	TP	Yes	Yes
Carbon dioxide	CO₂		Yes
Creatinine phosphokinase	CPK		Yes
Lipase	LPS		Yes
Uric acid	UA		Yes
Globin/albumin ratio	AGRATIO	Yes
Calcium	CA	Yes
Direct bilirubin	DBIL	Yes
Gamma-glutamyltranspeptidase	GTP	Yes
Phospholipid	PL	Yes
Triglyceride	TG	Yes

Abbreviation: NTP, National Toxicology Program.

Identification of a Threshold for LW/BW Associated With Tumorigenesis

Changes in LW/BW were determined across 8 time points for 75 chemicals at up to 3 doses as described in the Methods. Of the 3127 comparisons examined, there were 400 that had significant changes in LW/BW, 125 of which could be classified for tumorigenicity. Figure 2A shows the ratios of the 125 LW/BW changes separated into tumorigenic and nontumorigenic groups across time of treatment. Very few treatments caused increases in LW/BW at 1 day or shorter exposures consistent with our earlier analysis.¹³ The maximum LW/BW changes increased with time, whereas those values from the nontumorigenic group did not. Relative to the tumorigenic group, there were far fewer statistically significant changes in LW/BW that occurred in the nontumorigenic group.

Figure 2.

Identification and characterization of an LW/BW threshold for liver cancer. A, Distribution of LW/BW ratios across treatment times in the TG-GATES study. For each time point, the ratios were ranked for each of the tumorigenic and nontumorigenic groups. Horizontal lines represent the upper and lower thresholds derived from the TG-GATES training set. Red and blue dots are those treatments that either cause or do not cause liver tumors in chronic studies, respectively. Green-colored dots are those from treatments in which there were significant increases in genotoxicity derived from significant correlations (P value ≤ 1E-4) between the corresponding gene expression profile and the genotoxicity gene expression biomarker. Comparison number refers to the order in which the results are displayed. There were 16, 25, 27, 28, and 29 comparisons for the 1, 4, 8, 15, and 29 days’ treatment groups, respectively. B, Relationships between LW/BW threshold and activation of the 6 major MIEs for liver cancer. Chemical-dose from the TG-GATES study were categorized for tumorigenic class (Tum Class; red, positive; blue, negative), LW/BW threshold (LW/BW >1.17-fold; red bars), or activation of gene expression biomarkers (genotoxicity, AhR, CAR, ER, PPARα, cytotoxicity; -Log(P value) ≥ 4; red bars). One-dimensional clustering was used to group the 562 chemical-dose pairs (clades not shown). The findings were divided into 4 time points of exposure. AhR indicates aryl hydrocarbon receptor; CAR, constitutive activated receptor; LW/BW, liver weight to body weight; PPARα, peroxisome proliferator-activated receptor alpha.

To determine the basis of the LW/BW changes, we performed an analysis of the BW changes and the LW changes by themselves allowing an assessment of whether the changes in LW/BW were due to changes in BW, changes in LW or both. Of the significant changes, ∼12% were due to significant changes in both LW and BW, ∼33% were due to changes in LW only, and ∼9% were due to BW changes only. Almost all of the changes in BW were in the high dose group (data not shown). The rest of the changes were from treatments in which neither the LW nor the BW changed but were significant when assessed together. Thus, changes in BW by itself plays a minor role in determining the changes in LW/BW observed in this study.

Thresholds were identified using 2 independent sets of chemicals from the TG-GATES study to determine if the thresholds were similar across different sets of chemicals. The TG-GATES training set consisted of 38 chemicals, 9 of which were tumorigenic. Thresholds derived from the training set were compared to those from the test set consisting of 37 independent chemicals, 9 of which were tumorigenic. The greatest LW/BW change for any of the nontumorigenic chemical-dose pairs in the training set was 117%. Thresholds derived from the TG-GATES test set or the entire TG-GATES data set were similar or identical (116% and 117%, respectively). Using the threshold of 117% applied to the entire TG-GATES data set, there were 65 chemical-dose pairs from 14 chemicals (78% of the tumorigenic chemicals) that surpassed this threshold (Supplemental File 2). Statistically significant decreases in LW/BW were also observed for some of the tumorigenic and nontumorigenic treatments. A threshold of 83% was derived from the TG-GATES training set similar to the test set or the entire TG-GATES data set (77% for both). The threshold was only applicable to 4 chemical-dose pairs, all of which were from treatments with diethylnitrosamine (DEN), a genotoxic agent. DEN appeared to be unique among the genotoxic agents (Figure 2A, green dots) in being associated with decreases in LW/BW which exceeded the lower threshold.

We examined the relationships between the LW/BW threshold and activation of one or more of the 6 major MIEs for liver tumorigenesis. Liver weight to BW for each chemical-dose pair across 4 exposure time points was converted to a 1 or 0 based on whether the threshold of 117% was exceeded or not. Molecular initiating events were measured using gene expression biomarkers using the Running Fisher correlation test and were scored as 1 if exposure led to a -Log(P value) ≥ 4, a threshold for activation that we have used in past studies^13,21 (Figure 2B). One of the most notable features of the resultant clustering was the fact that chemical-dose pairs that were tumorigenic almost always exhibited activation of 1 or more of the biomarkers. There were a minor number of nontumorigenic chemical-dose pairs that exhibited activation of the biomarkers but were restricted mainly to conditions that activated CAR. Chemical-dose pairs that surpassed the LW/BW threshold were restricted to tumorigenic conditions, as expected and were associated with activation of AhR, CAR, ER, and PPARα. Very few treatments that led to activation of the cytotoxicity or the genotoxicity biomarkers also led to LW/BW surpassing the threshold, especially apparent at 15 days. Cytotoxicity is associated with regenerative proliferation to replace lost hepatocytes, and increases in LW/BW are not generally observed with cytotoxicants. Genotoxic agents would not be expected to increase LW/BW, as these agents do not cause liver tumors through a mitogenic MOA.¹²

The threshold for LW/BW derived from the TG-GATES training set and the cutoffs for the biomarkers (P value ≤ 1E-4) were used to determine the accuracy for predicting tumorigens. In predicting the tumorigenic chemicals in the test set, the analysis was performed on all 270 chemical-dose-times or on individual treatment times. The unit of prediction was on a chemical basis (see Methods). The predictive accuracies for the comparisons at the 4 individual time points was 93% (4 days), 96% (8 days), 89% (15 days), and 91% (29 days) (Table 2) with most of the errors being type 1 (FPs). Using individual comparisons as the unit of prediction, the accuracies were lower at each time point; the 8-day treatment still had the highest accuracy (93%) (Supplemental Table 1). In summary, an LW/BW threshold of 117% was established beyond which liver tumorigenesis occurs. The LW/BW threshold used with cutoffs for previously established gene expression biomarkers can accurately identify liver tumorigens.

Table 2.

Predictive Accuracies Using Thresholds for LW/BW and Gene Expression Biomarkers.^a

Training set	Test set	Unit of prediction	TP	TN	FP	FN	Sensitivity	Specificity	PPV	NPV	Balanced accuracy
Training set	Test set—All	Chemical	9	21	7	0	1.000	0.750	0.563	1.000	0.875
Training set	Test set—4 days	Chemical	8	27	1	1	0.889	0.964	0.889	0.964	0.927
Training set	Test set—8 days	Chemical	9	26	2	0	1.000	0.929	0.818	1.000	0.964
Training set	Test set—15 days	Chemical	8	25	3	1	0.889	0.893	0.727	0.962	0.891
Training set	Test set—29 days	Chemical	9	23	5	0	1.000	0.821	0.643	1.000	0.911

Abbreviations: FN, false negative; FP, false positive; LW/BW, liver weight to body weight; TN, true negative; TP, true positive.

^a Only TG-GATES data were used to determine predictive accuracy. The TG-GATES study was divided into training and test sets. The training set threshold for LW/BW (1.17) and P value thresholds for the gene expression biomarkers (P value ≤ 1E-4) were used in this analysis. Predictive accuracies were determined on either all time points (all) or individual time points (4, 8, 15, and 29 days). Accuracies were determined based on individual chemicals as described in the Methods.

The LW/BW threshold of 117% identified in our study is similar to values derived from other studies evaluating hypertrophy. Increases in LW/BW in rodent chemical exposure studies are mainly due to hepatocyte hypertrophy and hepatocyte hyperplasia. Hypertrophy can occur through a number of mechanisms. The most common mechanism is likely an increase in smooth endoplasmic reticulum proliferation associated with CYP enzyme induction.¹² While hepatocyte hypertrophy is not considered a risk factor by itself due to the fact that it is considered a reversible adaptive response, increases in hepatocyte proliferation is a KE in a number of liver cancer AOPs.³³ In rat studies carried out with 10 chemicals for approximately 2 weeks to 3 months, Amacher et al³⁴ found that centrilobular or panlobular hepatocellular hypertrophy was associated with increases in absolute LWs of >20%; they found no relationship between the magnitude of LW increase or hepatocellular hypertrophy and the degree of CYP enzyme induction. A European Society of Toxicological Pathology group found that an increase in LW of at least 20% was required to histologically detect a change in hepatocyte cell size¹² (corresponding to values in other publications discussed in reference³⁵. The threshold 117% indicates that the liver can increase in size to a limited extent in the absence of an associated increased risk of liver cancer. While it is likely the nontumorigenic liver increases are due in part to hepatocyte hypertrophy, we can’t say for certain that the increased LW is due to hypertrophy or hyperplasia without detailed histopathology, information that is currently lacking in the TG-GATES study.

Increases in LWs have been examined for relationships with liver tumor risk in 2-year carcinogenicity studies in rodents.^14,36 In a survey of 138 agrochemicals examined in mice, a relative increase in LW of ≥150% of control values after 1 year of treatment was positively correlated with the induction of liver tumors at 2 years.³⁶ The average relative LW of top dose animals in negative studies of 110% of control³⁶ was similar to our threshold of 117%. In a survey of mouse and rat NTP studies examining correlations between LW and/or histological parameters and carcinogenesis, Allen et al¹⁴ found that hepatocellular hypertrophy was the best single predictor of liver cancer in mice. The group found significant relationships between any (statistically significant) increases in LW across ∼170 studies and liver cancer in mice and rats. Using only increases in LW/BW, the authors could identify most of the liver tumorigens but with a high FP rate, not surprising, considering that in our study we found statistically significant increases in LW/BW under nontumorigenic conditions. In studies positive for a tumorigenic outcome, the LW/BW increase average was 150%.¹⁴ In a study examining early biochemical and tissue changes of 9 nongenotoxic NTP carcinogens in rats and mice, the authors define LW/BW as clearly positive for liver tumorigenicity (≥120%), equivocal results (110%-119%), and negative results (<110%).³⁷ Our threshold is in line with these earlier estimations but provides a more accurate increase for liver cancer induction because it is based on >50 nontumorigenic chemicals. In summary, the 117% threshold that we identified will be a useful indicator of the potential of chemicals to cause liver cancer in short-term studies using rats.

Identification of Liver Tumorigenesis Thresholds for ClinChem End Points

Thresholds for the ClinChem markers from the TG-GATES study were identified as described above for LW/BW. Figure 3A and B shows the treatment/control ratios for 10 of the ClinChem markers. An additional 10 ClinChem markers are shown in Supplemental Figure 2. Despite examination of responses across 416 comparisons, there were in general very few statistically significant changes in the nontumorigenic groups for any of the markers. Both upper and lower thresholds were determined for the training set, the test set, and the entire data set and are shown in Figure 3A and B and Supplemental Figure 2. In a number of cases, no thresholds could be determined, because no statistically significant values were found in the nontumorigenic group. For other thresholds, only 1 significant change was used to set the threshold (see Methods for a list of these end points). The thresholds derived for the ClinChem markers and LW/BW are summarized in Figure 3C. Overall, the thresholds were similar between the 2 independent sets of chemicals.

Figure 3.

Identification of thresholds for ClinChem end points. A and B, Ratios for each of the 10 indicated ClinChem end points from the TG-GATES study. Thresholds were determined as detailed in the Methods. The ratios were rank-ordered and divided into liver tumorigenic and nontumorigenic groups. Horizontal lines are at the derived upper and lower thresholds from the training set. C, The threshold values derived from the TG-GATES training set, TG-GATES test set, and the entire TG-GATES set are compared. The upper and lower thresholds are shown separately. The thresholds for the LW/BW are also shown. D, The ClinChem thresholds are similar to the absolute value of the high end of the range of historical controls. The upper and lower thresholds derived from the indicated end point were compared to the high or low end of the range of historical controls from 13 to 22 week old male Crl:CD(SD) rats. ClinChem indicates clinical chemistry; LW/BW, liver weight to body weight.

We reasoned that the thresholds would be close to or exceed the absolute values of the historical ranges for control rats. We compared our derived upper and lower thresholds to upper and lower values in ranges of ClinChem compiled from 448 to 666 control male Crl:CD(SD) rats 13- to 22 weeks-old (see Methods). For the upper thresholds, 11 of the 15 end points exhibited greater or about equal thresholds compared to the historical upper value in the reported range (Figure 3D, upper). Likewise, almost all of the lower thresholds were equal to or less than the historical lower value in the reported range (Figure 3D, lower). The outlier was TBIL in which both the upper and lower values in the range far exceeded the derived thresholds. Thus, for most of the end points, the thresholds exceeded or were similar to the upper or lower values of historical ranges.

The relationships between induction of ALT, AST, ALP, and TBIL to liver enzyme induction have been extensively reviewed, but there is little if any information about the levels associated with increases in rodent liver cancer.¹² Boone et al³⁸ concluded that increases in serum ALT activity in the range of 2- to 4-fold or higher in individual or group mean data when compared with concurrent controls should raise concern as an indicator of potential hepatic injury unless a clear alternative explanation is found. Increases in ALT activity of 2- to 3-fold should be considered indicative of hepatocellular damage based on the recommendations of a number of regulatory authorities.¹² These values are above the threshold we identified (1.41). In conclusion, we have identified thresholds for ClinChem markers beyond which liver tumors occur. These thresholds appear to be novel.

Characterization of Threshold Perturbation in a Time- and Dose-Dependent Manner

The number of times a threshold was exceeded for LW/BW or ClinChem markers in the entire TG-GATES data set is shown in Figure 4A. Thresholds for all end points never exceeded >3 incidences in the nontumorigenic conditions. In the tumorigenic conditions, some thresholds were often exceeded (> 15 times) and included upper thresholds for LW/BW, ALT, and AST as well as lower thresholds for PL, RALB, TC, and TP. Both upper and lower thresholds for BUN, CL, IP, K, NA as well as upper thresholds for CA, GLC, PL, and TC and lower thresholds for ALT, AST, and TBIL were not appreciably exceeded. For any of the end points, both upper and lower thresholds never exceeded more than 7 times each.

Figure 4.

Characterization of threshold frequency and time- and dose-dependence. A, Number of chemical-dose-times (incidences) the indicated upper or lower thresholds were exceeded across all of the comparisons in the TG-GATES data set. The graph shows that the thresholds were rarely exceeded under nontumorigenic conditions. Thresholds derived from the TG-GATES training set were used to determine the incidences across the chemical comparisons. B, The heatmap shows the number of chemical-dose-time incidences in which the indicated end points surpassed the derived upper or lower thresholds across time of exposure in the TG-GATES study. C, The heatmap shows the number of incidences in which the indicated end point surpassed the derived thresholds with increasing dose in the TG-GATES study. D, Relationship between tumorigenic chemical exposure and number of end point thresholds exceeded. Not shown are the number of comparisons from the tumorigenic and nontumorigenic groups that did not exceed any of the thresholds (33 and 404, respectively).

Treatments that resulted in thresholds being exceeded were characterized relative to exposure time (Figure 4B). Upper thresholds were not exceeded until 9 hours of exposure or after. The number of times a threshold was exceeded peaked at different exposure times including 8 (TBIL), 15 (ALT, AST), and 29 days (LW/BW, ALP, TP). All of the lower thresholds were not exceeded before 6 hours. The number of incidences peaked at 4 days (PL and TC) or later (RALB, TG, and TP). Thresholds were rarely exceeded in the nontumorigenic treatments and did not exhibit any time-dependence.

Dose-dependent changes in thresholds are shown in Figure 4C. There were dose-dependent increases in the number of incidences for a subset of upper thresholds (LW/BW, ALP, ALT, AST, TBIL, and TP) and lower thresholds (LW/BW, ALP, CA, PL, RALB, TC, and TP). Thresholds for other end points (RALB-upper, TC-upper, TG-lower) were exceeded only at the top dose. None of the end points exhibited obvious dose-dependence in the nontumorigenic group.

We also determined how many times LW/BW or ClinChem thresholds were exceeded for tumorigenic and nontumorigenic chemicals across the TG-GATES data set as measured by each chemical-dose-time comparison. A total of 17 end points were evaluated. Figure 4D shows that the number of thresholds exceeded per chemical-dose-time comparison ranged from 1 to 13, with 36 chemical-dose-time comparisons exceeding 2 end point thresholds. The frequency of chemical-dose-time comparisons exceeding thresholds decreased with the number of thresholds exceeded. Some nontumorigenic chemical-dose-time comparisons exceeded 1 end point threshold, suggesting that based on the number of end point thresholds exceeded alone, there might be greater certainty of tumorigenesis conditions if at least 2 end point thresholds are exceeded. In summary, the thresholds for a subset of the end points were exceeded in a time- and dose-dependent manner for tumorigenic chemicals, consistent with dose- and time-dependency of events in AOPs. Only tumorigenic chemicals exceeded more than 1 end point threshold in this subset. This analysis identified the ClinChem markers that most often exceed thresholds and thus might provide the greatest predictive value in short-term studies.

Relationships Between Changes in LW/BW or ClinChem Markers and the Activation of the Liver Cancer MIEs

We examined the similarity in responses across the ClinChem markers and LW/BW to determine their relatedness. All ratios from the TG-GATES 1311 comparisons for the 17 end points were compared to each other to derive a matrix of correlation coefficients used for 2-dimensional clustering (Figure 5A). Groups of end points positively correlated across the TG-GATES study included (1) ALP, AST, ALT, and TBIL, (2) TC, PL, BUN, K, and IP, (3) LW/BW, GLC, CA, TP, RALB, and NA, and (4) NA and CL. Other pairs of end points were notably negatively correlated and included (1) ALP and both K or NA, (2) ALT and both K or NA, (3) TBIL and CL, and (4) TC and both NA or CL. Many of the correlated end points were previously observed to increase together. For example, ALP, AST, ALT, and TBIL are all clinical indicators of liver damage.¹² Increases in LW/BW were associated with end points that include a liver-specific protein (RALB). The 2 indicators of blood lipid levels, TC and PL, also grouped together. Total cholesterol, PL, BUN, K, and IP all relate to interdependent kidney and liver function. The relationships between the remaining end points revealed here are potentially novel. These correlations are naive to potential covariances, which would require investigation of the relatedness of the variation in the end points. However, these correlations suggest that even in the absence of the complete set of ClinChem markers evaluated here, it may be possible to predict tumorigenicity in rats and that biologically linked ClinChem markers, such as those related to liver toxicity or kidney function, appear to correlate with one another.

Figure 5.

Relationships between changes in LW/BW, ClinChem markers, and MIEs across the TG-GATES study. A, Relationships between the LW/BW and ClinChem markers. The ratio values for each of the indicated end points across all of the chemical-dose-time comparisons in the TG-GATES study were compared to each other and correlation coefficients were derived. The values were clustered using 2-dimentional clustering (clades not shown). B, Relationships between changes in the LW/BW and ClinChem markers and the 6 gene expression biomarkers. Correlation coefficients were derived from the ratio values of the LW/BW and ClinChem markers versus the -Log(P value)s from the 6 gene expression biomarkers across all of the chemical-dose-time comparisons in the TG-GATES study. The values were clustered using 2-dimentional clustering (clades not shown). ClinChem indicates clinical chemistry; LW/BW, liver weight to body weight; MIE, molecular initiating event.

We next examined the relationships between changes in the ClinChem and LW/BW and changes in the MIEs assessed using the gene expression biomarkers. Correlations between the MIEs and the end points across the entire TG-GATES study were determined as described above (Figure 5B). For most of the markers, there was a similar pattern of correlations for AhR, CAR, genotoxicity, ER, and cytotoxicity, whereas the correlations with PPARα were noticeably different than the other MIEs. The highest positive correlations were found between the cytotoxicity biomarker and both AST and ALT, as expected and noted earlier.²¹ Other positive correlations (|correlation coefficient| ≥ 0.3) were between (1) the genotoxicity biomarker and AST or ALT, (2) the AhR biomarker and AST or ALT, (3) the CAR biomarker and LW/BW, (4) the ER biomarker and AST and ALT, and (5) the PPARα biomarker and ALP or LW/BW. Negative correlations were also noted between (1) the genotoxicity biomarker and NA, TP, or RALB; (2) the AhR biomarker and CA, TP, or RALB; (3) the ER biomarker and AGRATIO or NA; (4) the PPARα biomarker and K; and (5) the cytotoxicity biomarker and TP, NA, or RALB. The pairwise comparisons across the treatments between the ClinChem end points and MIEs are found in Supplemental Figure 3. In summary, the analysis shows that a subset of the end points exhibited patterns of changes that can be correlated to the activation of 1 or more of the MIEs.

There are numerous examples of associated events (AEs) whose increased expression/activity is associated with activation of the MIE. Examples include increased expression/activity of Cyp1a, Cyp2b, and Cyp4a protein family members as AEs for activation of AhR, CAR, or PPARα, respectively. In our analysis, we did not find compelling evidence to suggest that any of the end points could be classified as AEs with 1 or more AOPs. The highest correlations were between the cytotoxicity MIE, ALT, and AST, which was expected. However, even here, the correlations were less than perfect due to the known differences in the timing between increases in ALT/AST and activation of genes in the cytotoxicity biomarker which may be influenced by the increases in regenerative hepatocyte proliferation that occurs during the same time period.²¹ It was curious that there was fairly strong correlation (correlation coefficient ≥0.3) between AhR, genotoxicity, or ER activation and ALT and AST. This may be explained if the chemicals are not only inducing their respective receptors but inducing cytotoxicity, a reasonable assumption given that the highest dose level used in the TG-GATES study is the maximum tolerated dose. The LW/BW increases were associated with mitogenic AOPs through activation of CAR, PPARα, and to a lesser extent AhR. A surprising finding was that there wasn’t higher correlation between activation of PPARα and suppression of TG, given that agonists of PPARα are well known to decrease circulating lipid levels.³⁹ In summary, our comparisons between the ClinChem markers and MIEs revealed mostly weak correlations that do not support the use of specific ClinChem markers as AEs of the MIEs. The exception here is the use of ALT and AST as AEs for cytotoxicity. In addition, while further work is needed to determine why most of the ClinChem markers have thresholds associated with liver cancer, we speculate that many of the markers are altered in response to generalized stress responses that contribute to pathways leading to cancer.

The Thresholds Derived From the TG-GATES Study Approximate Those Found in Other Studies

We determined how the thresholds derived from TG-GATES study data compared to the highest values from the nontumorigenic comparisons in other sets of rat acute or subchronic studies. In the ToxRefDB data set, there were 258, 83, and 36 comparisons that were annotated for tumorigenicity and LW/BW, ALT, and AST changes, respectively. The highest LW/BW in the nontumorigenic group was 141% (compared to 117% from the TG-GATES study; Supplemental Figure 4A). A total of 28 of 210 nontumorigenic comparisons (13%) from 15 chemicals exceeded the threshold. Five of the comparisons were from treatments by 2 tumorigenic chemicals (acetaminophen and coumarin) at sub-tumorigenic doses. Exposure to 5 of the chemicals (1-[(5-nitrofurfurylidene)amino]hydantoin, coumarin, dipropylene glycol, endosulfan, p, p′-dichlorodiphenyl sulfone) representing 9 of the comparisons was shown to cause increases in steatosis in rats or mice (from data in ToxRefDB). For ALT, only 1 (1%) of the comparisons exceeded the threshold of 141% (coumarin) and none of the nontumorigenic comparisons exceeded the threshold of 195% for AST.

In data from studies carried out by the NTP, there were 133, 260, and 32 comparisons that could be annotated for tumorigenicity and LW/BW, ALT, or AST changes, respectively. There were 7 (10%), 6 (3%), and 1 (6%) nontumorigenic comparisons which exceeded the thresholds for LW/BW, ALT, or AST, respectively (Supplemental Figure 4B). The 7 comparisons which exceeded the LW/BW threshold were from 2 nontumorigenic chemicals/mixtures (kava kava extract, ginkgo biloba extract); in the NTP reports for these studies, focal fatty change (ginkgo biloba extract) or centrilobular fatty change (kava kava extract) were noted. Three chemicals exceeded the threshold for ALT (acetaminophen, acrolein, methacrylonitrile) and 1 exceeded the threshold for AST (acetaminophen).

The ClinChem end points in the DrugMatrix study were also evaluated. There were 1944 chemical-dose-times initially evaluated, but only 285 of the comparisons could be scored for tumorigenicity/nontumorigenicity. For the 19 ClinChem end points evaluated, statistically significant (P value <.05) changes were examined. Most of the measures did not exhibit clear thresholds. However, there were 7 measures that exhibited clear thresholds between tumorigenic and nontumorigenic groups (increases in ALB, ALP, AST, ALT, or TBIL and decreases in BUN, TC; Supplemental Figure 4C). The thresholds derived from the TG-GATES study were rarely exceeded for chemicals examined in the DrugMatrix study for the end points ALB, ALP, ALT, AST, and TBIL. These end points overlapped with those with the highest number of incidences surpassing thresholds described in Figure 4A.

In the Supplemental File 3, we discuss the relationships between the thresholds for LW/BW, ALT, and AST and liver tumorigenicity for the MARCAR study. In summary, of the 273 LW/BW comparisons from the nontumorigenic groups that we could find in ToxRefDB and NTP studies, ∼13% (35) exceeded the threshold. Causes of increases in LW/BW that may not be directly involved in tumorigenesis include fibrosis, abnormal storage of metabolism products, particles, or cleavage products and congestion.^12,40,41 One of the most obvious reasons though may be due to benign increases in fat accumulation in the liver. In fact, we found that in 16 comparisons, there was evidence of steatosis. Steatosis is considered a risk factor for liver tumorigenesis, and although is usually reversible, steatosis can progress to nonalcoholic steatohepatitis, cirrhosis, and cancer. If we remove these chemical-dose combinations that led to steatosis, only 7% of the comparisons exceeded the threshold. Thus, there may be other factors involved that contribute to LW/BW increases in these nontumorigenic conditions. These could include differences in whether the animals were fasted before sacrifice. Studies have shown that rodent LW increases in fed animals were maintained for up to 8 hours after feeding.⁴² Considerable interlaboratory variation in the practice of overnight fasting of animals prior to necropsy has been noted.¹² In the evaluation of a new chemical entity, the use of the LW/BW threshold will be guided by typical histopathological evaluation for fatty change. A LW/BW increase that exceeds the threshold would be less of a concern if associated with increases in steatosis. Additionally, it should be acknowledged that the threshold for LW/BW was calibrated for short-term studies and, while similar, may not reflect the true threshold at different ages and exposure times in the rat.

Thresholds for LW/BW and ClinChem Markers Can Be Used to Identify Liver Tumorigens

The thresholds for LW/BW and ClinChem markers derived from the TG-GATES training set were used to identify tumorigens in the TG-GATES test set. Using the training set to predict the test set, the predictive accuracies based on the entire 270 chemical-dose-times was 89% (Table 3). Examining individual time points, the highest predictive accuracy was at 8 (98%) days. Using individual comparison as the unit of prediction, the highest accuracy was using the 15-day samples (89%; Supplemental Table 2). Most of the errors were type 1. Thus, the thresholds for LW/BW and ClinChem markers derived from the training set can accurately identify liver tumorigens in an independent set of chemicals.

Table 3.

Predictive Accuracies Using Thresholds for 7 ClinChem Markers.^a

Training set	Test set	Unit of prediction	TP	TN	FP	FN	Sensitivity	Specificity	PPV	NPV	Balanced accuracy
TG-GATES training set	TG-GATES test set—All	Chemical	9	22	6	0	1.000	0.786	0.600	1.000	0.893
TG-GATES training set	TG-GATES test set—4 days	Chemical	8	27	1	1	0.889	0.964	0.889	0.964	0.927
TG-GATES training set	TG-GATES test set—8 days	Chemical	9	27	1	0	1.000	0.964	0.900	1.000	0.982
TG-GATES training set	TG-GATES test set—15 days	Chemical	9	25	3	0	1.000	0.893	0.750	1.000	0.946
TG-GATES training set	TG-GATES test set—29 days	Chemical	8	25	3	1	0.889	0.893	0.727	0.962	0.891
TG-GATES-all	DrugMatrix	Chemical	20	53	13	4	0.833	0.803	0.606	0.93	0.818

Abbreviations: ClinChem, clinical chemistry; FN, false negative; FP, false positive; TC, total cholesterol; TN, true negative; TP, true positive.

^a There are 2 sets of predictions found in this table. The first set uses the TG-GATES training set to predict tumorigenicity in the TG-GATES test set. The upper threshold for LW/BW and the upper and lower thresholds for the ClinChem markers were used. Predictive accuracies were determined on either all time points (all) or individual time points (4, 8, 15, 29 days). In the second set of predictions (last line), the thresholds derived from the entire TG-GATES data set were used to predict the tumorigenicity of chemicals in the DrugMatrix data set. In this case, only 5 upper thresholds (ALB, TBIL, ALT, ALP, and AST) and 2 lower thresholds (BUN, TC) were used in the predictions. Accuracies were determined based on individual chemical as described in the Methods.

We also determined whether the thresholds derived from the TG-GATES study could be used to identify tumorigens in the DrugMatrix study. Using only the 5 upper thresholds (ALB, TBIL, ALT, ALP, AST) and the 2 lower thresholds (BUN, TC) described in Supplemental Figure 4C including those that were most often altered (Figure 4A), the predictive accuracy using the chemical as the unit of prediction was 82% (Table 3). Using individual comparison as the unit of prediction the accuracy was 74% (Supplemental Table 2). Thus, a subset of the thresholds derived in 1 study could be used to make reasonable predictions of the ability of chemicals to cause liver tumors in 2-year bioassays based only on ClinChem markers in an independent study.

Summary

In our previous study, we showed that gene expression biomarkers that predict activation of the 6 major MIEs for liver cancer as well as individual genes in the biomarkers exhibit measurable thresholds in short-term studies beyond which liver cancer occurs.²¹ These studies also showed that the thresholds could be used in acute and subacute studies to accurately predict the occurrence of liver cancer in chronically treated rats. As not all laboratories incorporate gene expression profiling into chemical evaluation studies, we determined whether more typical measures taken in the course of rodent studies also exhibit thresholds and whether those thresholds can be used to predict cancer. The common measures that were examined included LW/BW and 21 ClinChem end points, which are routinely examined in short-term rodent studies. Our analysis was facilitated by the availability of large data sets which allowed us to examine changes in these parameters under dosing conditions known to lead to predictable liver cancer outcomes in 2-year studies. We found that LW/BW and a subset of the ClinChem markers exhibited thresholds that were consistent across treatment conditions for different sets of chemicals. For end points which exhibited thresholds, most exhibited dose- and time-dependent increases in the number of times a threshold was exceeded. Thresholds derived from the TG-GATES data set were consistent across other studies. Importantly, the thresholds could be used to predict liver cancer. The identified upper LW/BW threshold of 117% could be used in conjunction with gene expression biomarkers to predict liver tumorigenesis (up to 96% accuracy). Using only LW/BW and ClinChem thresholds derived from a TG-GATES training set, the accuracy of predicting liver tumors in the TG-GATES test set was 89%. These thresholds were most predictive when applied to 8-day treatments (98%). In the absence of the LW/BW threshold, a subset of ClinChem thresholds by themselves could be used to predict tumorigenesis across studies with reasonable accuracy (82%). Our study provides support that chemical-independent thresholds exist for a set of common end points in rodent studies and that the thresholds can be used to predict tumorigenic potential of chemicals. The mechanistic basis for why thresholds exist for a subset of ClinChem measures that are associated with liver tumor induction requires further study.

The thresholds identified in the present study have the potential to be applied to future rat studies to predict tumorigenic potential of chemicals and to provide guidance in setting doses for 2-yr bioassays, especially when used in conjunction with the gene expression biomarkers and their thresholds described in previous studies.^13,21,27 After preliminary short-term exposure studies, the toxicological thresholds could be used to help bracket the range of doses between the benchmark dose and the calculated dose at the tumorigenic thresholds. These predictions could be used to allow informed decisions to be made of doses to use in chronic studies to avoid tumor induction. Given that the rat liver tumor response is often the only target organ for industrial chemicals and potential pharmaceuticals,^19,43 90 days, 6 months, or 12 months of exposure often identify histopathology risk factors for liver cancer that can be further examined in short-term studies such as the ones described in the present analysis. The thresholds for LW/BW and ClinChem measures as well as assessment of induction of any MIEs and their thresholds assessed by global gene expression would provide a set of predictions to identify doses that would be tumorigenic and the chemical MOA. This information could be potentially used to make the case that because the MOA is identified, the 2-year bioassay is not required. Further studies would be required to understand the strengths and weaknesses of the approach. However, the approach is consistent with efforts to reduce carcinogenicity testing under the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH) S1 guidance modification initiatives. Modifications to ICH S1 Carcinogenicity Testing Guidance⁴⁴ proposes a more flexible approach to pharmaceutical carcinogenicity testing allowing for adequate assessment of carcinogenic risk without the need for always conducting a 2-year rat carcinogenicity study. This modification in the guidance may enable drug sponsors to gain 2-year rat carcinogenicity study waivers through a Carcinogenicity Assessment Document–based justification process. The approach is also consistent with EPA’s long-term goal to move toward making Toxic Substances Control Act decisions with NAMs in order to reduce, refine, or replace vertebrate animal testing (https://www.epa.gov/sites/production/files/2018-06/documents/epa_alt_strat_plan_6-20-18_clean_final.pdf; accessed August 21, 2019).⁴⁵

Supplemental Material

Supplemental Material, sj-docx-1-tpx-10.1177_0192623320960412 - Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals

Supplemental Material, sj-docx-1-tpx-10.1177_0192623320960412 for Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals by J. Christopher Corton, Katharine L. Korunes, Jaleh Abedini, Hisham El-Masri, Jason Brown, Katie Paul-Friedman, Ying Liu, Cari Martini, Shihan He and John Rooney in Toxicologic Pathology

Supplemental Material

Supplemental Material, sj-docx-2-tpx-10.1177_0192623320960412 - Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals

Supplemental Material, sj-docx-2-tpx-10.1177_0192623320960412 for Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals by J. Christopher Corton, Katharine L. Korunes, Jaleh Abedini, Hisham El-Masri, Jason Brown, Katie Paul-Friedman, Ying Liu, Cari Martini, Shihan He and John Rooney in Toxicologic Pathology

Supplemental Material

Supplemental Material, sj-tif-1-tpx-10.1177_0192623320960412 - Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals

Supplemental Material, sj-tif-1-tpx-10.1177_0192623320960412 for Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals by J. Christopher Corton, Katharine L. Korunes, Jaleh Abedini, Hisham El-Masri, Jason Brown, Katie Paul-Friedman, Ying Liu, Cari Martini, Shihan He and John Rooney in Toxicologic Pathology

Supplemental Material

Supplemental Material, sj-tif-2-tpx-10.1177_0192623320960412 - Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals

Supplemental Material, sj-tif-2-tpx-10.1177_0192623320960412 for Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals by J. Christopher Corton, Katharine L. Korunes, Jaleh Abedini, Hisham El-Masri, Jason Brown, Katie Paul-Friedman, Ying Liu, Cari Martini, Shihan He and John Rooney in Toxicologic Pathology

Supplemental Material

Supplemental Material, sj-tif-3-tpx-10.1177_0192623320960412 - Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals

Supplemental Material, sj-tif-3-tpx-10.1177_0192623320960412 for Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals by J. Christopher Corton, Katharine L. Korunes, Jaleh Abedini, Hisham El-Masri, Jason Brown, Katie Paul-Friedman, Ying Liu, Cari Martini, Shihan He and John Rooney in Toxicologic Pathology

Supplemental Material

Supplemental Material, sj-tif-4-tpx-10.1177_0192623320960412 - Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals

Supplemental Material, sj-tif-4-tpx-10.1177_0192623320960412 for Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals by J. Christopher Corton, Katharine L. Korunes, Jaleh Abedini, Hisham El-Masri, Jason Brown, Katie Paul-Friedman, Ying Liu, Cari Martini, Shihan He and John Rooney in Toxicologic Pathology

Supplemental Material

Supplemental Material, sj-txt-1-tpx-10.1177_0192623320960412 - Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals

Supplemental Material, sj-txt-1-tpx-10.1177_0192623320960412 for Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals by J. Christopher Corton, Katharine L. Korunes, Jaleh Abedini, Hisham El-Masri, Jason Brown, Katie Paul-Friedman, Ying Liu, Cari Martini, Shihan He and John Rooney in Toxicologic Pathology

Supplemental Material

Supplemental Material, sj-xlsx-1-tpx-10.1177_0192623320960412 - Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals

Supplemental Material, sj-xlsx-1-tpx-10.1177_0192623320960412 for Thresholds Derived From Common Measures in Rat Studies Are Predictive of Liver Tumorigenic Chemicals by J. Christopher Corton, Katharine L. Korunes, Jaleh Abedini, Hisham El-Masri, Jason Brown, Katie Paul-Friedman, Ying Liu, Cari Martini, Shihan He and John Rooney in Toxicologic Pathology

Footnotes

Acknowledgments

The authors thank Dr. Heidrun Ellinger-Ziegelbauer for data from the MARCAR study, Drs. Jim Klaunig, Dave Allen, and Dave Malarkey for critical review of the manuscript, and Molly Windsor and Keith Tarpley for assistance in creating the figures.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article. The information in this document has been subjected to review by the Center for Computational Toxicology and Exposure and approved for publication. Approval does not signify that the contents reflect the views of the Agency, nor does mention of trade names or commercial products constitute endorsement or recommendation for use.

Funding

Funding for this study came from the U.S. EPA Office of Research and Development.

ORCID iD

J. Christopher Corton

Supplemental Material

Supplemental material for this article is available online.

References

European Chemical Agency—List of Pre-registered Substances. Accessed May 17, 2017. http://apps.echa.europa.eu/preregistered/pre-registered-sub.aspx

Muir

Howard

. Are there other persistent organic pollutants? A challenge for environmental chemists. Environ Sci. 2006;40(1):7157–7166.

Bucher

Portier

. Human carcinogenic risk evaluation, part V: the national toxicology program vision for assessing the human carcinogenic hazard of chemicals. Toxicol Sci. 2004;82(2):363–366.

Gold

Manley

Slone

Rohrbach

Garfinkel

. Supplement to the carcinogenic potency database (CPDB): results of animal bioassays published in the general literature through 1997 and by the national toxicology program in 1997-1998. Toxicol Sci. 2005;85(2):747–808.

Waters

Jackson

Lea

. Characterizing and predicting carcinogenicity and mode of action using conventional and toxicogenomics methods. Mutat Res. 2010;705(3):184–200.

Gold

Slone

Bernstein

. Summary of carcinogenic potency and positivity for 492 rodent carcinogens in the carcinogenic potency database. Environ Health Perspect. 1989;79:259–272. doi: 10.1289/ehp.8979259

Haseman

. Using the NTP database to assess the value of rodent carcinogenicity studies for determining human cancer risk. Drug Metab Rev. 2000;32(2):169–186. doi: 10.1081/dmr-100100570

Boobis

Cohen

Dellarco

, et al. IPCS framework for analyzing the relevance of a cancer mode of action for humans. Crit Rev Toxicol. 2006;36(10):781–792.

USEPA. Guidelines for Carcinogen Risk Assessment; 2005.

10.

Edwards

Tan

Villeneuve

Meek

McQueen

. Adverse outcome pathways-organizing toxicological information to improve decision making. J Pharmacol Exp Ther. 2016;356(1):170–181.

11.

Ankley

Bennett

Erickson

, et al. Adverse outcome pathways: a conceptual framework to support ecotoxicology research and risk assessment. Environ Toxicol Chem. 2010;29(3):730–741.

12.

Hall

Elcombe

Foster

, et al. Liver hypertrophy: a review of adaptive (adverse and non-adverse) changes—conclusions from the 3rd international ESTP expert workshop. Toxicol Pathol. 2012;40(7):971–994.

13.

Rooney

Hill

3rd Qin

Sistare

Corton

. Adverse outcome pathway-driven identification of rat liver tumorigens in short-term assays. Toxicol Appl Pharm. 2018;356:99–113.

14.

Allen

Pearse

Haseman

Maronpot

. Prediction of rodent carcinogenesis: an evaluation of prechronic liver lesions as forecasters of liver tumors in NTP carcinogenicity studies. Toxicol Pathol. 2004;32(4):393–401.

15.

Igarashi

Nakatsu

Yamashita

, et al. Open TG-GATEs: a large-scale toxicogenomics database. Nucleic Acids Res. 2015;43(Database Issue):D921–D927.

16.

Fielden

Brennan

Gollub

. A gene expression biomarker provides early prediction and mechanistic assessment of hepatic tumor induction by nongenotoxic chemicals. Toxicol Sci. 2007;99(1):90–100.

17.

Conolly

Ankley

Cheng

, et al. Quantitative adverse outcome pathways and their application to predictive toxicology. Environ Sci. 2017;51(8):4661–4672.

18.

Knudsen

Keller

Sander

, et al. Future tox II: in vitro data and in silico models for predictive toxicology. Toxicol Sci. 2015;143(2):256–267.

19.

Hill

3rd Nelms

Edwards

, et al. Editor’s highlight: negative predictors of carcinogenicity for environmental chemicals. Toxicol Sci. 2017;155(1):157–169.

20.

Wittwehr

Aladjov

Ankley

, et al. How adverse outcome pathways can aid the development and use of computational prediction models for regulatory toxicology. Toxicol Sci. 2017;155(2):326–336.

21.

Corton

Hill

3rd Sutherland

Stevens

Rooney

. A set of six gene expression biomarkers identify rat liver tumorigens in short-term assays. Toxicol Sci. 2020:kfaa101. In press.

22.

Uehara

Minowa

Morikawa

, et al. Prediction model of potential hepato carcinogenicity of rat hepatocarcinogens using a large-scale toxicogenomics database. Toxicol Appl Pharm. 2011;255(3):297–306.

23.

Watford

Ly Pham

Wignall

Shin

Martin

Friedman

. Tox RefDB version 2.0: improved utility for predictive and retrospective toxicology analyses. Reprod Toxicol. 2019;89:145–158.

24.

Ellinger-Ziegelbauer

Gmuender

Bandenburg

Ahr

. Prediction of a carcinogenic potential of rat hepatocarcinogens using toxicogenomics analysis of short-term in vivo studies. Mutat Res. 2008;637(1-2):23–39.

25.

Romer

Eichner

Metzger

, et al. Cross-platform toxicogenomics for the prediction of non-genotoxic hepatocarcinogenesis in rat. PLoS One. 2014;9(5):e97640.

26.

Thomson

PDR

, ed. The Physicians’ Desk Reference, 51st ed. 1997:2998.

27.

Hill

3rd Rooney

Abedini

El-Masri

Wood

Corton

. Gene expression thresholds predictive of rat liver tumorigens in short-term assays. Toxicol Sci. 2020:kfaa102. In press.

28.

Kupershmidt

Grewal

, et al. Ontology-based meta-analysis of global collections of high-throughput public data. PLoS One. 2010;5(9):e13066.

29.

Oshida

Vasani

Jones

, et al. Identification of chemical modulators of the constitutive activated receptor (CAR) in a gene expression compendium. Nucl Recept Signal. 2015;13:e002.

30.

Oshida

Vasani

Thomas

, et al. Screening a mouse liver gene expression compendium identifies modulators of the aryl hydrocarbon receptor (AhR). Toxicology. 2015;336:99–112.

31.

Oshida

Vasani

Thomas

, et al. Identification of modulators of the nuclear receptor peroxisome proliferator-activated receptor alpha (PPA Ralpha) in a mouse liver gene expression compendium. PLoS One. 2015;10(4):e0124224.

32.

Oshida

Vasani

Waxman

Corton

. Disruption of STAT5b-regulated sexual dimorphism of the liver transcriptome by diverse factors is a common event. PLoS One. 2016;11(3):e0148308.

33.

Cohen

. Evaluation of possible carcinogenic risk to humans based on liver tumors in rodent assays: the two-year bioassay is no longer necessary. Toxicol Pathol. 2010;38(3):487–501.

34.

Amacher

Schomaker

Burkhardt

. The relationship among microsomal enzyme induction, liver weight and histological change in rat toxicology studies. Food Chem Toxicol. 1998;36(9-10):831–839.

35.

Ennulat

Walker

Clemo

, et al. Effects of hepatic drug-metabolizing enzyme induction on clinical pathology parameters in animals and man. Toxicol Pathol. 2010;38(5):810–828.

36.

Carmichael

Enzmann

Pate

Waechter

. The significance of mouse liver tumor formation for carcinogenic risk assessment: results and conclusions from a survey of ten years of testing by the agrochemical industry. Environ Health Perspect. 1997;105(11):1196–1203.

37.

Elcombe

Peffer

Wolf

, et al. Mode of action and human relevance analysis for nuclear receptor-mediated liver toxicity: a case study with phenobarbital as a model constitutive androstane receptor (CAR) activator. Crit Rev Toxicol. 2014;44(1):64–82.

38.

Boone

Meyer

Cusick

, et al. Selection and interpretation of clinical pathology indicators of hepatic injury in preclinical studies. Vet Clin Pathol. 2005;34(3):182–188.

39.

Corton

Cunningham

Hummer

, et al. Mode of action framework analysis for receptor-mediated toxicity: the peroxisome proliferator-activated receptor alpha (PPARalpha) as a case study. Crit Rev Toxicol. 2014;44(1):1–49.

40.

Carthew

Nolan

Edwards

Smith

. The role of cell death and cell proliferation in the promotion of rat liver tumours by tamoxifen. Cancer Lett. 1996;106(2):163–169.

41.

Maronpot

Yoshizawa

Nyska

, et al. Hepatic enzyme induction: histopathology. Toxicol Pathol. 2010;38(5):776–795.

42.

Leveille

Chakrabarty

. Diurnal variations in tissue glycogen and liver weight of meal-fed rats. J Nutr. 1967;93(4):546–554.

43.

Sistare

Morton

Alden

, et al. An analysis of pharmaceutical experience with decades of rat carcinogenicity testing: support for a proposal to modify current regulatory guidelines. Toxicol Pathol. 2011;39(4):716–744.

44.

International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use. Proposed Change to Rodent Carcinogenicity Testing of Pharmaceuticals—Regulatory Notice Document. Published 2015. Accessed on May 31, 2019. http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Safety/S1/S1_RND_Revisions_22Dec2015_Final.pdf

45.

Thomas

Bahadori

Buckley

, et al. The next generation blueprint of computational toxicology at the U.S. environmental protection agency. Toxicol Sci. 2019;169(2):317–332.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB

0.01 MB

0.88 MB

1.44 MB

2.48 MB

1.42 MB

0.01 MB

0.52 MB