The AOP Concept: How Novel Technologies Can Support Development of Adverse Outcome Pathways

Abstract

Although the introduction of the adverse outcome pathway (AOP) concept has already triggered a strong increase in the awareness that toxicological activity also needs to be studied at the molecular and cell biological level, the practical implementation of the AOP concept faces a multitude of challenges. The AOP concept first requires the dissection of the observed adverse outcome (AO) into individual events and their subsequent composition into simplified linear AOPs. However, for most adverse effects there is hardly sufficient information about the underlying molecular mechanisms available. To describe relevant events that lead to AOs, the use of high-throughput, high-content, and omics approaches seems to be particularly suitable, if they can be integrated in a standardized and systematic way. Subsequently, the complex molecular and cellular interactions of a linear AOP have to be combined in AOP networks to describe toxicological responses in an intact organism. Finally, essential key events (KE) within the whole network have to be identified and robust in silico, in chemico, or in vitro assays reflecting the molecular or cellular principles underlying these central KE have to be developed and integrated into reliable and predictive testing strategies. Here we discuss some of the challenges ahead that need to be addressed to allow the AOP concept to live up to its promises.

Introduction

Adverse outcome pathways (AOPs) are based on the mode of action (MoA) concept and provide a stepwise mechanistic overview covering different levels of biological organization. AOPs correlate a direct molecular initiating event (MIE), that is the interaction of a chemical with a biological system, and an adverse outcome (AO) via a series of key events (KE) that are encountered at different levels of biological organization.^1,2

Over the last years, the development and implementation of the AOP concept have become a central project of the Environment, Health and Safety Programme of the Organisation of Economic Co-operation and Development (OECD). In a proof-of-concept study, a first AOP on skin sensitization was published in 2012.^3,4 This AOP correlated covalent binding to skin proteins as the MIE leading to the activation of keratinocytes (KE 2) as well as dendritic cells (KE 3) and subsequent migration of dendritic cells into lymph nodes to activate T cells (KE 4) that finally mediate the immunological response on recurrent exposure to sensitizing chemicals. The release of this AOP structured and moved forward the development of in chemico and in vitro methods reflecting the underlying molecular and cellular principles and facilitated the approval of corresponding OECD test guidelines (OECD TG 442C, 442D, 442E).

In parallel, a much more comprehensive project was launched to establish a toxicological knowledge framework supporting chemical risk assessment based on mechanistic causality coordinated by the Extended Advisory Group on Molecular Screening and Toxicogenomics (EAGMST). To collect information on AOPs in a qualitative way, AOP Wiki has been established. AOP Wiki integrates knowledge and data from many different disciplines being reminiscent of popular crowdsourcing platforms. Up to now, more than 200 AOPs are listed in AOP Wiki, of which more than 50 are actively pursued in the AOP development program. They are subjected to a two-step internal and external review process before approval and publishing in the OECD series on AOPs. The first five AOPs were recently endorsed and published together with a user handbook.^5–10 Rather to intend the use of AOPs in a regulatory context, the endorsement of an AOP acknowledges the acceptance of the current state of science and the corresponding uncertainties. The endorsed AOPs will remain part of AOP Wiki and evolve as new data may be generated and added.

The development of the online software tools continuously progresses with, for example, the AOP Effectopedia that provides additional quantitative information about AOPs to complement the qualitative, linear AOP descriptions of AOP Wiki as well as the recent launch of the AOP knowledge base (http://aopkb.org/) as a main entry site, allowing to deal with the rapid increase in information and complexity. However, the experiences gained with the successful implementation of the AOP on skin sensitization as well as the development of additional AOPs revealed a number of challenges that urgently need to be addressed to ensure an efficient progress of the AOP program.

In the case of skin sensitization, it is important to keep in mind that test methods addressing the MIE or single KEs (TG 442C, 442D, 442E) are not intended and also not suitable to be used as stand-alone methods but rather need to be combined and used in testing strategies. Last year the OECD published both a general framework for the reporting of such strategies and a series of 12 so-called defined approaches (DA) described in a harmonized manner based on this framework.^11,12 Importantly, DAs are based on a fixed data interpretation procedure and can therefore be standardized. Different data interpretation procedures are combined with different data sources (e.g., in silico, in chemico, in vitro) to allow hazard and in some cases also potency prediction for skin sensitization. These DAs proved the applicability of the AOP concept.

Further work, however, is needed to achieve their regulatory acceptance. In particular, it is unclear how the applicability of the different DAs for regulatory decision-making in the various areas can be verified or validated. This question can indeed become a “bottleneck” in the whole process. Thus, it needs to be urgently addressed as it is expected that the increasing number of AOPs will result in the concomitant development of an even higher number of testing strategies or DAs requiring a standardized and efficient process for validation and implementation.

In addition, a high level of confidence and specificity of an AOP is important to derive test methods and DAs to be applied in a regulatory context.¹³ Particularly, the specificity of the MIE and the AO, the plausibility and consistency of experimental data supporting the biological relevance of KEs, and a transparent and comprehensive definition of uncertainties and data or knowledge gaps are fundamental requirements in AOP development to prevent oversimplification of biological diversity and complexity.¹⁴ Here it seems obvious that the relevance of an AOP highly depends on the availability of information on chemicals that trigger a specific adverse effect via the respective AOP. However, for most chemicals, the molecular mechanisms mediating toxicity are unknown and AOP development itself relies on the knowledge of the molecular MoA of a few substances only. Even an AOP based on limited experimental evidence can be very helpful to support targeted development and validation of methods addressing a potentially relevant KE.

Only after KEs are defined and appropriate methods are available they can be evaluated for their regulatory relevance. Subsequently, the applicability domain and relevance of the AOP will become apparent. Thus, the development of efficient high-content (HC) and high-throughput (HT) screening methods is of particular importance to verify the relevance of single KEs and KE relationships as well as the importance of an AOP itself. For now the selection of AOPs is triggered by the current knowledge of specific toxicity pathways and not necessarily by the regulatory need for the development of specific AOPs. Here, a targeted identification of relevant molecular mechanisms is needed and with the improvement of novel technologies in biosciences this seems feasible. In turn, AOPs help to collect and structure the current available knowledge. Therefore, AOPs are a useful tool to identify data or knowledge gaps and hence to prioritize future research to elucidate mechanisms of toxicity and to develop appropriate detection methods.

Finally, AOPs are predominantly described in a linear structure consisting of a series of consecutive steps poorly reflecting the complex in vivo situation that is in fact characterized by cross talks and positive or negative feedback mechanisms. Therefore, the AOP concept aims at the formation of larger networks sharing KEs that better represent the complexity of pathways leading to an AO. Such AOP-Networks (AOPN) could provide an even more comprehensive knowledge of toxicity pathways and would allow selecting the most essential KEs for further assay development and integration of these assays in testing strategies (or DAs).

In sum, the AOP project aims at nothing less but a comprehensive understanding of toxicity in general, which is a very ambitious project and still in its infancy. The applicability of the AOP concept for the dissection of complex toxicological networks, the integration of HC and HT data in the development of mechanism-based testing and assessment strategies, and the role of the AOP concept in a regulatory context obviously need much more work and discussion.¹³ At this point in time, it seems particularly important to reflect on the way AOPs are developed and how they can be informed from other areas of research.

How High-Throughput Screening and High-Content Screening Can Support AOP Development

There is a strong demand for HT-compatible in vitro tests for (1) developing AOPs, (2) targeted development of in vitro assays, and (3) testing compounds in multiple assays that currently lack proper toxicity data.¹⁵ In any case, the use of HT screening (HTS) and HC screening (HCS) methods is of high interest to accelerate the pace of toxicological risk assessment.

For purposes of definition: in HTS, usually a single parameter is measured in an in vitro assay for a large number of test compounds. In HCS—also known as automated microscope-based screening—multiple parameters are extracted from a cell-based assay yielding an HC of information.¹⁶ Compared to HTS, usually a lower number of compounds are tested in HCS due to the higher complexity of the assay, resulting data and evaluation.

HTS is a low-cost and fast testing strategy that has been proposed as a potential alternative to animal testing. In various disease-specific studies, a good correlation between the HTS bioassays and animal toxicity was observed.¹⁷ Prominent approaches for applying HTS in toxicity testing are the Toxicity Forecaster (ToxCast) program (phase I and II) and the Toxicology Testing in the 21^st Century (Tox21) collaboration, where large sets of chemically diverse compounds are tested in a large battery of in vitro assays to prioritize chemicals for further toxicity testing and to develop prediction models for toxicity, which can be used for grouping of compounds with similar AOs.¹⁸ The ToxCast program includes over 1600 different cell-based and biochemical assay endpoints from heterogeneous producers, experimental designs, readouts, and input file formats spanning up to 8200 unique chemicals.¹⁹

For toxicity assessment, compound profiles have to be tested for their applicability as predictive classification tools, that is, whether compounds with a similar MoA are grouped together and whether the results reflect or agree with in vivo data contained in the Toxicity Reference Database (ToxRefDB).²⁰ For AOP development, such compound profile could comprise results obtained from all KE-relevant in vitro assays.

Image-based HCS could also become a useful tool for toxicity testing as a higher amount of data can be extracted from each experiment. It is additionally of advantage that testing occurs in the living cellular environment. However, no image-based HCS method has yet been approved for toxicity assessment, which might be due to the novelty of the techniques and, thus, still existing difficulties in standardization as well as limited reproducibility and transferability.²¹

Image-based HCS has, though, often been applied for analyzing the effects of small molecules on cellular morphology and for profiling these compounds according to their MoA.^22–28 The subsequent calculation of compound profiles using the information extracted from image data required a diligent selection of various statistical methods, reviewed in Refs.^25,27 Ljosa et al. directly compared several profiling methods with regard to their ability to predict the MoA and found accuracies ranging from 64% to 94%, indicating the complexity of these approaches.²⁵ HCS has also been applied to predict toxicity such as nephrotoxicity as well as liver toxicity or injury.^26,29–32 The use of such profiling approaches is promising but challenging, a fact that is underlined by the results of an HCS-based profiling for drug-induced liver injury that yielded 50% sensitivity.²⁶

However, a prerequisite for regulatory applicability of HT or HC data is the availability of well-defined reference and performance controls. Moreover, the definition of standards would improve the comparability of data obtained from different platforms²¹ and beside; endpoint-specific controls should be defined for every assay.³³ So far, no comparisons starting from assay preparation to data evaluation have been performed between different laboratories, which would be a prerequisite for validation of HTS/HCS.

Summarizing, HTS/HCS is of high interest and has a high potential to fill data gaps in toxicity knowledge generation. However, to reach regulatory acceptance in the context of an AOP-testing battery, both technologies have to fulfill several requirements: it has to be decided (1) how many reference compounds should be included, (2) how sensitive a single assay (as a check for KE) or the sum of assays (AOP-testing battery) should be, and (3) how reproducibility and verification could be reached.

How Omics Technologies Can Inform AOP Development

Omics technologies are becoming more widely used in toxicology, necessitating their consideration in human health hazard and risk assessment programs. Especially for toxicogenomics a growing data basis is available.³⁴

For risk assessment, omics approaches are already used for formulating hypotheses regarding the mechanisms of action. Omics data, however, are not only promoting the development of new AOPs but AOPs or comparable working hypotheses are actually a prerequisite to link omics data to adversity as well.³⁵ If not linked to an AO at the organism level, omics data cannot be used in a regulatory context.³⁶ The challenge is no longer how to generate vast bodies of omics data but rather how to ensure the quality of said data and how to best extract relevant information.^37,38 For omics data to be used for AOP construction and in a regulatory context, several quality requirements have to be fulfilled, since the quality of the data highly depends on experimental design and sample preparation as well as the computational models used for their integration.

The diversity of the technological omics platforms, the complexity of biological systems, and the variety of analytical and computational methods, however, hamper the adoption of measures and procedures for ensuring data quality and reproducibility.³⁹ Recommendations range from common metadata checklists and data publications for omics studies⁴⁰ to overall strategies consisting of standardized biological samples and assessing within-run, within-laboratory, and between-laboratory reproducibility of omics analysis³⁹ to warrant the quality and the usability of generated data, and ultimately to circumvent the reproducibility crisis.

However, even if raw data are reproducible, different data analysis tools can lead to contradictory results. In 2013 and 2015, two reports compared identical transcriptomic data sets of human inflammatory diseases with data sets from mouse models using traditional gene-to-gene comparison techniques. Intriguingly, these analyses resulted in conflicting conclusions regarding the relevance of animal models for translational research.^41,42 This example demonstrates that standardization and the use of appropriate, unbiased data analysis strategies are indispensable.

Gene set enrichment analysis (GSEA) is one strategy that overcomes a lot of limitations associated with gene-to-gene comparison techniques.⁴³ GSEA uses all available data for all detected transcripts and statistically evaluates transcriptomic data at the level of a priori defined gene sets, for example, all genes previously shown to belong to a certain signaling pathway. In this manner, regulated gene sets (e.g., signaling pathways) can be detected.⁴³ Using GSEA to reanalyze the transcriptomic data sets mentioned above helped to identify the optimal mouse model for translational research, which may lead to a reduction in animal use in future toxicological research.⁴⁴ Moreover, GSEA is discussed to potentially inform the development of AOPs since it allows to derive sustained molecular alterations.^38,45,46

Outputs from different omics techniques can be combined, allowing confirmation of findings on different levels and the identification of signaling pathways that lead to AOs, which would in turn facilitate the use of omics methods in regulatory toxicology.^36,47

However, the harmonization of quality standards for experimental design, sample preparation and analysis, as well as statistical analyses is a crucial step toward regulatory acceptance of omics data. Moreover, it is important that the results are presented in a transparent and comprehensible way. In this manner, the potential for the translation of the obtained results into regulatory toxicology would be greatly enhanced, since toxicology requires solid reproducible information on potential adverse health effects.^47,48

AOPNs and Systems Biology

The capability of a linear AOP predicting the pathological/physiological response of an intact organism or at least of a particular tissue is severely limited because of the complexity of the underlying molecular networks. Hence, only the integration of individual linear AOPs into a complex regulatory network will allow to draw a full picture of pathological/physiological responses after exposure to a toxicant. The implementation of advanced molecular biosciences in combination with HCS/HTS and omics technology shifted the focus of toxicological research from single KE studies to molecular regulatory networks and opened up the field of systems toxicology.^49,50

Systems toxicology already combines principles of biology, chemistry, computer science, engineering, mathematics, and physics. It aims at the construction of toxicant-based computational models and thus may help to optimize AOP-based in vitro testing frameworks. In the coming years, the great challenge in developing AOPN will be to understand and to start modeling the complex molecular and cellular interactions underlying the toxicological response of an intact organism. In the long run, this development might transform toxicology from an observational to a predictive science in which outcomes and risks are inferred from accumulated mechanistic understanding.^51,52

For this purpose, different levels of regulatory networks, including transcriptional networks, protein–protein interaction networks, and metabolic networks, have to be considered. Due to the emerging complexity of such networks, single KEs do not sufficiently predict an AO. Thus, an appropriate strategy to predict the biological consequences for an intact organism is the identification of central KEs with strong KE relationships covering numerous linear pathways within the whole AOPN (Fig. 1).

FIG. 1.

How novel technologies can support development of AOPs and DAs. (A) Schematic representation of a linear AOP linking an MIE via a series of KE to an AO. (B) Increasing data and knowledge generation by HCS/HTS and omics technologies allows to refine existing AOPs and to create more complex AOP networks. The latter helps to identify central KEs and to select the most essential KEs for assay selection, targeted assay development, and integration of these assays in DA. DAs are based on a fixed set of information sources and a fixed DIP to convert inputs from the different information sources into a standardized prediction regarding a certain AO. AO, adverse outcome; AOP, adverse outcome pathway; DA, defined approaches; DIP, data interpretation procedure; HCS, high-content screening; HTS, high-throughput screening; KE, key events; MIE, molecular initiating event.

To achieve this requirement, a systematic two-side analysis of this systems biological question using both a top-down and a bottom-up approach might be a straightforward concept to generated progress in this novel scope of AOP development. “Top-Down” systems biology uses data delivered by HCS/HTS or omics technology, aiming to discover novel genes to characterize biological mechanism closer to the bottom.⁵³

Promising examples of modeling biological processes as well as adverse effects caused by chemicals are already provided by the ToxCast projects. There, the combination of databases providing detailed phenotypic in vivo data for a large number of chemicals and in vitro assay data resulted in the identification of relevant molecular targets associated with the adverse effect(s) and subsequently in the development of a the regulatory network underlying the phenotypic effects.⁵⁴

HCS/HTS test results, when coupled with publicly available databases that provide connections between the intended targets for the assays used and their consequent AOs, can support and accelerate the process of AOP development. Data from different programs were used to channel large-scale toxicogenomics information into a computationally predicted AOPN.^55,56 El Masri et al. described an approach in which life-stage physiologically based pharmacokinetic models were combined with AOP and environmental exposure models to screen for environmental hazards.⁵⁷

However, the data delivered by HCS/HTS or omics technology in a “Top-Down” approach alone do not suffice to unravel the dynamic interaction of different cellular responses or KEs resulting in toxicological responses of tissues and whole organisms. To overcome this drawback, a second, more mechanistic-based strategy should be integrated. The “Bottom-Up” approach starts with well-characterized molecular pathways containing high levels of mechanistic details and aims to integrate single pathways into a model for the entire system.⁵³ This strategy relies on a comprehensive characterization of the spatial and temporal molecular responses of the specific cell to allow the definition of cause/effect relationships, interdependence of different pathways, and the identification of key nodes in the regulatory network. Due to the rise of high-resolution and HC imaging techniques, this approach is now becoming more feasible.

A combination of both strategies might be a powerful tool to create complex AOPN.^58,59 Some of the existing computational systems biology tools could be adapted to suit the nature of the AOP framework and AOPs might help to use a modular approach to the development of systems toxicology.^34,60 Intensified outreach efforts, however, are needed to engage the systems biology modeling community in AOP and AOPN development.

Challenges Ahead

Although the introduction of the AOP concept has already triggered a strong increase in the awareness among toxicologists that toxicological activity also needs to be studied at the molecular and cell biological level, the practical implementation of the AOP concept faces a multitude of challenges.

The development of AOPs is dependent on our current understanding of the molecular mechanisms mediating adverse effects in vivo. For most chemicals and endpoints, this knowledge is rather limited and the use of modern omics technologies or HCS/HTS approaches bear the promise to allow an efficient elucidation of mechanisms, markers, and potentially even methods to unravel the molecular principles of a large number of AOPs in a timely manner (Fig. 1). However, as discussed above, these methods need standardization and harmonization to ensure the reproducibility and predictivity independent of technology and software to build confidence and warrant the acceptance of generated data.

The complexity of such approaches starting from cell-based omics or HCS/HTS assays to complex statistical analysis highlights the interdisciplinary demand of very specific expertise to actually be able to follow the approach itself, and also the numerous difficulties that arise with regard to reproducibility and transferability. Here a close translational cooperation of multiple disciplines, including toxicologists, cell and molecular biologists, chemists, computer scientists, and clinicians, is of key importance to target the efforts to toxicologically relevant effects and processes. Equally important is the thorough scientific review of AOPs and eventually AOPN. Given the vast variety of distinct AOPs targeting very different biological processes, an equally large pool of dedicated experts willing to support the scientific review process is needed.

The AOP concept ought to provide a pathway-based framework for collaborative crowdsourced collection of toxicologically relevant data and knowledge across a plethora of biological processes with various levels of complexity. However, for now it seems that only a small proportion of the scientific community is getting involved and a better communication of the regulatory need and the scientific importance of this program into the fields of basic or medical research is needed to get more expertise involved. Here the question arises how to motivate urgently needed experts to join efforts.

When implementing HTS and HCS methods, one should consider that beside expensive equipment such as robotics or microscopes for imaging-based analysis, appropriate software and IT-logistics have to be in place to allow an efficient processing of the enormous amount of data that will be generated by these technologies. For the latter, researchers also require tools and resources to facilitate the sharing and integration of data, and for linking of data to systems biology models.⁶¹ Large volumes of heterogeneous data have to be processed and stored.¹⁹ Data mining approaches have to be developed to integrate chemical parameters, endpoint-specific assay data, and HCS/HTS data from different types of databases. However, data are not always publically available illustrating that data accessibility and management have to be further improved.

Moreover, the specific expertise and equipment required to efficiently use HCS/HTS technologies are rare and most of these technologies are actually located in the (pharmaceutical) industry. However, verification of data is important in times of intense discussions about reproducibility and should be performed by independent bodies, but only a few academic and public laboratories can actually afford the necessary investments. Reproducibility, however, is also a problem regarding in vivo data as well as the in vitro extrapolation. For animal studies, it has been discussed that the quality is affected by poor study design, inappropriate statistical planning, selective reporting, the underrepresentation of negative results, and a high number of published false-positive results.⁶²

The U.S. government has devoted hundreds of millions of dollar to its Environmental Protection Agency (EPA) ToxCast and the cross-agency Tox21 projects yielding an enormous amount of data. It has already been proven that these data are suitable for the development of AOPs that are in turn applicable for regulatory purposes.^63–65 The European Union (EU) has funded the Safety Evaluation Ultimately Replacing Animal Testing (Seurat-1)^66,67 and the EU-ToxRisk project to promote the development of mechanism-based chemical safety testing strategies such as AOPs.⁶⁸ However, it seems that these efforts alone might not suffice to reach the ambitious goals of the AOP project and more funding mechanisms have obviously to be developed, especially to support interdisciplinary and holistic approaches in chemical safety assessment.

Finally, the success of the AOP project will also depend on the regulatory acceptance of AOP-based testing strategies. For now, only single methods have been validated and accepted as OECD test guidelines. In the case of skin sensitization, it is currently discussed how the different available DAs can be integrated into the test guideline program. To ensure regulatory acceptance of testing strategies or DAs, new ways of validation need to be established since existing validation processes cannot match the pace of development of new tests. Moreover, ring trials are difficult to accomplish when specialized equipment and expertise are required. DAs often comprise statistical predictive models for which technical aspects need to be described with enough detail for all major steps to be independently reproduced and to ensure the utility and reliability of the underlying models. Although AOP frameworks provide an elegant means for systematic and structured toxicity testing, their acceptance in regulation is not yet clearly foreseeable as strategies to combine multiple individual assays are either elusive or under debate.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

References

Ankley

, Bennett

, Erickson

, et al. Adverse outcome pathways: a conceptual framework to support ecotoxicology research and risk assessment. Environ Toxicol Chem, 2010:29; 730–741.

Vinken

. The adverse outcome pathway concept: a pragmatic tool in toxicology. Toxicology, 2013:312; 158–165.

Organisation for Economic Cooperation and Development. The adverse outcome pathway for skin sensitisation initiated by covalent binding to proteins. Part 1: scientific evidence. In: Series on Testing and Assessment; p. 168. Paris: OECD Publishing; 2012.

Organisation for Economic Cooperation and Development. The adverse outcome pathway for skin sensitisation initiated by covalent binding to proteins. Part 2: use of the AOP to develop chemical categories and integrated assessment and testing approaches. In: Series on Testing and Assessment; p. 168. Paris: OECD Publishing; 2012.

Organisation for Economic Cooperation and Development.. Users' handbook supplement to the Guidance Document for developing and assessing adverse outcome pathways. In: OECD Series on Adverse Outcome Pathways; p. 1. Paris: OECD Publishing; 2016.

Landesmann

. Adverse outcome pathway on protein alkylation leading to liver fibrosis. In: OECD Series on Adverse Outcome Pathways; p. 2. Paris: OECD Publishing; 2016.

Yauk

, Lambert

, Marchetti

, et al. Adverse outcome pathway on alkylation of DNA in male pre-meiotic germ cells leading to heritable mutations. In: OECD Series on Adverse Outcome Pathways; p. 3. Paris: OECD Publishing; 2016.

Villeneuve

. Adverse outcome pathway on aromatase inhibition leading to reproductive dysfunction (in fish). In: OECD Series on Adverse Outcome Pathways; p. 4. Paris: OECD Publishing; 2016.

Sachana

, Munn

, Bal-Price

. Adverse outcome pathway on chronic binding of antagonist to N-methyl-D-aspartate receptors (NMDARs) during brain development induces impairment of learning and memory abilities. In: OECD Series on Adverse Outcome Pathways; p. 5. Paris: OECD Publishing; 2016.

10.

Sachana

, Munn

, Bal-Price

. Adverse outcome pathway on binding of agonists to ionotropic glutamate receptors in adult brain leading to excitotoxicity that mediates neuronal cell death, contributing to learning and memory impairment. In: OECD Series on Adverse Outcome Pathways; p. 6. Paris: OECD Publishing; 2016.

11.

Organisation for Economic Cooperation and Development.. Guidance document on the reporting of defined approaches to be used within integrated approaches to testing and assessment. In: Series on Testing & Assessment; p. 255. Paris: OECD Publishing; 2016.

12.

Organisation for Economic Cooperation and Development.. Guidance document on the reporting of defined approaches and individual information sources to be used within Integrated Approaches to Testing and Assessment (IATA) for skin sensitisation. In: Series on Testing & Assessment; p. 256. Paris: OECD Publishing; 2016.

13.

Organisation for Economic Cooperation and Development.. Guidance document for the use of adverse outcome pathways in developing Integrated Approaches to Testing and Assessment (IATA). In: Series on Testing & Assessment; p. 260. Paris: OECD Publishing; 2016.

14.

Vinken

, Whelan

, Rogiers

. Adverse outcome pathways: hype or hope?. Arch Toxicol, 2014:88; 1–2.

15.

Judson

, Richard

, Dix

, et al. The toxicity data landscape for environmental chemicals. Environ Health Perspect, 2009:117; 685–695.

16.

Buchser

, Collins

, Garyantes

, et al. Assay development guidelines for image-based high content screening, high content analysis and high content imaging. In: Assay Guidance Manual. Sittampalam

, Gal-Edd

, Arkin

, et al. (eds.); Bethesda, MD: National Institutes of Health; 2012.

17.

Zhu

, Zhang

, Kim

, et al. Big data in chemical toxicity research: the use of high-throughput screening assays to identify potential toxicants. Chem Res Toxicol, 2014:27; 1643–1651.

18.

Judson

, Houck

, Kavlock

, et al. In vitro screening of environmental chemicals for targeted testing prioritization: the ToxCast project. Environ Health Perspect, 2010:118; 485–492.

19.

Filer

, Kothiya

, Setzer

, et al. tcpl: the ToxCast pipeline for high-throughput screening data. Bioinformatics, 2017:33; 618–620.

20.

EPA US. EPA Toxicity Reference Database. 2008.

21.

Trask

, Johnston

. Standardization of high content imaging and informatics. Assay Drug Dev Techn, 2015:13; 341–346.

22.

Feng

, Mitchison

, Bender

, et al. Multi-parameter phenotypic profiling: using cellular effects to characterize small-molecule compounds. Nat Rev Drug Discov, 2009:8; 567–578.

23.

Futamura

, Kawatani

, Kazami

, et al. Morphobase, an encyclopedic cell morphology database, and its use for drug target identification. Chem Biol, 2012:19; 1620–1630.

24.

Gustafsdottir

, Ljosa

, Sokolnicki

, et al. Multiplex cytological profiling assay to measure diverse cellular states. PLoS One, 2013:8; e80999.

25.

Ljosa

, Caie

, Ter Horst

, et al. Comparison of methods for image-based profiling of cellular morphological responses to small-molecule treatment. J Biomol Screen, 2013:18; 1321–1329.

26.

Persson

, Loye

, Mow

, et al. A high content screening assay to predict human drug-induced liver injury during drug discovery. J Pharmacol Toxicol Methods, 2013:68; 302–313.

27.

Singh

, Carpenter

, Genovesio

. Increasing the content of high-content screening: an overview. J Biomol Screen, 2014:19; 640–650.

28.

Bray

, Singh

, Han

, et al. Cell painting, a high-content image-based assay for morphological profiling using multiplexed fluorescent dyes. Nat Protoc, 2016:11; 1757–1774.

29.

Saito

, Okamura

, Takeuchi

, et al. High content analysis assay for prediction of human hepatotoxicity in HepaRG and HepG2 cells. Toxicol In Vitro, 2016:33; 63–70.

30.

, Xiong

, Zink

, et al. High-throughput imaging-based nephrotoxicity prediction for xenobiotics with diverse chemical structures. Arch Toxicol, 2016:90; 2793–2808.

31.

Peyre

, de Sousa

, Barcellini-Couget

, et al. High-content screening imaging and real-time cellular impedance monitoring for the assessment of chemical's bio-activation with regards hepatotoxicity. Toxicol In Vitro, 2015:29; 1916–1931.

32.

Persson

, Loye

, Jacquet

, et al. High-content analysis/screening for predictive toxicology: application to hepatotoxicity and genotoxicity. Basic Clin Pharmacol Toxicol, 2014:115; 18–23.

33.

Aschner

, Ceccatelli

, Daneshian

, et al. Reference compounds for alternative test methods to indicate developmental neurotoxicity (DNT) potential of chemicals: example lists and criteria for their selection and use. ALTEX, 2017:34; 49–74.

34.

Mc Auley

, Choi

, Mooney

, et al. Systems biology and synthetic biology: a new Epoch for toxicology research. Adv Toxicol, 2015:2015; 1–14.

35.

Lee

, Won

, Raisuddin

, et al. Significance of adverse outcome pathways in biomarker-based environmental risk assessment in aquatic organisms. J Environ Sci (China), 2015:35; 115–127.

36.

Marx-Stoelting

, Braeuning

, Buhrke

, et al. Application of omics data in regulatory toxicology: report of an international BfR expert workshop. Arch Toxicol, 2015:89; 2177–2184.

37.

Rouquie

, Heneweer

, Botham

, et al. Contribution of new technologies to characterization and prediction of adverse effects. Crit Rev Toxicol, 2015:45; 172–183.

38.

Dean

, Jay Zhao

, Lambert

, et al. Application of gene set enrichment analysis for identification of chemically-induced, biologically relevant transcriptomic networks and potential utilization in human health risk assessment. Toxicol Sci, 2017:157; 85–99.

39.

Bouhifd

, Beger

, Flynn

, et al. Quality assurance of metabolomics. ALTEX, 2015:32; 319–326.

40.

Kolker

, Özdemir

, Martens

, et al. Toward more transparent and reproducible Omics studies through a common metadata checklist and data publications. OMICS, 2014:18; 10–14.

41.

Seok

, Warren

, Cuenca

, et al. Genomic responses in mouse models poorly mimic human inflammatory diseases. Proc Natl Acad Sci U S A, 2013:110; 3507–3512.

42.

Takao

, Miyakawa

. Genomic responses in mouse models greatly mimic human inflammatory diseases. Proc Natl Acad Sci U S A, 2015:112; 1167–1172.

43.

Subramanian

, Tamayo

, Mootha

, et al. Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proc Natl Acad Sci U S A, 2005:102; 15545–15550.

44.

Weidner

, Steinfath

, Opitz

, et al. Defining the optimal animal model for translational research using gene set enrichment analysis. EMBO Mol Med, 2016:8; 831–838.

45.

Iida

, Fujii

, Uchida

, et al. Identification of aryl hydrocarbon receptor signaling pathways altered in TCDD-treated red seabream embryos by transcriptome analysis. Aquat Toxicol, 2016:177; 156–170.

46.

Sarma

, Kim

, Ryu

. Differential gene expression profiles of human leukemia cell lines exposed to benzene and its metabolites. Environ Toxicol Pharmacol, 2011:32; 285–295.

47.

Matthews

, Hanison

, Nirmalan

. “Omics”-informed drug and biomarker discovery: opportunities, challenges and future perspectives. Proteomes, 2016:4; 3.

48.

Dekant

. Toxicology and the reproducibility crisis: scientific publishing, hazard assessment and risk characterization. Toxicol Lett, 2016:263; 76–77.

49.

Barabasi

, Oltvai

. Network biology: understanding the cell's functional organization. Nat Rev Genet, 2004:5; 101–113.

50.

Waters

, Fostel

. Toxicogenomics and systems toxicology: aims and prospects. Nat Rev Genet, 2004:5; 936–948.

51.

National Academies of Sciences, Engineering, and Medicine, Division on Earth and Life Studies, Board on Environmental Studies and Toxicology, et al. Using 21st Century Science to Improve Risk-Related Evaluations. Washington, DC: National Academies Press(US); 2017.

52.

Wittwehr

, Aladjov

, Ankley

, et al. How adverse outcome pathways can aid the development and use of computational prediction models for regulatory toxicology. Toxicol Sci, 2017:155; 326–336.

53.

Bruggeman

, Westerhoff

. The nature of systems biology. Trends Microbiol, 2007:15; 45–50.

54.

Leung

, Phuong

, Baker

, et al. Systems toxicology of male reproductive development: profiling 774 chemicals for molecular targets and adverse outcomes. Environ Health Perspect, 2016:124; 1050–1061.

55.

Bell

, Angrish

, Wood

, et al. Integrating publicly available data to generate computationally predicted adverse outcome pathways for fatty liver. Toxicol Sci, 2016:150; 510–520.

56.

Oki

, Edwards

. An integrative data mining approach to identifying adverse outcome pathway signatures. Toxicology, 2016:350–352; 49–61.

57.

El-Masri

, Kleinstreuer

, Hines

, et al. Integration of life-stage physiologically based pharmacokinetic models with adverse outcome pathways and environmental exposure models to screen for environmental hazards. Toxicol Sci, 2016:152; 230–243.

58.

Cakir

, Khatibipour

. Metabolic network discovery by top-down and bottom-up approaches and paths for reconciliation. Front Bioeng Biotechnol, 2014:2; 62.

59.

Robinson

, Nielsen

. Integrative analysis of human omics data using biomolecular networks. Mol Biosyst, 2016:12; 2953–2964.

60.

Sturla

, Boobis

, FitzGerald

, et al. Systems toxicology: from basic research to risk assessment. Chem Res Toxicol, 2014:27; 314–329.

61.

Wolstencroft

, Owen

, Krebs

, et al. SEEK: a systems biology data and model management platform. BMC Syst Biol, 2015:9; 33.

62.

Mcleod

. Why animal research needs to improve. Nature, 2011:477; 511.

63.

Browne

, Judson

, Casey

, et al. Screening Chemicals for estrogen receptor bioactivity using a Computational Model. Environ Sci Technol, 2015:49; 8804–8814.

64.

Ellis-Hutchings

, Settivari

, McCoy

, et al. Embryonic vascular disruption adverse outcomes: linking HIGH throughput signaling signatures with functional consequences. Reprod Toxicol, 2017:71; 16–31.

65.

Benigni

, Battistelli

, Bossa

, et al. Endocrine disruptors: data-based survey of in vivo tests, predictive models and the Adverse Outcome Pathway. Regul Toxicol Pharmacol, 2017:86; 18–24.

66.

Daston

, Knight

, Schwarz

, et al. SEURAT: safety evaluation ultimately replacing animal testing—recommendations for future research in the field of predictive toxicology. Arch Toxicol, 2015:89; 15–23.

67.

Gocht

, Berggren

, Ahr

, et al. The SEURAT-1 approach towards animal free human safety assessment. ALTEX, 2015:32; 9–24.

68.

Daneshian

, Kamp

, Hengstler

, et al. Highlight report: launch of a large integrated European in vitro toxicology project: EU-ToxRisk. Arch Toxicol, 2016:90; 1021–1024.