Recommendations for Addressing Harm–Benefit Analysis and Implementation in Ethical Evaluation – Report from the AALAS–FELASA Working Group on Harm–Benefit Analysis – Part 2

Framework for evaluation of harm/benefit

First, the animal proposal that is used by committees to evaluate harm/benefit should be framed in a manner that illustrates why a particular study that uses animals can be expected to be of value, and should contain the details needed to allow the reviewers to determine the harms.

To aid in defining the harms, the WG chose, as did Mellor,^8,9 to consider harm factors that compromise animal subjects within the categories of the five freedoms ¹¹ (see Key in Table 1). This approach offers a comprehensive and broader view on animal harm and welfare, which we believe better safeguards the interests of the experimental animals ⁸ and identifies important areas for the application of the 3Rs. ¹⁰ The five freedoms^9,11 are used to define the overarching harms of the study.

The benefits for the study are defined using an overarching set of domains that was derived from the literature review (see Key in Table 1). It is clear that benefits from applied, immediately translational research are easier to define than possible benefits from basic research, but the importance of the benefit is not correlated to the ease of its definition.

The WG acknowledged that ethical review committees are presently well positioned to assess ‘harms’ but may be less well equipped to conduct a benefit analysis.^12–14 It may then be necessary to incorporate work from other review bodies such as scientific granting agencies and scientific peer/specialist review committees. However the summation of both ‘harm’ (Table 2a) and ‘benefit’ (Table 2b) tables needs to be linked in order to conduct a ‘harm–benefit’ analysis.

The following steps define the process for HBA:

Detail the harms and benefits at the top of both the harm (Table 2a) and benefit (Table 2b) tables.

OPEN IN VIEWER

Table 1.

KEY: Benefit Domains and Harm Factors.

OPEN IN VIEWER

Table 2a.

Harm table.

OPEN IN VIEWER

Table 2b.

Benefit table.

OPEN IN VIEWER

Table 3.

Summary of color gradient and score scheme for harm/benefit factors.

In the process of evaluating all potential harms and benefits in a systematic fashion, it is essential to realize that the weight or significance given to an individual harm/benefit will not be equal, and a single harm or benefit factor could dominate and steer the final outcome.

Definition of MFs for harms

Definition of MFs for benefits

Purported importance of outcome : ‘WHY’ is the study important? Although this cannot be defined with certainty an estimation of the importance of the outcome of the study should be made. This can be framed in terms of immediate and short-term benefits as well as the anticipated impact of the outcome for subsequent studies and long-term benefits.

Clarity of objectives : ‘HOW’ will the objectives be met? and ‘WHAT’ will the objectives be? The degree to which a sound hypothesis and clear objectives are elucidated can support the driving purpose of the study and ensure that the study outcome has value/benefit.

Translational potential : ‘WHO’ will the study benefit? and ‘WHEN’ will the benefit be realized? An assessment of how feasible the study is and how quickly the results can be expected to be applied to the benefit domain.

Likelihood of success : ‘HOW’ likely is it that you will obtain the objectives desired? In addition to the complexity and difficulty of proposed studies there are several other factors that affect the likelihood of success. These factors may include the existence of appropriate facilities, the expertise and competence of research and animal care and use personnel, as well as the level of resources and funding available to assure completion and continuity of the work. The track record of the study team should be considered in this evaluation.

Continuity of recognized scientific efforts : ‘WHAT’ is the larger body of knowledge this study contributes to? Consideration of how well this work amplifies/adds to the continuum of knowledge gained from previous studies, or indicates whether there is potential to continue to offer further benefits.

Quality of experimental design : ‘HOW’ will the objectives be obtained with high quality/effective use of resources (animals, time, etc.)? The quality of the experimental design should be considered in the benefit equation in that it ensures that the data collected are scientifically acceptable, and will validate the results obtained.

Innovation level : ‘HOW’ will this study advance science beyond the specific objectives of the study itself? Consideration of whether or not the proposed research will benefit other research through the conduct of novel and innovative processes and designs. This may include expected secondary benefits such as 3R advances.

Dissemination of results : ‘WHEN’ and ‘HOW’ will the results be distributed? How will the results be disseminated for maximum benefit? (e.g. are the results proprietary or public; presented or published, etc.).

Are all harms equal?

Drawing on information from the literature ⁶ and the WG professional experience, the WG offers the following points for consideration in the systematic analysis of harm.

There are inherent challenges in assessing procedural severity, and the application of professional judgment is often warranted. The level of harm is influenced by the quality of facilities, equipment, housing conditions (social versus single, quality of environmental enrichment, etc.), staff and investigator skills and competence, quality of veterinary observation and care, individual species and animal issues (phenotype and health status) as well as the definition and implementation of experimental endpoints. In summary, the level of harm is not related exclusively to the nature of the experimental procedure, but also to many other variables. Responsible entities should evaluate these elements carefully during HBA to ensure that high competence and appropriate compensatory provisions for procedures potentially impacting animal welfare and harm are applied in every instance to the fullest extent possible.

Example: Conducting procedures in a specialized center for studies with a particular species with unique requirements may result in far less stress to the animals involved than to animals used in similar studies conducted in research facility environments without similar personnel, knowledge, expertise and equipment resources suited to the species and research investigation. Responsible authorities should ensure that essential resources and expertise are in place before allowing studies to proceed.

The species and the behavior of the individual animal are potentially important factors when determining the level of harm. The same procedure may be scored differently depending on the species or the native reactivity and prior acclimatization. For example, a procedure conducted in a species that typically tolerates it poorly will normally be considered as more harmful than if conducted in a species that tolerates it well. Also, within a species, some individuals may be better acclimatized to experimental conditions or may have a naturally more cooperative disposition than others in behavioral studies.

Example: Comparable stereotaxic surgical procedures in neuroscience studies can be performed in non-human primates and in rats. Although in both cases the pain associated with the procedure itself can be abolished by means of appropriate anesthesia and analgesia, the level of stress/distress created by captivity may differ by species. In addition, differences in research environment, competency of personnel (see previous examples), housing conditions, etc. may affect the final level of harm associated with the procedure.

Consideration of harm should take into account the cumulative experience of the research animal in the research facility as well as in experimental procedures.

Example: Responsible entities may wish to develop specific guidance concerning repetitive procedures they will allow in animals that are maintained for specific purposes. For instance in colonies of animals instrumented for safety pharmacology studies significant factors that impact harm include: how long would animals be maintained, how many drugs would they be exposed to, and how long would they be ‘rested’ between procedures. The harm to animals would not be solely defined by the experiment proposed but also by the background of the use of the research subject.

Harm analysis should balance individual animal needs with those of the entire experimental group cohort used for data acquisition. Responsible entities should be receptive to assessing whether procedures of a greater severity to a few individuals are warranted instead of conducting procedures of less severity to a greater number of individuals in certain circumstances.

Example: Maintenance of a small colony of cannulated animals for repeated metabolism studies subjects a small number of animals to surgical procedures and repeated doses of compounds and procedures to which they become well adapted. This might result in less cumulative harm than would occur if multiple animals were used in single experiments for which the individual bleeding procedures were more stressful and for which they were not as well adapted.

For all animals, including genetically-modified animals, harm should be assessed using observation and scientific measures of pain and distress that occur in the research subjects and not by the assumption that alterations from the natural or wild state are deleterious a priori. Genetic modification does not necessarily result in experienced harm per se.

Harm should be assigned at the level of the whole research proposal submitted to the oversight body and should encompass all experimental procedures/conditions that potentially impact animal well-being.

There should be a mechanism of evaluating the actual level of harm regularly during the development of the protocol/project. If the harm seen is different from the prospective harm assessment, responsible entities should re-evaluate and take appropriate action. Moreover, responsible entities cannot reliably be expected to achieve a sound review process if the assessment is entirely conceptual and is limited to just a paper review.

Pilot studies are useful for determining the in vivo experimental approach and types of procedures to optimize data collection. Sometimes, however, very little prior knowledge is available to predict expected outcomes and the harm experienced by the animals. Thus, special attention in the evaluation and the actual conduct of pilot studies should be provided.

Humane endpoints can reduce the level of pain and negative impacts on the five freedoms. In some cases, researchers need some preliminary data to determine effective, early endpoints. The capacity to define early endpoints clearly impact the HBA.

Are all benefits equal?

At a cursory glance, benefits of improving health caused by serious diseases are easy to support. ¹⁶ However, the nature of the underlying cause of a specific disease can be an issue for discussion. ¹⁶ Is the disease caused by predetermined factors (e.g. genetics), factors out of the individual’s control (e.g. contagious disease, intrauterine environment or exposure, or accident), or is it a consequence of a certain lifestyle (e.g. smoking) where one might expect the patient to have some influence on the outcome, or is it a result of another deterministic factor? For many types of disease there are no discrete and identifiable influences but rather undefined and complex mechanisms that are at play, and caution must be taken not to categorically devalue benefits for diseases that have a ‘lifestyle related’ ethology. ¹⁷ Animal experiments are usually used as one tool, together with in vitro methods, epidemiological studies, clinical research or other scientific approaches. Therefore, in such cases the HBA should consider the use of animals as an additional factor in the approaches used to improve health.

Questions can be raised regarding routine product testing. Is the product of substantial importance for the improvement of the consumer’s quality of life or is the product being developed to satisfy human pursuit for luxury items or for vanity reasons? This distinction is not always clear. Product testing clearly is designed to protect health, and some products are used for several purposes, for example the botulinum toxin is used both for treating wrinkles as well as for treating neurological diseases. The European Commission has already limited animal use in favor of alternatives in some circumstances through the registration, evaluation, authorization and restriction of chemicals (REACH) legislation ¹⁸ Also, placing restrictions on the commercial or intellectual freedoms involved in pursuing new drugs to improve performance and reduce side-effects is difficult, and few would propose defining a minimal increment of improvement necessary to justify the use of research animals in drug development. Responsible entities will undoubtedly be faced with assessing difficult ethical quandaries akin to the above on an individualized basis.

Scientific discovery efforts are often met with failure and the communication of these failures should be encouraged and facilitated and reported to ensure that negative findings bear some benefit. The publication of negative results may be regarded as counterproductive and a waste of time, potentially stigmatizing the laboratory and the research sponsor, and drawing all possible sources of failure of the laboratory into question. However, negative results are highly relevant because they reveal important knowledge and may prevent subsequent unproductive or poorly conceived inquiries in the same area, subjecting additional animals to futile experimentation. According to Claude Bernard, the founder of modern medicine, ‘there are no unsuccessful experiments … the results are always the true consequence of the conditions of the experiment’. ¹⁹

The likelihood of success is a relevant dimension in the discussion of benefits. This does not relate to the uncertainty implicit in basic research, but to what extent the experiments are based on good scientific principles, a clear hypothesis and problem formulation, systematic review of existing knowledge, selection of appropriate methods, and research design to generate reliable data. Also, the likelihood of success depends on the research group’s expertise and available resources (knowledge, skills, personnel facilities, etc.). Likelihood of success or quality of the experiments using the chosen methods and models was presented as a separate dimension from harm and benefit in the Bateson cube model.^20,21 We found it appropriate to discuss this under the likelihood of achieving the desired benefits and as an MF among the benefits, but did not give it its own dimension. Failure in design can lead to an unnecessary use of animals, publication of invalid results, and subsequent experiments being based upon flawed hypotheses.

There are enduring approaches to the systematic analysis of scientific problems and their investigation. The analysis logically begins with the systematic review of scientific literature relevant to the problem of interest.^22–24 Such literature reviews should be structured, thorough and transparent. Once the problem in question and experimental hypothesis are clearly formulated against the backdrop of a thorough literature review, the strategic selection of methods may proceed. This is important because any harm to animals can only be justified if it is really necessary to answer the question. This approach to the conduct of research was emphasized by Bernard ²⁵ who stated, ‘Like investigations in any field of science, the merit of animal experiments ultimately depends on rigid adherence to principles of the scientific method.’

Animal research projects funded through public resources and foundations are usually subjected to the critical, peer review of the research by experts in the field who declare their independence and absence of any conflict of interest in the conduct of their duties. The WG believes that peer review of this nature may constitute a factor important to a project’s benefits and may serve as the nucleus of the responsible entities’ final evaluation of benefit in the HBA process.

Simplied HBA

HBA using the approach described here can be a time and labor-intensive task. Therefore, responsible entities may wish to prioritize experiments according to the intensity of review and analysis deemed appropriate. A simplified Bateson square^20,21 can be used to devise a ‘quick and simple’ way to sort experiments. Animal experiments can be simply categorized as ‘low harm–high benefit’, ‘low benefit–high harm’, ‘low harm–low benefit’ or ‘high harm–high benefit’. The low harm–high benefit experiments elicit minimal controversy. Terminal procedures (animals anesthetized for the whole experiment and then killed under anesthesia) for a beneficial purpose experience minimal harm, assuming that high standards of care are addressed. However, sacrificing a large number of animals in an experiment of this type would still raise ethical concerns if adequate scientific justification for high animal numbers were lacking. Experiments categorized as low benefit–high harm might also be easily decided. Very likely such experiments would not be ethically justified and the application would be rejected. Review of such a research proposal should prompt greater clarity on the nature and urgency of the expected benefits and should focus on reducing harm (refinement/3R). Experiments defined as low or moderate harm–low benefit or high harm–high benefit might stimulate the most discussion. Should the responsible entity accept the justification for an experiment where the benefit is poorly defined, even if harm for the animal is trivial? Experiments that are obviously beneficial but that also cause much harm are also difficult to assess. If newly emerging severe diseases occur, this might cause an urgent specific research need. Because experience with a new disease is limited it may involve some trial and failure before a research model is adequately refined to reduce harm. Finally, perhaps a majority of animal experiments (and animals) fall into a gray zone of uncertain potential benefit and experience harm levels that are not severe. In some instances, it may be permissible for responsible entities to reduce or waive the HBA review requirements in studies of this nature.

Responsibility for HBA Outcome

The model suggested in this presentation does not say anything about who should take part in the HBA and make the final decision. However, as subjective opinions influence our evaluations^27–30 we think a broad representation of competent persons is the only way to give a balanced HBA process and decision. This applies both for the harms and the benefits.

The WG favors and recommends the use of consensus rather than voting for the decision method of the responsible entities in conducting HBAs. The responsible entities should be able to project transparency in how they evaluated harm and benefit and how they reached their final conclusion. This transparency will aid external review by outside groups if warranted, improve information exchange, and build broader, more informed consensus and aid communication on why animals are used to the general public.

Reaching an agreement about the relevant parameters of harm and methods to palliate them is far less challenging than reaching an agreement on what constitutes a meaningful benefit resulting from a research animal study. This is clearly evident in the chart summarizing the consideration of harms and benefits in the literature for research animal studies in which the two most cited categories, benefits to humans and the quality of the research, are identified as pertinent.^3,21,26,31 Identified benefits to animals, benefits to the environment, and knowledge benefits are also important considerations.^3,26

The WG contends that the researchers must be able to define and describe some primary benefits for their inquiry while recognizing that there is no guarantee that projected benefits will be realized. Communications from both scientists and members of responsible entities with the WG have emphasized that the definition and assessment of benefits, particularly tangible benefits, can be a daunting and unsatisfying task in some studies. They have argued that for scientific projects that have undergone authoritative, external scientific peer review successfully and been deemed worthy of support with public funding, this should constitute an adequate, if not definitive, statement of the project’s benefits. This approach should help address the growing administrative burden that scientists face from the responsible entity overseeing research and expedite decision-making. However, unless harm to animals has also been carefully addressed in the peer review process evaluating benefits, the conclusions of the external peer review may be brought into question and the ethical implications of animal use for the specific project has not been addressed.

Finally it is important to recognize that the final comparison and evaluation of HBA will be influenced by both attitudes and competence of those making the decision. ³²