Evaluating the Risks of Clinical Research: Direct Comparative Analysis

Abstract

Objectives:

Many guidelines and regulations allow children and adolescents to be enrolled in research without the prospect of clinical benefit when it poses minimal risk. However, few systematic methods exist to determine when research risks are minimal. This situation has led to significant variation in minimal risk judgments, raising concern that some children are not being adequately protected. To address this concern, we describe a new method for implementing the widely endorsed “risks of daily life” standard for minimal risk. This standard defines research risks as minimal when they do not exceed the risks posed by daily life activities or routine examinations.

Methods:

This study employed conceptual and normative analysis, as well as use of an illustrative example.

Results:

Different risks are composed of the same basic elements: Type, likelihood, and magnitude of harm. Hence, one can compare the risks of research and the risks of daily life by comparing the respective basic elements with each other. We use this insight to develop a systematic method, direct comparative analysis, for implementing the “risks of daily life” standard for minimal risk. The method offers a way of evaluating research procedures that pose the same types of risk as daily life activities, such as the risk of experiencing anxiety, stress, or other psychological harm. We thus illustrate how direct comparative analysis can be applied in practice by using it to evaluate whether the anxiety induced by a respiratory CO₂ challenge poses minimal or greater than minimal risks in children and adolescents.

Conclusions:

Direct comparative analysis is a systematic method for applying the “risks of daily life” standard for minimal risk to research procedures that pose the same types of risk as daily life activities. It thereby offers a method to protect children and adolescents in research, while ensuring that important studies are not blocked because of unwarranted concerns about research risks.

Introduction

Clinical research with children and adolescents poses an ethical dilemma. Society's obligation to protect those who cannot give their own informed consent suggests that they should be enrolled in research only when it offers the prospect of clinical benefit. However, this approach would prohibit research vital to improving medical care for children, as well as other groups who cannot consent. For example, this approach could prohibit pharmacokinetic and pharmacodymanic studies that are essential for identifying proper medication dosing in children.

Prominent guidelines and regulations, including the the World Medical Association's Declaration of Helsinki, attempt to avoid this concern by allowing “nonbeneficial” research with individuals who cannot consent, when the risks are minimal (Emanuel et al. 2000; Council for International Organizations of Medical Sciences 2002; World Medical Association 2013). For example, regulations in the United States allow research with children that offers no prospect of direct benefit when an institutional review board (IRB) finds that the research involves no more than minimal risks. If the IRB judges the risks to be minimal, parents then decide whether to enroll their children based on their own judgment of the associated risks (unless the IRB waives the requirement for informed consent) (United States Department of Health and Human Services 1991).

Many guidelines and regulations define minimal risks based on the risks posed by specified comparator activities. The Council of International Organizations of Medical Sciences (CIOMS) defines risks as minimal when they do not exceed the risks posed by routine examinations (Council for International Organizations of Medical Sciences 2002). Regulatory guidance in many countries – including the United States, Canada, India, Nepal, South Africa, and Uganda – define risks as minimal when they do not exceed the risks ordinarily encountered in daily life or during routine examinations (United States Department of Health and Human Services 2009; Canadian Institutes for Health Research 2010; Indian Council of Medical Research 2006; Nepal Health Research Council 2001; South African Medical Research Council 2002; Uganda National Council for Science and Technology 2007). Furthermore, some commentators propose that risks in pediatric research should be considered minimal when they do not exceed the risks posed by acceptable charitable activities for children or adolescents, such as participating in a soccer game for a charity fundraiser (Wendler 2005).

Comparing the risks of research interventions with the risks posed by other activities provides a context in which to evaluate the risks of “nonbeneficial” research procedures and studies. However, absent a systematic method for making these comparisons, investigators, sponsors, and IRBs must rely largely on intuition alone. This approach is problematic for several reasons. First, intuitive judgments about risk do not systematically take account of the relevant data. These judgments are, therefore, less likely to reflect the actual risks faced by study participants. Second, intuitive risk judgments are subject to cognitive biases, which in turn are influenced by numerous contextual factors (Tversky et al. 1974; Slovic 1987; Weinstein 1989; Kahnemann 2012). These factors increase the chances that IRB assessments of research risks are unduly affected by subjective and, possibly, irrelevant factors. Third, as a result, intuitive judgments about which research risks are acceptable vary significantly, leading to inconsistent judgment across individuals, research sites, and studies (Shah et al. 2004). Such variation can be a particular challenge for pediatric studies that frequently need to recruit from several sites.

In some cases, IRBs may have no or extremely limited data on the risks posed by a research intervention. For example, it can be difficult to predict whether a first-in-humans trial of a novel compound poses no, some, or a high risk of harm to children or adolescents. In cases of significant uncertainty about the risks of research interventions, IRBs should err on the side of caution and categorize interventions as having more than minimal risk. More frequently, IRBs have some data on the risks posed by the research intervention, but are unsure whether these risks qualify as minimal or greater than minimal risks. For example, it may be known that a “punch” skin biopsy typically poses a 1 per 100 risk of bleeding that will require another clinical visit, as well as a 1–10 per 1000 risk of infection that can require treatment with oral antibiotics (Rid and Wendler 2011). In these cases, protecting children without inadvertently blocking valuable research requires IRBs to determine whether the risks are minimal or greater than minimal.

Relying on intuitive judgment alone to make these determinations can leave children and adolescents insufficiently protected in some studies, whereas other studies may be rejected out of unwarranted concerns about their risk level. To address this concern, we describe a systematic method, direct comparative analysis (DCA), that IRBs and others can use to categorize the risk level of research interventions that pose the same types of risks as one or more specified comparator activities. To illustrate how DCA works in practice, we consider how it can be used to determine under United States regulations whether a respiratory CO₂ challenge poses minimal or greater than minimal risk in children and adolescents.

Clarifying the Minimal Risk Standard

United States regulations define minimal risk as follows: “The probability and magnitude of harm or discomfort anticipated in the research are not greater than those ordinarily encountered in daily life or during the performance of routine physical or psychological examinations or tests” (United States Department of Health and Human Services 1991). As written, one could interpret this definition as specifying that research procedures qualify as having greater than minimal risk whenever children do not ordinarily encounter the procedure in daily life. However, this interpretation suggests that a study on the impact of seated meditation in attention-deficit/hyperactivity disorder (ADHD) necessarily poses greater than minimal risk in children who do not ordinarily meditate in their daily lives. Therefore, whether an intervention poses minimal or greater than minimal risk depends upon whether the intervention's risk level exceeds the level of risk children ordinarily encounter in daily life or during routine examinations, not on whether children undergo the intervention in daily life. To make this determination, reviewers need a way to systematically compare the level of risk posed by research interventions with the level of risk posed by activities in daily life or during routine examinations and tests.

Others might interpret the risks “ordinarily encountered in daily life” as referring to the risks ordinarily encountered by the individuals enrolled in the study. However, this interpretation would allow investigators to justify risky research in children and adolescents who live in war-torn areas on the grounds that the risks are no greater than those they face in their daily lives. To avoid exploiting children's circumstances in this way, we will use for the present analysis the widely endorsed “objective” interpretation of the “risks of daily life” standard. This interpretation defines risks as minimal when they do not exceed the risks of activities ordinarily encountered in daily life by average, healthy, normal children and adolescents (Field and Behrman 2004; Kopelman 2004). In addition, we assume that risks are minimal only when they do not exceed the risks involved in appropriate daily life activities.

Direct Comparative Analysis

To systematically compare the risks of research procedures to the risks in daily life for average, healthy children, it is helpful to recall that all risks are composed of the same basic elements: Type of potential harm, magnitude of potential harm (including duration), and likelihood that the potential harm will be realized. This fundamental fact suggests the possibility of systematizing the comparison of different risks by resolving them into their basic elements and comparing the respective elements to each other. For example, under the minimal risk standard endorsed in the United States, IRBs and others would determine the type, magnitude, and likelihood of the potential harms posed by the research intervention and compare these elements to the type, magnitude, and likelihood of harms that result from one or more daily life activities or routine examinations for average, healthy, normal children.

To date, we are aware of only one systematic method that has been developed to help IRBs make these comparisons (Rid et al 2010). This method allows IRBs to compare harms of very different types. For example, it allows IRBs to compare the risks of infection from a research biopsy to the risks of bone fracture from riding a bicycle. As a result, it can be used to assess the risk level of almost any research intervention. However, this flexibility also makes the method complex to implement (Rid et al 2010). This complexity raises the need for methods that are both systematic and relatively straightforward to use.

DCA offers such a method when the risks of the research intervention in question are of the same type as the risks of one or more activities of daily life, or any other appropriate comparator activity specified by relevant guidelines or regulations (e.g., routine examinations). Specifically, DCA includes five steps that enable IRBs to compare the likelihood and magnitude of the harm occurring as a result of the research intervention to the likelihood and magnitude of the same type of harm occurring as a result of an appropriate comparator activity (Table 1). The five steps of DCA are:

1. Identifying the potential harms of the research intervention

2. Estimating the magnitude and likelihood of each potential harm based on available data

3. Identifying an appropriate comparator activity, or appropriate comparator activities, that pose the same type(s) of harm

4. Estimating the magnitude and likelihood of the harms occurring as a result of the comparator activity, drawing on the available data

5. Comparing the magnitudes and likelihoods of the harms from the two activities. If the likelihoods and magnitudes of the potential research harms are all lower than the likelihoods and magnitudes of the comparator harms, then the risks of the research intervention do not exceed the risks of the comparator activity, and the intervention qualifies as being of minimal risk.

Table 1.

Direct Comparative Analysis for Evaluating the Risks of Research Interventions that Pose the Same Type of Harm as Appropriate Comparator Activities

Step	Example	Normative judgment
	Intervention: respiratory CO₂ challengeComparator activity: recommended routine examinations/procedures
1. Identify the potential harms of the research intervention	Anxiety	• Which data are relevant to the study and population in question?
2. Estimate the magnitude and likelihood of each potential research harm	∼50% likelihood of experiencing mild to moderate anxiety after 5 min of 5% CO₂	• Are the available data sufficient to enable confident judgments?
3. Identify appropriate comparator activity that poses the same type(s) of harm	Childhood vaccinations	• Are the potential harms of the comparator activity really of the same type? • When are comparator activities appropriate for evaluating the risks of “non-beneficial” research?
4. Estimate the magnitude and likelihood of the harms occurring as a result of the comparator activity	Average child experiences moderate anxiety from undergoing vaccination	• Are the available data sufficient to make confident judgments?
5. Compare the magnitudes and likelihoods of the harms from the two activities	Anxiety less severe and less likely from CO₂ challenge than from childhood vaccinations	• When some likelihoods for different magnitudes of research harm are higher and others lower than the comparator activity, does the intervention still qualify as involving minimal risk? • When an intervention is associated with many potential harms, does it still qualify as involving minimal risk?

The example uses the “routine examinations” standard for minimal risk that is stipulated by many research regulations. The given data represent rough estimates. More details on the available evidence can be found in the text.

Given that subjective experiences of negative emotions (e.g., anxiety, distress) are often comparable across different activities, DCA is especially useful for evaluating procedures that pose risks of psychological harm. Therefore, to illustrate how DCA can be used in practice, we use it to assess whether CO₂ challenge studies in children and adolescents pose minimal or greater than minimal risks based on the widely endorsed “risks of daily life” standard.

Example: CO₂ Challenge in Children and Adolescents

Panic disorder (PD) is a debilitating condition that is characterized by sudden attacks of terror, and affects>6,000,000 people in the United States alone (Kessler et al. 2005; Roy-Byrne et al. 2006). PD presents as various subtypes, with one of the most notable subtypes involving perturbed respiration as manifested by heightened sensitivity to inhalation of CO₂-enriched air in the clinical or research setting. Distinct PD subtypes often respond differently to treatment, and novel treatments may be identified through research on subtype-specific perturbations in physiology (Briggs et al. 1993; Klein 1993; Pine et al. 2005a).

Because the first signs of PD often manifest during adolescence, research on physiological risk traditionally focuses on children and adolescents. Studies in this area typically expose participants to CO₂-enriched air using the same procedure as is used to identify a respiratory subtype of PD in adults. This involves breathing through a face mask, initially with room air, followed by 5% CO₂. The procedure can be stopped at any time by lowering the face mask. In addition, participants are asked to rate their subjective levels of anxiety or fear during the procedure, typically using ordinal Likert Scales.

Because CO₂ inhalation offers no prospect of clinical benefit for healthy children, IRBs in the United States can approve its use in healthy children only if it poses minimal risk (United States Department of Health and Human Services 1991). It is, therefore, essential to determine whether CO₂ inhalation poses minimal or greater than minimal risk.

Applying Direct Comparative Analysis

Many activities in children's daily life (e.g., being home alone) and routine examinations (e.g., physical examination) pose a risk of anxiety. The anxiety experienced during these activities is often measured in the same way it is measured in the research context, namely by asking children to evaluate their subjective level of anxiety on ordinal Likert scales. DCA, therefore, offers a way to evaluate whether undergoing a CO₂ challenge poses minimal or greater than minimal risk, by directly comparing the magnitude and likelihood of the anxiety with the magnitude and likelihood of the anxiety resulting from activities in daily life.

Step 1: Identifying the potential harms posed by the research intervention

Respiratory challenge with 5% CO₂ can cause transient anxiety in some children that is associated with a range of symptoms, including shortness of breath, mild chest pain or discomfort, numbness or tingling, and mild headache. The anxiety and most associated symptoms are fully reversible within several minutes of stopping the challenge; mild headaches may last up to several hours. Previous studies report no other adverse events and no long-term adverse effects following a 15 minute 5% CO₂ challenge in healthy and at-risk children (Pine et al. 2005b). In particular, there are no reports of exacerbation of ongoing anxiety, emergence of new anxiety symptoms related to participation, or reports of new-onset panic disorder up to 3 years after undergoing a 15 minute 5% CO₂ challenge (Pine et al. 2005b). These findings are consistent with data from ∼1000 adults, which show no long-term adverse effects up to 4 years after undergoing 35% CO₂ challenge (Gorman et al. 1984, 1988, 1994; Fredholm et al. 2000).

Step 2: Estimating the magnitude and likelihood of the potential harms

The only data of which we are aware regarding the likelihood and magnitude of CO₂-induced anxiety comes from previously unpublished data that were collected as part of a study involving 197 children and adolescents, ages 9–20 years (Table 2; other results from the study are published elsewhere) (Pine et al. 2000; Roberson-Nay et al. 2010). Of the 197 participants, 161 were at risk for PD because of a positive family history, and 36 were healthy (i.e., not at risk to the best of the investigators' knowledge). Participants were exposed to a maximum of 15 minutes of CO₂-enriched air using the procedure described. They were also asked to rate their state anxiety before, 5 minutes into, and immediately following the CO₂ exposure. A 0–10 Likert scale was used for this purpose, with “0” indicating no anxiety and “10” indicating the most severe anxiety the child or adolescent could imagine.

Table 2.

Risk of Experiencing Anxiety from a Respiratory Challenge with 5% CO₂ in Subjects 7–20 Years of Age

		Healthy subjects (n=36)		Subjects at risk ^a (n=161)
5% CO₂ (min)	Anxiety score 0–10 Likert	n (%)	Missing n	n (%)	n
0 (no mask)	Mean (SD)	1.06 (1.66)	0	0.96 (1.70)	0
	0	21 (58.3)		92 (57.1)
	1–3	13 (36.1)		57 (35.4)
	≥4	1 (2.8)		12 (7.5)
0 (with mask)	Mean (SD)	0.72 (1.49)	0	0.68 (1.12)	0
	0	23 (63.9)		100 (62.5)
	1–3	12 (33.3)		52 (32.5)
	≥4	1 (2.8)		8 (5.0)
5	Mean (SD)	1.00 (1.93)	1	1.04 (1.41)	1
	0	20 (57.1)		82 (51.3)
	1–3	13 (37.1)		63 (39.4)
	≥4	2 (5.7)		15 (9.4)
≤15 (after termination)	Mean (SD)	0.78 (1.48)	3	0.70 (1.20)	11
	0	17 (53.1)		96 (64.0)
	1–3	14 (43.8)		45 (30.0)
	≥4	1 (3.1)		9 (6.0)

Subjects at risk: Parents diagnosed with the respiratory or cognitive type of panic disorder.

Healthy controls in this study self-reported a mean anxiety score of 1.00 (SD 1.93) after 5 minutes of CO₂ inhalation, the point at which the CO₂ challenge is fully effective and the anxiety peaks in most individuals. Children or adolescents at risk for PD reported a mean of 1.04 (SD 1.41). These scores did not differ significantly from each other and from the measures at 0 and 15 minutes of the CO₂ challenge.

The likelihood of experiencing no anxiety (anxiety score=0) was ∼50–60% in both groups. The likelihood of experiencing moderate to significant anxiety (anxiety score≥4) was 5–10%, with the highest reported scores of 8 out of 10 in one healthy participant, and 6 in three participants at risk for PD. Only two of the participating children and adolescents, one a healthy control and one at risk for PD, ended the procedure before 5 minutes of CO₂ inhalation.

Step 3: Identifying an appropriate comparator activity that poses the same type of harm as the research intervention

Average, healthy, normal children ordinarily engage in a number of appropriate activities that pose a risk of anxiety, including taking tests in school, watching the TV news, and receiving recommended vaccinations. Although anxiety is a complex phenomenon, it seems plausible to conceptualize the anxiety associated with these activities to be of the same type of harm as the anxiety associated with undergoing a CO₂ challenge. In particular, although anxiety can be caused by a wide range of triggers, the negative, subjective experience is relatively similar across different triggers. This assumption is supported by the literature on the clinical effects of anxiety, which establishes that reports of the level of anxiety experienced during a range of activities predict long-term impairment (Pine et al. 1998), and treatments to mitigate these impairments target the experienced anxiety (Walkup et al. 2008).

Step 4: Estimating the magnitude and likelihood of the harm occurring as a result of the comparator activity

We conducted a literature search using PubMed, Cumulative Index to Nursing and Allied Health (CINAHL), PsycINFO, Scopus, and Google Scholar to identify studies that provide self-report data on 1) the anxiety, worry, fear, or distress that 2) healthy children or adolescents (3–18 years of age) experience in the context of 3) appropriate activities of daily life or recommended routine examinations or procedures using 4) ordinal Likert-type scales. In consultation with the coauthors, one author (E.A.) reviewed the search results, selected the studies that met the inclusion criteria, and extracted the data. The search yielded 22 publications, and we selected data points on 67 daily life activities that we deemed appropriate and 41 routine examinations (Table 3 and Supplementary Tables, Appendices I–III) (Supplementary Tables, are available in the online version of this article at www.liebertpub.com/jcap.) (Ollendick 1983; Alvesalo et al. 1993; Doogan and Thomas 1992; Kearney et al. 1993; Neff and Dale 1996; Stickler 1996; von Baeyer et al. 1997; Wong et al. 1998; Cohen et al. 1999; Muris et al. 2000; Peretz and Efrat 2000; Smith and Wilson 2000; Cohen et al. 2002; Mindell and Barrett 2002; Smith and Wilson 2002; Lindahl et al. 2005; Rantavuori et al. 2005; Howard and Freedman 2007; Lacey et al. 2008; van der Molen and Bushman 2008; Laing et al. 2009; Nicholas et al. 2010). With the exception of four publications (Mindell and Barrett 2002; Lacey et al. 2008; van der Molen and Bushman 2008; Nicolas et al. 2010) the included studies reported the mean anxiety or fear experienced by children and adolescents and not the likelihood of experiencing low, moderate, or severe anxiety from the activities. The available data, therefore, primarily allowed estimates of the level of anxiety that the average child would experience with a likelihood of 100% during various daily life activities and routine procedures.

Table 3.

Risk of Experiencing Anxiety in Daily Life or During Routine Examinations in Children (Abridged Version) ^*

Study	Location	n	Population	Age (years)	Activity	Anxiety/Fear/Worry/Stress ^a
						0–10 Likert scale (mean)
Anxiety
van der Molen and Bushman 2008	Netherlands	572	School children	8–12	Common threats on TV news	5.38
Howard and Freeman 2007	UK	475	School children	8–12	Dentist generally	1.58
					Teeth scraped and polished	3.38
Muris et al. 2000	Netherlands	153	School children	4–12	Scary dream	5.7
Smith and Wilson 2000	US	88	School children	6–11	TV news/Crime story	3.75
Neff and Dale 1996	Canada	48	Children in “community facilities”	7–12	Hearing parents quarrel	5.63
					Having best friend angry	5.43
					Being made fun of in class	5.15
					Losing a pet	7.98
Fear
Nicolas et al. 2010	France	1303	School children	5–11	Going to the dentist	2.2
Laing et al. 2009	UK	142	School children and adolescents	7–16	Lifts/small spaces	3.33
					Lonely places	5.33
					Nightmares	6
Lacey et al. 2008	US	107	Patients at general pediatric clinics (excludes children with chronic illness/cognitive disability, ≥4 medical procedures in the last 2 years, language barriers)	4–6	Going to doctor's office	2.59
					Getting a shot	5.25
van der Molen and Bushman 2008	Netherlands	572	School children	8–12	Common threats on TV news	4.78
					Common threats in fictional TV shows	5
Rantavuori et al. 2005	Finland	1212	Children participating in a population study of dental caries	6–15	Going to the dentist	1.45
					Dental instruments in mouth	2.45
von Baeyer et al. 1997	Canada	32	Children getting their ears pierced (first time)	5–11	Getting 1st ear pierced	3.98
					Getting 2nd ear pierced	4.85
Stickler 1996	US	113	School children	∼10–12	Scary movie	4.75
					Strange or mean-looking dogs	4.5
					Parents criticizing	4.3
					Nightmares	3.8
Alvesalo et al. 1993	Finland	828	School children	∼12–13	Injections	2.75
					Having someone put instruments in their mouth	1
Ollendick 1983	US	217	“Normal” children (excluded children with previous mental-health contacts and/or special education placements)	8–11	Giving an oral report	4.75
					Taking a test	3.9
					Being teased	2.9
					Being criticized by others	4.95
Worry
Cohen et al. 2002	US	61	Children being immunized at a health department	3–7	Getting a shot	6.23
Cohen et al. 1999	US	39	School children	8–11	Getting a shot	4.21
Stress
Lindahl et al. 2005	Sweden	46	School children (excluded children with language barriers)	∼12–15	Reading test	3
					Math test	4.5
Kearney et al. 1993	US	567	School children	7–15	Answering a question in class	3.53
					Getting in trouble at home	2.5
					Playing sports well	4.38

See Supplementary Tables, Appendices I–III for the full version and additional explanation. Supplementary Tables, are available in the online version of this article at www.liebertpub.com/jcap.

Data converted from original recordings. See Supplementary Tables, Appendices II and III.

Because the existing studies used different scales, we normalized all ratings to a common scale. We chose a 0–10 Likert scale to allow for comparisons with the data on CO₂-induced anxiety. For each scale, we determined the number of segments between no anxiety – set at “0” corresponding to the 0–10 Likert scale used in the CO₂ challenge study – and the highest possible anxiety. We then used a simple fraction calculation to normalize the reported mean to a 0–10 Likert scale with 10 segments, and “10” as the number for the highest possible anxiety. For example, in one study children reported a mean anxiety of 1.51 on a five point smiley face, where “1” signified not worried and “5” signified very worried (Howard and Freeman 2007). This scale has four segments. Adjusting the reported mean to a scale from 0 to 4, where “0” signifies no anxiety, yielded a rating of 0.51. To normalize this finding, we calculated (0.51/4 )=(x/10), yielding a score of 1.28. In another study, children were asked how anxious they were when watching TV news. They reported a mean anxiety of 1.5 on a five point Likert scale that ranged from “not at all scared” (0) to “very very scared” (4) (Smith and Wilson 2000). We normalized the finding as above, calculating (1.5/4)=(x/10) to yield a score of 3.75.

The median of the normalized mean anxiety ratings was 3.37 on a 0–10 scale for all identified data. This suggests that the average child experiences overall mild to low-moderate anxiety (3.37 out of 10) during a broad range of appropriate daily life activities and routine examinations and procedures.

Step 5: Comparing the likelihoods and magnitudes of the harm

The mean anxiety resulting from CO₂ inhalation in our data set was lower than the mean anxiety reported in all of the 67 appropriate daily life activities. This applied to both the healthy controls and children or adolescents at risk for PD. In addition, the mean anxiety from CO₂ challenge was lower than the mean anxiety for 33 of the 41 activities associated with routine examinations. Children's fear of certain aspects of routine dental procedures was lower in two publications (Alvesalo et al. 1993; Peretz and Efrat 2000). However, the same publications and several others reported higher levels of anxiety associated with other aspects of dental visits, suggesting that obtaining dental care overall is likely to exceed the anxiety associated with a respiratory CO₂ challenge (Ollendick 1983; Alvesalo et al. 1993; Wong et al. 1998; Peretz and Efrat 2000; Rantavuori et al. 2005; Howard and Freeman 2007; Laing et al. 2009; Nicolas et al. 2010).

Even though the mean anxiety is lower for CO₂ challenge, a finding that CO₂ challenge exposes a significant number of children and adolescents to significantly greater anxiety than they experience during appropriate activities of daily life activities would suggest that CO₂ challenge poses greater than minimal risk. Therefore, in addition to comparing the mean anxiety ratings, we also compared the percentages for children and adolescents who experience more significant anxiety. For CO₂ challenge, 8.7% (17/195) children or adolescents in our data set reported anxiety scores of 4/10 or higher, all of which were fully reversible within several minutes of stopping the challenge. In comparison, in the limited number of studies that recorded these data, the percentage of children and adolescents reporting higher anxiety scores from activities of daily life was 7–35%. These activities included watching the news and being vaccinated (Mindell and Barrett 2002; Lacey et al. 2008; van der Molen and Bushman 2008; Nicolas et al. 2010).

In addition, two studies reported a higher mean anxiety rating for two comparator activities – getting immunized and losing a pet – than the highest score recorded in two out of the 195 children undergoing a CO₂ challenge in our data set (Neff and Dale 1996; Cohen et al. 2002). Hence, this comparison equally suggests that the risk of experiencing anxiety from a 15 minute challenge with 5% CO₂ is lower than the risk of anxiety from activities of daily life and routine examination. Overall, this analysis suggests that respiratory CO₂ challenge qualifies as a minimal risk procedure in children and adolescents, under United States regulations (United States Department of Health and Human Services 1991).

The Need for Judgment

Although DCA provides a systematic method to evaluate the risks of research interventions that pose the same types of risk as appropriate comparator activities, it does not eliminate the need for reviewers to use their judgment. First, the risks of some comparator activities may differ in morally significant ways from the risks of research participation. For example, the risks of many daily life activities (e.g., mountain biking) are accepted because the activities offer children and adolescents the potential for direct benefit. As a result, these risks do not necessarily provide an appropriate standard for evaluating the risks of research interventions that do not offer subjects a prospect of direct benefit (Field and Behrman 2004; Wendler 2005).

One way of addressing this concern is to compare the risks of research interventions with the risks of activities that are designed to benefit others. In the present case, we compared the risks of CO₂ challenge with the risks of a broad range of daily life activities. Some of these activities seem appropriate for children and adolescents as part of charitable fundraisers that primarily aim to benefit others (e.g,. taking a test, playing sports) (Wendler 2005).

Second, investigators and IRBs must determine that the comparator activity poses a sufficiently similar type of harm. In the present case, we assumed that the anxiety experienced during CO₂ challenge was essentially the same type of harm as the anxiety experienced as the result of activities of daily life. Although subjective experiences of psychological harm will typically be comparable across activities, other research harms may not be similar enough for using DCA. In this case, IRBs should use an alternative systematic method for evaluating research risks (Rid et al. 2010).

Third, investigators and IRBs need to use their judgment to decide whether, given the strength and relevance of the available data, they can be confident of the risk assessments. When there is significant uncertainty about the risks of an intervention because the available data are insufficient, IRBs should err on the side of caution and categorize the procedure as entailing more than minimal risk. This is because uncertainty implies large confidence intervals or standard deviations for likelihood estimates of potential research harms, as well as an increased likelihood of previously unknown harms. Under conditions of significant uncertainty, the risks of an intervention are, therefore, likely to be greater than minimal.

Fourth, reviewers need to exercise their judgment to estimate and compare the magnitude and the likelihood of the harms resulting from the research intervention and the comparator activity. For example, for the purposes of ethical evaluation, a mean anxiety rating of 1.4 on a 0–10 Likert scale seems normatively equivalent to a rating of 1.5. Reviewers may also need to balance likelihoods against harms, provided the available data are of sufficient quality. For example, the likelihood of experiencing moderate anxiety may be somewhat greater in research than in the comparator activity, whereas the likelihood of experiencing serious anxiety may be significantly lower. In that case, reviewers need to judge whether the decrease in the one risk compensates for the increase in the other.

Finally, although important to protecting pediatric research subjects, a finding that a research intervention poses minimal risk does not necessarily imply that it is acceptable. IRBs and others also need to ensure that additional ethical requirements are satisfied, including obtaining assent and fair subject selection (Emanuel et al. 2000). With regard to the present case, it is important to assess whether it is ethically appropriate for researchers to intentionally provoke negative symptoms in participants, even when the risks are minimal. To address this question, investigators and IRBs should consult the literature on the ethics of intentionally provoking some negative symptoms in participants (Miller and Grady 2001; Hope and McMillan 2004).

Practical Implementation

Given the relative complexity of making systematic risk evaluations, it may make most sense for dedicated committees with the expertise and access to the relevant data to use DCA to develop default risk determinations for common research interventions (Rid et al. 2010). Interventions that primarily pose risks of psychological harm (e.g., completing anxiety-provoking questionnaires, undergoing an MRI) would be ideal candidates. IRBs could then consider whether there is anything about the study or population in question that implies that the risks might be greater than the default judgment suggests.

Conclusions

Determining whether research interventions pose minimal or greater than minimal risk is essential for protecting pediatric research participants while allowing acceptable research to proceed. Many regulations and guidelines define minimal risks in comparison to the risks posed by the activities of daily life and routine examinations. However, there are few systematic methods available to systematically compare the risks of research procedures with the risks posed by these activities.

DCA provides a method to make these comparisons when the research risks are of the same type as the risks of activities of daily life. As a result, DCA is particularly valuable for assessing procedures that primarily pose risks of psychological harm, such as anxiety. This suggests that DCA is especially valuable for ensuring that pediatric research subjects are not exposed to excessive risks, while allowing important and appropriate research in pediatric psychiatry and psychopharmacology to proceed and benefit future patients.

Footnotes

Acknowledgments

The authors thank Frank Miller (Department of Bioethics, National Institutes of Health [NIH] Clinical Center) for connecting them to each other, and Karen Smith (NIH Library) and Lorelei Woody (The Taubman Health Sciences Library, University of Michigan) for support with the literature search. These individuals received no compensation for their contributions.

Disclosures

No competing financial interests exist. Annette Rid received financial support from the University of Zurich (Nachwuchsförderungskredit) for this project. The sponsor had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.

References

Alvesalo

, Murtomaa

, Milgrom

, Honkanen

, Karjalainen

, Tay

: The Dental Fear Survey Schedule: A study with Finnish children. Int J Paediatr Dent, 3:193–198, 1993.

Briggs

, Stretch

, Brandon

: Subtyping of panic disorder by symptom profile. Br J Psychiatry, 163:201–209, 1993.

Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of Canada, Social Sciences and Humanities Research Council of Canada: Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans. Montreal; 2010.

Cohen

, Bernard

, Greco

, McClellan

: A child-focused intervention for coping with procedural pain: Are parent and nurse coaches necessary?. J Pediatr Psychol, 27:749–757, 2002.

Cohen

, Blount

, Cohen

, Schaen

, Zaff

: Comparative study of distraction versus topical anesthesia for pediatric pain management during immunizations. Health Psychol, 18:591–598, 1999.

Council for International Organizations of Medical Sciences (CIOMS): International Ethical Guidelines for Biomedical Research Involving Human Subjects. Geneva: CIOMS; 2002.

Doogan

, Thomas

: Origins of fear of dogs in adults and children: The role of conditioning processes and prior familiarity with dogs. Behav Res Ther, 30:387–394, 1992.

Emanuel

, Wendler

, Grady

: What makes clinical research ethical?. JAMA, 283:2701–2711, 2000.

Field

, Behrman

: Institute of Medicine. Ethical Conduct of Clinical Research Involving Children. Washington, DC: The National Academies Press; 2004.

10.

Fredholm

, Arslan

, Halldner

, Kull

, Schulte

, Wasserman

: Structure and function of adenosine receptors and their genes. Naunyn Schmiedebergs Arch Pharmacol, 362:364–374, 2000.

11.

Gorman

, Askanazi

, Liebowitz

, Fyer

, Stein

, Kinney

, Klein

: Response to hyperventilation in a group of patients with panic disorder. Am J Psychiatry, 141:857–861, 1984.

12.

Gorman

, Fyer

, Goetz

, Askanazi

, Liebowitz

, Fyer

, Kinney

, Klein

: Ventilatory physiology of patients with panic disorder. Arch Gen Psychiatry, 45:31–39, 1988.

13.

Gorman

, Papp

, Coplan

, Martinez

, Lennon

, Goetz

, Ross

, Klein

: Anxiogenic effects of CO₂ and hyperventilation in patients with panic disorder. Am J Psychiatry, 151:547–553, 1994.

14.

Hope

, McMillan

: Challenge studies of human volunteers: Ethical issues. J Med Ethics, 30:110–106, 2004.

15.

Howard

, Freeman

: Reliability and validity of a faces version of the Modified Child Dental Anxiety Scale. Int J Paediatr Dent, 17:281–288, 2007.

16.

Indian Council of Medical Research (ICMR): Ethical Guidelines For Biomedical Research On Human Participants. New Dehli: ICMR; 2006.

17.

Kahnemann

: Thinking, Fast and Slow. London: Penguin; 2012.

18.

Kearney

, Drabman

, Beasley

: The trials of childhood: The development, reliability, and validity of the Daily Life Stressors Scale. J Child Fam Stud, 2:371–388, 1993.

19.

Kessler

, Chiu

, Demler

, Merikangas

, Walters

: Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry, 62:617–627, 2005.

20.

Klein

: False suffocation alarms, spontaneous panics, and related conditions. An integrative hypothesis. Arch Gen Psychiatry, 50:306–317, 1993.

21.

Kopelman

: Minimal risk as an international ethical standard in research. J Med Philos, 29:351–378, 2004.

22.

Lacey

, Finkelstein

, Thygeson

: The impact of positioning on fear during immunizations: Supine versus sitting up. J Pediatr Nurs, 23:195–200, 2008.

23.

Laing

, Fernyhough

, Turner

, Freeston

: Fear, worry, and ritualistic behaviour in childhood: Developmental trends and interrelations. Inf Child Dev, 18:351–366, 2009.

24.

Lindahl

, Theorell

, Lindblad

: Test performance and self-esteem in relation to experienced stress in Swedish sixth and ninth graders—saliva cortisol levels and psychological reactions to demands. Acta Paediatr, 94:489–495, 2005.

25.

Miller

, Grady

: The ethical challenge of infection-inducing challenge experiments. Clin Infect Dis, 33:1028–1033, 2001.

26.

Mindell

, Barrett

: Nightmares and anxiety in elementary-aged children: Is there a relationship. Child, 28:317–322, 2002.

27.

Muris

, Merckelbach

, Gadet

, Moulaert

: Fears, worries, and scary dreams in 4- to 12-year-old children: Their content, developmental pattern, and origins. J Clin Child Psychol, 29:43–52, 2000.

28.

Neff

, Dale

: Worries of school-age children. J Soc Pediatr Nurs, 1:27–32, 1996.

29.

Nepal Health Research Council: National Ethical Guidelines for Health Research in Nepal. Ramshah Path, Katmandu; 2001.

30.

Nicolas

, Bessadet

, Collado

, Carrasco

, Rogerleroi

, Hennequin

: Factors affecting dental fear in French children aged 5–12 years. Int J Paediatr Dent, 20:366–373, 2010.

31.

Ollendick

: Reliability and validity of the Revised Fear Survey Schedule for Children (FSSC-R). Behav Res Ther, 21:685–92, 1983.

32.

Peretz

, Efrat

: Dental anxiety among young adolescent patients in Israel. Int J Paediatr Dent, 10:126–132, 2000.

33.

Pine

, Cohen

, Gurley

, Brook

, Ma

: The risk for early-adulthood anxiety and depressive disorders in adolescents with anxiety and depressive disorders. Arch Gen Psychiatry, 55:56–64, 1998.

34.

Pine

, Costello

, Masten

. Trauma, proximity, and developmental psychopathology: The effects of war and terrorism on children. Neuropsychopharmacology, 30:1781–1792, 2005a.

35.

Pine

, Klein

, Coplan

, Papp

, Hoven

, Martinez

, Kovalenko

, Mandell

, Moreau

, Klein

, Gorman

: Differential carbon dioxide sensitivity in childhood anxiety disorders and nonill comparison group. Arch Gen Psychiatry, 57:960–967, 2000.

36.

Pine

, Klein

, Roberson–Nay

, Mannuzza

, Moulton

3rd , Woldehawariat

, Guardino

: Response to 5% carbon dioxide in children and adolescents: relationship to panic disorder in parents and anxiety disorders in subjects. Arch Gen Psychiatry, 62:73–80, 2005b.

37.

Rantavuori

, Lahti

, Seppa

, Hausen

: Dental fear of Finnish children in the light of different measures of dental fear. Acta Odontol Scand, 63:239–244, 2005.

38.

Rid

, Emanuel

, Wendler

: Evaluating the risks of clinical research. JAMA, 304:1472–1479, 2010.

39.

Rid

, Wendler

: A proposal and prototype for a research risk repository to improve the protection of research participants. Clin Trials, 8:705–715, 2011.

40.

Roberson–Nay

, Klein

, Mannuzza

, Moulton

3rd , Guardino

, Pine

: Carbon dioxide hypersensitivity in separation-anxious offspring of parents with panic disorder. Biol Psychiatry, 67:1171–1177, 2010.

41.

Roy–Byrne

, Craske

, Stein

: Panic disorder. Lancet, 368:1023–1032, 2006.

42.

Shah

, Whittle

, Wilfond

, Gensler

, Wendler

: How do institutional review boards apply the federal risk and benefit standards for pediatric research?. JAMA, 291:476–482, 2004.

43.

Slovic

: Perception of risk. Science, 236: 280–285, 1987.

44.

Smith

, Wilson

: Children's comprehension of and fear reactions to television news. Media Psychol, 4:1–26, 2002.

45.

Smith

, Wilson

: Children's reactions to a television news story: The impact of video footage and proximity of crime. Communic Res, 27:641–673, 2000.

46.

Stickler

: Worries of parents and their children. Clin Pediatr, 35:84–90, 1996.

47.

South African Medical Research Council (SAMRC): Guidelines on Ethics for Medical Research: General Principles, including research on children, vulnerable groups, international collaboration and epidemiology. Tygerberg, South Africa: SAMRC; 2002.

48.

Tversky

, Kahneman

: Judgment under uncertainty: Heuristics and biases. Science, 185:1124–1131, 1974.

49.

Uganda National Council for Science and Technology: National Guidelines for Research Involving Humans as Research Participants. Kampala, Uganda: Uganda National Council for Science and Technology; 2007.

50.

United States Department of Health and Human Services: U.S. Code for Federal Regulations, 45 CFR 46; revised. Washington, DC; 2009.

51.

van der Molen

, Bushman

: Children's direct fright and worry reactions to violence in fiction and news television programs. Journal Pediatr, 153:420–424, 2008.

52.

von Baeyer

, Carlson

, Webb

: Underprediction of pain in children undergoing ear piercing. Behav Res Ther, 35:399–404, 1997.

53.

Walkup

, Albano

, Piacentini

, Birmaher

, Compton

, Sherrill

, Ginsburg

, Rynn

, McCracken

, Waslick

, Iyengar

, March

, Kendall

: Cognitive behavioral therapy, sertraline, or a combination in childhood anxiety. New Engl J Med, 359: 2753–2766, 2008.

54.

Weinstein

: Optimistic biases about personal risks. Science, 246:1232–1233, 1989.

55.

Wendler

: Protecting subjects who cannot give consent: toward a better standard for “minimal”. risks. Hastings Cent Rep, 35:37–43, 2005.

56.

Wong

, Humphris

, Lee

: Preliminary validation and reliability of the Modified Child Dental Anxiety Scale. Psychol Rep, 83:1179–1186, 1998.

57.

World Medical Association (WMA): Declaration of Helsinki – Ethical Principles for Medical Research Involving Human Subjects (first version adopted 1964). Geneva: WMA; 2013.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.05 MB

0.03 MB

Evaluating the Risks of Clinical Research: Direct Comparative Analysis

Abstract

Objectives:

Methods:

Results:

Conclusions:

Introduction

Clarifying the Minimal Risk Standard

Direct Comparative Analysis

Example: CO2 Challenge in Children and Adolescents

Applying Direct Comparative Analysis

Step 1: Identifying the potential harms posed by the research intervention

Step 2: Estimating the magnitude and likelihood of the potential harms

Step 3: Identifying an appropriate comparator activity that poses the same type of harm as the research intervention

Step 4: Estimating the magnitude and likelihood of the harm occurring as a result of the comparator activity

Step 5: Comparing the likelihoods and magnitudes of the harm

The Need for Judgment

Practical Implementation

Conclusions

Footnotes

Acknowledgments

Disclosures

References

Supplementary Material

Example: CO₂ Challenge in Children and Adolescents