Opinions on the Preclinical Evaluation of Novel Therapies for Spinal Cord Injury: A Comparison between Researchers and Spinal Cord-Injured Individuals

Introduction

W hat preclinical evidence of safety and efficacy is needed to justify the translation of an experimental therapy into human clinical trials of spinal cord injury (SCI)? Carefully considering this question is important for the SCI field for a number of reasons. First, clinical trials of interventions to improve neurologic function after SCI can take many years to complete (Fawcett et al., 2007; Tator, 2006), and cost an enormous amount of money, as recently illustrated by Geron Corporation's decision to prematurely terminate their embryonic stem cell trial (Frantz, 2012). Second, while the SCI research community absorbs these direct costs of the clinical trial, it also bears the potential opportunity costs of being unable to evaluate other (possibly even more promising) treatments because of limited money, patients, and clinical research centers with the capacity to conduct acute SCI trials. And finally, while SCI individuals who participate in an early (Phase I or IIa) experimental trial might acknowledge that the trial's focus is on confirming the safety of a therapy, it is understandable that they invariably hope that it will be of some benefit to them. It is therefore the responsibility of the research community to ensure that the decision to move a treatment forward into clinical trials is based on careful consideration of its supporting preclinical evidence (Dietrich, 2003).

In 2009 we conducted a survey to ask researchers for their perspectives on this question of what preclinical evidence was needed to justify the translation of a therapy into SCI clinical trials (Kwon et al., 2010). We received 324 responses from clinicians, clinician-scientists, scientists, trainees, industry, and regulatory agencies. Respondents revealed strong support for the need for large animal studies and independent replication. In general, the survey revealed that the research community had high expectations for the extent of preclinical evidence that was needed prior to human translation. It also illustrated the gap between what the research community felt was needed and what was actually being produced in terms of preclinical efficacy prior to human trials being initiated. The survey results highlighted the need for the design of a rational preclinical pathway for SCI therapies on their way to human translation, and sparked the development of a grading scale to objectively evaluate the published preclinical scientific literature on acute SCI treatments (Kwon et al., 2011).

While this initiative provided the perspectives of the research community about the translation of experimental SCI therapies, it lacked the voice of an important constituent in this dialogue: that of individuals living with SCI. It is certainly conceivable that the opinions of SCI individuals may differ from those of the scientific community, and that these opinions may provide tremendous insights for translational research (Anderson, 2004; Dietrich, 2003; Illes et al., 2011; Reimer and Illes, 2009). For example, while the research community engages in vigorous internal debate about the need for demonstrating efficacy in large animal models and independently replicating promising findings, individuals living with SCI might themselves be satisfied with the translation of a therapy that has shown promising results in a single rodent study. Given that they are ultimately the end-users of the promising neuroprotective and neuroregenerative technologies that emerge from the laboratory, it seemed appropriate to seek to understand their perspectives on preclinical research. We therefore undertook this survey to determine what patient expectations were with respect to the extent of preclinical evidence necessary to translate an experimental therapy into a clinical trial.

Methods

Results

Demographic characteristics of SCI individuals and researchers

SCI individuals

In all, 214 responses were received from individuals with SCI (response rate of 29%). The average age of the respondents was 42 years, and on average, 5.5 years had passed between their SCI and the conduct of this survey. In addition, 74% were males, and 62% suffered injuries of the cervical spinal cord. The provincial registry from which the individuals were identified provided baseline neurologic impairment data: 48% were AIS grade A, 15% were AIS grade B, and 37% were AIS grades C/D. The respondents were asked to report their current level of neurologic impairment, and reported it as: 33% AIS A, 12% AIS B, and 55% AIS C/D at the time of the survey.

A wide spectrum of educational backgrounds were observed among the respondents (Fig. 1). Respondents reported obtaining and accessing information about SCI research from many sources, with the Internet being the most common (Fig. 2). The respondents were asked to rate on a scale of 1 to 10 how informed and knowledgeable they felt they were about SCI research and emerging treatments (with 1 being “not at all informed” and 10 being “extremely well informed”). On this question 63% of the respondents rated themselves between 1 and 5, suggesting that the majority deemed themselves to be poorly informed about SCI research and emerging treatments (Fig. 3). In fact, 41% of the respondents were at the lower end of the spectrum of being poorly informed (1, 2, or 3), while only 13% described themselves as being at the upper end of the spectrum of being extremely well informed (8, 9, or 10).

OPEN IN VIEWER

FIG. 1.

Highest level of education attained by the respondents.

OPEN IN VIEWER

FIG. 2.

Sources of information about spinal cord injury research used by the respondents.

OPEN IN VIEWER

FIG. 3.

Self-reported level of the extent to which respondents feel informed about spinal cord injury research and emerging treatments.

SCI researchers

From our previous survey of the SCI research community (Kwon et al., 2010), 105 respondents classified themselves as “Scientific Principal Investigators” running a laboratory-based research program, 34 as “Clinician Scientists” (clinicians who additionally operate a laboratory-based research program), 76 as “Spinal Surgeons,” 20 as “Clinicians” (non-surgical), 69 as “Trainees” (graduate student, post-doctoral students, and research associates), and 20 as “Others” (industry or research foundation representatives, clinical research assistants, and regulatory officials). For the purposes of this current analysis, we combined the 105 who identified themselves as scientific principal investigators of a laboratory-based SCI research program, and all 130 respondents who possessed a clinical background (clinician scientists, spinal surgeons, or non-surgical clinicians). This comprised the cohort of 235 SCI researchers. Demographic information about the researchers themselves was not collected (e.g., age, gender, years in the scientific field or in clinical practice, or educational background).

Perspectives on preclinical scientific evidence and safety assurance

Non-invasive experimental drug therapies in acute SCI

For these questions, individuals with SCI were presented with the scenario of a researcher approaching them in the emergency room shortly after their injury to propose their participation in a clinical trial of an experimental drug administered intravenously. They were asked to consider what evidence would be “enough evidence from the laboratory to warrant trying the drug in persons like yourself.” Researchers were asked to consider what demonstration of safety and efficacy was needed for such a non-invasive experimental drug before proceeding with a clinical trial. The comparisons between SCI researchers and individuals with respect to the efficacy and safety of non-invasive drug therapies are shown in Figure 4.

OPEN IN VIEWER

FIG. 4.

Comparison between spinal cord injury (SCI) individuals and SCI researchers on the need for demonstrating effectiveness and safety of non-invasive drug therapies. Significant differences between SCI individuals and researchers are denoted by p<0.001 (by chi-square analyses).

A higher proportion of SCI individuals than researchers were in agreement (49% versus 40%) with the statement that efficacy in rodent studies was sufficient for clinical trials (p<0.001 by chi-square analysis; Fig. 4A). Conversely, however, the majority of both SCI individuals and researchers (62% and 67%) agreed with the statement that efficacy in large animal models was necessary (Fig. 4B), and considerably more individuals (68%) than researchers (39%) agreed that primate studies were needed (p<0.001 by chi-square analysis; Fig. 4C). Eighty-four percent of the SCI individuals agreed with the need for independent replication of efficacy, compared to 96% of researchers (Fig. 4D). On the issue of demonstrating safety, 70% of SCI individuals agreed with the need for primate testing, compared to 52% of the researchers (Fig. 4G).

Invasive cell transplantation therapy

For these questions, individuals with SCI were asked to consider what evidence they felt was “enough evidence from the laboratory to warrant trying the stem cells in persons like yourself.” Researchers were asked to consider what demonstration of safety and efficacy was needed for such an invasive cell transplantation therapy before proceeding with a clinical trial. Comparisons between SCI researchers and individuals with respect to the efficacy and safety of invasive stem cell therapies are shown in Figure 5.

OPEN IN VIEWER

FIG. 5.

Comparison between spinal cord injury (SCI) individuals and SCI researchers on the need for demonstrating effectiveness and safety of invasive stem cell therapies. Significant differences between SCI individuals and researchers are denoted by p<0.001 (by chi-square analyses).

Similarly to the drug therapies, a higher proportion of SCI individuals than researchers were in agreement (45% versus 17%) with the statement that efficacy in rodent studies was sufficient for clinical trials (p<0.001 by chi-square analysis; Fig. 5a). These sentiments were also reflected in the question about demonstrating safety, for which a higher proportion of SCI individuals than researchers were in agreement (41% versus 21%) with the statement that safety in rodent studies was sufficient (p<0.001 by chi-square analysis; Fig. 5e). Accordingly, SCI researchers had stronger agreement than SCI individuals for the need for demonstrating efficacy and safety in large animal studies (efficacy: 81% versus 59%, p<0.001 by chi-square analysis, Fig. 5b; safety: 82% versus 64%, not significant, Fig. 5f). The majority of SCI individuals and researchers agreed to the need for primate studies to demonstrate both efficacy and safety (Fig. 5c and g). Again, we note the contradictory nature of the responses of the SCI individuals, with 45% agreeing that efficacy in rodent studies was sufficient, but then 59% and 69% agreeing that large animal and primate studies were needed.

Discussion

Increasing impatience with basic science by the public has created urgency for researchers to rapidly translate their work into clinical benefits. While the case for novel experimental treatments for SCI is no exception, such expectations are especially complicated by the relatively low incidence of SCI compared to traumatic brain injury or stroke, and the resulting high costs and constraints on the duration of trials and the ability to recruit suitable participants (Geisler et al., 2001; Lammertse et al., 2007; Tuszynski et al., 2007). These factors create a scenario in which the research community realistically only has the capacity to test the efficacy of a handful of treatments at any given time, and imposes an opportunity cost on the field, as promising new therapies may not move forward into human trials simply because fiscal or human resources are already committed elsewhere. It is thus imperative that the most judicious and evidence-based criteria are applied to determine which treatments are taken forward to human trials (Blesch and Tuszynski, 2009; Dietrich, 2003). Similar sentiments have also been voiced in the stroke field, which has suffered considerable frustration in translating promising neuroprotective therapies into effective human treatments (Fisher et al., 2009; O'Collins et al., 2006).

We undertook this study to determine the perspectives of individuals with SCI on the issue of “What preclinical evidence of safety and efficacy is needed to justify the translation of an experimental therapy into human clinical trials of spinal cord injury (SCI)?” Having already surveyed the opinions of researchers in the SCI field on this topic, we felt that it was important to garner the voice of individuals living with paralysis, and determine if significant differences existed between the research community and the end-users. We recognize that the level of knowledge of SCI research was quite variable among the individuals with SCI who responded to this survey (as they themselves reported). As such, it would not be expected that they would understand many of the logistical, financial, and regulatory challenges associated with conducting large animal or primate research and replication studies. However, the survey was not intended to test individuals with SCI on their knowledge of translation in SCI research. It merely asked for their opinions about the research being conducted on novel therapies that they (and newly injured people just like them) will be asked to participate in testing in clinical trials. While these opinions may not be as informed as the opinions of the research community, it seems appropriate to collect and consider them in the consideration of translational studies that lead to clinical testing of novel therapies. The questionnaire instrument is included as the Supplementary Questionnaire for those interested in how we posed the questions to the individuals with SCI (see online supplementary material at http://www.liebertonline.com). We obviously attempted to use non-technical language in the generation of the questions, but acknowledge that (like any survey of this nature) the level of understanding of each question likely varied among respondents.

The data suggest that SCI individuals were more likely than SCI researchers to agree that preclinical safety and efficacy data in small animal models (e.g., rats or mice) represent sufficient evidence to proceed with clinical trials for both non-invasive neuroprotective drug therapies and invasive stem cell therapies. However, despite this sentiment, the majority of individuals with SCI still expressed the opinion that demonstrating efficacy and assuring safety of drug and cell therapies in large animal models and primates was needed. Over 80% of individuals with SCI agreed that independent replication of the efficacy of promising therapies was needed prior to initiating human trials.

The responses of individuals with SCI concerning evidence in rodent models alone versus the need for large animal or primate studies were somewhat contradictory. Compared to the researchers, SCI individuals had a greater acceptance of rodent studies alone for demonstrating efficacy, as reflected by the higher proportion of those agreeing with, and the lower proportion of those disagreeing with, the statement that rodent studies were sufficient. However, in a subsequent question, SCI individuals reported a greater demand for primate studies (as reflected by the higher proportion of SCI individuals agreeing with, and the lower proportion of those disagreeing with, the statement that primate models were needed). The somewhat lower proportion of researchers who felt that primate studies were needed likely reflects their awareness of the high costs and the lack of established models, as well as the ethical issues associated with such studies.

This contradiction was likely related to the order in which the questions were presented to the respondents in the survey. Upon being posed the question of whether efficacy in rodent studies alone was sufficient, the instinctive response from individuals with SCI might be to agree with that statement; but then in subsequent questions they might grasp the rationale for additional testing in large animal and primate studies, and agree with the additional need for those studies without going back to alter their initial response to the rodent studies. It is possible that the responses might have changed if we had asked the questions in the opposite order. Despite these issues, we feel that the balanced perspective on these results is that while SCI individuals were more likely than researchers to accept rodent studies as being sufficient, they still had a strong desire to see efficacy demonstrated in large animal and primate studies.

It is recognized that many cellular substrates have been developed as potential therapies for SCI (Tetzlaff et al., 2011), and that not all of these are actually stem cells. However, for the survey administered to individuals living with SCI we termed these as “stem cell therapies,” because we felt that this is how the overwhelming majority of individuals with SCI view cellular therapies, and we did not feel that the questionnaire was the appropriate medium to try to educate people about these differences. In the survey given to the SCI research community, we of course described these more broadly as “cell transplant therapies.” It is recognized that not all cell therapies (stem cell or otherwise) pose the same risks, and not all have to be delivered surgically. However, to achieve some clarity around the context of the stem cell therapy that we were asking SCI individuals to consider, we specifically indicated that this would be a “stem cell therapy” that would be implanted into their injured spinal cord in an additional surgical procedure, and then outlined a series of potential complications (see “Questionnaire for Individuals with SCI” in the Supplementary Questionnaire; see online supplementary material at http://www.liebertonline.com). Given the awareness of SCI individuals about clinical trials such as the Geron hESC-derived oligodendrocyte progenitors, and the myriad of unproven “stem cell treatments” being offered at clinics around the world, we felt that this context of a surgically-implanted stem cell therapy was a reasonable one to frame the questions related to stem cell therapies.

Without question, there is an urgent need to bring experimental therapies into the clinical realm where their efficacy can be tested. It is evident to most researchers in the field that large-scale clinical trials to definitively test the efficacy of such therapies are major undertakings. However, the question we asked in our survey of the research community and of the end-users (SCI individuals) was related to the justification for merely initiating the human evaluation of a potentially promising treatment; in other words, the first step of bench-to-bedside translation into a Phase I study. The purpose of such preliminary small-scale clinical trials is primarily to confirm safety and feasibility, and not to conclusively demonstrate efficacy. Thus one might call into question the necessity for such severe emphasis on preclinical efficacy, when the decision being made is merely to initiate a safety-confirming Phase I study, and not a full-scale efficacy-testing multicenter trial. Along these lines, we have encountered the following questions: “Why should the efficacy of ‘Therapy X’ need to be demonstrated not just in a rodent model but also in a large animal model or even primate model, and then be replicated in an independent laboratory, when all we want to do is a Phase I clinical trial to demonstrate its safety? Why set the preclinical bar so high?”

The problem with setting the preclinical bar lower becomes apparent at the end of Therapy X's Phase I clinical trial, after which it is quite likely that its safety and feasibility will be confirmed. Variability in spontaneous neurologic recovery typically makes it nearly impossible to detect whether the therapy is imparting a neurologic benefit in such small trials; but for the same reason, detecting a neurologic detriment is similarly difficult. The naturally high rate of adverse events and complications in acute SCI patients (Aito, 2003) makes it difficult to discern whether the adverse events observed in patients treated with the therapy are related to the therapy, or are simply part of the natural course of events in a complex patient population. So, in all likelihood, Therapy X's Phase I trial will end with the rather bland conclusion that it was feasible to administer and did not harm the subjects that received it. Now the big decision must be made to commit to the large, definitive study to show efficacy. However, the extent to which Therapy X's preclinical efficacy was demonstrated may be no different than what existed before the Phase I study began. If the evidence for preclinical efficacy was thin at the outset of the Phase I study, it is quite likely to be no more persuasive at the conclusion of the Phase I study. Translationally-relevant questions around dose optimization, the time window of efficacy, and the robustness of functional benefits that were never answered before going into the clinical trial now loom large. While this might be good reason to pause and actually conduct some of these important studies, considerable pressure at this stage will now exist to proceed with the large-scale trial to evaluate efficacy, given the significant effort that went into conducting the Phase I study. This form of “bait and switch” puts the field at risk of expending significant time and energy doing a large-scale trial on a therapy that many researchers and patients would consider premature, due to the absence of more extensive preclinical testing (based on their responses on the survey). We therefore contend that the more appropriate time to consider and establish the robustness of a therapy's efficacy is in the preclinical stage, prior to initiating any clinical trial.

As we have pointed out before, we do not wish to see insurmountable barriers erected in preclinical research requirements that would prevent all therapies from ever reaching human translation (Kwon et al., 2010). Neither do we wish to replicate the experience of the stroke field, and its plethora of failed neuroprotection clinical trials in spinal cord injury (O'Collins et al., 2006). At some point, however, only so much can be gleaned from the animal models and transitioning into humans is necessary. It is true that we have had clinical trials in SCI fail to demonstrate the efficacy of a therapy that showed promise in the laboratory setting. But from this we have learned a great deal about how to design clinical trials in SCI (Steeves et al., 2007), and tremendous advances have been made in establishing outcome measures that can be utilized in trials to help seek out therapeutic effects (Alexander et al., 2009). These should improve our chances of successfully demonstrating efficacy in current and future clinical trials. These improvements in trial design and human outcome measures will of course mean little if we push poorly-studied treatments into clinical trials with little or no preclinical regard to basic considerations such as optimal dose or time window of therapeutic efficacy. Clearly, we need to balance the obvious urgency for new therapies and our improved ability to test them with a rational and scientifically robust strategy for establishing them. Our data support the contention that individuals with SCI expect high levels of preclinical robustness before experimental therapies are offered to them in clinical trials, a sentiment that is also shared by the majority of the research community. Initiatives to establish a rational and balanced framework for determining how much preclinical evidence is enough to warrant translation of neuroprotective and neuroregenerative strategies would be welcomed by all.