Abstract
Little quantitative data exist concerning barriers that impede translation from bench to bedside. We systematically reviewed synthetic or biosynthetic polymer nerve scaffolds for peripheral nerve repair to study a defined research area that is beyond the discovery phase and has potential for clinical application. Using electronic and manual search methods, we identified published English language articles, where scaffolds were tested in preclinical animal models. A systematic review of these 416 reports estimated all costs related to the use of animals, surgery, and evaluation methods. The research studied 17 different nerves in eight animal species, with use of 65 evaluation methods at an estimated cost of $61,264,910 for the preclinical studies. A total of 127 surveys were sent to authors, of whom 12 could not be accessed electronically and 45 (39%) responded. Major causes for failure to translate included lack of a commercial partner, insufficient financial resources, a research program not involved in translation, and lack of expertise in regulatory affairs. This review emphasizes the urgent need for standardization of preclinical models and the need to establish better collaboration between laboratory investigators, clinicians, and the companies involved in commercialization. It identifies important areas for education of future investigators in the process of translation from discovery to improved health such as those funded by the National Institutes of Health Clinical and Translational Science Awards.
Introduction
B
The new discipline of comparative effectiveness research, driven by the Agency for Health Care Quality and Research, uses evidence-based medicine to compare the effectiveness of tests, treatments, procedures, and healthcare services without directly considering cost. 7
Despite these cost estimates, few researchers have attempted to study why translation moves slowly. The process of translation from bench to bedside has been divided into stages that delineate the progression of taking a research discovery toward preclinical models (T1), through preclinical models into humans (T2), and into general use that benefits the health of communities (T3).
Balas and Boren 8 in 2000 demonstrated that the average time from discovery to widespread use was ∼17 years. They analyzed nine cases of medical discoveries that improved healthcare and the factors that governed the progress from original research to clinical use. In each case, the major cause identified for slow translation was knowledge transfer. This transfer of knowledge occurred mainly at the last stage, from publication in reviews and textbooks to implementation, or T3.
Pharmaceutical companies and medical researchers also identify T2 as a difficult step. Cost and regulatory barriers are often quoted as major reasons for slow process at this level.5,6 The barriers to translation at T2 have not been extensively studied.
In the field of tissue engineering of devices for organ or tissue repair, the final step in laboratory development of a device intended for use in humans is to test it in animals. We used a recently completed systematic review of a specific medical application, 9 peripheral nerve repair with a synthetic or biosynthetic conduit, to study the barriers between preclinical animal studies and translation to use in humans. This area represents a distinct platform to study the question of barriers to translation using a systematic review process.
Publications concerning nerve scaffolds are circumscribed so that all publications may be identified using standard searching criteria, thereby identifying all publications reporting on the use of synthetic or biosynthetic nerve scaffolds in animals. Performance of an animal study is a strong indicator that the investigators intend for the device to move into clinical use. We used a cohort of these studies to contact investigators and inquire why their discovery had not been taken forward into clinical use. We also estimated the cost of this translational effort.
Results
Description of studies in the systematic review
Data were generated using a recently completed systematic review. 9 The literature search on animal models yielded 416 studies that met the inclusion criteria. 9 Eight animal species were studied: 308 with rats, 31 with rabbits, 31 with mice, 14 with cats, 17 with dogs, 10 with monkeys, 4 with sheep, and 1 with guinea pigs. Seventeen different nerves were used in these studies, including sciatic, peroneal, tibial, facial, median, radial, ulnar, alveolar, cavernous, hypogastric, and saphenous. The sural, optic, phrenic, recurrent laryngeal, femoral, and lingual nerves were each chosen in one study. More than 70 materials were used in the 416 studies (table 4 of Ref. 9); more than 21 different experimental approaches were used to assess the results of nerve regeneration. 9
Survey results
Of the 416 studies included in the systematic review, 229 were published in the 10-year window between January 1, 1999, and December 31, 2008 (Fig. 1). The electronic address of the corresponding author was identifiable for 195 of the publications. Thirty-two of these studies represented an author–polymer combination that occurred more than once. One survey (Table 1) was sent out for the most recent publication from each unique author–polymer combination. Of 127 surveys sent, the corresponding authors of 12 publications could not be reached because mail was undeliverable or they had nonfunctional e-mail addresses. Forty-five authors responded, for an overall response rate of 38%. Included among the nonresponders were two investigators who had published results on several different polymers and chose not to complete a survey for each publication.
Survey participant selection design.
The majority of the 45 investigators who participated in the survey were trained in laboratory research (80%) and were senior-level researchers at the time (56%) (Table 2). Less than half (47%) had received research funding through a government mechanism. Institutional intramural funding (26%) and not-for-profit foundations (23%) were reported as providing the balance of funding. Only one investigator (2%) reported that the study was funded by a for-profit private or commercial organization.
Among responders, 82% classified the results of their study as positive, 16% as equivocal, and 2% as negative. Slightly more than half of respondents (51%) reported that the material was still under investigation. An interesting anomaly was that 20% reported that the material was clinically available, but this was true for only two materials. We have no explanation for this unless investigators thought that the material was moving forward into clinical trials that have not yet been realized.
When asked to identify all barriers in the translation of their work to bedside, responders most often cited “insufficient financial resources” (18%), a response that was followed closely by “lack of a commercial partner” (16%) (Fig. 2a). The third most frequently cited reason for not translating positive work was that the survey responders' “research program was/is not involved in translation” (12%). Lack of appropriate clinical collaborators (7%) and no access to an appropriate patient population (7%) were also factors. Responders reported that the most important barriers to translation of their work were the lack of a commercial partner (21%) and insufficient financial resources (21%) (Fig. 2b). Only 9% of responders reported that their study results suggested that the material would not be clinically useful.
Participant responses.
Investigators were also given the option of a free-text response to identify any other factors that they believed were important. One author stated that “commercialization relies upon strong industry–academic partnerships, which are difficult to forge and manage.” Another commented that when the researchers approach commercial partners, the companies “seem wary of the regulatory hurdles that must be overcome, and it seems most would rather wait for cell therapies to ‘mature’ before engaging with this kind of technology.” Others cited as barriers the difficulty in obtaining funding for large-animal studies and the lack of commercially available cell sources for human use.
Study cost results
The cost of the studies was calculated using standard charges and salaries at Mayo Clinic in fiscal year 2011. The greatest cost was for paying the personnel who conducted the studies. The cost for a senior research technologist qualified to carry out or assist with animal surgeries, animal care, tissue harvesting, and other experimental assessments was a direct cost of $88,640 per year, including benefits. The NIH postdoctoral stipend scale is used at Mayo Clinic. The annual cost for a midlevel (year 3) postdoctoral fellow, including benefits, is $61,494 (in 2012). It was assumed that 100% of the effort of a fellow and 100% of the effort of a technologist would be engaged for the entire study observation period. In addition, 3 months of 100% of their time and effort was allotted for postmortem tissue harvesting, processing, and analysis. Allotment of time and effort was thereafter based on the actual length of the studies reported. The total cost was $13,462,131 for postdoctoral fellows and $19,404,875 for technologists.
The types and numbers of animals studied and the average study duration are summarized in Table 3. Rats accounted for 77% of the total animals used and were the sole animal used in 308 (74%) of the 416 studies. Mice were the second most common species, followed by rabbits, sheep, monkeys, dogs, and cats in order of use. Guinea pigs were used in only one study, of four animals (Table 3). Total costs for purchasing all animals were $195,808; this expense, together with shipping costs, accounted for 6% of the total cost. Per-diem costs of animal care made up an additional 7% of the estimated total.
In general, larger animals were used for both longer nerve gaps and longer periods of regeneration.
The total estimated direct cost of all studies in the review was $38,604,228. Adding the US federal facilities and administrative (F + A) indirect recovery cost rate used at Mayo Clinic brings the total to $61,264,910. The majority of this total is reflective of the associated labor investment. The combined cost of hired research technicians and postdoctoral fellows working on the studies was $32,867,006 in direct costs or $52,159,938, including indirect recovery, comprising 85% of the total amount. This cost only covers the preclinical animal model studies and does not include any costs for the extensive laboratory-based research into biomaterial synthesis or the design and fabrication of the scaffolds.
Discussion
There have been a number of excellent reviews delineating and discussing the barriers to translation in the field of tissue engineering. Often these are discipline or tissue specific and typically represent the opinions and perspective of leaders in the field.10–14 Orthopedic applications are prominently represented in these publications. We have used a novel approach, where we used a systematic review to contact a broad spectrum of researchers. In the field of epidemiology, this would be described as a population-based study. We defined a cohort of investigators and addressed the whole population with an internet-based survey. We used a specific type of circumscribed technology, scaffolds for peripheral nerve repair. Use of standard systematic review methods enabled us to capture virtually the entire published literature in English in this area. Unpublished research, especially that performed by companies, was not captured and is not accessible because it is in the confidential master files or their equivalent at such regulatory agencies as the Food and Drug Administration (FDA) in the United States. Research published in other languages was not captured by this process. However, since English is the lingua franca of biomedical research, we have to assume that the majority of important work is now published in English. The present study is also focused because it addresses one step in the process of translation, the preclinical animal model. It is assumed that if a study is performed in an animal model, it is because the research team is determining whether the device may be useful for humans. Interestingly, the population-based data generated in this way confirmed the opinions of previous expert reviews, and allowed an estimate of the cost of these barriers. As far as we are aware, this approach has not been used previously in the medical device field.
Using the actual number of animals and the length of the study, we had a realistic way to estimate true costs. However, a lot of assumptions were made in determining the cost of the research. The first was the use of a 2010–2011 cost basis. We believe this is a reasonable way to compress 10–15 years of research into comparable estimates. The use of Mayo Clinic estimates for salaries and animal costs provides a conservative measure. Mayo Clinic uses NIH stipend scales for postdoctoral trainees. Animal prices were from the largest vendors that are used nationally in the United States. Stipends for technologists are standardized across the institution and set to be competitive for comparable employment in the US Midwest. This cost basis is, therefore, regional but is a reasonable representation of the United States. It is most likely an underestimate compared with some European countries and an overestimate compared with some Asian nations.
We made assumptions about the level of trainees and the extent of their involvement, which we believe is reasonable and conservative. Many studies may be carried out by graduate students who cost less, but we have underestimated the real amount of time involved in studies by limiting it to the study period plus 3 months for analysis. Our study also estimated the total cost by not including any estimates for developmental work, pilot studies, or the time of the principal investigator. We included the F + A indirect recovery cost. The rate is used by all federal granting agencies supporting biomedical research in the United States and is an estimate of all costs involved in supporting the infrastructure of a research institution. This support includes everything from associated educational activities and electricity to legal and bioinformatic support. The rate is negotiated individually in a rigorous process between institutions and the US Department of Health and Human Services. The range of rates varies from 30% to 100% of direct costs (personal communication from NIH personnel). The variability is largely dependent on geography. Mayo Clinic at 58.7% is in the middle range for high-intensity biomedical research institutions. Although this result is US-centric, it is probably the closest estimate possible for the actual, total cost of performing research.
The total cost estimate for performing this focused area of research was $61,264,910. Although conservative, the sum represents a considerable investment of resources by public and private granting agencies and institutions. This field of research has resulted in three materials now in clinical use: collagen, polycaprolactone, and polyglycolic acid. The available scaffolds are derived from other biomedical applications, such as tissue fillers and resorbable sutures.
Currently available synthetic or biosynthetic polymer nerve scaffolds for peripheral nerve repair are only useful for repairing short gaps (<3 cm) in small sensory nerves. The medical and surgical communities are in agreement that a device to repair longer gaps in motor and mixed nerves is needed. The design of these animal studies, the majority of which evaluated mixed nerves, such as the sciatic nerve, is clearly directed toward meeting this need.
We wanted to understand the barriers to moving materials forward in the translational process as perceived by the investigators. We took advantage of electronic database searching for the systematic review. We chose a cohort of studies for the 10 years of 1999 through 2008 because electronic contact information for authors would be available, most senior authors would still be active in research, and the research would have had time to advance to the next steps. As far as we are aware from informal contacts within the research community and the FDA, no novel materials for nerve tubes are moving through the formal approval process. We used the Research Electronic Data Capture (REDCap) (http://project-redcap.org) system developed in the NIH—Centers for Translational Science Awards (CTSA) consortium. REDCap has been used in more than 25,750 studies around the world over the past 5 years. The response rate of 39% is typical for survey research of this type. It is possible that this introduces bias because those who respond may have strong opinions about why the research did not progress. We tried to increase the response rate by recontacting authors. We also used PubMed and Google to identify corresponding authors who may have moved to new institutions. This did not substantially increase the response rate.
The results of the questionnaire revealed that 21% of respondents cited “lack of a commercial partner” as a top reason for the absence of translation of their findings and 6% cited that they had the “expectation that a company would translate” their work (Fig. 2b). Together, these responses indicate a disconnect between expectation of responsibility for translation and the reality of the process. This finding begs the questions of whose role it is to ensure that findings are translated and what are the responsibilities of each partner involved in the translational process (i.e., researcher, institution, funding agency, and commercial entity). Given the nature of translational research, we believe there are means of working toward clarification of each individual's and group's role and toward bridging the “valley of death” 15 that is the handoff between researchers and industry.
The attitude of many basic scientists appears to be that their contribution ends with the publication of their findings. This attitude is apparent in our findings, in which 18% of respondents do not know whether their biomaterial is still being researched and 29% do not know whether it is available clinically (Table 2). Therefore, a shift is needed in the paradigm of researchers' thinking that publication is the end of the process. It is essential that they see translation into the clinic as the end point. Investigators with positive results should remain actively engaged in the translation of their findings, regardless of whether their research emphasis changed. Many of the hurdles identified as top barriers to translation, including the lack of access to appropriate patient populations (9%), clinical collaborators (3%), and large animal testing facilities (9%), are problems of nonmonetary resource availability.
A number of potential solutions can address these issues. The process of change should begin with the education of biomedical research students (MD and PhD) on translational methods and processes. New doctoral, masters, and certificate programs in clinical and translational science supported by the NIH-CTSA should fulfill this need if the programs are implemented rigorously. All students should be able to articulate the path to translation for the research in which they are involved. At a more advanced level, the institutions will need to actively participate in educating investigators on the resources available to them to engage in the translational process. Issues that institutions need to address include availability of patent and intellectual property offices, the staff of these offices, and the resources the offices can provide to researchers in the basic and laboratory sciences. In addition, we suggest that each institution involved in translational research develop a procedure for passing findings from investigators to those who can locate clinical or commercial, or both, collaborators so as to streamline the process. Although these mechanisms exist, they are underused.
Institutions should consider incentives that are both appropriate for their culture and will drive investigators toward translation. These incentives will be most effective if an individual's academic success is facilitated by demonstration that he or she is successful in the process of translating discoveries into improved community health. For basic laboratory scientists, this process might involve demonstrating active collaborations with clinicians that inform their research. For clinician investigators and clinicians, involvement in clinical trials and collaborations with basic scientists leading to demonstrable advances in health would be markers of success. Such change would involve a paradigm shift away from using grants and publications as the benchmarks for success.
Government, academic, and commercial entities need to adjust their responsibilities in research and translation, as well as their expectations of one another. Although the NIH and other funding agencies are putting greater weight on the significance of proposed research, no mechanism is in place that requires or persuades a researcher or institution to carry successful or promising research forward in the translational process. A section on the future directions of the proposed research should also be an influential part of the scoring of a grant application. Researchers should be forced to think through the next steps of carrying their research toward the clinic and to describe potential collaborators in clinical practice or industry. This thought process is especially relevant to research specifically intended to develop a product for human use, such as the nerve scaffold research described herein. The statement “this discovery may be of importance in our understanding of disease x” should not be an adequate justification for sponsoring research. The new NIH scoring system that puts weight on the impact of research should evaluate whether a proposal clearly articulates the pathway by which a discovery will impact the field. This evaluation would include the pathway to translation if it involves biomedical research.
Another area for improving the translational process is data sharing and aggregation. A major problem highlighted by our study is the diversity of approaches used to study a common problem. With 70 different materials tested in eight species, 17 nerves, and a wide range of nerve gaps, direct comparison is impossible. Virtually no attempt has been made to standardize animal models used in preclinical testing. The lack of such an attempt could be remedied if funding agencies demanded that for any animal study, a standardized model be used. We believe that it is the responsibility of the biomedical research community to develop such standardized models that can then be used to justify animal use. The NIH is showing leadership in this field, as demonstrated by its new approaches to funding primate research, 16 which was driven by a report from the Institute of Medicine. 17 A similar approach by funding agencies to insist on the use of standardized models as a justification for all animal research would make data sharing practical and useful.
Methods and Materials
Inclusion criteria
Inclusion criteria were (1) in vivo experimental study of peripheral or cranial nerve grafting; (2) animal species used as experimental model; (3) use of a synthetic or biosynthetic nerve conduit, including biodegradable materials, nonbiodegradable materials, and materials processed from biological sources (e.g., collagen); and (4) English language article. We included experimental studies in which additional agents were also used (e.g., systemic administration of drugs, ultrasonographic or electrical impulse), as well as studies in which the implant was filled with a matrix material or a growth factor.
Exclusion criteria included (1) absence of a gap between the proximal and distal stumps of the injured nerve, (2) use of an autologous or heterologous tissue (e.g., vein, artery, muscle, nerve, perineurium) as a material to synthesize the nerve scaffold, and (3) human clinical studies. Because of the paucity of articles available on sheep as an animal model, we included studies that used a scaffold as a sleeve without a gap between the injured stumps.
Study identification
PubMed and Scopus were systematically searched for English language articles (January 1950 to December 2009) by entering the following search terms and Boolean operators: “rat,” “rats,” OR “mouse,” “mice” OR “rabbit,” “rabbits” OR “dog,” “dogs” OR “cat,” “cats” OR “sheep” OR “pig,” “pigs” OR “monkey,” “monkeys” as medical subject heading terms and combining them with the text AND “nerve tube,” “nerve tubes,” OR “nerve conduit,” “nerve conduits” OR “nerve guide,”“nerve guides” OR “nerve scaffold,” “nerve scaffolds.”
Selection of articles
Titles or abstracts were evaluated for study inclusion. When a title or abstract could not be discarded with certainty, the full text of the article was acquired.
Assessment of study quality and data extraction
Each experimental study was independently analyzed by two of the authors to grade the quality of the study design. This selection included the reporting of studies on the following criteria: adequacy of experimental design, quality of outcome measures, and eligibility criteria. For each eligible study, two reviewers extracted all available and relevant data for the experimental groups. These data included demographic and physical information about the animal used (species, weight, and sex); the number of animals included; the injured nerve model; the type of material used; the length of the gap and the nerve scaffold; the characteristics of the experimental groups; and the assessments performed.
Survey development
A nine-question, self-administered e-mail questionnaire was designed with the aid of the Mayo Clinic Survey Research Center using REDCap. The questions were related to potential barriers to the translation of the nerve scaffold studied, the funding source for the study in question, and general demographic characteristics of the researcher responding to the survey. The survey was piloted by contacting eight leaders in the field of biomaterials research and asking for their input on the questions. This input was incorporated into the final survey design.
Corresponding authors from a subset of studies in the systematic review were surveyed if the following criteria were met: (1) the study was published in 1999 through 2008; (2) the full text article was accessible through PubMed; and (3) the e-mail address of the corresponding author was given (Fig. 1). Additionally, the corresponding authors having more than 1 study that met these criteria were sent one survey referencing their most recent study per type of nerve scaffold studied. If an e-mail address was nonfunctional and an alternate e-mail address for the corresponding author was identified, the survey was then sent to the alternate address.
Analysis of survey results
Data were exported from REDCap in .csv format. Quantitative analysis and visualization of survey data were conducted using R software. 18
Study cost
Following the systematic review, animal use numbers in the described studies were totaled and included in subsequent cost analysis. Studies were organized by animal subgroup (i.e., mouse, rat, guinea pig, rabbit, cat, dog, sheep, or monkey). Species-specific ordering prices, crate, and shipping costs were applied individually to each animal subgroup. Monetary values were obtained in consultation with Mayo Clinic Department of Comparative Medicine and Mayo Clinic Animal Care Facility technicians and in collaboration with independent vendors. Per-diem costs for animal care were collected and used in concordance with the average length of study, calculated from the articles listed in the review. The costs of embedding and sectioning were approximated for one sciatic nerve sample ($62) and were used as a representative standard for scaffold surgery costs. The costs of employment of one research technician and one postdoctoral fellow were calculated using Mayo Clinic employment numbers, included for each study, and adjusted for the length of study time.
Absorbable nerve conduits with current FDA and Conformite Europeenne clearance include NeuraGen (type I collagen; Integra NeuroSciences, Plainsboro, NJ), NeuroMatrix and NeuroFlex (type I collagen; Collagen Matrix, Inc, Oakland, NJ), NeuroTube (polyglycolic acid; Synovis Micro Companies Alliance, Inc., Birmingham, AL), and Neurolac (poly(65/35 (85/15 L/D)-lactide-ɛ-caprolactone); Polyganics BV, Groningen, The Netherlands).
Footnotes
Acknowledgments
The authors gratefully acknowledge the following investigators who contributed to their online survey: W.F. Neiss, Bai-Shuan Liu, Ryo Sasaki, Yueh-Sheng Chen, Jin Ho Lee, Tetsuro Tamaki, James Phillips, Susan Hall, Martin Kanje, Liang Zhou, Shimon Rochkind, B.C. Vasconcelos, C. Gay-Escoda, Ryosuke Kakinoki, Xavier Navarro, Stephen W.P. Kemp, Paul H. Wooley, Suleyman Kaplan, Koen Jansen, Jamal A. Mohammad, Gregory H. Borschel, Stefano Geuna, G. Keilhoff, Koji Ryoke, Sung-Tsang Hsieh, Li Haohuan, A.M.B. Martinez, and I. Barakat-Walter.
Funding
This project was supported by a CTSA award from the National Center for Advancing Translational Science. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH. Internal funding was received from the Mayo Foundation for Medical Education and Research.
Disclosure Statement
No competing financial interests exist