Translating Stem Cell Research from the Bench to the Clinic: A Need for Better Quality Data

T he Irish mathematical physicist Lord Kelvin (1824–1907), is famously quoted as saying, “when you can measure what you are speaking about and express it in numbers you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind: it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science.” Surely, cell biology is a science, but it has yet to reach a level of knowledge on par with physics and chemistry. Biological systems must follow the laws of physics, but the large number of molecular events and interactions, the significant role of stochastic fluctuations, and the emergent properties that arise from this complexity, conspire to make it particularly challenging to discover the rules that allow prediction of biological phenomena.

The required achievements for viable cell therapies include, among others, specified cell replacement, appropriate development, and localization. These formidable goals would ideally be achieved through detailed knowledge of the biochemical events that are directly related to a given manipulation. In fact, such knowledge is incomplete, so progress is necessarily guided by experience and improved results rather than a predetermined, optimized production. Even more challenging is the predictability of undesired outcomes through events that are indirectly related to the initial manipulation. Achieving that depth of knowledge and understanding is too large a task for individual research laboratories. Clearly, there is the need to share, reproduce, and build on the data of others.

The scientific method demands that the same materials manipulated with the same protocol produce the same result. Researchers talking about their protocols state that, “it works for us” and “the results are reproducible in our lab.” However, it is important to precisely and numerically delineate what works mean, and define thoroughly, through the use of sufficiently controlled experiments, the conditions under which the observations are reproducible in any laboratory. One necessity is the reporting of more experimental and procedural details than are currently commonly provided; and another is more reporting of control experiments. Would discovery be more successful and efficient if more and better measurements were being reported? Or would the redundancy of extensive control experiments slow the process down? A recent commentary [1] indicated that only a very small percentage of landmark studies in cancer research can be reproduced in pharmaceutical laboratories that are interested in developing new treatments. Yet, many of these articles are highly cited, and influence the performance of, or avoidance of, related research in other laboratories. Observations such as these suggest that progress would be more efficient and cost-effective if more emphasis was placed on rigorous performance and reporting of experiments. The resulting increased reproducibility of results, robustness of reagents and procedures, and verifiable evidence of accuracy with appropriate controls and analytical procedures would surely minimize bias and uncertainty.

Reproducible, accurate, and robust measurements will allow the field to reap greater benefits from our national investment in research in the biomedical sciences. Perhaps, much of the data that are submitted for publication are based on measurements that are reproducible, accurate, and robust; but rarely are these supporting data reported. Data submitted for publication represent the culmination of a series of experiments, some of which might have worked the first time they were performed, but many of which will have required extensive optimization [2]. To the extent that such data sets exist they are of value, but are currently languishing in individual laboratories. These unpublished data sets likely include control data from pilot experiments that tested reagents, for example, as well as replications of the experiment. Within the current publication model, there is clearly no space to include these auxiliary data sets with the final publication. However, as we move increasingly to an online publication model, there is no reason why these data could not form part of the supplementary materials, and perhaps even be part of a searchable database. We must evolve our enterprise to promote reporting of metrics and experimental details associated with complex laboratory activities such as cell culture and derivation of cell lines. These details are not independent of, but rather enhance, the hypothesis-driven experiment, and elevate our understanding and characterization of stem cell research to a more rigorous level. We similarly need to evolve our statistical analysis armetorium so as to be able to harvest useful information and make relevant interpretation of the huge data sets that could be made available.

Currently, the biomedical research enterprise is focused on discovery. Potentially, groundbreaking research reports in top tier journals have their Methods section relegated to supplementary information, or, at best, the end of the research article. Articles that report methods development and validation are usually published in more specialized journals. However, the how by which a laboratory process was carried out is as important as the result. Unless sufficient control experiments are performed and understood, the findings are susceptible to misinterpretation.

With the sharing of sufficient high-quality data, laboratories can communicate accurately and completely the parameters and conditions that are critical to the biological phenomena being observed. High quality in this context does not mean headline-grabbing, but refers to carefully, rigorously achieved, and precisely reported properly controlled data sets. Such controlled data are unarguably more valuable than impactful, but irreproducible studies. The path from bench to clinic will be accelerated not by short cuts, but through thorough experimentation and responsible reporting.