Research Perspective on Utilizing and Valuing Tumor Biobanks

Abstract

Tumor biobanks have become critical components of the cancer research infrastructure. Consideration of how to place appropriate values on tumor biobanks is important for all stakeholders. At the level of individual biobanks, value is important in determining how to contribute to, utilize, and fund biobanks. At the level of the research system, value is important in determining how to evaluate, rationalize, and sustain or modify the investments in this infrastructure. This review considers approaches and indicators for evaluation of a biobank with a particular focus on utilization, as one important indicator of value, from the perspective of the researcher and funder. The patterns of utilization and the influence of different phases and approaches of research, and types of biobank are described, as well as strategies for biobanks to increase utilization and therefore their value to research.

Introduction

More personalized decisions, and more accurate therapies, guided by more informative biomarkers, these are the goals we are striving to realize with precision medicine.¹ To get there we need to continue to invest in a health research enterprise that is vibrant enough to come up with great ideas and to be able to test them in robustly designed studies, to generate deeper and more reproducible knowledge. However, health research also needs other ingredients beyond ideas, such as tools, technologies, and increasingly biobanks of human biospecimens and associated data. While development of biobanks and their collections has accelerated over the past few decades, there has been insufficient consideration given to the valuation of biobanks and building organizational frameworks to evaluate, rationalize, and sustain or modify biobank investments.

Value is a relative property and may be perceived, projected, measured, or a combination of all three. When assessed, value is based on one or more qualitative and quantitative metrics that are usually selected on the basis of importance to the stakeholder. There are many value metrics that can be applied to a biobank, but some are more relevant than others depending not only on the perspective of the stakeholder but also on the type, goals, and maturity of the biobank. This article focuses on one measure of value: utilization rates.

In this review, we will define biobanks as types of research infrastructure and discuss the valuation of biobanks. Given the wide scope of the topic, we will narrow the main focus here to consider the value of human tumor biobanks from the perspective of the academic research user and funder, and in the context of a broad definition of biobanks (as cancer research infrastructures), some important qualifiers (phase of research and type of biobank), and one key metric (patterns of utilization). The scheme proposed provides a framework to guide the evaluation of biobanks by listing and categorizing the array of indicators of value and impact that could be considered. Given the very limited literature on the valuation of biobanks, we have provided references where possible, with the remainder of the discussion representing the collective opinion of the authors.

Defining Types of Tumor Biobanks

In some contexts, it is useful to restrict the term biobank or biorepository to a defined facility with professional standards.² However, in the context of valuing tumor biobanks from the perspective of different stakeholders, the definition needs to be broadened to include any collection of human biospecimens and annotating data arising from the process of biobanking in support of cancer research.

This broad definition therefore includes all collections that are associated with research projects, studies, clinical trials, or formal infrastructure projects. This is because first, the purpose of all these forms of collection is essentially the same, to provide input fuel for cancer research. Second, the process of “biobanking” through which these collections arise is very similar and typically involves all or most of the same steps (such as enrollment, consent, collection, processing, annotation, storage, and release from short- or long-term storage for specific research). Third, these different forms of collection are frequently indistinguishable from the perspectives of the funder and researcher. Many funders draw little distinction between different types of biobanks in terms of requirements for performance.

Similarly, most researchers are almost completely agnostic to the type of biobank they access. Within the ethical and legal scope, researchers obtain, use, and publish data derived from biospecimens independent of the type of biobank. For the most part, reviewers do not appreciate any collection differences and journals do not apply any requirements for the nature or quality of the biospecimen source.

Biobanks and Cancer Research Infrastructure

Tumor biobanks are just one of many types of cancer research infrastructure. Research infrastructures have been defined as facilities, resources, and services used by the science community to conduct research and foster innovation.^3,4 Research infrastructures include: a single item or suite of major scientific equipment, e-infrastructures such as data and computing systems, and resources such as collections, archives, or scientific databases. Research infrastructures can be single-sited (a single resource at a single location), distributed (a network of distributed resources), or virtual (the resource is created by electronic linkage).

In common with other types of research infrastructures, human tumor biobanks primarily exist to support health research. This means that the overall investment in biobanks and the scope of biobanks in terms of numbers and size should be proportionate to the scale of research, and the design must fit the needs of current and projected future research demand. The anticipated life cycle of biobanks might be expected to follow that of typical research infrastructures, with an initial implementation phase, an operation and maturation phase, and then an evolutionary phase that involves upgrading, reorientation, or termination. For a biobank, upgrading might mean adding a new collection, reorientation refers to shifting the operational emphasis (see section “Factors that could increase utilization of existing biobanks”), and termination means complete consumption/exhaustion of the collection or a legacy event such as loss of funding or leadership.⁵

Biobanks are also distinct from many forms of research infrastructures in several ways. First, the activity of biobanking that leads to the creation of biobanks spans many arenas and groups, including society, patients, clinical medicine, health research, law, and ethics. Second, the main product of biobanks is not purchasable and is not owned by the infrastructure, as human tissue laws and/or ethics guidelines in Canada and Australia and in most countries state that human biospecimens may not be “sold.”^6,7 Finally, the inherent value of the main product, a tumor biospecimen, generally (1) relates to the newness and prevalence of the treatment strategy, (2) increases with time, (3) achieves maximum value only after many years once outcomes data mature (3–20 years depending on the tumor type and treatment), and then (4) declines to a lower but tangible level.

Phases of Cancer Research

Investigators, at least good ones, tackle important problems and identify a broad or specific hypothesis to address it. They then develop a strategy to test the hypothesis. This often involves pursuing a series of linked questions that progress through several phases toward a conclusion or solution. In reality, research progression typically occurs through multiple, gradual, branching, or iterative steps and is steered by supportive and/or contrary findings. Accordingly, the needs for types of biospecimens vary depending on the phase and nature of the research question, along what is called a research pipeline.⁸

To illustrate this, we consider a research pipeline to be a highly simplified process comprising three progressive research phases (Table 1). In the initial “discovery” phases of biomarker-focused cancer research, the focus is on the cellular level. The demand for sets of biospecimens is based on tissue level criteria that are selected on the basis of a range of biospecimen level requirements, such as representative pathology in the tissue in a variety of preservation formats, and with a range of biospecimen compositions. In the subsequent “translation” and “validation” phases, the research focus moves to disease and then patient levels. In the translation phase, this involves exploration of biological relevance in more specifically defined cohorts and then establishing potential clinical relevance in preclinical studies. Then, approval is obtained to move into phased and randomized clinical studies and trials in the validation phase. This involves gaining licensing approval, showing clinical impact, and achieving clinical adoption.

Table 1.

Emphasis of Biospecimen Requirements by Phase of Research

	Discovery	Translation	Validation
Level	Cell	Disease	Patient
Scale	Case (small numbers)	Cohort (intermediate numbers)	Population (large numbers)
Focus	Exploration of cellular localization and mechanisms of action	Determining the clinical relevance of a mechanism or biomarker	Validation of the clinical impact of measurement of a mechanism or biomarker
Cohort selection	Tissue-level criteria (e.g., >75% invasive tumor cells)	Disease-level criteria (e.g., invasive lobular-type breast cancers with >10 years median follow-up data)	Patient-level criteria (e.g., cases representative of multiple hospital sites in a health region)
Biospecimen requirements	Range of specifications (e.g., larger samples, range of tissue composition, stages of disease)	More restrictive (e.g., typical clinical presentation format)	Relatively restricted (e.g., single sample type, typical clinical preservation format)
Data requirements	Biospecimen-level data (e.g., collection, composition, and pathology data; emphasis on BRISQ criteria⁷)	Disease-level data (e.g., diagnosis, retrospective treatment, and outcomes data; emphasis on REMARK criteria⁸)	Patient-level data (e.g., defined patient population, standardized clinical diagnosis, specific treatment, and prospective outcomes data; emphasis on CONSORT criteria⁹)

BRISQ, Biospecimen Reporting for Improved Study Quality; REMARK, REporting recommendations for tumour MARKer prognostic studies; CONSORT, Consolidated Standards of Reporting Trials.

The priorities for biospecimens and for data used in the research change, as well as the recommended criteria for reporting data associated with biospecimens, cohorts, and populations studied to generate the results described in publications.^9–11

For an individual biobank, it is important to recognize which research phase(s) the biobank is designed to support, as the scale of cohorts and selection and quality criteria change with research phases. This can influence the biobank design and the utilization of the biobank, as well as researchers' perspectives on biobank value.

As one general example, discovery phase researchers previously placed a high value on biobanks with frozen format biospecimens. However, advances in technology have meant that the value of frozen biospecimens is diminishing, whereas the value of biobanks that can facilitate access to formalin fixed and paraffin embedded blocks in pathology archives is growing.¹² Pathology archives themselves are becoming the most highly valued form of biobank for discovery phase research as well as other phases. As another example, a current theme in translational phase research is to explore blood plasma factors, such as circulating tumor DNA.¹³ This is a driver of value for biobanks that can specifically provide cases with primary tumor specimens linked to preoperative and postoperative plasma samples.

The Type of Tumor Biobank

There are many types and designs of biospecimen collections, and there remains no universally accepted classification scheme to adequately communicate the diversity and therefore specific value to researchers.^14,15 However, some specialized classifications and terminologies have been developed for related purposes. For example, we have previously developed and implemented a classification scheme to serve as the basis for applying standardized requirements for different types of biobank in the CTRNet certification program.^16–18 This particular scheme categorizes biobanks into three types (mono-, oligo-, and poly-user biobanks) based on scale, leadership, and intended use. But classification to help distinguish different types of biobank for consideration of value by individual researchers, who are trying to find the right biospecimens and cohorts for their research study, is perhaps best based on the type of design for accrual.

In general, a research question that needs biospecimens to address it is posed to address unknowns or problems. However, several factors such as the maturity of the line of investigation and data already generated, the amount and duration of funding secured to pursue it, and the need for initial selection of biospecimens on the basis of treatment and outcomes data determine if the question can be feasibly addressed by a research design that involves obtaining biospecimens from an existing collection or from a prospective collection.¹⁹

In other words, research questions involving biospecimens can be considered as either “retrospective” or “prospective” in nature, and biobank design can be categorized by this important feature. As an example, where access to a retrospective biobank is needed, a researcher may want to examine the outcome of a biomarker in a subset of breast cancers (e.g., estrogen receptor-positive breast cancers) treated by both hormone therapy and a chemotherapy regimen. It is not feasible to prospectively collect cases because of the very long time periods and large cohort size required to obtain relevant outcomes data.

As another example, a researcher may apply to access biospecimens from a prospective clinical trial biobank. In such cases, the researcher will usually be asked to first provide supporting evidence from data generated from more accessible and less valuable retrospective collections. Alternatively, as an example where a prospective biobank is needed, a researcher may wish to study a biomarker related to response to a new treatment where previously collected biospecimens do not exist. As another example, the biomarker under study may require a new and specialized collection protocol not typically employed in prior collections.

There are also research questions that might be addressed either retrospectively or prospectively. For example, a researcher may wish to study a relatively rare disease or an uncommon subset of a common disease. A single local biobank is unlikely to be able to prospectively collect sufficient cases in a reasonable period of time. However, another biobank might be located that specializes in this rare disease and can supply a retrospective cohort. Similarly, a functional network of biobanks might be able to either compile a reasonable sized retrospective cohort from their existing stocks or commit to prospectively collect such a cohort in a short period based on the large enrollment capacity across multiple sites. Issues relating to the selection criteria needed to be able to identify a rare condition in time to collect biospecimens for research, and the length of outcomes data available may influence the choice of best approach.

Patterns of Utilization

A researcher with a new question requiring analysis of human biospecimens has three main choices to secure biospecimens: create, comb, or contract. In addition to funder and grant reviewer stipulations, there are advantages and disadvantages to each choice that are influenced by the particular phase within the spectrum of the research pipeline and the options for different accrual designs of biobanks, as well as a set of factors intrinsic to the researcher. These three choices will be discussed in turn:

Creation of a new biobank collection: This is the most common approach used by researchers when a prospective study design is needed and is feasible in the context of the funding and timelines. But this approach is also often undertaken by researchers even when a retrospective design requiring access to outcomes data is envisaged. The desire to create the collection independently and “direct” or “own” (in the legal sense of having full title over) the final collection is a strong motivation for researchers who are trained to be innovative, original, and to compete. However, when as is sometimes the case, the researcher has no prior experience in biobanking, this desire often leads to an underestimation of the challenges in creating and then maintaining the collection, in terms of time, effort, expertise, and funding.

This approach also encourages publication of data based on what can just be feasibly collected from a local site and in the funding timeframe. As a result, this approach tends to promote creation of cohorts of “convenience” with local clinical practice biases rather than cohorts of sufficient scale, scope, and design to minimize local population biases and allow robust statistical analyses. It is often the nature of funding mechanisms in health research that expenditure on biobanking is a line item that is not subject to any specific or detailed justification. Therefore, this option continues to be most prevalent, despite the fact that in many instances other collection options might be better from a scientific perspective. This leads to the continued duplication of collections and increased challenges for the research system to sustain these collections.

Combing or finding an existing biobank collection: This is the next most common approach, especially when a retrospective study design is optimal (such as when it is necessary to complete the study in the shortest time). This choice is usually pursued by one of three approaches in the following order of prevalence: (1) combing the literature for relevant cohorts used in published studies and establishing a collaboration, (2) combing the collective wisdom of local colleagues for “biobanker” contacts, or (3) combing the Internet using a biobank search engine or biobank network locator or locator service. While biobank networks have put much effort into improving the pathway to the latter approach, stimulating the inclination to find and improving the ease of querying locators remain significant challenges.

However, many biobanks, especially mono-user and oligo-user type research biobanks, are not part of any network or represented on any locators because of the original intent of such collections. The absence of a generally understood classification scheme for biobanks that can be used by locators and the description of relevant quality features of biospecimens at a sufficient level of detail for a researcher are also barriers to identifying quality biospecimens. Wider adoption of standardized reporting of biospecimen cohort variables, such as the Biospecimen Reporting for Improved Study Quality recommendations⁹ and biobank source,²⁰ in scientific articles and by biobanks advertising cohorts within their collections, and by the locators on which they are represented, would also improve the efficiency of combing approaches.

Contracting with an existing biobank: In this approach, a biobank is asked to conduct part or all aspects of the biobanking needed to compile a new custom collection. This approach is perhaps least frequently used by researchers. However, this can be an efficient approach from many angles, striking a good balance between the needs of the individual researcher and the costs to the research system and therefore maximizing investments in biobank infrastructures and specialized biobanking expertise. The Cooperative Human Tissue Network model in the United States is perhaps the largest and most compelling example of the effectiveness of this prospective collection model and overall approach.²¹ However, many biobanks that were initially established to create collections to offer in support of retrospective studies could benefit from setting targets for their retrospective stocks and redirecting their expertise and infrastructure to promoting this type of service.

A list of the different considerations underlying the choices between these different approaches that a researcher can take to secure biospecimens is summarized in Table 2. Clearly, the choice for securing biospecimens for studies is dependent on research circumstances, and this also influences the utilization of a collection. One hundred percent utilization of a new biobank or cohort can be achieved if a researcher decides and is able to create or contract with an existing biobank for a specific study purpose and conducts analysis of every case collected. However, utilization rates can also be reduced in this scenario due to unforeseen circumstances, such as loss of funding/resources or unexpected adverse events in a clinical trial.

Table 2.

A List of Considerations from the Researcher and Study Perspective Regarding Different Approaches to Secure Biospecimens

Considerations for the researcher

Time to obtain

Effort to obtain

Cost to obtain

Considerations for the research study

Ability to ensure relevant quality

Ability to apply multiple selection criteria

Pace of research

Ability to link to adequate outcomes data

Level of evidence provided by research data

Combing for biobank collections also enhances the utilization of existing biobanks. If a researcher can find an appropriate cohort for their study, establish the fit with the inclusion criteria for the new research question and ascertain the level of quality from known features, and negotiate access, then the utilization of cases (involving at least some aliquots of biospecimens and precompiled data from selected cases) will occur. The rate of utilization can again be as high as 100% when the cohort located is a mono- or oligo-user biobank. However, it is often the case that the only accessible or feasible source for some research studies is a poly-user biobank.

By definition and design, most poly-user biobanks are set up with a plan to create a large enough collection that it can provide easy access to specific subsets of cases with known quality and selected by multiple selection criteria, including biospecimen, pathology, clinical, and outcomes data features. Nonetheless, the biobank often cannot know which specimens will be most useful in the future since this is dependent on availability of outcomes data, future research technologies and/or directions, and funding priorities. As such, the biobank may often expand its target for biospecimens above and beyond those required for immediate researcher needs. The difficulty in predicting future demand means that biospecimen utilization rates will always remain <100% and often <50% by design. Otherwise, the biobank may not be fulfilling its plan and goal of adequately supporting retrospective research and the ability to select cases by new criteria.

As described above, biobank utilization rates expressed as use of a proportion of the overall biobank collection thus vary according to the intended primary use and the type of subsequent use. The plan for use should be integral to the research or business plan, and utilization rates should be considered only in relation to the scientific justification in the plan. If the plan for primary intended use is to consume the entire biospecimen for each case, then utilization rates should be close to 100%. If there is no primary intended use or this initially consumes only a portion of the biospecimen for each case and the collection is stored for future use, then utilization rates should not only be considered for the initial primary use but also separately in relation to the stored bank of biospecimens.

Utilization should also be considered in relation to other features, such as the scale and nature of the collection/storage cost per case, the proportion of all cases utilized over time, and the proportion of cases ever utilized relative to the proportion never used.

As one example, a biobank may be based on a cohort that is created for a primary defined purpose and has 100% initial utilization and is then maintained for another 10 years during which a subset of 10% of cases are selected and all used for five subsequent studies. This might be assessed as a highly successful biobank if based on 100% utilization of all cases at some point in time and several subsequent uses of just a subset of cases. However, the continued storage and maintenance of 90% of the cases that are never used after the initial primary use suggest low overall value of maintaining the entire collection. Another biobank may be based on no plan for primary use but is created and maintained for 10 years to allow annotation with outcomes data. The utilization rate might be assessed as 0% until 10 years. Then in the following year, five unrelated studies may have selected and partially used mostly different sets of cases involving an overall 10% of cases. This might be assessed as less successful than the first biobank based on only 0% utilization at year 10 or 10% overall utilization at year 11. However, it might also be argued that utilization rates are not a relevant metric while the biobank matures given the plan to compile cases with 10 years outcomes data. The fact that 90% of its cases are unused after 11 years may not be an important metric given that the biobank has been able to support many researchers to select very specific new cohorts on many occasions, which just happen to involve 10% of the collection.

Therefore, as illustrated by the nuances and complexity of the examples above, it should be acknowledged that there are limitations on focusing too much on utilization as a measure of value because many factors such as biobank type, plan, and maturity need to be considered. Nevertheless, at some point in time, the need to store all unused cases in both biobanks should be reviewed and culling should be considered.

Factors That Could Increase Utilization of Existing Biobanks

At the level of individual biobanks, utilization rates for individual biobanks vary for many reasons. Rates cannot be “judged” without detailed knowledge of several factors, such as the biobank plan, design, and stage of maturity. Nonetheless, many biobanks that were originally designed to support retrospective studies are underutilized. This is due to the emergence of donor outcomes over time on a case level, and emerging trends in research and funding priorities on a systemic level. Where relevant, diverting biospecimen researchers from creating to combing for existing biospecimen stocks, or to contracting to collect new biospecimens when justified, would allow existing biobanks to leverage past investments or the opportunity costs of utilizing their platforms to more efficiently create prospective collections for others.

The issue of underutilization of biobanks has been discussed in the literature²² and in international forums such as the Marble Arch Working Group for Biobanking²³ and the International Society for Biological and Environmental Biorepositories.²⁴ As discussed briefly here and in other articles in this Journal, the topic is complex but important. Clearly, there are some simple and immediate operational steps and some more difficult and longer range adjustments that an existing individual biobank can take to improve utilization of its stocks or the investment in its biobanking capacity. Simple steps include changing an over-restrictive or over-protective access policy, increasing promotion of the biobank's collection and services, joining a biobank locator service(s), and establishing better linkages to other biobanks within networks to contribute to filling their user requests. There are also several more strategic adjustments that can be considered:

Increased use of biobank and biospecimen locators: It is clear that at the level of the entire research biobank system there is significant duplication and inefficiency within regions and across countries. This is of particular importance in cases where disease incidence is the rate-limiting step for biospecimen accrual, and researchers must seek more than one source of biospecimens to enable statistically relevant cohorts. This can be due to the lack of biobank standardization as well as ethical, legal, and IT challenges to connecting (1) users with biobanks, (2) biobanks across networks, and (3) biospecimens within distributed virtual biobanks into accessible cohorts. Important efforts and significant advances have been made to advance harmonization in the area of biospecimen governance.²⁵ Similarly, early prototypes of biobank locators are now being improved by linkage to registration strategies to ensure that all collections are entered and descriptive fields adequately completed. These efforts are leading to significantly improved user-driven search capabilities at the level of biobanks and biospecimens.^26–28

While it will be important to gather data to show how widely accessed and effective locators are for researchers, individual biobanks should and could easily expand their visibility and potential accessibility to researchers and biospecimen brokers by contributing to such locators.

External QA indicators: An obvious step for a biobank to convey value is to enroll in an external quality assurance scheme that can provide potential users with assurance as to levels of quality. However, the value of designations such as CTRNet certified, CAP accredited, or ISO accredited^16,29–31 are not yet widely understood or appreciated by researchers, journals, or funders. Better communication and promotion of these values by the biobank and funding communities, and especially indications of value of the different levels of external QA, will be important.

Establishing internal stock targets: For biobanks intended to support retrospective studies, utilization rates have often declined over time as stock levels increase beyond the need of local researchers. This occurs because many biobanks do not set a target for their collection. Notwithstanding the difficulty in predicting future researcher requirements, careful consideration of the targets for stocks and revision of these targets linked to culling over time will ensure that such biobanks achieve a better balance between stocks and demand, and this will be reflected in stable or increased utilization rates.³²

Linkage to other biobanks in a functional network can enable further modification and reduction in unnecessarily large stocks within individual biobanks. To be able to provide a reasonable sized cohort of cases to a researcher where the cohort needs to be selected using several criteria, a single biobank will often collect and store 5–10 times more cases in stock than it expects to select. But if the same reasonable sized cohort could be selected and provided with contributions from multiple biobanks across a network, each biobank would be able to reduce its overall stock target intended to support specific areas of research. The freed resources could then be redirected to increase targets of other stock components or invested in achieving higher quality outputs. For this to work, the biobank network must be cohesive and standardized.

Shift of operational emphasis to incorporate prospective contract collection services: As suggested above, utilization rates within the entire portfolio of an individual biobank can also be enhanced when a target stock size is reached. This shifts the balance of operational focus to just maintain this target stock level while increasing the focus on offering contract-based custom collection services to support more prospective research projects and studies. This approach allows a biobank to maintain its enrollment capability, which takes time and effort to develop and establish in collaboration with clinical services. This enrollment capability also makes it more attractive for researchers to contract with biobanks for services rather than collect their own biospecimens.

Shift of operational emphasis to create stocks with more complex qualities: To maintain value, tumor biobanks need to evolve with the changing needs of research users. At the cutting edge of cancer research, users are pursuing important current questions around tumor evolution and tumor heterogeneity and the characteristics of metastatic tumors.^33,34 This discrepancy between the quality characteristics of current stocks and research needs is a growing contributor to underutilization.

There are many aspects to biospecimen quality and different ways to enhance this quality beyond just those aspects integral to a single biospecimen. Quality features that pertain to the inherent quality of the biospecimen include detailed composition, or the data about the associated individual, the manner in which the specimen was banked, and the immediate clinical data and consent status linked to the biospecimen.

Another set of quality features that are more extrinsic to the biospecimen and more closely related to the host are also important. We have previously described these as “complex” qualities.³⁵ An example of a complex extrinsic quality is a biospecimen that is linked to collections of other biospecimens from the same host and/or other clinical events over time. Complex qualities often serve to further enhance the selectable criteria of individual biospecimens within cohorts used for research studies. However, in general, tumor biobanks contain an oversupply of relatively simple quality cases (e.g., primary tumor sample and initial diagnostic data) with some treatment and outcomes data, and a paucity of more complex cases.

Beyond the level of individual biobanks and individual research users, funders and journals as stakeholders in the research system also have important roles to play in improving biobank utilization rates and therefore improving both systemic efficiency and quality. Encouraging utilization of existing collections provides benefits for biobanks and also reduces resource expenditure for researchers, through avoiding the development of new collections. It also provides funders with potentially greater outputs for their investment. Funders should require strong scientific justifications for any use of research funds to create new biobank collections and any restrictions on access policies, and the registration of all new and continuing collections to improve future accessibility.

Consideration should also be given to incentives to increase the use of these registered collections (e.g., Ref.³⁶). Journals should require more details on the sources of biospecimens used to generate data in articles and evidence for the quality level required for the work described. This would increase pressure on biobanks to directly enroll in external quality assurance programs and participate in biobank locators, and on researchers to utilize established biobank infrastructures.

Valuing Investment in Tumor Biobanks

Rationalizing investment in biobanks, where the criteria for evaluation of biobanks are poorly defined, is almost as complex a topic as investment and valuation of research itself.

In Canada, for example, the Canadian Cancer Research Alliance of >30 cancer research funders³⁷ has estimated that, in 2011, the total funding for all aspects of cancer research amounted to CAN$577M including operating funds for research grants (54%), equipment and infrastructure (32%), and career and trainee salary awards (14%). Formal funding for national and provincial tumor banks and networks within the operating and equipment budget was ∼$10M (1.8%). Since then, total funding had fallen 17% by 2015, and most of this decline was attributable to a 40% reduction of spending on infrastructure.³⁸ But it has been estimated that between 15% and 25% of the operating funding, which has remained relatively unchanged, also involved some component of biobanking in the context of research studies and trials. Certainly, the appetite for biospecimens reflected in the data in research publications is much larger than the formal funding level.³⁹

However, while immediate evaluation of the overall investment in research occurs through the main currency of research, namely articles and grants awarded, and through additional well-established processes including reports, and peer-review, the same is not necessarily true of research infrastructure investments. This may be partly due to infrastructure being considered a secondary contributor and as only a source of materials to research or the low-cost barrier to entry for small biobanks.

However, while there are requirements for infrastructures such as animal tumor xenograft models and animal facilities to be itemized in research budgets, adhere to national standards, and to both register and carefully justify animal cohort numbers, no similar requirements exist for biospecimens and biobanks. This leads to duplication and wasted investment reflected in underutilization as well as uncertain reproducibility of data generated using biospecimens.

Approaches to Valuing Tumor Biobanks

Ultimately, investment in cancer research infrastructure and its value are linked to perceived research impact. In Canada, the largest research infrastructure funding agency is the Canadian Foundation for Innovation (CFI).⁴⁰ Although the numbers of previously funded projects that specifically involve biobanks are low, it is instructive to consider how such a dedicated infrastructure funder approaches evaluation of other forms of infrastructure and the associated impact of funding.

The CFI uses a variety of approaches that are integrated within an outcomes measurement study methodology that covers the planning process and influences on organizational strategy, research capacity building, highly qualified personnel development, research productivity, and extrinsic benefits reflected in broad societal impacts.⁴¹

In the specific field of biobanking, Hofman et al.⁴² have previously published a well-considered scheme for evaluating the performance and contribution of hospital integrated tumor biobanks. This scheme considers and assigns scores in four overall categories—quality, activity, productivity, and visibility—and includes sets of indicators within each of these categories.

A comparison of these two approaches identifies not only substantial overlap but also some differences. These differences could be explained by the desire to place a stronger emphasis on the category of research/scientific productivity by experts in the field of biobanking. In comparison, a funder of research infrastructures may place additional emphasis on aspects of strategy, capacity, and leverage created around the infrastructure. This is reflected in additional indicators such as the external effects of the business plan on the local environment and organization, indicators of new highly trained personnel created and recruitment/retention of researchers attributable to the investment within the organization, and indicators of complementary investments.

We present a modified combined scheme of biobank outcomes measures in Table 3. This is based on the CFI categories described above but with expanded indicators and metrics, including utilization. Many of these added or more granular indicators fall under the research productivity category and were mostly introduced by drawing from the Hofman scheme. In addition, some new quality indicators are specifically proposed to promote aspects of evaluation that place high value on strategies to improve biobank utilization.

Table 3.

Indicators of Value and Impact of Tumor Biobanks

Categories of outcomes	Value indicators	CFI metric	Hofman metric	New metric	Examples of features and specific metrics relevant to each indicator
Strategic research planning	Strategic plan or business plan and process	x	a		Quality of original scheduled review and updates, revisions over time, financial plan including stock targets and cost recovery approach
	Internal effects of the plan	x			Consistency between plan and actual operations; delineation of focus on supporting retrospective vs. prospective research
	External effects of plan	x			Changes in related or linked infrastructures resulting from the plan
	Complementary investments	x			Investments (human and financial) and commitment from other partners (e.g., institution, department, or other research group leaders)
Research capacity	Investment value of infrastructure	x			Total investments over time (funds, in-kind, etc.)
	Capabilities (technical and operational)	x	v		Combination of expertise, experience, equipment, and enrollment capabilities
	Recruitment and retention of researchers	x			Recruitment/retention of researchers that arises as a result of the biobank
	Linkages/visiting researchers	x			Visiting researchers/biobankers/others, purpose, and extent
	Linkage of infrastructure	x			Linkage to other similar biobanks that increases actual or potential research utilization, representation on locators
	Multidisciplinarity	x			Capacity to involve multiple research and health areas
HQP	Number/type of trainees	x			Types include undergraduate/postgraduate trainees
	Nature of training—type and relevance	x	v		Types of training include formal training, informal observation, group lectures/workshops, etc.
	Knowledge transfer through HQP	x			Current status of trainees in relation to biobanking roles
Research productivity	Competitiveness	x			Access policy, access conditions, uniqueness of retrospective collection or prospective collection capability, enhanced value through return of data/materials
	Sharing of infrastructure	x			Provision of access to infrastructure components (e.g., sharing of storage space, equipment, database, servers, storage with related or unrelated entities)
	Research productivity generated by supported users	x	p		Knowledge output relevant to research field supported by users of biobank (e.g., talks, seminars, workshops, abstracts, reviews, articles, and status/impact factors of each output)
	Research productivity generated by biobank	x	p		Knowledge output relevant to biobanking (e.g., talks, seminars, workshops, abstracts, reviews, articles)
	Retrospective research support		a		Products for retrospective study support (stock numbers, cases, biospecimens, aliquots)
	Prospective research support		a		Products for prospective study support (case numbers, types, enrollment)
	Performance capability (for users)		a		Customer response/turn-around time (times from application to decision, initiation, completion of study support), return users, profiles of users
	Performance capability (for funders)		a		Utilization rates (over time, and by study, samples, aliquots, and by retrospective- and prospective-type study support), culling rates
	External quality metrics		v		Participation in external quality assurance programs (proficiency, certification, accreditation programs)
	Internal quality metrics for operations		q		Enrollments (rate, consent yes/no performance), biospecimens (size, quality, storage monitoring), data (entry time, query capability)
	Internal quality metrics for staff				Staff training and education plan, schedule, records
	Internal quality metrics for products (simple qualities)		q		Enrollments (nature of consent), biospecimens (composition, extract QC indices), data (completeness, validation from multiple sources, extent of data)
	Internal quality metrics for products (complex qualities)			x	Enrollments (ability to recontact), biospecimens (extent of selectability and linkage by format, multiple locations, multiple timepoints, host interventions), data (from multiple timepoints) cohorts (degree of representation of population)
	Indirect research outputs		p		Numbers of applications, collaborations, grants, and/or funding secured, which arises as a result of the biobank
	Indirect infrastructure outputs		p		Numbers and types of operational tools, databases, IT data linkages that arise as a result of the biobank
Innovation/extrinsic benefits	Leverage of expenditures	x			Success of parent or related organization in the relevant research field
	Stakeholder involvement			x	Commitment of governance structure, participant, and community engagement in governance, local, and external user feedback on customer experience
	Type and amount of knowledge and technology exchange	x	v		Interactions with similar infrastructures, participation and role in relevant societies/organizations
	Key innovations (economic, social, organizational)	x			Number, nature, and importance of changes in health care that can be linked to better outcomes
	Evolving industrial clusters	x	v		Partnerships with industry

“Categories of outcomes” are all derived from the CFI schema. “Value indicators” include all relevant proposed indicators. Those indicators reflected in the CFI scheme and/or in the Hofman scheme or that are new are identified in the “CFI metric” column (“x”), the “Hofman metric” column (further delineated as associated with “q” quality, “a” activity, “p” productivity, and “v” visibility in the Hofman scheme), and the “New metric” column.

CFI, Canadian Foundation for Innovation; HQP, highly qualified personnel.

Conclusion

Valuing biobanks involves consideration of many factors and outcomes measures in the context of the type of biobank. To maximize biobank utilization, all collections of human biospecimens and annotating data for cancer research require careful up-front planning to determine the phase(s) of research that they will support. A retrospective, prospective, or combination design for biospecimen accrual should form part of the biobank's business plan, with all these different approaches essential to span the full scope and needs of cancer biospecimen research.

Researchers can access biospecimens for their studies through three described mechanisms: creating, combing, or contracting. Of these, combing or contracting by researchers should be promoted to increase utilization of existing biospecimens or biobank infrastructures, respectively, in turn benefiting the entire research system by leveraging past investments.

Although biospecimen utilization rates cannot be assessed as a simple metric at an individual biobank level, many existing biobanks could benefit from taking a number of short-term and strategic approaches to increase their utilization rates. The model proposed here provides a broad framework to evaluate the value and impact of biobanks.

Footnotes

Acknowledgments

P.H.W. and L.M. gratefully acknowledge support for this work by the Biobanking and Biospecimen Research Program at BC Cancer (supported by the Provincial Health Services Authority), the Canadian Tissue Repository Network (funded by grants from the Institute of Cancer Research, Canadian Institutes of Health Research and the Terry Fox Research Institute, and from the Canadian Cancer Research Alliance), and the Office of Biobank Education and Research, University of British Columbia (supported by the Department of Pathology and Laboratory Medicine, University of British Columbia). PhD studies of A.R. are supported by a NSW Health PhD scholarship and a NSW Health Accelerator grant. J.A.B. gratefully acknowledges grant support from the Cancer Institute NSW Biobanking Stakeholder Network.

Author Disclosure Statement

No conflicting financial interests exist.

References

Mordente

, Meucci

, Martorana

, Silvestrini

Cancer biomarkers discovery and validation: State of the art, problems and future perspectives. In: Scatena

(ed). Advances in Cancer Biomarkers: From Biochemistry to Clinic for a Critical Revision. Dordrecht, Netherlands: Springer; 2015: 9–26.

Hewitt

, Watson

. Defining biobank. Biopreserv Biobank, 2013; 11:309–315.

European Commission: Research Infrastructures. Available at http://ec.europa.eu/research/infrastructures/index_en.cfm?pg=what Accessed August 22, 2014.

European Strategy Forum on Research Infrastructures. Available at www.esfri.eu Accessed August 7, 2018.

Matzke

LAM

, Fombonne

, Watson

, Moore

. Fundamental considerations for biobank legacy planning. Biopreserv Biobank, 2016; 14:99–106.

British Columbia Human Tissue Gift Act. Available at www.bclaws.ca/civix/document/id/complete/statreg/96211_01#section10 Accessed October 3, 2013.

New South Wales Human Tissue Act. Available at www.legislation.nsw.gov.au/inforce/dae79cae-8043-47df-9c37-fe8d1d7e38b3/1983-164.pdf Accessed October 3, 2013.

Ioannidis

JPA

, Bossuyt

PMM

. Waste, leaks, and failures in the biomarker pipeline. Clin Chem, 2017; 63:963–972.

Moore

, Kelly

, Jewell

, et al. Biospecimen reporting for improved study quality. Biopreserv Biobank, 2011; 9:57–70.

10.

Altman

, McShane

, Sauerbrei

, Taube

. Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK): Explanation and elaboration. PLoS Med, 2012; 9:e1001216.

11.

Moher

, Schulz

, Altman

, CONSORT Group (Consolidated Standards of Reporting Trials). The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA, 2001; 285:1987–1991.

12.

Gaffney

, Riegman

, Grizzle

, Watson

. Factors that drive the increasing use of FFPE tissue in basic and translational cancer research. Biotech Histochem, 2018; 93:373–386.

13.

Wan

JCM

, Massie

, Garcia-Corbacho

, et al. Liquid biopsies come of age: Towards implementation of circulating tumour DNA. Nat Rev Cancer, 2017; 17:223–238.

14.

Watson

, Barnes

. A proposed schema for classifying human research biobanks. Biopreserv Biobank, 2011; 9:327–333.

15.

Watson

PH.

Biobank classification: Communicating biorepository diversity. Biopreserv Biobank, 2014; 12:163–164.

16.

Matzke

EAM

, O'Donoghue

, Barnes

, et al. Certification for biobanks: The program developed by the Canadian Tumour Repository Network (CTRNet). Biopreserv Biobank, 2012; 10:426–432.

17.

Rush

, Christiansen

, Farrell

, et al. Biobank classification in an Australian setting. Biopreserv Biobank, 2015; 13:212–218.

18.

Ling

, Rush

, Carter

, et al. An Australian Biobank Certification Scheme: A study of economic costs to participating biobanks. Biopreserv Biobank, 2018; 16:53–58.

19.

Atherton

, Sexton

, Otali

, Bell

, Grizzle

. Factors affecting the use of human tissues in biomedical research: Implications in the design and operation of a biorepository. In: Grützmann

, Pilarsky

(eds). Cancer Gene Profiling: Methods and Protocols. New York, NY: Springer; 2016: 1–38.

20.

Howard

, Mascalzoni

, Mabile

, Houeland

, Rial-Sebbag

, Cambon-Thomsen

. How to responsibly acknowledge research work in the era of big data and biobanks: Ethical aspects of the Bioresource Research Impact Factor (BRIF). J Community Genet, 2018; 9:169–176.

21.

CHTN—Cooperative Human Tissue Network. Available at www.chtn.nci.nih.gov Accessed August 2, 2013.

22.

Scudellari

Biobank managers bemoan underuse of collected samples. Nat Med, 2013; 19:253.

23.

Marble Arch Working Group for Biobanking for medical research. Available at www.marblearchgroup.org Accessed August 7, 2018.

24.

ISBER. International Society for Biological and Environmental Repositories. Available at www.isber.org Accessed November 7, 2013.

25.

Tassé

A-M.

A comparative analysis of the legal and bioethical frameworks governing the secondary use of data for research purposes. Biopreserv Biobank, 2016; 14:207–216.

26.

Biobanking and Biomolecular Resources Research Infrastructure. Available at http://bbmri-eric.eu Accessed August 22, 2014.

27.

NCI Specimen Resource Locator. Available at www.specimens.cancer.gov Accessed August 2, 2013.

28.

The Biobank Resource Centre Home. Available at www.biobanking.org/brc Accessed August 2, 2013.

29.

McCall

, Branton

, Blanc

, et al. The College of American Pathologists Biorepository Accreditation Program: Results from the first 5 years. Biopreserv Biobank, 2018; 16:16–22.

30.

Furuta

, Allocca

, Schacter

, Bledsoe

, Ramirez

. Standardization and innovation in paving a path to a better future: An update of activities in ISO/TC276/WG2 biobanks and bioresources. Biopreserv Biobank, 2018; 16:23–27.

31.

Barnes

, Shea

, Watson

. The Canadian Tissue Repository Network Biobank Certification and the College of American Pathologists Biorepository Accreditation Programs: Two strategies for knowledge dissemination in biobanking. Biopreserv Biobank, 2017; 15:9–16.

32.

Meredith

, Slotty

, Matzke

, Babinszky

, Watson

. A model to estimate frozen tissue collection targets in biobanks to support cancer research. Biopreserv Biobank, 2015; 13:356–362.

33.

Jamal-Hanjani

, Quezada

, Larkin

, Swanton

. Translational implications of tumor heterogeneity. Clin Cancer Res Off J Am Assoc Cancer Res, 2015; 21:1258–1266.

34.

Shah

, Morin

, Khattra

, et al. Mutational evolution in a lobular breast tumour profiled at single nucleotide resolution. Nature, 2009; 461:809–813.

35.

Watson

PH.

Biospecimen complexity-the next challenge for cancer research biobanks?. Clin Cancer Res Off J Am Assoc Cancer Res, 2017; 23:894–898.

36.

Office of Health and Medical Research. Available at www.health.nsw.gov.au/ohmr/Pages/biospecimen-collection-grants.aspx Accessed August 8, 2018.

37.

Canadian Cancer Research Alliance. Available at www.ccra-acrc.ca Accessed August 22, 2014.

38.

Canadian Cancer Research Alliance; Cancer Research Investment in Canada. Available at www.ccra-acrc.ca/index.php/publications-en/investment-reports-annual Accessed August 8, 2018.

39.

Castillo-Pelayo

, Babinszky

, LeBlanc

, Watson

. The importance of biobanking in cancer research. Biopreserv Biobank, 2015; 13:172–177.

40.

Canada Foundation for Innovation. Available at www.innovation.ca Accessed August 8, 2018.

41.

Tremblay

, Zohar

, Bravo

, Potsepp

, Barker

. The Canada Foundation for Innovation's outcome mesurement study: A pioneering approach to research evaluation. Res Eval, 2010; 19:333–345.

42.

Hofman

, Ilie

, Long

, et al. Measuring the contribution of tumor biobanks to research in oncology: Surrogate indicators and bibliographic output. Biopreserv Biobank, 2013; 11:235–244.