Abstract
Detailed documentation of the experimental materials and methods is essential for the validation of scientific papers. Human biospecimens are increasingly utilized as materials in cancer research and information about the biospecimens used is a component of this documentation. We hypothesized that previously reported biospecimen data are inadequate for accurate replication and/or validation of a substantial proportion of studies. To examine this issue, we analyzed biospecimen reporting in a representative cross section of publications over the past 12 years (1998, 2004, 2010) in the journals, Cancer Research (CR, n=46) and Clinical Cancer Research (CCR, n=73). We assessed biospecimen data in relation to the standards outlined as the Tier 1 recommended data elements from the Biospecimen Reporting for Improved Study Quality (BRISQ), in addition to ethics criteria. These data elements encompass features of biospecimens influenced by the patient, medical procedure, and biospecimen acquisition, handling and storage processes. Analysis found that while there was a significant increase in the reporting of ethics board approval status (p<0.008) and name of the ethics board (p<0.0001), there were no significant differences between these journals or over this period in reporting other biospecimen-related data elements. Of the 15 Tier 1 data elements assessed in CR and CCR, the data elements commonly obtained from the “Clinical Chart” (8/15 elements) were significantly better reported than elements that would typically be obtained from the “Biobank” (p<0.0001). Our findings demonstrate that reporting of biospecimen-related data elements has been incomplete. As one part of the solution to this issue, we propose the use of an online data-elements reporting tool (www.biobanking.ca) by biobanks. This BRISQ Report tool aims to help biobanks provide the relevant biospecimen-related data as a structured report, and to promote its inclusion as supplementary material in publications to improve the quality of future research studies.
Introduction
We hypothesized that previously reported biospecimen data in cancer research publications are inadequate for accurate replication and/or validation of studies, and that the adoption of improved reporting standards requires the development and dissemination of an electronic tool to aid biobanks and researchers to compile and report the information.
Our short term goals were to 1) conduct a baseline assessment of the current reporting status of biospecimen information in research publications, and 2) to develop an open access online tool based on the criteria listed in the BRISQ manuscript that could allow biobanks to generate a “BRISQ Report”. Ideally, this report would accompany material (biospecimen and data) released to researchers and become part of the supplementary data accompanying article submission for publication. Our future goal is to assess the impact of such a tool on the adoption of BRISQ reporting recommendations.
Our analysis of biospecimen reporting in two scientific journals over the past decade shows that there are indeed variations and deficiencies in the reporting of elements determined to be important in the BRISQ manuscript. These deficiencies have an impact on the ability to validate studies, and new approaches are needed to improve the standards for reporting of biospecimen-related data elements.
Materials and Methods
Literature review and inclusion criteria
We set out to analyze biospecimen reporting in publications using human biospecimens over the past decade. The selection criteria for these journals were as follows: must be well-established; must be listed on PubMed, the leading biomedical citation database; must have a recent impact factor in the top 25% of oncology journals (>4.5 in 2010); and must have an editorial policy encouraging a wide variety of cancer research areas. The two journals were selected on the basis that together they represent a broad spectrum of basic, Cancer Research (CR), and translational and clinical cancer research, Clinical Cancer Research (CCR).
Using 2010 as a recent year preceding the initiation and publication of the SPREC (2010) and BRISQ (2011) articles, and for which we had full access to all papers in both CR and CCR journals, three publication years at 6-year intervals were selected to cover more than a decade: 1998, 2004, and 2010. For each year, we identified and analyzed all publications using biospecimens within the first issue (January 1 issue). Biospecimens were defined as any human biospecimens or biological material comprised of whole solid tissues, cells isolated from solid tissues, and fluids or blood. 8
Analysis of reported data elements
The accuracy and reporting frequency of 15 elements from the BRISQ criteria that comprised the first tier of elements that were “recommended to be reported” was assessed. The 15 Tier 1 BRISQ data elements were previously categorized by the BRISQ Committee as information that might influence the integrity, quality, and/or molecular composition of biospecimens; they related to five phases based on the lifecycle of a biospecimen. We modified and further defined the Shipping temperature element number 14 as two phases; 14A) from the patient to location of preservation or research use, and 14B) from storage to location of research use. Three additional non-BRISQ elements, which are not directly relevant to biospecimen quality but are relevant to use of biospecimens, related to reporting of research ethics information were also included in our analysis to provide a comparison with other data elements intrinsic to most biospecimen research studies (Table 1). The reporting status for each of these 15 BRISQ and 3 non-BRISQ data elements was determined using the following definitions: 1) Not reported, 2) Unclear, inferred, or assumed, 3) Reported, or 4) Not applicable (Table 2). Specific definitions and guidelines for each data element and each reporting status were developed (Appendix Table A1 with a small pilot set of articles and through group discussion by the following three individuals described. The review and analysis was performed by two trained individuals, one Bachelor of Science graduate with 3 years' experience as a summer student in a tumor biobank and research laboratory, and one Master of Science graduate with 2 years' experience in a research laboratory and 1 year in biobanking (AC and SC). Any disagreements in classification or difficult-to-interpret studies were reviewed and arbitrated by a physician with a specialty degree in anatomic pathology and over 20 years' experience in cancer research and tumor banking (PW).
Adapted from BRISQ Recommended List.6
Not part of Tier 1 BRISQ Recommended List.
Statistical analysis
Statistical analysis of literature review results was conducted using Chi-squared tests for significance. All statistical tests were performed using GraphPad Prism 5 (GraphPad Software Inc., La Jolla, CA).
Development of a BRISQ report tool
We developed an online tool that provides a structured and comprehensive checklist for reporting BRISQ data elements. The BRISQ Report tool was developed using an open source software/web-enabled database, MySQL (Oracle Corporation, Redwood Shores, CA) and is hosted by the University of British Columbia (UBC) Faculty of Medicine IT network. The software stacks that make the core of the system include Apache and PHP in Windows 2008 R2 platform. The BRISQ Report tool is designed to allow users to selectively enter the biospecimen-related data elements recommended in the BRISQ guidelines, with tabs separating the elements by each of the three tiers. Users are not required to fill in all fields but can populate specific fields relevant to their biospecimens. Some fields consist of drop-down lists to facilitate standardized data entry where possible. Upon completion of the form, submission of the data to the database enables generation of a summary of the all data elements entered and an option to download a portable document format (PDF) of the report. The values in the dropdown list (or controlled data) are taken from examples in the BRISQ and/or SPREC articles.
Results
Proportion of articles using biospecimens
In total, 234 articles from both journals were reviewed from the three selected publication years and 51% (119/234) were found to have used human biospecimens. The breakdown of articles using biospecimens was 36% (46/127) in CR and 68% (73/107) in CCR (Table 3).
Reporting of ethics criteria relevant to use of biospecimens
All 119 articles were assessed for 3 ethics-related data elements. There was a significant trend in better reporting of the ethics criteria over the period studied, with increasing frequency of documentation of Research Ethics Board (REB)/Institutional Review Board (IRB) approval status (Fig. 1B, p<0.008) and the name of the REB/IRB (Fig. 1C, p<0.0001), but there was no significant change in documentation of consent (Fig. 1A). Analysis of all three ethics criteria combined showed a significant improvement in reporting frequency over time (Fig. 1D, p<0.0001).

Reporting status of ethics criteria data elements in Cancer Research and Clinical Cancer Research journals from 1998-2010.
Reporting of Tier 1 BRISQ elements relevant to use of biospecimens
The reporting of the 15 Tier 1 BRISQ elements was assessed in both CR (Fig. 2A) and CCR (Fig. 2B) journals. Overall, we found that there was no change across this period in the reporting frequency of the data elements in either journal. The CCR articles had overall higher reporting numbers than CR across most of the criteria examined. For both journals, the three most frequently reported data elements were Biospecimen type (100% and 100%, in CR and CCR, respectively), Disease status (98% and 100%), and Clinical diagnosis (89% and 97%). In CR, the criteria that were reported infrequently (40%–60%) were Clinical characteristics at 41% (19/46), Pathological diagnosis at 48% (22/46), and Type of long-term preservation at 48% (22/46). Elements that were rarely/never reported (<20%) were Storage duration at 15% (7/46), Type of stabilization at 9% (4/46), and Shipping temperatures at 0%. In contrast, CCR had high reporting rates for Clinical characteristics at 84% (61/73), Pathological diagnosis at 67% (49/73), and Type of long-term preservation at 70% (51/73). Similarly, elements rarely/never reported included Storage temperature at 18% (13/73), Type of stabilization at 14% (10/73), and Shipping temperatures at 0%.

Reporting frequency of Tier 1 BRISQ criteria in
Comparison of data elements typically obtained from the “Clinical Chart” versus the “Biobank”
We classified the data elements into those that are typically obtainable from the “Clinical Chart” (I. Preacquisition and II. Acquisition categories) and those typically only obtainable from the biospecimen collection process or the “Biobank” (III. Stabilization/Preservation, IV. Storage/Transport, and V. Quality Assurance). The reporting status of data elements from the Clinical Chart were 80% (757/952) Reported, 8% (72/952) Unclear, inferred, or assumed, and 13% (123/952) Not reported. Biobank-obtained data elements had reporting frequencies of only 39% (328/850) Reported, 1% (12/850) Unclear, inferred, or assumed, and 60% (510/850) Not reported. Using these two distinctions for comparison, it was found that data elements commonly obtained from the Clinical Chart were reported at a significantly higher rate than those elements obtained from the Biobank (p<0.0001) (Table 4).
Data elements typically obtained from the Clinical Chart are reported at a higher rate than those from the Biobank, p<0.0001.
Discussion
Over the past 2–3 decades, there has been a rapid expansion of research using human biospecimens such that these are now essential materials in all levels of research from discovery to clinical validation. Replication and validation are essential steps in all research; one important example is the specific process of translating basic research findings in cancer research into prognostic and predictive biomarkers that can impact cancer care. Recognition of the need for reporting standards around patient cohorts used for biomarker studies that could facilitate replication of results and minimize cohort selection bias, stimulated the development of the REMARK criteria and the recent publication of several biospecimen quality-focused articles.9–12 Subsequently, appreciation of the potential influence of additional biospecimen variables on creating bias in biospecimen-based research raised the issue of developing complementary additional reporting standards focused on clinical and biobanking variables relevant to biospecimens. In light of this, the BRISQ list of recommended elements was generated through the deliberations of an expert committee that included members from the National Cancer Institute Office of Biorepositories and Biospecimen Research (OBBR), the International Society for Biological and Environmental Repositories (ISBER), and the REMARK group. As detailed in the original BRISQ article and listed on the EQUATOR site, 7 potentially relevant criteria were identified and separated into three tiers, with Tier 1 elements recommended to report, Tier 2 considered beneficial to report, and Tier 3 considered as other potentially relevant elements. In this study, focused only on reporting of Tier 1 elements, we have found that reporting of biospecimen-related data elements used in cancer research studies is frequently incomplete and has not improved over a period of more than a decade, despite a greater than 2-fold increase in use of biospecimens in this period. 8 Of the 15 Tier 1 BRISQ data elements considered to be essential for validation of research findings involving biospecimens, 6 we found a significant deficiency in reporting of those elements that would typically be obtained from the biobank or biobank processes.
The impact of failure to report data clearly related to some of the specific BRISQ elements can be debated. The relevance of some elements may vary with the focus of the research; some that are based on stable biological factors such as DNA mutation may be less important to document in replicating findings than those derived from RNA expression profiles. For example, we have noted that the Tier 2 BRISQ element Collection time can differentially influence expression of some breast tumor biomarkers, and that different biobanks can have very different distributions of collection times across their stock. 2 Therefore, both collection time for individual biospecimens and overall biobank of origin for cohorts can be factors to consider in assessing and validating studies, but only in some circumstances. Definition of what constitutes a “lack of clarity” in reporting detail may also relate to the expertise of the reviewer, but it should be noted that the authors here encompass a range of experience from biobank technician to graduate student to established investigator—all with knowledge of biospecimens. The feasibility of detailed documentation of some factors also needs to be taken into consideration.
Our study supports previous findings of poor reporting of other recommended elements relevant to tumor biomarker studies,13,14 and confirms that replication of experiments in a significant proportion of articles in leading cancer research journals may be hindered by incomplete reporting of data relevant to biospecimens: the underlying issues are multifactorial. Refinement of which elements are most important to report based on experimental data is important to validate or modify the initial BRISQ criteria, which are largely based on empirical existing standards, which provide a necessary starting point. Awareness of these elements and reporting standards should be communicated widely among researchers, journal reviewers, and editors. Review and dissemination of protocols to improve BRISQ criteria should reduce the effort involved and facilitate and promote the adoption of better standards. Finally, although it is anticipated that the impact of improved reporting will result in greater success in the translation and validation of research in the future, processes to measure and validate this assumption will also be required.
One approach to promote the delineation of the BRISQ elements by researchers is for biobanks that support research users to also provide a detailed summary of the relevant biospecimen data. The BRISQ Report tool was created to facilitate reporting quality by enabling biobanks to provide a structured and standardized report of biospecimen data based on all tiers of the BRISQ recommended list of elements. The first release of this tool is openly accessible for any biobank, and available online at www.biobanking.ca (Fig. 3). In this first iteration, users must enter all data on each occasion to generate the report. Plans for the second release of the tool include additional features such as user account and login capability authenticated by username/password, individual data storage for each biobank user, and user-defined templates allowing reduction of data entry time for similar multiple data releases. We anticipate that biobank users will value the ability to create a PDF BRISQ data element report that can then be provided to research users as an additional ‘product’ (along with typical products such as biospecimens, extracted nucleic acids, patient and clinical data reports, etc.). Once the research study has been completed and submitted for publication, this BRISQ Report can be included with other supplementary data as an appendix. This may facilitate review by the journal by clearly delineating biospecimen factors and, perhaps, assist in successful publication.

The BRISQ Report tool. Screenshot of part of the first page of an online tool to allow users to enter biospecimen-related information based on the BRISQ recommended list and generate a summary report. Placement of the cursor into the data field box displays the definition of the data element.
In conclusion, our literature review and analysis demonstrates that reporting of Tier 1 BRISQ recommended elements obtained from biobanks has been poor. To overcome this, we have created a simple tool for biobanks to report biospecimen-related data that will offer a complete and informative representation of the biospecimens used in research. Better reporting of biospecimen information allows researchers to have a clearer representation of studies conducted, improve reproducibility of results, and ultimately increase the overall quality of biospecimen-based research.
Footnotes
Acknowledgment
The authors would like to acknowledge the BRISQ committee for their contribution to defining important biospecimen variables for reporting which was the basis for development of the BRISQ Report tool.
Author Disclosure Statement
The authors have no conflicts of interest of financial ties to disclose.
| Reporting Status | |||||
|---|---|---|---|---|---|
| Data Element | Not Reported | Reported | Unclear, Inferred, or Assumed | Not Applicable | |
| 1. | Biospecimen type | No mention of biospecimen type used | Biospecimen type clearly stated | ||
| 2. | Anatomical site | No mention of anatomical site biospecimen taken from | Clearly states where the biospecimen was taken from. Must be specific enough that another scientist could accurately replicate the project |
Information indirectly inferred/assumption made/unclear |
|
| 3. | Disease status of patients | No mention of whether patient has disease or is a healthy control | Disease status of patient clearly stated |
||
| 4. | Clinical characteristics of patients | No clinical characteristics of patients mentioned |
Clinical characteristics of patients are mentioned in methods, results or supplementary data | ||
| 5. | Vital state of patients | No mention of whether patient was alive or deceased at time of biospecimen acquisition | Vital state of patient clearly stated or can be clearly assumed from collection method |
Information indirectly inferred/assumption made/unclear |
|
| 6. | Clinical diagnosis of patients | No mention of clinical diagnosis of patient, and a reasonable assumption cannot be made with the information provided | Clinical diagnosis is clearly stated with supporting information. May be obtained from the paper itself or supplementary data |
||
| 7. | Pathology diagnosis | No mention of pathology diagnosis or histological review | Paper clearly states biospecimens were reviewed by a pathologist, (e.g., Dr.____), was histologically analyzed, or reviewed and graded according to WHO guidelines. | Information indirectly inferred/assumption made/unclear |
|
| 8. | Collection mechanism | No information provided regarding biospecimen collection mechanism. | Method of biospecimen collection is clearly stated. | Information indirectly inferred/assumption made/unclear |
|
| 9. | Mechanism of stabilization | No information on type of stabilization. |
The type of stabilization between the time biospecimen taken from patient prior to preservation or use and location in which stabilization occurred at point of harvesting is stated. Must state that stabilization occurs at point of harvesting (e.g., in the OR or frozen section room). | ||
| 10. | Type of long-term preservation (Format of biospecimen) | No information on format of biospecimen for long-term preservation | Article states whether biospecimen is frozen, FFPE or fresh. | Information indirectly inferred/assumption made/unclear | No long-term preservation was required, biospecimen used immediately in research assay. |
| 11. | Constitution of preservative | No information on what the biospecimen is preserved in |
Article states what the biospecimen is preserved in. | Biospecimen used immediately in research assay. |
|
| 12. | Storage temperature | No information on temperature provided | Storage temperature of biospecimens is clearly stated |
Information indirectly inferred/assumption made/unclear |
Biospecimen used immediately in research assay. |
| 13. | Storage duration | No information on when biospecimen was collected, or storage duration | Article provides information on storage duration or period of time when biospecimens were obtained | Biospecimen used immediately in research assay. |
|
| 14. | Shipping Temperature | ||||
| A Phase 1 | No information on shipping temperature | Shipping temperature clearly stated |
|||
| B Phase 2 | No information on shipping temperature | Shipping temperature clearly stated |
Biospecimen used immediately in research assay. | ||
| 15. | Composition assessment and Selection | Reasons for choosing specific biospecimens in experiments is not stated | Selection criteria or composition assessment of biospecimens is stated |
||
| 16. | Documentation of consent | No statement made regarding patient consent | Patient consent to participate is stated or waiver of consent obtained from REB/IRB | ||
| 17. | Documentation of REB approval | Article does not state REB/IRB approval | Paper clearly states REB/IRB approval |
||
| 18. | Name of REB/IRB | No name of REB/IRB institution stated | Paper clearly states name of REB/IRB board | Infer REB/IRB board used |
|
For any of the data elements:
• if article references a previous article by the same group with all appropriate information, status will be “Reported.”
• if article references a previous article by the same group with inaccurate or incomplete information, status will be “Unclear, inferred or assumed.”
• if article references a previous article by a different group, then status will be “Not reported.”
