The College of American Pathologists Biorepository Accreditation Program: Results from the First 5 Years

Introduction

Quality biospecimens are suitable for their downstream purpose, whether clinical or research. For testing, biospecimens must be sufficiently intact to represent the in situ biological state of the chosen analyte. Thus, biospecimens require quality systems for procurement, processing, storage, curation, and distribution. In pathology (the medical area that focuses on the medical diagnosis of disease through the analysis of tissue, cell, and body fluid samples), quality systems must meet federal standards described in the Clinical Laboratory Improvement Amendments (CLIA).^1,2 Compliance with CLIA standards must be verified by an external entity for a U.S.-based pathology practice to bill the federal government for diagnostic services. Areas of quality important for clinical diagnostic laboratories include staff training/competency, up-to-date standard operating procedures, maintenance of equipment, chemical and laboratory safety, information system services of data integrity, preservation, and security.

All the areas outlined, including the use of standard operating procedures for biospecimen handling before testing in the clinical realm are aimed at decreasing preanalytical variance. This in turn helps maximize the accuracy and precision of the diagnostic test. Clearly, minimizing preanalytical variance is also critical to translational research, since new diagnostic assays are derived from this groundwork. The need for standardization in specimen preanalytical conditions and the overall alignment with the College of American Pathologists' (CAP) mission were the primary reasons for establishment of the CAP Biorepository Accreditation Program (BAP).

The CAP is the primary organization for board-certified pathologists in the United States, and each year, 89% of newly board-certified pathologists become CAP fellows. The United States government (through the Centers for Medicare and Medicaid Services [CMS]) has given the CAP deemed status for regulatory inspections; accreditation of a clinical laboratory by the CAP is sufficient for certification under CLIA. ¹ Currently, 40% of pathology laboratories in the United States (including the majority of academic medical center laboratories) utilize CAP accreditation to fulfill CLIA requirements. After leading the laboratory accreditation process for more than 50 years, CAP now accredits over 8000 laboratories in 50 countries. ³ CAP also offers a service to help laboratories earn accreditation under ISO 15189 for Medical Laboratories. ⁴

Therefore, while many pathology practices and diagnostic laboratories are familiar with the CAP Laboratory Accreditation Program (LAP), the CAP BAP is relatively new. The CAP recognized the need for pathology partnership in biorepository quality in 2010 and began compiling best practices through 2011 and early 2012. Guidance was obtained from the International Society for Biological and Environmental Repositories (ISBER) Best Practices, ⁵ the National Cancer Institute's Best Practices for Biorepositories, ⁶ the Organisation for Economic Cooperation and Development, ⁷ the CLIA regulations, ² and the CAP's own 50-year history of laboratory accreditation standards.

The CAP initially created 275 checklist requirements for biorepositories, including 98 assigned to the “Laboratory General” checklist and 177 assigned to the more specific “Biorepository” checklist. For comparison, the 2017 edition of the CAP LAP-specific checklist for “Anatomic Pathology” contains 187 items; this laboratory is also subject to the “Laboratory General” checklist. CAP also provides a “Standards for Accreditation” document outlining high-level requirements for leadership and oversight of the laboratory or biorepository.

Like CAP LAP, CAP BAP requirements include specific personnel qualifications, training, and competency assessment, quality management program integration within the larger institutional administrative system, and general physical space, cleanliness, and safety policies. The biorepository-specific checklist contains standards specific for the protection of research participants (e.g., consent requirements), specimen collection and handling requirements, including biospecimen chain of custody, institutional policies (e.g., materials transfer agreements, governance documents), and long-term preservation of biospecimens (e.g., equipment preventive maintenance, temperature monitoring, and backup systems requirements).

CAP accreditation programs are different from some other certification/accreditation programs in that a peer-based inspection process is utilized. Indeed, a requirement of program enrollment in both LAP and BAP is the commitment to perform reciprocal inspections on behalf of the CAP. For BAP, the on-site inspection team leader is generally the pathologist, PhD, or manager/director of their respective institutional biorepository, and all team inspectors must have current experience in an active biorepository and successfully complete the CAP inspector training program. Volunteer inspectors new to the program are always paired with experienced CAP staff inspectors for their first few inspections to insure consistency of the process. The total inspection team size and duration of inspection are dependent on the size and complexity of the biorepository being inspected.

Accreditation Process

The accreditation process begins with enrollment in CAP BAP, which requires an application, an annual enrollment fee, and submission of the biorepository's activity menu. Examples of activities include informed consent, specimen handling, extraction of nucleic acids, cryostorage, information technology, histology, and sample distribution activities. Technical specialists from the CAP conduct a “readiness assessment conference” with the applicant biorepository. Individualized feedback is provided with either recommendations for further development before initial inspection or selection of the on-site inspection date (Fig. 1).

OPEN IN VIEWER

FIG. 1.

The three-year cycle of the CAP Biorepository Accreditation Program. CAP, College of American Pathologists.

Inspection teams are selected from peer institutions based on overlapping areas of biorepository scope and activity. Inspection team members undergo formal training by CAP in the use of the R.O.A.D. technique (Read, Observe, Ask, Discover) for assessing checklist item compliance. The R.O.A.D. technique was established by the CAP for its LAP in 2010 with the goal of improving consistency in the inspection process. ⁸ During the day of inspection, selected repository documents are read/reviewed, biorepository practices are observed, questions are asked, and discoveries are documented regarding compliance to specific standards.

Checklists containing standards are applied during the inspection process. The standards require evidence of compliance, but are not proscriptive in how the standard is achieved. For example, the BAP.08300 standard for freezer preventative maintenance states that “there is a procedure for freezer preventative maintenance.” The standard makes suggestions, but does not prescribe a specific procedure. Rather, the biorepository is free to outline the appropriate maintenance program for their freezer fleet. This could consist of employee-performed cleaning and maintenance as well as professional service visits. The repository must be able to produce records of employee training on their specific program and maintain results of any professional service calls. ⁹ Standards are assigned as phase I or phase II with the understanding that the severity of a phase II deficiency is greater than a phase I deficiency.

After the initial on-site inspection, results are reported to the CAP and the biorepository is given time to respond in writing to any noted deficiencies. A CAP technical specialist reviews the documentation and refers the matter to the CAP regional commissioner who makes decisions on accreditation. On the 1- and 2-year anniversaries of the on-site inspection, self-inspections are conducted by the biorepository with the submission of results to CAP followed by a teleconference. At the third anniversary of accreditation, the cycle begins again with application/activity menu submission and on-site inspection.

Biorepositories whose activity menus expand while under accreditation must report this to the CAP and may be subject to requests for additional documentation or an out-of-cycle on-site inspection. Similarly, biorepositories whose on-site or self-inspection discloses significant deficiencies may be subject to additional investigation and/or on-site inspections.

Results

A pilot set of biorepositories with diverse activities and backgrounds were inspected the first year after the launch of the program in 2012, and results of the inspections were closely monitored by the BAP Committee and the CAP Board of Governors. Successful completion of this pilot phase led to formal opening of the program to all United States biorepositories in 2013.

Currently, 53 biorepositories are fully CAP BAP accredited and 13 are in preparatory or review phases with the goal of attaining accreditation. Most of the biorepositories in the program are hospital/medical center-based; however, there are commercial/private and governmental biorepositories represented as well (Fig. 2).

OPEN IN VIEWER

FIG. 2.

Demographics of biorepositories enrolled in the CAP Biorepository Accreditation Program.

A total of 90 inspections were completed between the launch of the program in May 2012 and December 2016. Of these, 61 were initial inspections and 29 were re-inspections. Reinspections either represented biorepositories entering their second 3-year accreditation cycle or represented a second visit inspection to follow-up on identified areas of concern. In total, 527 deficiencies were identified in the areas of Equipment/Instrumentation (22%), Information Technology (18%), Specimen Handling and QC (15%), Quality Management (16%), Personnel (11%), Safety (10%), Facilities (6%), and Regulatory (2%) (Fig. 3).

OPEN IN VIEWER

FIG. 3.

Breakdown of cited deficiency type by year of inspection.

Discussion

The CAP BAP Committee conducts face-to-face meetings thrice annually in conjunction with the CAP Laboratory Accreditation Committees. During these gatherings, the BAP Committee further refines the checklist requirements and standards of accreditation as part of its continuous improvement activities. While the checklists are updated annually, a specific biorepository anticipating on-site or self-inspection is provided with the version of the checklists that will be applied to them months in advance. This ensures that biorepositories have appropriate lead time to incorporate new or changing requirements into their practices.

The majority of deficiencies observed over the first 5 years cover areas in which standards are straightforward (equipment/instrumentation 22%, IT systems 18%, specimen handling 15%, safety 10%, facilities 6%, and regulatory 2%). However, 27% of deficiencies were identified in the areas of quality management (16%) and personnel (11%). These areas reference the more complex themes that are included as part of the accreditation process. For example, specific requirements under the heading of quality management include having key performance indicators for performed activities, documenting and tracking quality and protocol deviations, and demonstrating sufficient initial training and ongoing competency assessments for staff.

When broken down by years (grouping 2012–2013 together to equalize inspection totals), higher numbers of “quality management” deficiencies are cited in the 2012–2013 group compared with 2014 and 2015. Higher numbers of deficiencies in “quality management” were again noted in 2016 (Fig. 3). This may reflect the result of a second-cycle focus on quality monitoring and continuous quality improvement.

Biorepositories undergoing initial inspection are expected to be functioning under standard operating procedures and policies sufficient to address all checklist items. Furthermore, they are expected to have written and begun to implement an overarching quality management plan. Early in their first 3-year cycle, it is understood that biorepositories may not have had the time to cycle through their complete quality management plan (which can include alternating attention among different key performance indicators and gathering data through proficiency testing programs or other means). By the time of the second on-site inspection (year 4), the biorepository is expected to demonstrate evidence of their quality management plan “in-action,” including responses to acceptable or unacceptable levels of performance. Another possible explanation for the higher number of quality management deficiencies seen in 2016 is the addition of new biorepositories to the program.

In general, the CAP has a greater level of concern for a biorepository that is cited for the same deficiency during sequential inspections. Over the 5-year period, a total of 23 recurring deficiencies (defined as a deficiency cited in consecutive inspections at the same biorepository) were identified. These 23 events encompassed 17 unique checklist items in the areas of: (1) Equipment, Maintenance and Function Check documentation; (2) Personnel and Competency Assessment Record updates; and (3) Quality Assurance Audit documentation (Table 1). Affected biorepositories received additional attention from CAP BAP support staff, who provided verbal guidance and printed resources. Biorepositories with recurring deficiencies were also more likely to receive an out-of-cycle on-site inspection (for the specific purpose of assessing progress in the area(s) of concern).

Overall, however, the rate of successful accreditation among enrolling biorepositories is high. Only one biorepository was denied accreditation after multiple on-site inspections secondary to repeated failure to demonstrate compliance to key BAP requirements. Four biorepositories voluntarily dropped out of the program after one unsuccessful on-site inspection despite having the opportunity for improvement and reinspection. Of the 57 biorepositories who have successfully entered the program during the first 5 years, the rate of attrition is also low; only four previously CAP-accredited biorepositories have voluntarily elected to terminate their involvement in the program. Of these four, one biorepository is known to no longer be in operation. Additional exit interview data was not available for this study.

In keeping with the educational focus of CAP BAP, members of the CAP BAP committee and other program affiliates have developed many resources for biorepositories during the past 5 years. An introductory guide to biorepositories for community pathologists has been created. ¹⁰ A summary of external quality assurance options for biorepositories was published. ¹¹ A CAP-published book entitled Developing and Organizing an Institutional Biospecimen Repository was released in 2014 with revisions for a second edition presently occurring. ¹² Joint efforts between the CAP BAP Committee and the CAP Clinical Informatics Committee has generated a list of preanalytical variables that should ideally be captured in the biorepository information management system. ¹³ Vignettes of current activities and financial structures in biorepositories have been published. ¹⁴

Importantly, the CAP BAP “Biorepository” checklist, created in conjunction with many best practices documents, is now crossreferenced in the latest version of the NCI Best Practices published in 2016. ¹⁵ CAP resources have been utilized to perform surveys relevant to the biorepository community. ¹⁶ Importantly, CAP BAP Committee members and others have presented their efforts at scientific meetings and published them in the pathology and biorepository literature. An analysis of the CAP BAP as a major method of knowledge dissemination in biobanking has recently been published. ¹⁷

The unique strength of a peer-reviewed accreditation process is the bidirectional communication between inspection teams and the biorepositories, which is critical to educational success. Best practices can be directly taught and/or observed during the inspection process, and this can occur at the team leader/biorepository director level or at the level of the inspection team member and the biorepository staff member. On-site inspectors may elect to interview supporters and end users of the biorepository's services, which can produce additional relevant recommendations. Summation conferences conducted by the inspection team leader can be attended by institutional sponsors and stakeholders. This process can serve as external validation of both excellent practices and areas requiring additional institutional support.

Several biorepository program participants have provided qualitative feedback about the peer review process to BAP leadership through the Biorepository Accreditation Post-Inspection Critique process. Feedback examples include: “The Inspection Team was very knowledgeable and offered good recommendations throughout the inspection process.” “Our inspection team was thorough and professional. The experience and tone were educational and positive.” “Being a new lab, I found this process very enlightening—difficult to wade through at times—but there was always help available.”

Examples of qualitative negative feedback have included vagueness of larger checklist requirements, including the requirement for an overarching quality management plan and program costs. CAP BAP leadership has made additional educational material and example quality management plans available to enrollees. Concerns regarding costs could represent two separate issues. First, there is a cost associated with CAP BAP participation on an annual basis. This cost is set by CAP leadership to cover infrastructure for program development (e.g., in-person meetings and teleconferences), CAP administrative overhead, and travel expenses for the inspection teams. Second, there are costs associated with bringing the biorepository into compliance with the checklist standards.

As previously noted, CAP BAP checklist items do not specify exact methodology—any solution that satisfies the requirement is acceptable. This flexibility allows biorepositories to utilize either less expensive manual or more expensive automated or computerized solutions. For example, the document control requirements for standard operating procedures could be met by using paper records and manual sign-offs or by using an electronic document control system. On the bench, the requirement to assess quantity and quality of extracted nucleic acids could be met by using manual spectrophotometry/gel electrophoresis or by using a fully automated commercial analyzer. Meeting the requirement for pathologist-performed histological assessment of tissue specimens can require significant investment if this practice has not previously been performed. However, this critical standard supported by biobanking best practice documents,^5,15 is necessary for patients' molecular diagnostics assays performed in CAP-accredited diagnostic laboratories, ¹⁸ and was required by The Cancer Genome Atlas project. ¹⁹ A biorepository desiring CAP accreditation could mitigate this cost by asking a pathologist to assess tissue specimens just before their distribution, rather than assess every specimen in the existing inventory. With additional time and growth of the CAP BAP, deeper examination of formal customer feedback and further examination of total program costs should occur.

Expansion and acceptance of the CAP BAP by the United States biorepository community has the potential to help create a standard of quality that enhances competitiveness for funding opportunities. Indeed, the CAP BAP has gathered national and governmental recognition since its launch in 2012. As indicated above, the latest version of the National Cancer Institute's Best Practices in Biorepositories now includes the “Biorepository” checklist as an appendix. ¹⁵ In September 2015, after U.S. Department of Health and Human Services announced a “Proposal to Improve Rules Protecting Human Research Subjects” (Notice of Proposed Rulemaking for Revisions to the Common Rule), the CAP BAP was directly referenced in posted public comments. ²⁰

However, there is still a need for increased awareness and understanding of the CAP BAP. In the 2015 Precision Medicine Initiative (PMI) request for applications, biorepositories applying to become the “PMI Cohort Program Biobank” were requested to use processes, which were “consistent with CLIA” or “CLIA-compliant,” even though no accepted definition or surrogate accreditation program for this standard was suggested. ²¹ This, along with general misunderstanding of CAP and CLIA accreditation, suggests the need for additional education and clarification. CLIA regulations apply to all testing performed for the diagnosis, prevention, or treatment of any disease in a specific patient ¹ —i.e., the practice of diagnostic laboratory medicine. By definition, a biorepository's primary purpose is not the performance of diagnostic laboratory medicine. However, terms such as “CLIA-compliant” recognize that application of CLIA (or more stringent) requirements to biorepositories will ensure the biospecimen quality and reproducibility required for the development of new diagnostic assays. Omitting CAP BAP from this discussion fails to recognize the CAP's deemed status to inspect for CLIA under CMS, making CAP BAP uniquely capable of externally validating a biorepository's adherence to CLIA-equivalent quality processes. With continued growth and acceptance of the program, this concept can be better addressed in grant announcements and public policy documents.