Abstract
BACKGROUND:
Along with the dramatic development of molecular diagnostic testing for the detection of oncogene variations, reference materials (RMs) have become increasingly important in performance evaluation of genetic testing.
OBJECTIVE:
In this study, we built a set of RMs for genetic testing based on next-generation sequencing (NGS).
METHOD:
Solid tumor tissues were selected as the samples of RMs for preparation. NGS was used to determine and validate the variants and the mutation frequency in DNA samples. Digital PCR was used to determine the copy numbers of RNA samples. The performance of the RMs was validated by six laboratories.
RESULTS:
Thirty common genetic alterations were designed based on these RMs. RMs consisted of a positive reference, a limit of detection reference, and a negative reference. The validation results confirmed the performance of the RMs.
CONCLUSION:
These RMs may be an attractive tool for the development, validation, and quality monitoring of molecular genetic testing.
Introduction
Based on the associations between oncogene mutations and their sensitivity to anticancer targeted drugs, genetic testing of these oncogenes is recommended before deciding anticancer treatment. Accordingly, molecular diagnostic testing has undergone a dramatic development in the detection of oncogene variations in recent years. Many technologies used for the detection of DNA sequence variations have been developed for cancer research. These technologies can be grouped into three general approaches: polymerase chain reaction (PCR), hybridization, and next-generation sequencing (NGS) [1].
NGS is a high-throughput technology for large scale DNA parallel sequencing and could be conducted with different platforms by their respective technical principles [2]. At present, there are three major NGS platforms in the Chinese market, including HiSeq/MiSeq from Illumina, Ion Torrent PGM/Proton from Life Technologies, and BGISEQ-500 from Beijing Genomics Institute (BGI). Illumina have adopted a sequencing-by-synthesis (SBS) technology by using fluorescently labelled reversible-terminator nucleotides [3]. The amplified DNA templates were immobilized on the surface of a glass flow cell to realize the synthesis and sequencing. In contrast, the Ion Torrent platform use a semiconductor sequencing technology to detect the protons released as nucleotides are incorporated during synthesis [4]. Besides, BGISEQ-500 is powered by combinatorial Probe-Anchor Synthesis (cPAS) and improved DNA Nanoballs (DNB) technology. In the clinical setting, NGS is typically used for sequencing specific gene or panels of genes rather than for entire genomes. Thus different NGS platform have chosen difference enrichment methods. In the case of the HiSeq/MiSeq and BGISEQ-500, they harness the hybrid capture targeted technologies. Conversely, Ion Torrent adopt PCR amplification methods to prepare the libraries.
Nanoliter-sized droplet technology paired with digital PCR (ddPCR) is an absolute quantitatively methods to directly measure the precise amount of the initial nucleic acids template, based on limiting partition of the PCR volume and on Poisson statistics. It works by partitioning a DNA/cDNA samples into many individual, parallel PCR reactions through a water oil emulsion technique [5]. During amplification, TaqMan chemistry with dye-labelled probes is used to detect sequence-specific targets. Being allowing for high-sensitive absolute quantification of target sequence, ddPCR could have a high potential in detecting genetic variants. There are two major digital PCR in the present market, the Droplet Digital PCR from Bio-Rad (California, USA) and the QuantStudio 3D PCR from Thermo Fisher Scientific (Waltham, MA, USA).
Many diagnostic laboratories are using EGFR KRAS NRAS, BRAF, PIK3CA, MET and ALK genetic testing as companion diagnostics for relevant anticancer therapeutics. These tests also serve as a tool to help hospital diagnostic laboratories monitor gene therapy patients and their outcomes. Likewise, in China, molecular diagnostic testing products specific to these oncogene mutations have been stepping up the research and development by many companies. However, laboratories performing genetic tests for medical purposes are required to establish quality assurance practices to ensure confidence in test integrity and accuracy [6]. An integral part of quality assurance is the use of characterized reference materials (RMs). Thus the thoroughly characterized RMs are essential for quality assurance, assay validation and standardization.
At present, there is a complete set of RMs preparation strategies, including samples selection for RMs, samples sub-package and performance evaluation. Ideally, a RM should resemble a patient sample, and contain the mutation types or variant alleles that the assay is designed to detect. In the performance evaluation, sub-packed samples were sent to volunteer laboratories for characterization of the accuracy, specificity, stability and uniformity of RMs. However, clinical genetic tests are available for over 3,000 conditions RMs are not available for the vast majority of these assays. Especially for oncology-related RMs, public available RMs is limited and expensive in the world around, and worse in China. Due to the lack of publicly available RMs for oncogene mutation detection in China, it is necessary to establish well-characterized quality controls for monitoring the test performance and proficiency of DNA-based genetic tests.
In the current study, we aimed to establish a set of RMs to evaluate the performance of EGFR KRAS NRAS BRAF PIK3CA MET and ALK mutation testing based on NGS, digital PCR (ddPCR), and real-time fluorescence qualitative PCR (RT-PCR). The RMs we developed are suitable for oncogene mutation test specimens of formalin-fixed paraffin-embedded (FFPE) or fresh frozen tumor tissues. The current study presents a successful technology method for RMs preparation from the resident cases’ specimens for the quality assurance needs of molecular diagnosis laboratories.
Materials and methods
Cancer gene alteration selection
Cancer-related gene alterations that are widely used in the clinic and have been clearly reported in the literature were selected. These gene alterations included variants in EGFR KRAS NRAS BRAF PIK3CA, MET amplification, and ALK fusion. All of these genes play a role in anticancer therapeutics.
Samples and DNA extraction
FFPE samples from 53 patients with pathologically diagnosed solid tumors were retrieved from Shanghai Pulmonary Hospital (Shanghai, China) between October 2015 and September 2017. Among these samples, 38 were mutant type (MT) and used as the positive reference and the limit of detection (LOD) reference, while 15 were wild type (WT) and used as the DNA negative reference. The H1299 cell line was obtained from the American Type Culture Collection (ATCC) (Manassas, VA, USA) and prepared as the RNA negative reference.
Genomic DNA was extracted from enrolled subjects using the QIAamp DNA FFPE Tissue Kit (Qiagen, Valencia, CA). RNA was prepared using the Qiagen RNeasy FFPE Kit. All DNA/RNA samples were checked for quality control using a Thermo Scientific NanoDrop 2000C spectrophotometer (Life Technologies, Carlsbad, CA, USA). The concentration of each DNA/RNA sample was recorded using a Qubit 3.0 fluorometer (Life Technologies, Carlsbad, CA). The NGS assay performed on three main platforms (Illumina, NexSeq500; Life technology, Ion Proton; BGI, SEQ-500) and ddPCR performed on two main platforms (Bio-rad, QX200 Droplet Reader; Thermofisher, QuantStudio 3D Digital PCR) were used to determine and validate the variants and the mutation frequency in DNA samples. The final mutation frequency was from the mean value of different results from the testing laboratories. ddPCR (QX200 Droplet Reader) was used to determine the copy numbers of RNA samples.
RMs preparation
RMs were prepared as 12-
Six participating centers (Darui Technology, Guangzhou, China; Annoroad Gene Technology, Beijing, China; Novegene, Beijing, China; Burning Rock DX, Beijing, China; BGI-Shenzhen, Shenzhen, China; Geneplus Technology, Beijing, China) covering three NGS assay platforms were invited to assess the accuracy and specificity of detecting the above mentioned oncogene variants of the RMs. All detection methods were performed following a standard operating procedure and instructions of the relevant oncogene variant detection kit of each laboratory. No more than one failure of the RMs sample was permitted during library construction. Among the RMs, MET amplification and ALK fusion of DNA samples were not suitable for NGS with PCR amplification methods; RNA samples (ALK fusion) were not available for the NGS assay with hybrid capture targeted technology. Detection of all genetic mutations from the positive-reference illustrated its high accuracy. No mutations were detected in the negative reference, confirming its specificity. The LOD requirement was as follows: variants in the 5% LOD reference and 2.5% LOD reference, 4 copy number changes in MET amplification, 400 copies/30 ng and 100 copies/30 ng of ALK fusion should be detected correctly. (a genetic testing kit with an LOD less than 2% should prepare the LOD reference using positive samples in the RMs and WT genomic DNA. And this kit detecting the lower percentage LOD were regarded as agree with the LOD requirement). All detected results that showed the accuracy, specificity and LOD of a genetic testing kit were gathered to determine the performance of the RMs.
The DNA/RNA samples were kept at
Results
In total, we selected 30 common gene mutations (Table 1) in EGFR, KRAS, NRAS BRAF, PIK3CA MET and ALK according to clinical diagnostic data for relevant anticancer therapeutics. These mutations included 6 KRAS variants, 2 NRAS variants, 1 BRAF variant, 2 PIK3CA variants, 14 EGFR mutations, 1 MET amplification, and 4 ALK fusions. Both DNA and RNA samples of ALK fusion were included in the RM. According to the information in the online database of MyCancerGenome, the carrier rates of KRAS mutation in lung adenocarcinoma, EGFR, NRAS, BRAF, and PIK3CA mutation, MET amplification and ALK fusion in NSCLC were summarized. The database also revealed the mutation allele frequency of mutated gene, which could indicate the distribution of the oncogene variants in cancer patients. Among these, G12C were the most common variants in KRAS mutations, while Q61L in NRAS, E545K in PIK3CA; T790M and INDEL in exon 19 were the most common variants in EGFR with mutation frequency of 50% and 48%, separately. The mutation frequency of each gene in the present study, which was derived from the detection results of the mean value from two testing platforms (NGS and ddPCR) of five laboratories, is shown in Table 1. These mutation frequency were set as positive reference in the RMs.
Information on gene mutations in EGFR, KRAS, NRAS, BRAF, PIK3CA, MET, and ALK
Information on gene mutations in EGFR, KRAS, NRAS, BRAF, PIK3CA, MET, and ALK
When five laboratories were invited to detect the mutation frequency and variant sites of the samples, score coefficient was set to classify the grade level of samples. For the classification, most of samples revealed a good uniformity. And the mean values from five laboratories of these samples were calculated as the final mutation frequency. However, results of five mutations were shown grater differences from 5 detection laboratories, with two-fold difference between maximum value and minimum value (Table 2). A further analysis indicated that results from Illumina sequencing platform and BGI sequencing platform shown a good uniformity, with a lower detection value, while results from Life Science sequencing platform and ddPCR platform were found a good uniformity, with a higher detection value. For this situation, means values from Illumina platform and BGI platform were selected as the final mutation frequencing.
A total of 180 RMs were prepared for the application of clinical laboratories in our study. As shown in Table 3, the RMs consisted of the positive reference (
Mutation frequency of the reference materials from 5 testing laboratories
Characteristics of the reference materials
MET amplification of 4 copies and ALK fusions of 400 copies/30 ng and 100 copies/30 ng were also designated as LOD references. For ALK fusion, five RNA samples were used in the RM. For insufficient samples, KRAS Q61H, PIK3CA H1047 and EGFR T790M were only set as LOD reference, rather than positive reference.
Six participant laboratories were invited to validate the accuracy and precision of the RMs. The results shown that all positive reference samples were detected, while all negative reference samples were undetected by six laboratories. However, on the detection of LOD reference samples, 45-EGFR-
As a transmembrane tyrosine kinase receptor, EGFR expression is associated with poor prognosis in several cancers, including lung cancer, colorectal cancer, hepatocellular carcinoma, and so on. RAS-RAF-MAPK and PI3K-PTEN-AKT, as the downstream signaling pathways of EGFR, also play significant roles in tumorigenesis [7, 8, 9]. Interfering with oncogene signaling should be a promising therapeutic approach. Tumor suppressor genes and proto-oncogenes participating in these signalling pathways, such as EGFR, MET, KRAS, NRAS, BRAF, and PIK3CAT, had been considered as targeted genes in the targeted therapy. In the past decade, several anti-EGFR drugs (known as EGFR tyrosine kinase inhibitor, EGFR-TKI) have been approved for the treatment of tumors, especially for non-small cell lung cancer (NSCLC) and colorectal cancer (CRC). However, more and more clinical trials have indicated that different types of oncogene mutations have different responses to anticancer therapeutics [10]. Gefitinib is the most successful molecular targeted therapeutic drug for late-stage lung cancer [11, 12]. Reports shown that the T790M mutation in EGFR exon 20 leads to drug resistance to gefitinib [12], while patients with mutations located in exons 18, 19, and 21 in EGFR are sensitive to gefinitib treatment [13, 14]. Mutant KRAS is associated with resistance to anti-EGFR antibodies [15]. Besides, mutations in NRAS, BRAF, and PIK3CA, have also revealed negative effects against anti-EGFR antibodies [16, 17]. MET amplification is a well-established mechanism of acquired resistance to EGFR inhibitors in NSCLC and CRC [18, 19]. ALK can also be oncogenic by forming a fusion gene with any of several other genes. ALK fusion is resistant to EGFR-TKIs in lung cancer therapeutics [20]. Crizotinib was the first ALK inhibitor to receive FDA approval for the treatment of patients with ALK-positive NSCLC [21]. Therefore, concomitant diagnosis for these gene mutations had been required before the use of targeted therapeutic drugs.
With the development of molecular genetic testing in oncogene diagnosis and personalized treatment, the requirement for RMs of relevant oncogene mutations continues to play an important role in both clinical diagnostic and research laboratories. According to previous reports and clinical specimens’ information, 7 oncogenes associated with cancer targeted therapeutic drugs and common mutational types were chosen to prepare RMs to evaluate the performance of these genetic mutation tests, including 30 common gene mutations in EGFR KRAS NRAS BRAF, PIK3CA MET and ALK.
Among these oncogene alterations, previous studies have reported that KRAS mutations account for 15–25% of cases of lung adenocarcinoma [22], with the most common mutations located in codons 12 and 13 of exon 2, which accounts for 85–90% of cases with mutated KRAS. In this study, KRAS mutations (G12D, G12V, G12C, G12A, and G13D) were in common mutation regions. Similarly, the proportion of EGFR variants was 10% in NSCLC of USA populations and 35% in NSCLC of Asian populations [23, 24, 25], with the most common EGFR mutations located in exons 18, 19, 20, and 21. This study also selected EGFR mutations in these locations. According to previous studies, NRAS BRAF, and PIK3CA mutations, and ALK fusion account for 1% [26], 1–4% [27, 28, 29], 1–3% [30, 31], and 3–7% [32] of all cases of NSCLC, respectively. MET amplification accounts for 2–4% of cases of previously untreated NSCLC [33] and 5–20% of patients with EGFR-mutated tumors and acquired resistance to EGFR-TKIs [34, 35]. Herein, we chose typical abnormal sites in these genes. In general, the genetic mutations selected in this study cover the common oncogene mutation types and meet the evaluation requirement of most genetic testing kits.
When the mutation frequency of clinical samples was determined, we found that the detection results of 5 samples were significantly different among the 5 laboratories, with a difference of greater than two-fold between the maximum value and the minimum value. Among these results, data from Illumina platform of laboratory is similar with data from BGI sequencing platform, with a lower mutation frequency; while results from Life Science sequencing platform of laboratory kept consistent with results from ddPCR platform, with a higher mutation frequency. As we know, Illumina and BGI sequencing platform adopted the same NGS enrichment method of hybrid capture targeted technologies, while Life Science sequencing platform used multiple PCR amplification to prepare library. Both the multiple PCR amplification methods and ddPCR adopt PCR methods to amplify target fragments. Consequently, the different detection results of one samples were caused by the different methods of obtaining the targeted fragments. Usually, tumor tissue specimens, as part of standard pathology practice, are routinely processed and stored as formalin-fixed paraffin-embedded (FFPE) blocks. Although this method is beneficial to preserve samples for a long time, this also presents many challenges to the diagnostic laboratory. FFPE samples are typically a variable mix of neoplastic and normal cell tissue (stroma). The DNA extracted from these samples is often limited in fragmented, quantity and of poor quality. In addition, it may contain artefactual sequence alterations arising from formalin crosslinking and deamination of cytosine nucleotides. Do et al. reported that these problems can be mitigated by the use of shorter amplicons, de-crosslinking steps and treatment with uracil-DNA glycosylase, a DNA repair enzyme, which has been shown to markedly reduce the number of sequence artefacts in damaged FFPE DNA when used prior to PCR amplification [36, 37]. Therefore, targeted mutation fragments obtained from hybrid capture targeted technologies is less than that from multiple PCR amplification and ddPCR methods, which may explain the phenomenon that mutation frequency detected from Illumina and BGI sequencing is lower than Life Science and ddPCR platform. The degree of the discrepancy reflected the degree of DNA fragmentation, cross-linking and deamination of FFPE samples.
The original mutation frequency of the tissue samples analyzed in this study was ranged from 6% to 65%. Since these samples were all obtained from clinical tumor tissues, the mutation frequency could reflect the actual situation of clinical oncogene mutations. Actually, it is not easy to identify and obtain appropriate samples for poor collection of clinical samples. Tumor cell contained a larger number of low-frequency mutation, and the detection for these low-frequency mutations were quite important for clinical significance [38]. The variants allele frequency detection of oncogene that used for diagnostics was lower than 5%. So high sensitivity and specificity for detection technology is required. Therefore, LOD reference is necessary and important for the RMs of oncogene testing In our study, LOD reference were also set in the RMs for the sensitivity and specificity evaluation of the detection system. Accordingly, 5% and 2.5% of the LOD reference and 400 copies/30 ng and 100 copies/30 ng of the LOD reference were prepared for evaluating the sensitivity of the testing system in genetic detection. For a low-frequency LOD reference (i.e., 1% or 0.1%), laboratories could prepare it using the mutant samples and WT genomic DNA or RNA in the RMs.
Performance evaluation of the RMs is a necessary step for the available RMs preparation. Validation of the oncology-related RMs should firstly consider the application of international reference materials. If not, a well characterized proposal should be developed from the relevant standard, guidance, and criterion. In our study, there no international reference materials for the selected oncogenes mutation condition. Thus, six laboratories located within China and accredited by the National Center for Clinical Laboratories certified were selected to assistant the performance evaluation of the RMs. The results revealed that four variants in 2.5% LOD reference were failed to be detected by two laboratories. According to the re-checked reports, limited testing performance of the two laboratories lead to these failure detections. Therefore, validation results from six laboratories still confirmed the accuracy and precision of the RMs. Besides, applications of the RMs for the laboratory quality management revealed the reliability and stability of the RMs
Conclusion
We successfully prepared RMs for molecular genetic testing from the clinical specimens, which could provide reliable and suitable calibration to evaluate the performance of EGFR KRAS NRAS BRAF PIK3CA MET and ALK genetic analyses based on NGS, ddPCR, and RT-PCR. Since these oncogene mutations are associated with the drug sensitivity of anti-cancer therapeutics, our RMs should accelerate the development of the detection methods of these oncogene variants. However, since residual patient specimens are not readily available or renewable, RMs prepared from these samples were with limited amount. Therefore, cell line materials will be investigated in further studies on this kind of RMs preparation.
Ethics approval
The study was approved by the Ethics Committee of Shanghai Pulmonary Hospital (Protocol number 094BL). Patient consent was obtained for publication.
Supplementary data
The supplementary files are available to download from https://dx-doi-org.web.bisu.edu.cn/10.3233/THC-220102.
Footnotes
Conflict of interest
None to report.
