Abstract
Objective
Both PCR and Hybrid Capture II (HCII) have been used for identifying cervical dysplasia; however, comparisons on the performance between these two tests show inconsistent results. We evaluated the performance of HCII and PCR MY09/11 in both screening and diagnostic populations in sub-sample of 1,675 non-pregnant women from a cohort in three clinical centers in the United States and Canada.
Methods
Sensitivity, specificity, positive predictive value, negative predictive value, and concordance between the two tests were calculated.
Results
Specificity of HCII in detecting low-grade squamous intraepithelial lesion (LSIL) was higher in the screening group (88.7%; 95% CI: 86.2%–90.8%) compared to the diagnostic group (46.3%; 95% CI: 42.1%–50.6%); however, specificity of PCR was low in both the screening (32.8%; 95% CI: 29.6%–36.2%) and diagnostic (14.4%; 95% CI: 11.6%–17.6%) groups. There was comparable sensitivity by both tests in both groups to detect high-grade squamous intraepithelial lesion (HSIL); however, HCII was more specific (89.1%; 95% CI: 86.8%–91.0%; 66.2%; 95% CI: 62.0%–70.1%) than PCR (33.3%; 95% CI: 30.2%–36.5%; 17.9%; 95% CI: 14.8%–21.6%) in the screening and diagnostic groups, respectively. Overall agreement for HPV positivity was approximately 50% between HCII and PCR MY09/11; with more positive results coming from the PCR MY09/11.
Conclusion
In the current study, PCR MY09/11 was more sensitive but less specific than HCII in detecting LSIL, and HCII was more sensitive and specific in detecting HSIL than PCR in both screening and diagnostic groups.
Keywords
Introduction
Cervical cancer is an important public health issue worldwide, yet the incidence and mortality rates of cervical cancer have substantially declined in the United States and other countries where cytologic screening programs using the Papanicolaou (Pap) test have been widely employed.1,2 The limitations of the Pap test (ie, high false positivity) have resulted in the addition of HPV DNA testing to screening programs to improve their efficacy.3–5 HPV DNA testing has been recommended to be used as a screening tool, either in combination with cytologic Pap test or alone;6,7 or to triage women with abnormal cytologic test results to determine whether they should be referred for colposcopy; 4 or to follow-up women under treatment for high grade cervical intraepithelial neoplasia (CIN) to detect residual disease or predict recurrence. 8
Two methods that have been widely used for HPV DNA detection are polymerase chain reaction (PCR) and Hybrid Capture II (HCII). In the PCR method, HPV types are amplified by different types of consensus primers such as MY09/11, GP5+/6+, PGMY09/11, to detect the presence of HPV DNA in the clinical sample (reviewed in Brink et al 9 ). HCII is a signal amplification method and is one of three tests currently approved for commercialization by the US Food and Drug Administration (FDA).10–12 Previous comparisons of the performance of PCR and HCII have yielded inconsistent results.13–17 Among those studies, only two studies13,14 were conducted in both screening and diagnostic settings.
Determining the test with better accuracy is important in identifying cervical precancerous lesions in diverse clinical settings, particularly in the context of using HPV testing with triage by Pap testing. The objective of the current study was to evaluate the performance of HCII and PCR MY09/11 in the detection of HPV DNA and cervical dysplasia in both the screening and diagnostic settings in a sub-sample of 1,675 non-pregnant women from a cohort in three clinical centers in the United States and Canada.
Materials and Methods
Study Population
From October 1998 to November 2005, 1,850 non-pregnant women aged 18 years and older were enrolled in a phase II clinical trial at three locations: the University of Texas MD Anderson Cancer Center, Lyndon Baines Johnson General Hospital, and British Columbia Cancer Agency. The trial was conducted to evaluate fluorescence and reflectance spectroscopy, an emerging technology using an optical probe, to detect precancerous lesions in the cervix. Women who reported that they had no history of abnormal Pap smear results were enrolled in the screening group, and women who reported that they had an abnormal Pap test at any previous time were enrolled in the diagnostic group. Women who had a history of cervical cancer or CIN were excluded. Details on recruitment strategies and socio-economic characteristics of study participants were described elsewhere. 18 The study protocol was approved by the Institutional Review Boards at the three study locations, and women provided written informed consent before participating in the study.
Data Collection
The study protocol included a comprehensive clinical exam and routine tests for screening and detection of gynecologic disease. Study participants were asked to provide complete medical history, including risk factors of cervical cancer. During the pelvic exam a series of procedures were conducted, including cytologic Papanicolaou smear, cervical cultures to test for chlamydia and gonorrhea, specimens for HPV testing, and a colposcopic exam of the vulva, vagina and cervix, which included two to four fluorescence and spectroscopic measurements. 18
Specimen Collection
Endocervical cytobrushes were used to collect clinical specimens for different laboratory procedures (ie, HPV DNA and RNA, Hybrid Capture II, HPV typing, quantitative cyto- and histopathology). DNA specimens were placed in 250 μL PBS with 0.02% sodium azide immediately after cytologic sampling and were stored at –80 °C until extracted. Specimen extraction was performed approximately one month after collection. 19 Biopsies were taken from colposcopicly abnormal and normal sites for histopathologic confirmation of disease, as described elsewhere.18,19 Briefly, the colposcopist took one or two directed biopsied from an area of overall abnormal colposcopic impression. He/she then took one or two biopsies of squamous and columnar epithelium from an area of normal appearance. The biopsy specimens were fixed in buffered formalin and embedded in paraffin blocks. 20
Pathology Review
Pathology review was described in details in previous reports.19,20 Briefly, the first pathology review was done at each institution by one of the gynecologic pathologists who were on clinical duty. A second blinded pathology review was done by one of the study pathologists. If there was a discrepancy between two readings, the slide was reviewed a third time by our study pathologist to achieve the final consensus diagnosis. Similarly, liquid-based cytology was also read by one of the cytopathologists on clinical duty and by a study cytopathologist at each institution. Discrepancies between readings were evaluated by a third cytopathologist to attain the final consensus diagnosis. In case of disagreement between cytologic and histologic result, the worse diagnosis was taken as the consensus between the two diagnoses.
HPV DNA Detection Using Hybrid Capture II Test
Hybrid Capture II (DIGENE, then QIAGEN Corporation) was performed by a clinical laboratory (Laboratory Corporation of America), following the protocol recommended by the manufacturer. Briefly, HPV DNA was denatured and incubated with RNA probe A for low-risk types and probe B for high-risk types to form RNA-DNA hybrids. The hybrids were captured in a solid phase with antibodies specific for RNA-DNA hybrids and were attached by antibodies conjugated to alkaline phosphate. The emitted light (or product of chemiluminescence obtained from the conjugated antibody-hybrid that corresponds to the amount of DNA in the sample) was measured by a luminometer as relative light units (RLUs). A sample was classified positive when the RLUs were equal to or greater than the mean value of the positive control (1 pg/mL). 18
HPV DNA Isolation and Detection
A commercially available kit (QIAamp DNA Mini Kit, Qiagen, Valencia, CA) was used to extract viral DNA from cervical cytobrush specimens. Details on DNA extraction were reported elsewhere. 21 Briefly, 20 μL Proteinase K and 400 μL Buffer AL was added to the tube containing the specimen. The tube was incubated for 10 min at 56 °C, and 400 μL ethanol was then added. The cytobrush was left in the tube to this point to obtain as much DNA as possible from the sample. The cytobrushes were then removed from the tubes using forceps before DNA elution. Forceps were flamed between each sample to prevent contamination. The sample was applied to a QIAamp Spin Column in 2 steps and centrifugation was performed at 8,000 rpm for 1 min after each application. The spin column was then washed with 500 μL of Buffer AW1 and AW2 and centrifuged for 1 min at 8,000 rpm and 3 min at 14,000 rpm, respectively. To remove residual fluid and dry the filter, the fluid was removed from the collection tube, and the column was centrifuged for 1 min at 14,000 rpm. The filter was placed in a clean microcentrifuge tube and 110 μL AE buffer was added. The mixture was incubated at room temperature for 5 min and then centrifuged for 1 min at 8,000 rpm to elute. Extracted DNA was stored at –80 °C before PCR was performed. Following the methods of Manos et al, 22 we analyzed the samples for HPV DNA using MY09/11 consensus HPV primers that amplify a 450 bp region of the L1 open reading frame of at least 28 different HPV types. PCR products were resolved by agarose gel electrophoresis, transferred to nylon membranes (Bio-Rad Laboratories, Hercules, CA) and hybridized to a 32P-labeled HPV consensus probe. Consensus probe-positive samples were then hybridized to 32P-labeled specific HPV-16 and HPV-18 probes on separate nylon membranes. Sample positivity was assessed by audioradiography following hybridization. DNA extracted from HPV18-positive HeLa cells, HPV16-positive CaSki cells, and a negative control without DNA were used as controls in the PCR and subsequent hybridization. 21
HPV DNA Detection by Real-Time Polymerase Chain Reaction
cDNA (20 ng) extracted from clinical samples was analyzed by real-time PCR to determine the presence or absence of HPV via melting curve analysis and then to type the samples as HPV-16 or HPV-18 positive using molecular beacons. Methods for these analyses were derived partially from the work of Cubie et al 23 and Szuhai et al. 24 For HPV typing using molecular beacons, the following primers were used with the MY09/11 consensus primers to amplify a 450-bp fragment in the L1 reading frame. 24
HPV16: 5′-CGCCTCAATGCTGCTGCTGTACTAC-GAGGCG-3′
HPV18: 5′-CGCCTCTATTAGTGAAGTAATGGGA-GACGAGGCG-3′
Amplification was performed using SYBR Green PCR Master Mix (BioRad), and the fluorescence spectra generated from amplification were recorded at the end of each cycle using the iCycle iQ Detection System (BioRad). 24 For the melting curve analysis, DNA was amplified using the MY 09/11 primers as described by Manos et al 22 in the presence of SYBR Green and then the PCR products were melted using a single cycle of 70 °C for 10 seconds to 95 °C for one second at a transition rate of 0.1 °C per second. Samples were called HPV-positive if they had a melting peak between 79 °C and 83 °C degrees based on profiles obtained from DNA amplified from HeLa (HPV-18), CaSki (HPV-16) cells, and an HPV 6/11 mixture obtained from DNA cloned into plasmids. HPV-positive samples were typed using fluorescence-labeled probes specific for HPV-16 (MY133) 22 and a custom-designed probe for HPV-18 in a multiplex reaction. Amplification curves obtained for each set of samples were used to ascertain positive samples.
Disease Definition
We used 2001 Bethesda Classification System to define disease outcome in current study. 25 Specifically, histologic diagnosis was grouped into four categories. A high-grade squamous intraepithelial lesion (HSIL) was defined as those having a cytologic diagnosis of moderate dysplasia or a histologic diagnosis of cervical intraepithelial neoplasia (CIN) 2, severe dysplasia (CIN 3) or carcinoma in situ (CIS). A low-grade squamous intraepithelial lesion (LSIL) was defined as those having a cytologic diagnosis with HPV associated changes or a histologic diagnosis of mild dysplasia (CIN 1). Samples that were negative for dysplasia by both cytology and histology were defined as normal. Samples with a diagnosis of atypical squamous cells of undetermined significance (ASCUS) were kept as a separate group because of their potential histologic importance in showing a transition from a normal to an abnormal stage.
Statistical Analysis
Socio-demographic characteristics (ie, age, race, education, marital status and employment status), clinical characteristics (Papanicolaou smear results, menopausal status) and risk factor data (smoking status and alcohol intake) were compared between the screening and diagnostic groups. Student's t-test or Pearson χ 2 test was used to determine the difference between two groups for continuous variables (ie, age) and for categorical variables (ie, the remaining variables), respectively. The performance of each test (PCR or HCII) was evaluated separately in the screening and diagnostic groups. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) and their respective 95% confidence intervals (CIs) were calculated. 26 The level of agreement between the two tests according to the Kappa statistic (κ) was defined excellent, good, and marginal reproducibility if κ > 0.75, 0.40 < κ < 0.75, and κ < 0.40, respectively.26,27 All tests were two-sided and were considered statistically significant if P < 0.05. All statistical analyses were performed using STATA 12.0 (Stata Corp., College Station, Texas).
Results
A total of 1,675 participants who were tested with both HCII and PCR and had valid cytology and histology diagnoses were available for evaluation. The mean age and standard deviation of women in the screening group was significantly higher than that of women in the diagnostic group (44.09 ± 12.15 vs. 36.50 ± 11.61, respectively, P < 0.001). In the screening group, approximately 66% of participants had normal cytology/histology and only 1.65% had a diagnosis of HSIL; however, the disease groups were fairly equally distributed in the diagnostic population (Table 1).
Demographic and clinical characteristics of participants in screening and diagnostic groups.
Mean and standard error.
HCII detected 108 and 379 HPV infections in the screening and diagnostic groups, respectively; while PCR detected 604 and 639 HPV infections in the screening and diagnostic groups, respectively (Table 2).
Frequency of HPV test results and histologic diagnoses of patients in screening and diagnostic groups.
PCR appeared to be more sensitive but less specific than HCII in detecting LSIL. HCII detected a lower prevalence of HPV compared to PCR in LSIL and comparable prevalence of HPV to PCR in HSIL, both in the screening and diagnostic groups (LSIL: 1.2% vs. 7.6% and 10.7% vs. 21.7%, respectively; HSIL: 1.6% vs. 1.6% and 27.7% vs. 27.7%, respectively). In detecting LSIL, PCR was also more sensitive than HCII in both groups [(64.5%; 95% CI: 54.6% to 73.3%) and (84.8%; 95% CI: 78.7% to 89.4%) in the screening group and diagnostic group, respectively, by PCR versus (16.8%; 95% CI: 10.5% to 25.5%) and (41.9%; 95% CI: 34.9% to 49.2%) in the screening group and diagnostic group, respectively, by HCII]. The specificity of HCII in detecting LSIL was higher in both the screening group (88.7%; 95% CI: 86.2% to 90.8%) and diagnostic group (46.3%; 95% CI: 42.1% to 50.6%), in comparison to PCR [(32.8%; 95% CI: 29.6% to 36.2%) and (14.4%; 95% CI: 11.6% to 17.6%) in the screening group and diagnostic group, respectively]. Both tests had comparable PPV and NPV in the screening and diagnostic groups (Table 3). In detecting HSILs a comparable sensitivity was found in both tests; however, HCII was more specific than PCR (89.1%; 95% CI: 86.8% to 91.0% and 66.2%; 95% CI: 62.0% to 70.1% versus 33.3%; 95% CI: 30.2% to 36.5% and 17.9%; 95% CI: 14.8% to 21.6%, in screening group and diagnostic group, respectively).
Performance of the HCII and PCR to detect low-grade squamous intraepithelial lesions (LSILs) and high-grade squamous intraepithelial lesions (HSILs) in screening and diagnostic groups.
The crude agreement for HPV positivity between PCR and HCII was 48.78% and their reproducibility was marginal (Kappa statistic κ = 0.15; 95% CI: 0.12 to 0.18) (Table 4).
Concordance of results of HCII and PCR for HPV detection.
Discussion
We found PCR MY09/11 to be more sensitive than HCII, but HCII was more specific than PCR MY09/11 in detecting LSILs in both the screening and diagnostic groups. For the detection of HSILs, the two tests had comparable sensitivity in both the screening and diagnostic groups; however, HCII was substantially more specific than PCR, in both groups. We also found that even though the crude agreement between the two tests was good, the reproducibility was marginal.
The higher sensitivity and lower specificity of PCR MY09/11 over HCII was not surprising because while PCR requires only 500–1,000 copies of the HPV genome to generate a positive result, HCII requires approximately 5,000 copies of the HPV genome. 9 Our findings in the screening group were consistent with previous studies13,17 but inconsistent with others.15,28 For example, in a study at 4 clinical centers in the US, Schiffman et al 28 reported that HCII is more sensitive (93.6% vs. 89.3%, P < 0.0005) but less specific (41.2% vs. 48.5%, P < 0.0005) than PCR in detecting CIN3 or cancer. Also in a study in the former Soviet Union in 2003, Kulmala et al 15 reported that HCII is more accurate than PCR [(sensitivities: 85.2%; 95% CI: 82.9%-87.5% versus 74.0%; 95% CI: 71.1%-76.9%; respectively); (specificity: 67.2%; 95% CI: 64.1%-70.3% versus 64.1%; 95% CI: 58.8%-69.4%; respectively)]. Our findings most likely differ from these two studies15,28 due to the usage of different PCR consensus primers ie, MY09/11 versus GP05+/06+. 15 Another important difference is the use of different gold standards (ie, either cytology or histology in our study versus cytology in previous studies).15,28 Cytology, even conducted in an excellent quality laboratory, has also its own limitations, as pointed out by Castanon et al 29 that negative cytologic results yielded higher misclassification of cancer than negative histologic results (14% vs. 5%, respectively).
We also found good agreement but marginal reproducibility between the two tests. This result is similar to previous finding by Giovannelli et al 30 in which 1,100 specimens were tested to compare the performance between HCII and PCR (both PGMY09/11 and GP5+/6+ primers). They found the crude agreement between PCR PGMY and HCII was 88.5% but had marginal agreement (κ = 0.17). One possible explanation is that PCR might be able to detect HPV sub-types more often than HCII.
The small number of false positives detected in our study could be due to the following three reasons. First, different specimens were used for each of the two tests. The specimens for HCII were collected first and stored in the transport medium provided by the manufacturer. The specimens for PCR were collected second and were stored in PBS until DNA extraction was performed. There could be a cellular sampling issue whereby the cells collected for HCII were more or less likely to contain HPV than those subsequently sampled for PCR analysis. Second, the use of separate transport media for each test could also contribute to discrepancies in the test results. Castle et al 31 reported that, while there was no effect on subsequent HPV DNA detection by HCII in samples stored for up to 8 years, there was an effect on detection of internal control genes used for PCR assays. Perhaps using the same medium will prevent the discrepancy in the future. Third, equivocal results might be due to low copies of viral genome, infection with multiple HPV types or undetectable HPV types. HCII was not designed to detect HPV 30, 55, and 56, which were considered non-targeted HPV types in cervical cancer screening.
A major strength of our study is the large sample of participants that allows us to compare the performance of the two tests in both the screening and diagnostic settings. Also in the current analysis, misclassification of disease status is kept to a minimum as stated in the pathology review protocol, we utilized the histologic results from the biopsy, which was evaluated by two independent pathologists and, in case of any discrepancy, resolved by a third pathologist.
In a previous study, we reported the advantage of using HCII with Pap testing to screen for cervical cancer. 18 The present analysis supports the use of HCII over PCR MY09/11 in the detection of HSILs as, in both the screening and diagnostic groups, HCII proved to have comparable sensitivity and substantially higher specificity than PCR MY09/11. However, our data did not support the use of HCII in the detection of LSILs because in both the screening and diagnostic settings, HCII was less sensitive and but more specific than PCR MY09/11. More recently, PCR PGMY09/11, an updated version of PCR MY09/11, has become more widely used in both the research and clinical settings. Even though the MY09/11 primer sets for PCR analyses are not being used as frequently, our results still support the notion that PCR-based techniques may perform better for identification of LSILs while the technology behind HCII may be more apt for identifying HSIL. Furthermore, even newer technologies for HPV detection are being studied for integration into screening algorithms. For example, the COBAS HPV Test and APTIMA® HPV Assay, two PCR-based assays, are FDA approved for commercial use; however, one of the major concerns is whether they can be affordable for use in organized cervical cancer screening programs, particularly in limited-resources settings (ie, developing countries). Finally, the clinical decision of which type of test to choose for screening and diagnostic testing should ultimately be based upon an array of factors, including economic and epidemiologic evidence.
Author Contributions
Conceived and designed the experiments: HNL, KS, LMD, MF, MES. Analyzed the data: HNL, KS, LMD, MF, MES. Wrote the first draft of the manuscript: HNL, KS, LMD, MF, MES. Contributed to the writing of the manuscript: HNL, KS, LMD, MF, MES. Agree with manuscript results and conclusions: HNL, KS, LMD, MF, MES. Jointly developed the structure and arguments for the paper: HNL, KS, LMD, MF, MES. Made critical revisions and approved final version: HNL, KS, LMD, MF, MES. All authors reviewed and approved of the final manuscript.
Funding
This work was supported the National Institutes of Health, grant number P01CA082710, the University of Texas School of Public Health-Cancer Prevention and Research Institute of Texas (CPRIT) grant #RP101503 (to HNL) and in part by a Dan L. Duncan Cancer Center Scholar Award (to MES). The content is solely responsibility of the authors and does not necessarily represent official views of the CPRIT.
Competing Interests
Author(s) disclose no potential conflicts of interest.
Disclosures and Ethics
As a requirement of publication the authors have provided signed confirmation of their compliance with ethical and legal obligations including but not limited to compliance with ICMJE authorship and competing interests guidelines, that the article is neither under consideration for publication nor published elsewhere, of their compliance with legal and ethical guidelines concerning human and animal research participants (if applicable), and that permission has been obtained for reproduction of any copyrighted material. This article was subject to blind, independent, expert peer review. The reviewers reported no competing interests.
Footnotes
Acknowledgements
We would like to thank all of the women who participated in this study. Without them, this study would not have been possible. We also wish to thank the support from staff of the P01 in both Houston and Vancouver, especially Laura Dillon and Nan Earle for their help with sample analysis and data management.