Test Performance of Sputum microRNAs for Lung Cancer: A Meta-Analysis

Abstract

Background: Lung cancer is the leading cause of cancer-related deaths worldwide. Early detection and precise diagnosis are critical for the patients with lung cancer. Increasing evidence has suggested that microRNAs (miRNAs) play important roles in the diagnosis of lung cancer. To evaluate the overall diagnostic performance of sputum miRNAs for the detection of lung cancer, a meta-analysis was performed. Methods: A systematic search for published literature evaluating the diagnostic accuracy of sputum miRNAs in lung cancer was performed to determine pooled sensitivity and specificity. A summary receiver operating characteristic curve was constructed to assess the overall test performance. Subgroup analysis was utilized to explore potential sources of heterogeneity in the included studies. Results: Eight studies with a total of 514 patients and 491 controls were included in this meta-analysis. Sputum miRNAs had a pooled sensitivity of 0.70 (95% confidence interval [95% CI]: 0.66-0.70) and a pooled specificity of 0.89 (95% CI: 0.86-0.91) for the detection of lung cancer, with an area under the summary receiver operating characteristics curve of 0.83. Significant interstudy heterogeneity for specificity was observed, with miRNA profiles being a possible source. Conclusion: Sputum miRNAs are potentially useful noninvasive markers for diagnosis of lung cancer. The diagnostic specificity of sputum miRNAs may be influenced by the miRNA profiles. It would be important for further work to evaluate the generalizability of our results by methodologically rigorous studies on a well-defined patient population.

Introduction

Lung cancer is the most common cause of cancer deaths in the world. The severity of lung cancer is proved by the following fact that the 5-year survival is only about 10% (Parkin et al., 2005). The survival of the patients relies primarily on early detection. Recent study demonstrated that the 5-year survival of patients with operable stage IA lung cancer can be as high as 55-80% (Wang et al., 2010). Therefore, early diagnosis is essential for the lung cancer patients to receive effective therapies as early as possible (Vosa et al., 2013).

Traditional methods such as chest radiograph and computed tomography (CT) have been used for early detection of lung cancer, however, neither of them has good diagnostic performance (Frost et al., 1984; Lu et al., 2010; Aberle et al., 2011; Shen et al., 2013). The development of new noninvasive tools that can be applied to early diagnosis of lung cancer is thus clinically important.

MicroRNAs (miRNAs) are small noncoding RNA molecules, about 22 nt long, involved in various cellular biological processes, including proliferation, apoptosis, development, and differentiation (Pritchard et al., 2012). It has been proven that miRNAs are stably present in sputum, and the measurement of miRNA expression aberrations could be potentially useful for lung cancer diagnosis (Xie et al., 2010). Although several studies have reported the diagnostic value of sputum miRNAs in lung cancer, the results are still inconsistent (Xie et al., 2010; Xing et al., 2010; Yu et al., 2010; Roa et al., 2012; Anjuman et al., 2013; Yang et al., 2013; Li et al., 2014; Shen et al., 2014).

The objective of this meta-analysis was to evaluate the overall diagnostic performance of sputum miRNAs as noninvasive biomarkers for the detection of lung cancer, which to our knowledge, has not previously been reported.

Materials and Methods

Inclusion and exclusion criteria

Studies were included in the meta-analysis if they met the following criteria: (1) Individual study containing two comparator groups: lung cancer group and control group. (2) Diagnostic studies using sputum miRNAs for lung cancer. (3) The diagnosis of lung cancer was based on histopathological biopsy. (4) Sufficient data could be obtained to calculate sensitivity and specificity.

In addition, the following types of studies were excluded: (1) conference abstracts, review articles, letters, case reports; (2) studies with patients 20 or less; (3) studies without available data or lacking control groups; (4) studies assessed with low quality; and (5) studies involving patients who received therapy before the screening test.

Search strategy

Systematic computerized searches (up to 25th December, 2013) were conducted in MEDLINE/PUBMED, EMBASE, the Cochrane Library, and Chinese Biomedical Literature Database (CBM) without language limitation. Three groups of search terms used to identify relevant studies were as follows: (1) the primary disease—lung neoplasm, lung cancer, lung carcinoma, pulmonary neoplasm, pulmonary cancer, pulmonary carcinoma, and NSCLC; (2) the diagnostic test—microRNA and miRNA; and (3) the outcome—sensitivity, specificity, diagnosis, detection, and accuracy. The searches were limited to human studies. Furthermore, bibliography mentioned in the identified literatures was reviewed by hand-searching, to include additional literature that was not indexed.

Data extraction

The following items were extracted from the identified studies: the author, country, publication year, number of patients and controls, miRNA assay, miRNA expression profiles, specimen, and raw data [the number of true-positive (TP), false-positive (FP), false-negative (FN), and true-negative (TN) subjects]. Disagreements were resolved by two authors. If a consensus could not be reached, another author was consulted and a final decision made by a majority of votes.

Assessment of methodological quality

The quality of all selected studies was assessed by the 14-item Quality Assessment of Diagnostic Accuracy Studies (QUADAS) criteria (Whiting et al., 2003). The answer to each question contained “yes,” “no,” or “unclear.” The responses were recorded as “yes” scored 1, “unclear” scored 0, and “no” scored 1, with a maximum value of 14.

Statistical analysis

The Meta-DiSc 1.4 analysis software (Zamora et al., 2006) and STATA version 12.0 (Stata Corporation, College Station, TX) were utilized to analyze the data. A p-value <0.05 was considered as statistically significant. Study heterogeneity was assessed by the I² index and χ² test. The I² index is a measure of the percentage of total variation across studies due to heterogeneity rather than chance, a value 50% or more was considered indicative of heterogeneity (Huedo-Medina et al., 2006). In the χ² test, p-value<0.05 was considered to indicate that there was apparent heterogeneity between studies. If heterogeneity was demonstrated, a random-effects model (Fleiss et al., 2003) was used for the primary meta-analysis to obtain a summary estimate for sensitivity and specificity with corresponding 95% confidence interval (95% CI) using Meta-DiSc. In addition, the positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR) with respective 95% CI were calculated for each study. The summary receiver operating characteristic curves (SROCs) were estimated to describe the global test performance. Subgroup analysis was performed to investigate the potential sources of variation in the study results. Publication biases were estimated by Deeks' funnel plots (Deeks et al., 2005).

Results

Literature search and study characteristics

The bibliographic searches and manual review identified 660 records for consideration. After reviewing the titles and abstracts, 40 articles were chosen for full-text review. Of these, eight articles fulfilled all the inclusion criteria and were considered appropriate for the meta-analysis (Xie et al., 2010; Xing et al., 2010; Yu et al., 2010; Roa et al., 2012; Anjuman et al., 2013; Yang et al., 2013; Li et al., 2014; Shen et al., 2014). The final set of 8 articles consisted of 12 independent studies, with a total of 514 lung cancer patients and 491 controls. Detailed selection procedures are demonstrated in Figure 1.

FIG. 1.

The flowchart of study selection.

Among the 12 studies included in this meta-analysis, 4 assessed the diagnostic value of the combination of miR-31 and miR-210 in lung cancer (Anjuman et al., 2013; Li et al., 2014; Shen et al., 2014), while 8 evaluated other miRNA profiles (Xie et al., 2010; Xing et al., 2010; Yu et al., 2010; Roa et al., 2012; Yang et al., 2013). The characteristics of each included study are recorded in Table 1.

Table 1.

Main Characteristics of the Studies Included in the Meta-Analysis

								Test results
Author	Publication time	Country	Method	LC/Non-LC	Specimen	QUADAS	microRNA profile	TP	FP	FN	TN
Yu et al. (1)	2010	USA	PCR	36/36	Sputum	11	miR-21, 486, 375, 200b	29	3	7	33
Yu et al. (2)	2010	USA	PCR	64/58	Sputum	11	miR-21, 486, 375, 200b	45	12	19	46
Xie et al.	2010	USA	PCR	23/17	Sputum	10	miR-21	16	0	7	17
Xing et al. (1)	2010	USA	PCR	48/48	Sputum	10	miR-205, 210, 708	35	2	13	46
Xing et al. (2)	2010	USA	PCR	67/55	Sputum	10	miR-205, 210, 708	48	3	19	52
Roa et al.	2012	Canada	PCR	24/6	Sputum	10	miR-21, 143, 155, 210, 372	20	0	4	6
Yang et al. (1)	2013	China	PCR	24/24	Sputum	9	miR-21	20	6	4	18
Yang et al. (2)	2013	China	PCR	24/24	Sputum	9	miR-155	18	4	6	20
Li et al.	2014	USA	PCR	35/40	Sputum	12	miR-31, 210	23	6	12	34
Anjuman et al.	2013	USA	PCR	39/42	Sputum	9	miR-31, 210	24	4	15	38
Shen et al. (1)	2014	USA	PCR	24/24	Sputum	11	miR-31, 210	20	6	4	18
Shen et al. (2)	2014	USA	PCR	24/24	Sputum	11	miR-31, 210	18	4	6	20

FN, false negative; FP, false positive; LC, lung cancer; PCR, polymerase chain reaction; QUADAS, Quality Assessment of Diagnostic Accuracy Studies; TN, true negative; TP, true positive.

Quality of the selected studies

In the present meta-analysis, patients with lung cancer were confirmed based on a histopathological examination. The polymerase chain reaction (PCR) assay was used to determine miRNA expression profiles in the included studies. All the studies clearly declared that samples of lung cancer patients were collected before any therapy. Of the 12 included studies, 9 had a higher QUADAS score (≥10). Table 1 shows the result of the QUADAS assessment.

Diagnostic accuracy

Figures 2 and 3 show the sensitivity and specificity of sputum miRNAs in detecting lung cancer, respectively. The point sensitivity for the pooled data was 0.70 (95% CI: 0.66-0.70) and showed no heterogeneity (p=0.487; χ²=10.49; I²=0.0%). The point specificity for the pooled data was 0.89 (95% CI: 0.86-0.91) and the area under the SROC (Fig. 4) was 0.83. The pooled DOR was 17.53 (95% CI: 12.22-25.14). The results corresponded to the PLR of 5.63 (95% CI: 4.14-7.66) and the NLR of 0.35 (95% CI: 0.30-0.40). Significant heterogeneity in specificity was found (p=0.042; χ²=20.29; I²=45.8%). To explore the potential source of heterogeneity, subgroup analyses were performed.

FIG. 2.

Forest plot of sensitivity of sputum miRNAs for the diagnosis of lung cancer. The point estimates of each study are shown as solid circles. Error bars represent 95% confidence intervals (95% CIs).

FIG. 3.

Forest plot of specificity of sputum miRNAs for the diagnosis of lung cancer. The point estimates of each study are shown as solid circles. Error bars represent 95% CIs.

FIG. 4.

Summary receiver operating characteristic curve (SROC) of sputum miRNAs for the diagnosis of lung cancer. Each solid circle in the SROC curve represents one study. The size of each solid circle indicates the sample size of each study.

Subgroup analysis

Based on miRNA profiles, we performed subgroup analysis (combined use of miR-31 and miR-210 vs. other miRNA profiles). No heterogeneity was observed among studies with combined use of miR-31 and miR-210 [specificity 0.89 (95% CI: 0.84-0.93); I²=0.0%; χ²=0.72; p=0.869]. However, noticeable heterogeneity was found among studies with using other miRNA profiles [specificity 0.89 (95% CI: 0.84-0.92); I²=64.2%; χ²=19.57; p=0.007]. Therefore, we identified miRNA profiles as a possible source of heterogeneity in sputum miRNAs.

Publication bias

To evaluate the potential publication bias, Deeks' funnel plot was designed using the log (DOR) of individual studies against their sample size. The funnel plot for sputum miRNAs is shown in Figure 5. In detail, no evidence of a small-study effect was observed for sputum miRNAs (p=0.34), indicating symmetry in the data and a low probability of publication bias.

FIG. 5

Deeks' funnel plot asymmetry test for the studies. Each circle represents one study. The size of each solid circle indicates the sample size of each study. The curves indicated a low likelihood of publication bias (p-value=0.34).

Discussion

The results of our analysis showed an overall moderate test performance of sputum miRNAs for diagnosis of lung cancer. The meta-analysis determined a point sensitivity of 0.70 (95% CI: 0.66-0.70) and specificity of 0.89 (95% CI: 0.86-0.91), respectively. Heterogeneity in specificity was noted (p=0.042; χ²=20.29; I²=45.8%). Subgroup analysis indicated that miRNA profiles might be a source of heterogeneity in the diagnostic specificity.

Sputum is one of the most easily accessible body fluids (Saccomanno et al., 1965; Thunnissen, 2003). It has been demonstrated that endogenous miRNAs were stably present in the sputum and could reliably be measured by PCR (Xie et al., 2010). In recent years, several studies reported that sputum miRNAs could be used as noninvasive biomarkers for diagnosis of lung cancer. However, the results were inconsistent among the studies. This meta-analysis showed an overall moderate test performance as evaluated by the area under the SROC (0.83). The pooled specificity was favorable at 89% in comparison with reported summary results for CT of 0.61 (Aberle et al., 2011). However, the sensitivity of sputum miRNAs (0.70) was still not yet efficient for routine clinical practice. In addition, it should be noted that the miRNAs could not indicate the difference between early and advanced stages of lung tumors. Moreover, it is difficult for sputum-based biomarkers to localize tumor masses in the lungs. This limitation of sputum miRNAs can be overcome by CT, which can provide structural information and identify lung cancer at a small size (Bach et al., 2007; Welch and Black, 2010). The combined use of the miRNA biomarkers and CT can potentially augment CT specificity for lung cancer diagnosis, reflecting that the sputum miRNAs could be potentially useful in improving CT for diagnosis of lung cancer.

Subgroup analysis suggested that the diagnostic specificity of sputum miRNAs might be influenced by miRNA profiles. This result was in line with the previous report that detection of a combination of different miRNAs might be a better predictor, with a higher diagnostic efficiency, when compared with that of a single miRNA (Yu et al., 2010). Consequently, it was necessary to identify the microRNA combination in sputum that could provide an optimal diagnostic accuracy. We also used funnel plot to detect potential publication bias, and it showed a low likelihood of publication bias.

There were limitations in the meta-analysis. First, the selected studies had limited methodological quality. For most studies, the spectrum of patients did not represent the patients who received the test in practice. This may lead to heterogeneity in specificity among studies. Second, some articles included in our meta-analysis did not describe a number of important characteristics influencing the performance of miRNAs, such as the stage of lung cancer and the smoking status of patients. Third, some studies recruited healthy controls while others did not, this may result in a great variance among the control group. Due to insufficient data for the controls, we could not investigate the effect caused by the makeup of the controls.

In conclusion, this meta-analysis was the first report that assessed the overall diagnostic value of sputum miRNAs, which are the widely studied noninvasive diagnostic methods used for detecting lung cancer. Sputum miRNAs are potentially useful noninvasive biomarkers for diagnosis of lung cancer and the diagnostic specificity may be influenced by miRNA profiles. It would be important for future work to evaluate the generalizability of our results by methodologically rigorous studies on more clearly defined patient populations.

Footnotes

Acknowledgments

This study was financially supported by the Presidential Foundation of the School of Public Health and Tropical Medicine, Southern Medical University, China (No. GW201225).

Author Disclosure Statement

No competing financial interests exist.

References

Aberle

, Adams

, Berg

, et al. (2011) Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med, 365:395-409.

Anjuman

, Li

, Guarnera

, et al. (2013) Evaluation of lung flute in sputum samples for molecular analysis of lung cancer. Clin Transl Med, 2:15

Bach

, Jett

, Pastorino

, et al. (2007) Computed tomography screening and lung cancer outcomes. JAMA, 9:953-961.

Deeks

, Macaskill

, Irwig

(2005) The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol, 9:882-893.

Fleiss

, Levin

, Paik

(2003) The comparison of proportions from several independent samples: estimation of the marginal mean proportion. In: Fleiss

, Levin

, Paik

(eds) Statistical Methods for Rates and Proportions. 3rd edn. John Wiley and Sons, New Jersey, pp. 212-217.

Frost

, Ball

Jr ., Levin

, et al. (1984) Early lung cancer detection: results of the initial (prevalence) radiologic and cytologic screening in the Johns Hopkins study. Am Rev Respir Dis, 4:549-554.

Huedo-Medina

, Sanchez-Meca

, Marin-Martinez

, et al. (2006) Assessing heterogeneity in meta-analysis: Q statistic or I² index?. Psychol Methods, 11:193-206.

, Ma

, Guarnera

, et al. (2014) Digital PCR quantification of miRNAs in sputum for diagnosis of lung cancer. J Cancer Res Clin Oncol, 140:145-150.

, Yang

, Zhang

, et al. (2010) Meta-analysis of serum tumor markers in lung cancer. Zhongguo Fei Ai Za Zhi, 12:1136-1140.

10.

Parkin

, Bray

, Ferlay

, et al. (2005) Global cancer statistics, 2002. CA Cancer J Clin, 2:74-108.

11.

Pritchard

, Cheng

, Tewari

(2012) MicroRNA profiling: approaches and considerations. Nat Rev Genet, 5:358-369.

12.

Roa

, Kim

, Razzak

, et al. (2012) Sputum microRNA profiling: a novel approach for the early detection of non-small cell lung cancer. Clin Invest Med, 5:e271

13.

Saccomanno

, Saunders

, Archer

, et al. (1965) Cancer of the lung: the cytology of sputum prior to the development of carcinoma. Acta Cytol, 9:413-423.

14.

Shen

, Liao

, Guarnera

, et al. (2014) Analysis of microRNAs in sputum to improve computed tomography for lung cancer diagnosis. J Thorac Oncol, 9:33-40.

15.

Shen

, Wang

, Yang

, et al. (2013) Diagnostic value of circulating microRNAs for lung cancer: a meta-analysis. Genet Test Mol Biomarkers, 5:359-366.

16.

Thunnissen

(2003) Sputum examination for early detection of lung cancer. J Clin Pathol, 56:805-810.

17.

Vosa

, Vooder

, Kolde

, et al. (2013) Meta-analysis of microRNA expression in lung cancer. Int J Cancer, 12:2884-2893.

18.

Wang

, Nelson

, Bogardus

, et al. (2010) Five-year lung cancer survival: which advanced stage nonsmall cell lung cancer patients attain longterm survival?. Cancer, 6:1518-1525.

19.

Welch

, Black

(2010) Over diagnosis in cancer. J Natl Cancer Inst, 9:605-613.

20.

Whiting

, Rutjes

, Reitsma

, et al. (2003) The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol, 3:25

21.

Xie

, Todd

, Liu

, et al. (2010) Altered miRNA expression in sputum for diagnosis of non-small cell lung cancer. Lung Cancer, 2:170-176.

22.

Xing

, Todd

, Yu

, et al. (2010) Early detection of squamous cell lung cancer in sputum by a panel of microRNA markers. Mod Pathol, 8:1157-1164.

23.

Yang

, Zhang

, Sun

, et al. (2013) Diagnostic value of the detection of microRNAs in sputum of patients with non-small lung cancer. J Clin Pulm Med, 2:226-228.

24.

, Todd

, Xing

, et al. (2010) Early detection of lung adenocarcinoma in sputum by a panel of microRNA markers. Int J Cancer, 12:2870-2878.

25.

Zamora

, Abraira

, Muriel

, et al. (2006) Meta-DiSc: a software for meta-analysis of test accuracy data. BMC Med Res Methodol, 6:31