MicroRNAs in Human Cerebrospinal Fluid as Biomarkers for Alzheimer’s Disease

Subjects

The Institutional Review Board of Oregon Health & Science University (OHSU) approved all of the donor procedures (IRB 6845); all subjects provided written informed consent. Participants underwent detailed clinical and laboratory evaluation, including cognitive testing and interview with a collateral historian. The samples were banked at the Oregon Alzheimer’s Disease Center (OADC), the core program of the Layton Aging & Alzheimer’s Disease Center, supported by the National Institute onAging.

CSF sample collection

The OADC has standardized their CSF collection protocol to correspond to that used in other AD centers engaged in CSF biomarker research [17]. All CSF examinations are done in the morning under fasting conditions, in the lateral decubitus position, with a 24-gauge Sprotte spinal needle that minimizes the discomfort of the procedure and reduces the incidence of lumbar puncture headache. The first 3 to 5 mL of CSF collected is sent to the clinical lab for cell count, and determination of glucose and total protein levels. Next, serial syringes of 5 mL CSF are collected, mixed, transferred to polypropylene tubes in 0.5 mL aliquots, and the tubes numbered to account for any gradient effect in subsequent experiments. All CSF tubes have an OADC subject number, but no other identifying information, in order to facilitate later collaborations and sharing of samples. Immediately after aliquots are transferred, the CSF is frozen on dry ice and stored in a –80°C freezer.

Apolipoprotein E genotyping

DNA was isolated from blood and amplified by Touchdown PCR with 250 μM dNTPs, 1 Unit Taq DNA Polymerase, buffer, 1X Q-solution (Qiagen), and 0.5 μM forward and reverse primers for ApoE exon 4 (E4 allele). A product size of 443 nucleotides identified on a 1% agarose 1X TBE gel was excised, cleaned with ExoSAP-IT reagent (Affymetrix), and sequenced on a model 377 automated fluorescence sequencer (Applied Biosystems). Chromatogram traces were examined and nucleotide sequences determined using FinchTV (Geospiza, Inc.).

RNA isolation and amplification

Total RNA was extracted from 0.5 mL of each CSF sample using the mirVana™ PARIS™ RNA and Native Protein Purification Kit (ThermoFisher Scientific), modified to include two aqueous extractions during the organic phase extraction steps in order to maximize RNA recovery [13]. RNA samples were concentrated using the RNA Clean & Concentrator™-5 Kit (Zymo Research) and eluted in 9 μL RNAse/DNase-free water. RNA concentrations were initially measured on a set of test CSF samples using the Quant-iT™ RiboGreen^® RNA Assay Kit (ThermoFisher Scientific). The average concentration for the test group was 133 pg/ μL with a total RNA recovery of approximately 2 ng/mL CSF. The concentrated RNA samples were converted to cDNA and pre-amplified using a T-100 thermocycler (Bio-Rad, Hercules, CA) with Megaplex™ RT Primers, Human Pool Set v3.0, as per the manufacturer’s protocol (“Megaplex Pools For microRNA Expression Analysis”), following instructions for detection of miRNA with pre-amplification. The pre-amplification products were diluted into a prescribed final volume of 100 μL and stored at –20°C until ready for the final detection PCR reactions. Real-time PCR reactions followed the manufacturer’s protocol, using 18 μL of diluted pre-amplificationproduct.

MicroRNA qRT-PCR arrays

The expression profile of miRNAs in CSF samples was determined using the TaqMan^® Array Human MicroRNA A + B Cards Set v3.0 (ThermoFisher). The arrays consist of two cards (A and B), each containing a total of 384 TaqMan^® MicroRNA Assays per card, including potential endogenous control RNAs (U6 snRNA, RNU44, RNU48). For these arrays, there is an n = 1 technical replicate for each RNA probe. The qRT-PCR amplifications and data acquisition were performed on a QuantStudio™ 12K Flex Real-Time PCR System (ThermoFisher) using automated baseline and threshold values determined by Expression Suite™ software v1.2.2 (ThermoFisher). Amplification data were imported into ExpressionSuite and cycle time (Ct) values were calculated. The expression data were analyzed as separate batches according to the TLDA card lot number. The Ct value for each well was reported along with the amplification score (AmpScore) metric. Quality control filtering of the Ct values consisted of the following steps: i) PCR products were considered below the detection threshold and censored if Ct ≥36 or if Expression Suite reported the Ct value as “Undetected”; ii) individual assays were excluded if AmpScore <0.9; iii) all other Ct values were accepted as reported by Expression Suite. RNU44 and RNU48 detection was inconsistent across all cards and samples, and excluded from further analysis. U6 snRNA detection was consistent within cards, across cards, and across samples, making it a strong candidate for quality assessment and normalization strategies (see below).

All qRT-PCR data and donor-specific metadata have been submitted to the exRNA Atlas [18] repository and will be available following publication of results. This PCR dataset with title “TLDA miRNA Screen for Candidate AD Biomarkers in CSF from Living Donors” and accession number “EXR-JSAUG1UH2001-AN” can be accessed via the Datasets link in the menubar at http://exrna-atlas.org/.

Statistical analysis of miRNA expression in control and AD CSF

Initial analysis of the TaqMan^® data was performed using a default statistical analysis and routine procedures for qRT-PCR data processing. To improve identification of robust and reproducible miRNA candidates, we developed a stringent analytical approach which involved passing the complete miRNA data set through multiple quality and performance filters in order to hone in on candidate AD miRNAs that: i) were frequently expressed within the dynamic range of qRT-PCR measurements for CSF samples; ii) whose Ct values trend strongly with the AD condition; iii) were able to discriminate AD from control at the patient level. Note that built into this analytic approach were rigorous internal validation procedures designed to more accurately estimate classification performance of the top miRNA candidates prior to measuring them in independent cohorts.

Test (i) was addressed by thorough data filtering using ExpressionSuite error flags. After exclusion of technical failures, we further excluded miRNAs with low (< ∼20%) detection rates in samples in order to restrict focus to targets with adequate biomarker potential. Although the amplification experiments for this study were allowed to run to 40 cycles before termination, empirical data-quality metrics indicated that virtually no amplifications occurred at or after 36 cycles, so we censored the raw Ct values at a threshold of 36; thus, censored values may be viewed as miRNA expression below a lower limit of detection. Ct values were then normalized relative to the U6 snRNA reference and transformed onto an expression scale by taking the multiplicative inverse (i.e., 1 divided by the value) and scaling by the maximum value so that higher values indicate relatively greater quantities of miRNA expression; censored values were assigned a value of zero on this expression scale. 151 candidate miRNAs passed the initial quality filters and proceeded on to further analysis.

To address tests (ii) and (iii), we performed a four-step battery of inferential test procedures. In order to mitigate inferential bias due to testing method, we imposed the rule that a candidate miRNA would not be carried forward unless it had passed at least two of the four steps. The testing was performed using Stata Release 14 (http://www.stata.com/) and R Version 3.2.3 (https://cran.r-project.org/) software.

The first two steps employed log-rank testing of normalized Ct values and predictions from univariate logistic regression on the transformed expression values using models that accounted for censoring. The goal in these phases was to select candidate miRNAs according to their strength of association with AD status, and selection cutoffs were chosen to achieve an average false discovery rate (FDR) of  30% under correction for multiple comparisons. This FDR heuristic was chosen to reflect our anticipation that selected candidates would, on average, be about twice as likely to be associated as not to be associated with AD status.

The last two steps of testing were designed to find candidate miRNA markers whose expression profiles would provide a reliably clear signal of AD or non-AD status across the observed range of total miRNA expression contexts; the goal in these phases was to gauge the relative predictive importance of each miRNA while accounting for interactions with the other miRNAs. To test the signal strength, we repeatedly sampled random groups of miRNAs and assessed their ability to classify the CSF samples into “AD” and “non-AD” bins. If the classification does worse when a certain miRNA is excluded from the mix, that miRNA is probably contributing somesignal, and is viewed as relatively more important than other miRNAs. For the testing we used two different random-forest classifiers [19]— one based on conditional inference trees [20] and the other based on CHAID trees [21]— in order to make the selection process more robust to the choice of underlying classification method; selection in both phases was based on an appropriate method-specific ‘importance score’ that could be calculated for each classifier.

Upon completion of these tests, we identified 36 candidate miRNAs that passed at least two of the four steps, and this set was carried forward for the final evaluation step: multimarker classification performance, which was assessed by evaluating linear combinations of all possible subsets of the top miRNA biomarker candidates via best-subsets logistic regression, employing multiple imputation of missing values. Estimated inclusion probabilities for each miRNA and computed area under the ROC curves (AUC) [22] fit from each model were calculated and used to rank targets.

Prospective assessment of candidate miRNAs

To get a more accurate estimate of the prospective performance of the top 36 candidates from our rigorous analytic pipeline, we implemented a repeated leave-one-out (LOO) procedure [23] to internally validate our findings. This is a standard method for projecting what the performance of a classifier would be in a new cohort; it attempts to ascertain how well the 36 candidates would replicate in an independent cohort. We applied this procedure in a repeated manner to the m = 30 MI datasets previously generated for multimarker classification performance evaluation in the final stage of selection testing (see above). Since each MI dataset is a simulation of the total set of miRNA expression values on these 36 candidates that could have been obtained from a cohort drawn from the same population as ours, each can thus be viewed as a small perturbation of our observed data. On each of these MI datasets we performed a LOO nonparametric nearest-neighbor discriminant analysis (using the three nearest neighbors by Canberra distance on Mahalanobis-normalized expression values [23]), where each subject is classified as AD or non-AD in turn by leaving that subject out of the calculation of the discrimination rule and then using the resulting rule, based on the kept-in subjects only, to classify the left-out subject. Averaging over the MI replicates using the standard combining procedure for multiple-imputation (MI) [24] estimation to the LOO sensitivity and specificity estimates calculated for each MI dataset, we obtain an improved estimate of the projected sensitivity and specificity of our set of 36 miRNA biomarker candidates if applied to a new cohort. This provides an internally validated estimate of the discriminating ability of this set ofmarkers.

Replication study of candidate miRNAs

Reproducibility of phenotype associations under similar measurement technology platforms is a necessary feature for any proposed biomarker. In order to assess the reproducibility of our initial findings on retest, we generated custom TaqMan^® miRNA assays to examine the expression of candidate AD miRNAs in a subset of 16 AD and 16 control CSF samples that were analyzed in the original array assays. This subset of CSF samples was chosen based on the availability of duplicate sample aliquots from the OADC bank, and the ability to include 32 CSF samples (representing 1/3 of the original CSF samples) for these test/retest reliability studies. The custom arrays included assays for 46 candidate miRNAs (original analysis), 6 miRNAs not changed between AD and control CSF, 4 miRNAs not detected in CSF, and U6 snRNA. All miRNA assays were plated in triplicate to allow for analysis of n = 3 technical replicates/RNA in the replication study. Only 20 of the 36 candidate miRNAs selected by the four-step statistical evaluation were included in this study, for reasons we discuss in more detail below (see Replicationstudies).

Donor characteristics

All donor characteristics for the 50 AD and 49 control subjects considered in this phase of the biomarker study, including sex, age at CSF collection, Mini-Mental State Examination (MMSE), and ApoE genotype, are shown in Table 1. The 49 control subjects were community volunteers in good health, with MMSE [25] scores between 28 and 30 (29.22±1.28), Clinical Dementia Rating scores of 0, and no evidence or history of cognitive or functional decline [26, 27]. The 50 AD participants were recruited from OHSU Neurology clinics and diagnosed with probable AD according to ADRDA-NINDS criteria [28, 29] at a consensus conference of OADC clinicians, with mean MMSE scores between 18 and 19 (18.28±6.4), and Clinical Dementia Rating scores of 1-2. ApoE genotyping was done for the 90/99 CSF donors who also donated blood samples (44 control and 46 AD). The genotype revealed that of the control group, 54.55% had zero, 38.64% had one, and 6.82% had two ApoE4 alleles. In the AD group 17.39% had zero, 58.70% had one, and 23.91% had two ApoE4 alleles, so that the E4 genotype was over-represented in AD as expected [30].

Identification of candidate miRNAs that discriminate AD from control CSF

The TaqMan^® miRNA arrays were assessed for uniformity of miRNA expression within the AD group, and within the control group, by correlating the expression levels for each individual CSF sample assay to the average expression for entire sample group. The miRNA expression from one sample was compared to the miRNA expression for all samples in the same group to show the correlation between one individual assay relative to the average of the miRNA assay across each group for control and AD (data not shown). These data showed that miRNA expression is consistent within each group (control versus AD) and that regulation of miRNAs is not random in individual samples.

For a miRNA to be included in the statistical analysis, it had to be present in at least 20% of all 99 CSF donor samples, which resulted in 151 miRNA candidates chosen for analysis. The discriminating abilities of the 151 miRNAs that passed our PCR quality control filtering were assessed using four statistical testing steps. The log-rank and ROC AUC tests assess the group differences and discriminating ability of each miRNA individually, while the two random forest tests (CART and CHAID) evaluate the classification performance of each miRNA as a member of a group. The p-values for the log-rank tests, AUC values from the ROC tests, and the relative importance ranks from the CART and CHAID tests are presented in Table 2. We summarized these results in a “Multitest Score” for each miRNA that indicates the number of steps where the miRNA was selected as a strong performer. MiRNA that performed well in at least two statistical tests (Multitest score = 2, 3, or 4) were considered top biomarker candidates. We also include for each candidate the fraction of samples in control and AD groups with detectable levels of that miRNA (% Detected), along with the log fold change and 95% confidence interval (CI) of the detected values in AD versus control samples.

Linear combinations of miRNAs increase sensitivity/specificity

We evaluated linear combinations of subsets of the 36 selected miRNAs via best-subsets logistic regression (employing multiple imputation of values missing due to technical dropout), and computed AUC to assess multimarker classification performance. Top-performing linear combinations of 3 and 4 miRNAs attain AUC of 0.80–0.82 (Fig. 1). Addition of ApoE genotype status to miRNAs further increased sensitivity/specificity as an AD biomarker (Fig. 2). ApoE alone has an AUC of 0.73 in this group ofsubjects, and 3 miRNAs had an AUC of 0.80. However, 3 miRNAs and ApoE genotype increase the AUC to 0.84. Thus, a combination of new (miRNA) and a reference (ApoE) biomarker increased classification performance.

Prospective discriminating ability of miRNA candidates

Estimates of classification performance obtained in discovery datasets are often too optimistic about real-world classification performance on new test cases because models trained on a particular set of data sometimes tune themselves too finely to that data, capturing idiosyncratic features that may not be reflective of typical features in the target population. A standard means of improving the accuracy of prospective classification assessment is a leave-one-out (LOO) classification procedure, whereby one data point is left out of the model estimation and the resulting model, based on the rest of the data, is used to guess the class of the left-out point. This procedure is repeated for each data point in turn, and the overall accuracy of the guesses is taken as the estimate of what the discriminating ability would be in a new dataset. Having generated m = 30 multiple-imputation (MI) datasets in a previous analysis step, we were able to repeat the LOO procedure in all 30 MI datasets and average the estimates using the standard combining rules to obtain an estimate that is not only validated internally but also robust to likely patterns of missing values because the variance across the MI datasets simulates the variance caused by having missing values. The MI-averaged estimates of LOO performance are 60% sensitivity at 73% specificity. We plotted this projected point of sensitivity and specificity for AD discrimination by our set of 36 candidates, along with a 95% joint confidence region, in Fig. 3. We also extrapolated a plausible ROC curve passing through that point, assuming a typical monotonic shape for the curve, and calculated the projected AUC (0.72) that would be implied by the curve. From Fig. 3, we can see that not only is our set of candidates projected to discriminate better than blind guessing with good confidence, but also that the performance is at least on par with ApoE, and possibly much better.

Replication studies

Our original (what we call a “default”) analysis of the 754 miRNAs was based on simple statistical criteria such as significance values, and led to a liberal selection of 46 candidates (only partially overlapping with our current set of 36). During this process we recognized the need for more robust procedures, including better data filtering and more stringent selection criteria, but also needed to move forward with verification testing while our more rigorous pipeline was being developed. Taking the 46 identifications as our best candidates thus far, we attempted to reproduce the observed associations with AD using custom TaqMan^® arrays to measure differential expression of miRNAs in AD versus control CSF. The original set of 99 CSF samples had been run on TLDAs with n = 1 technical replicate/RNA probe, while the custom TLDAs were designed with n = 3 technical replicates/RNA probe using 32 of the 99 CSF samples (16 AD versus 16 control). These 32 CSF samples were identical sample aliquots from the original 99 CSF samples in the OADC bank, and represent 1/3 of the original CSF samples. We found that 20 of the original 46 candidates held up as expected and 26 did not (Fig. 4). After our more rigorous analytical pipeline, which included improved raw data processing and quality control as well as a stronger battery of statistical selection procedures, was completed, we used it to reanalyze the entire miRNA set (Fig. 4). The new pipeline selected 36 miRNAs that included the same 20 that had replicated and, interestingly, rejected all of the failed 26, mostly on data quality grounds. This demonstrates the need for strong data quality filtering procedures in biomarker selection pipelines. Figure 4 shows a flowchart of the original analysis and the stringent statistical reanalysis, and the overlap between the finding from both that strongly supports the inclusion of the top candidates in the validation phase: all 20 of the 20 targets from the original analysis that held up in the verification study would also have been selected for replication by the stringent statistical reanalysis.