Reproducibility and variability of quantitative magnetic resonance imaging markers in cerebral small vessel disease

Abstract

Brain imaging is essential for the diagnosis and characterization of cerebral small vessel disease. Several magnetic resonance imaging markers have therefore emerged, providing new information on the diagnosis, progression, and mechanisms of small vessel disease. Yet, the reproducibility of these small vessel disease markers has received little attention despite being widely used in cross-sectional and longitudinal studies. This review focuses on the main small vessel disease-related markers on magnetic resonance imaging including: white matter hyperintensities, lacunes, dilated perivascular spaces, microbleeds, and brain volume. The aim is to summarize, for each marker, what is currently known about: (1) its reproducibility in studies with a scan–rescan procedure either in single or multicenter settings; (2) the acquisition-related sources of variability; and, (3) the techniques used to minimize this variability. Based on the results, we discuss technical and other challenges that need to be overcome in order for these markers to be reliably used as outcome measures in future clinical trials. We also highlight the key points that need to be considered when designing multicenter magnetic resonance imaging studies of small vessel disease.

Keywords

Magnetic resonance imaging cerebral small vessel disease marker brain volume atrophy white matter hyperintensities microbleeds lacunes perivascular spaces reproducibility repeatability variability

Introduction

Cerebral small vessel disease (SVD) is a major source of cognitive impairment and is the second most common cause of dementia in older aged people.¹ SVD is typically sporadic and related to aging and vascular risk factors, but can also be caused by genetic conditions.² The most frequent sporadic forms of SVD are arteriolosclerosis and cerebral amyloid angiopathy (CAA). A significant proportion of patients with SVD has associated neurodegenerative diseases such as Alzheimer’s disease (AD), which can lead to a mixed dementia.³ In therapeutic trials for dementias, surrogate imaging markers of both pathophysiological processes should ideally be used to account for the potential mixed origin.

Pathological features of SVD are heterogeneous and out of reach of current in vivo imaging techniques. Nonetheless, parenchymal lesions presumed to be caused by small vessel changes have been adopted as markers of SVD, giving neuroimaging a central role for diagnosis and characterization.⁴ The spectrum of brain magnetic resonance imaging (MRI) manifestations of SVD is wide, ranging from minor white matter hyperintensities (WMH) often seen in population-based studies to numerous subcortical lesions. Indeed, MRI using various sequences allows an evaluation of several imaging markers related to SVD, including recent small subcortical infarcts, WMH, lacunes, enlarged perivascular spaces (PVS), cerebral microbleeds (CMB), and brain atrophy, as summarized by an international working group defining neuroimaging standards (STRIVE) for research into SVD.⁵

The uncertain or lower reproducibility of these MRI markers, especially across centers, is a major concern in cross-sectional and longitudinal observational studies as well as in clinical trials. Indeed, a key challenge is to minimize the variability caused by technological and subject-related factors. Such variability may confound the detection of disease- or treatment-related MRI changes, thereby limiting the power to detect and follow the progression of imaging markers of SVD. From the patient being imaged to the final imaging marker of interest, which is quantified by an expert rater or an automatic image-processing algorithm, there are several sources of variability. These can include subject-related physiological changes or instrument-related factors such as field strength, head coil, gradients, and sequence parameters (see Figure 1). Each can affect the reproducibility of the marker. Additional variation may be introduced at the image post-processing stage, depending on which software is used for inhomogeneities correction or tissue segmentation, for example. The definition of lesions and the interpretation of images by several experts may be other sources of variability. The measurement of each MRI marker therefore depends on several factors, the effects of which need to be minimized, corrected, or taken into account in a posteriori analyses for studies to be comparable.

Figure 1.

Schematic view of the different sources of variability in a multicenter MRI study (adapted from www.imaios.com).

The aims of this review are to summarize current knowledge regarding the reproducibility and variability of quantitative imaging markers of SVD including WMH, lacunes, PVS, CMB, and brain volume in both single center and multicenter contexts. To highlight technical solutions for common problems and to identify the remaining challenges that need to be overcome for such markers to be considered valid and reliable. Other SVD-related MRI markers, due to their acute manifestation (recent small subcortical infarct) or their so far non-quantitative assessment (cortical superficial siderosis, cortical microinfarcts) have not been examined in reproducibility studies to our knowledge and hence were not included in the literature search. Post-processing tools or expert ratings that also contribute to the global variability and accuracy of the final marker measurement are beyond the scope of this review. For each MRI marker of SVD, we review: (1) its reproducibility in studies with a scan–rescan design in single or across multiple centers; (2) the acquisition-related sources of variability; and, (3) the techniques that can be used to minimize methodological limitations. We also discuss those factors that need to be considered to deal with image variability inherent in MRI markers in multicenter design studies such as clinical trials or longitudinal studies.

Methods

Search strategy and selection criteria

This review was designed to focus on the reproducibility and variability of well-defined MRI markers of SVD: WMH, lacunes, PVS, CMB, and brain volume.

For each biomarker, taking into account the vocabulary heterogeneity of each marker (see supplementary material), a PubMed search has been conducted up to 14 August 2015 with the associated terms “MRI” AND (“variability” OR “reproducibility” OR “reliability” OR “consistency”). Papers published before 2002 were not considered to focus on recent MR technology. Reviews, conference abstracts, and articles not written in English were excluded from analysis. Articles from our personal databases were also incorporated to the search. To be included in the reproducibility summary (Table 1), studies had to include subjects who underwent a scan–rescan procedure either at the same center or/and at different centers. Filtering was firstly based on review of the title and abstract for relevance. Studies were not mandatorily related to SVD (e.g. multiple sclerosis or AD) to include the largest number of studies about variability and reliability for the MRI markers of interest.

Table 1.

Summary of reproducibility studies (within-centre or/and between-center scan–rescan procedure).

Study		Marker of interest	Between-center		Within-center		Magnetic field	Analysis tool	Between-center reproducibility	Within-center reproducibility
First author	Year published	Marker of interest	Nb subjects rescanned	Nb of MRI sites	Nb subjects rescanned	Nb rescans per subject	Magnetic field	Analysis tool	Between-center reproducibility	Within-center reproducibility
Brain volumetry
Schnack	2004	ICV, BV, GMV, WMV	6	4	5	2	1.5T, 1T	Own segmentation	BV: ICC = 0.98	BV: ICC = 1
Ewers	2006	GMV, WMV, CSFV	1	10	NA	NA	1.5T	SPM2	CoV=5% for all volumes	NA
Jovicich	2009	ICV	15	3	15	2	1.5T, 3T	Freesurfer	NA	Reproducibility error=2.56%
Reig	2009	ICV, GMV, WMV, CSFV	5	5	NA	NA	1.5T	SPM-based	ICV: CoV=1.9%, ICC=0.99	NA
Wonderlick	2009	BV, GMV, WMV	11	2	NA	NA	3T	Freesurfer	BV: ICC=0.99	NA
de Boer	2010	BV, GMV, WMV, CSFV	NA	NA	30	2	1.5T	FSL, SPM5, kNN	NA	BV: CoV=0.39% with FSL, CoV=0.72% with SPM5
Huppertz	2010	BV, GMV, WMV, CSFV, ICV	1	6	1	3	1.5T, 3T	SPM5	BV: CoV=3.78%; ICV: CoV=2.97%	BV: CoV=0.5%; ICV: CoV=0.6%
Kruggel	2010	ICV, BV, GMV, WMV	172	59	41	2	1.5T, 3T	Fantasm	BV: median RAVD=4.76%, 90th percentile=12.34%; ICV: median RAVD=1.66%, 90th percentile=3.92%	BV: median RAVD=0.30%, 90th percentile RAVD=1.09%; ICV: median RAVD=0.29%, 90th percentile RAVD=1.27%
Sampat	2010	BPF	NA	NA	18	2	1.5T	Own segmentation	NA	BPF: CoV=0.46%
Cover	2011	Normalized brain volume	NA	NA	671	2	1.5T	SIENAX	NA	BV: median RAVD=0.96%, 90th percentile RAVD=5.11%
Shokouhi (1.5T)	2011	ICV, GMV, WMV, CSFV	13	3	13	2	1.5T	BrainVISA	BV: reliability^a=0.97	BV: reliability=0.99
Shokouhi (3T)	2011	ICV, GMV, WMV, CSFV	11	5	11	2	3T	BrainVISA	BV: reliability=0.88	BV: reliability=0.96
Landman	2011	GMW, WMV	NA	NA	21	2	3T	Java imaging science toolkit	NA	GMV: CoV=2.6%; WMV: CoV=1.5%
Chalavi	2012	GMW, WMV	3	2	3	2	3T	Freesurfer, SPM	NA	NA
Huang	2012	WMV, CSFV	6	2	NA	NA	3T	FSL	WMV: ICC=0.62; CSFV: ICC=0.87	NA
Pfefferbaum	2012	Supratentorial volume	114	2	NA	NA	1.5T, 3T	Freesurfer	BV: ICC=0.81	NA
Brain atrophy
Smith	2007	PBVC	NA	NA	68	2	1.5T	SIENA, BSI	NA	SIENA: median RAVD=0.16%; BSI: median RAVD=0.17%
De Bresser	2011	PBVC	NA	NA	10	2	1.5T	SIENA, SPM8, kNN	NA	Coefficient of repeatability^b=0.92% for SIENA, 2.64% for SPM8, 0.31% for kNN
Cover	2011	PBVC	NA	NA	385	2	1.5T	SIENA	NA	PBVC: median RAVD=0.35%, 90th percentile RAVD=1.37%
Durand-dubief	2012	Atrophy	9	2	NA	NA	1.5T	7 segmentation algorithms	SEM: 1.77 for Freesurfer, 0.98 for BSI, 0.85 for SIENA	NA
White-matter hyperintensities
Wei	2002	WMH volume	NA	NA	20	2	1.5T	3 pipelines: PVEC, TDS, TDS+	NA	CoV range: 2.6–7.5% according to pipeline
Sicotte	2003	WMH volume	25	2	NA	NA	1.5T, 3T	threshold-based segmentation	Median RAVD: 12.5%	NA
de Boer	2010	WMH volume	NA	NA	30	2	1.5T	auto trained kNN	NA	CoV: 5.87%
Theysohn	2011	WMH grading	20	2	NA	NA	1.5T, 7T		75% of subjects with equal grading at 7T compared to 1.5T, 13% superior and 13% inferior	NA
Ramirez	2013	WMH volume (total subcortical, deep white, periventricular)	NA	NA	20	2	1.5T	Lesion Explorer	NA	ICC=1; mean AVD in 10 older subjects=213 mm³
Cerebral microbleeds
Stehling	2008	CMB number	25	2	NA	NA	1.5T, 3T		More CMB at 3T versus 1.5T	NA
Nandigam	2009	CMB contrast index and diameter	4	2	NA	NA	1.5T, 3T		Smaller contrast index and diameter at 1.5T versus 3T	NA
Conijn	2011	CMB number	34	2	NA	NA	1.5T, 7T		More CMB at 7T versus 1.5T	NA
Theysohn	2011	CMB number	20	2	NA	NA	1.5T, 7T		More CMB at 7T versus 1.5T	NA

ICV: intracranial volume; BV: brain volume (GMV + WMV); GMV: gray matter volume; WMV: white matter volume; CSFV: cerebrospinal volume; BPF: brain parenchymal fraction; WMH: white matter hyperintensities; CMB: cerebral microbleeds; ICC: Intraclass correlation coefficient; CoV: coefficient of variation; NA: not available; Reproducibility error: The group reproducibility error for each structure is derived averaging the reproducibility errors across subjects, where for each subject the error is estimated as the absolute test–retest volume percent change relative to the mean test–retest volume; kNN: k-nearest-neighbors segmentation method; RAVD: relative absolute value of the difference; PBVC: percent brain volume change.

Reliability in this study has been computed as the ratio of the variance excluding the contribution from the factor (within-center or between-center) to the total variance.

Coefficient of repeatability: 1.96 times the standard deviation of the differences between two measurements (Bland–Altman), SEM: standard error of measurement, AVD: absolute value of the difference.

Term and metrics definitions

As the development and implementation of quantitative imaging biomarkers has been hampered in part by inconsistent use of terminology, an interdisciplinary group of radiologists, statisticians, physicists, and other researchers worked to develop a comprehensive terminology to serve as a foundation for quantitative imaging biomarker claims.⁶ Definitions agreed are reproduced in supplementary material.

In order to simplify the manuscript reading, we will only use the terms “within-center reproducibility” (strictly speaking, the repeatability) to report results from scan–rescan procedures in a single center; and “between-center reproducibility” to report results from multicenter studies.

Several metrics are used to measure variability or reproducibility across different studies. A brief description of each is included in supplementary material.

Results

Variation in the number of studies and in the evaluation of the reproducibility

Number of studies

There was large variation in the number of studies conducted for assessing the reproducibility and variability of each SVD MRI marker. Brain volume and brain atrophy measures have received the most attention (19 studies with within or/and between center repeated scans of the same subjects).^7–22 This is probably related to the significant imaging research efforts in AD community during the last decade in which cerebral atrophy was of primary concern. Particularly, the Alzheimer’s disease neuroimaging initiative (ADNI) permitted the most important studies on this aspect, including repeated examinations of the same subject on the same or on different scanners.^14,17 Only five studies were found that evaluated the within-center or between-center variability of WMH measures^13,23–26 (Table 1). Few studies examined the potential sources of variability of WMH volumes, despite a large number of image-processing techniques attempting to quantify them accurately. Finally, there were only four studies on CMB^25,27–29 and no study on the variability of PVS and lacunes. This probably reflects the absence of consensus until recently about radiological terminology and definitions as well as difficulties to automatically quantify these markers.³⁰ Indeed, computational segmentation of lacunes and PVS has not yet been validated in a specific study. Overall, the reproducibility of measure of longitudinal change, i.e. the evolution of a given marker as a function of time, has only been evaluated for brain volume change (brain atrophy) in four studies^9,17,19,22 and not for other markers.

Evaluation of reproducibility

Several studies with a scan–rescan procedure selected in this review included different measures of variability in volumetric measurements that were not consistent across studies: intraclass coefficients, coefficients of variation, standard deviations, mean absolute values of differences. Hence, most of these studies were not directly comparable (Table 1). This prevented a quantified comparison between the multiple factors that may influence the measurements. It would be valuable for the community to harmonize measures of reproducibility and make sure that all important aspects of reproducibility are included.

Reproducibility and variability of MRI markers

White-matter hyperintensities (WMH)

Definition and interest in SVD

From a radiological perspective, WMH of presumed vascular origin have a higher signal intensity compared to the normal-appearing white matter on proton-density (PD)-weighted, T2-weighted, and fluid-attenuated inversion recovery (FLAIR) sequences and may appear iso- or hypointense on T1-weighted images.⁵ It is widely accepted that WMH are important for clinical outcome, in terms of cognitive and functional impairment.^5,31 Many qualitative and quantitative techniques have been used to measure WMH. Qualitative approaches are based on visual rating scales completed by well-trained readers.^32–34 Quantitative techniques involve image-processing algorithms to obtain volumetric measures or spatial distribution of WMH lesions.^35–39 In this review, we only focused on quantitative techniques, as they offer a continuous measure that is a more reliable and sensitive alternative to visual rating scales.⁴⁰

Within-center reproducibility

Several studies found good to excellent reproducibility of WMH segmentation and volumetry for different computational image analysis methods using a scan–rescan procedure in a single site^13,23,26 (Table 1). For example, in the study recruiting the largest number of subjects, de Boer et al.¹³ assessed the within-center reproducibility of WMH segmentation by comparing computational segmentation in 30 subjects scanned twice within a short interval at 1.5T. They found with their best segmentation pipeline a mean (sd) WMH volume difference expressed in % of intracranial volume of 0.01% (± 0.05%) and a coefficient of variation less than 6%.

Between-center reproducibility (effect of various vendors, coils)

No report was found about between-center reproducibility of WMH measures in large cohorts evaluated in several centers and using the same magnetic field.

Magnetic field effect

It is not clear how the magnetic field strength impacts the measurement of WMH volume beyond the expected improved resolution at higher field. Indeed there are relatively few direct comparisons of different scanner field strengths in any situation.⁴¹ In 10 patients with SVD, Theysohn et al.²⁵ showed that FLAIR images acquired at 7T highlighted WMH known from 1.5T with comparable extent. However, WMH were not adequately quantified in this study as neither number nor volume was assessed. On the contrary, in patients with multiple sclerosis, it has been reported that 3T scans showed a mean 10% higher WMH volume compared to 1.5T scans in total lesion volume based on PD-weighted images.²⁴

Sequence effect

WMH detection and automated quantification usually rely on the FLAIR sequence, which produces T2-weighted images with cerebrospinal fluid suppression. It has been recently shown that 3D FLAIR sequence parameters could be optimized to increase lesion detection and identification in multiple sclerosis compared with default 3D FLAIR or 2D FLAIR (Figure 2).⁴² This approach might also be used in SVD. Indirectly, this study showed that MR parameters (echo time, inversion time) influence the tissue contrast between the normal white matter and white matter lesions on which the algorithms to segment WMH heavily depend.

Figure 2.

Enhanced lesion detection in 3D FLAIR sequence (courtesy of Polak et al.⁴²). Sagittal slices from (a, d) 3D FLAIRED (a FLAIR sequence optimized in order to enhance detection of lesions); (b, e) GE 3D FLAIR; and (c, f) 2D FLAIR. Note increased lesion detection in the frontal lobe (1) deep white matter (2) and juxtacortical (3) regions with 3D FLAIRED. Improved deep white matter lesion (4) detection and resolution in 3D FLAIRED compared to other sequences. Cerebellar lesion (5) detected in 3D FLAIRED and GE 3D FLAIR, but uncertainly discerned in 2D FLAIR.

Some pipelines have used an interleaved PD/T2 acquisition for WMH segmentation, especially as this dual spin echo sequence is more sensitive to WMH in the deep basal ganglia and thalamus than FLAIR and also allows better detection of PVS.^43,44 Multisequence image analysis approaches to quantify WMH, e.g. which combine FLAIR with T2* or T1 or T2 or several sequences together, are increasingly used and emphasize the need to obtain several key standard sequences when studying SVD.

Partial volume effects

Partial volume effect is the combination of signals from different tissue types in a single voxel whose image intensity is thus dependent on proportions of each tissue. Partial volume effects with clinically feasible resolutions complicate delineation of the lesion borders.⁴⁵ Particularly, smaller voxel sizes allow more accurate estimation and yield larger total lesion volumes, especially for patients with small lesions.⁴⁶ For example, in 10 patients with multiple sclerosis, lesion volumes were reported to be on average 9% greater with 3 mm compared to 5 mm slice thickness.⁴⁶

Other factors with some effects

The approach to post-processing for bias field (or B1 inhomogeneities) correction can have an important effect on WMH volume quantification according to the computational method used.⁴⁷

Reproducibility in longitudinal studies

No study was found to assess the reproducibility of any longitudinal measurement of WMH.

Lacunes and PVS

Definition and interest in SVD

From a radiological perspective, lacunes of presumed vascular origin are round or ovoid, subcortical, fluid-filled cavities of between 3 mm and 15 mm in diameter, consistent with a previous acute small subcortical infarct or hemorrhage in the territory of one perforating arteriole.⁵ Number, size, shape, and location are measures of interest for lacunes. To date, very few automatic image-processing methods exist to segment lacunes, and most studies rely on semi-automatic computational approaches or manual segmentations by trained observers.^37,48–51

PVS are fluid-filled spaces that follow the typical course of a small vessel as it goes through gray or white matter and have a signal intensity similar to CSF on all sequences.^5,52 Number, size, shape, and location are measures of interest for PVS. Rating methods have been developed to describe these markers.^53,54 To date, very few computational image-processing methods exist to segment PVS though some are in development.^37,55–57 The first image-processing methods for both lacunes and PVS detection were not fully robust and efficient, requiring time-consuming user manual intervention but new methods are more automated and much faster.

Within- or between-center reproducibility

The literature is very sparse concerning the reproducibility of measurements describing lacunes or PVS.⁵² In 20 subjects scanned twice on a 1.5T scanner, one study assessed a good reproducibility for lacunar volume with the used dedicated software but the extent or numbers of lacunes per subject were not specified.²⁶

High-resolution images should be more sensitive for detecting lacunes and PVS because they are small. Therefore, their volume assessment may heavily depend on spatial resolution and geometrical inaccuracies induced by gradient nonlinearities.

Reproducibility in longitudinal studies

No study was found to assess the reproducibility of any longitudinal measurement for both lacunes and PVS.

CMB

Definition and interest in SVD

From a radiological perspective, CMB associated with SVD are small hypointense lesions on paramagnetic sensitive MR sequences such as T2*-weighted gradient-echo (GE) sequence and are most frequently located in the cortico-subcortical junction, deep gray or white matter in the cerebral hemispheres, brainstem, and cerebellum.^5,58 CMB are associated with SVD and lobar CMB are often seen in patients in memory clinics who have CAA.^5,44,59,60 Most studies suggest that they are associated with impaired cognitive function,^61,62 although their direct clinical impact remains uncertain. Several rating scales have been designed to minimize observer variation^63,64 along with several post-processing algorithms for semi-automatic or automatic quantification.^65–69 We consider in this review all methods to study CMB as a few computational methods have been published and tested in reproducibility studies. Many MRI acquisition factors are expected to influence CMB detection as their size is a consequence of susceptibility effects that are highly dependent on MRI sequence parameters. In fact, hypointensities are typically larger than the physical size of the corresponding histopathological lesion (blooming effect).⁷⁰

Within- or between-center reproducibility

No study was found to assess CMB detection reproducibility through a scan–rescan procedure at the same center or between centers with the same imaging protocol. However, several studies examined various factors involved in CMB detection such as magnetic field strength or sequence parameters.

Magnetic field effect

Intravoxel dephasing due to susceptibility effects and associated signal loss in gradient-echo images increases with higher magnetic fields.⁷¹ Several studies hence demonstrated an increased number of CMB at higher field strength^25,27,29 with an increase in contrast-to-noise ratio^27,28 and CMB size^25,28 as illustrated in Figure 3. For example, mean increases from 1.5T to 3T were 48% for contrast-to-noise ratio and 12% for CMB size on 119 CMB detected in four subjects.⁷⁰ In addition, more CMB are found at higher field strength; Theysohn et al.²⁵ found 101 CMB at 7T (mean size: 3.2 mm) against 33 at 1.5T (mean size: 1.4 mm) in 10 subjects.

Figure 3.

Effect of slice thickness, sequence and magnetic field on CMB detection (courtesy of Nandigam et al.²⁸). The pairs of images illustrate comparisons of thick-section GRE (a) versus thin-section SWI (b); thin-section GRE (c) versus thin-section SWI (d); thick-section GRE (e) versus thin-section GRE (f) (all preceding images at 1.5T); and SWI at 1.5T (g) versus SWI at 3T (h). The black arrows in (a) and (b) illustrate a CMB prospectively counted on both sequences, whereas lesions denoted by white arrows were initially identified only on the SWI image. The black arrows in the remaining images highlight lesions on the paired images for comparison.

Sequence effect

3D T2*-weighted gradient-echo images reveal significantly more CMB in more subjects compared to 2D T2*-weighted gradient-echo images, primarily because of higher spatial resolution.⁷² In 200 older aged participants, Vernooij et al.⁷² reported 71 subjects with at least one microbleed (median number 2.5) using the 3D sequence compared to 42 subjects (median number 1.0) with the 2D sequence. Likewise, the susceptibility-weighted imaging (SWI) technique, which uses phase images to enhance susceptibility-related contrasts in the resulting image, is particularly sensitive to paramagnetic and ferromagnetic substances.⁷³ CMB are better delineated on SWI images compared to classic T2* images^25,28,74,75 (Figure 3). Another study also confirmed that SWI detected more CMB than conventional T2* images (prevalence of 40% vs. 23%, total number was 284 vs. 219 in 141 patients).⁷⁶ To note, the differentiation between CMB and vessels might be more difficult with SWI compared to T2* images although this is a matter of debate.^75,77 A last point is that susceptibility-weighted sequences are different according to vendors (e.g. “SWI” for Siemens, “SWAN” for General Electric, “VenoBOLD” for Philips) and only the Siemens “SWI” technique uses phase data. Therefore, the effect of vendor’s susceptibility-weighted technique on CMB detection needs further assessment.

Sequence parameters effect

The echo time (TE) reflects the timescale on which dephasing occurs, with long TEs enabling more time for dephasing thereby enlarging the susceptibility effect. Thus, choice of TE directly impacts on the number and volume of microbleeds detected as well as the number of structures that may be mistaken for microbleeds (calcium or iron deposits, flow voids, etc.). Increasing TE improves the sensitivity to detect CMB but more artefacts are generated near air-tissues interfaces causing uncertainty about rating some lesions.^78,79 Otherwise, thin MR imaging sections compared with thicker sections were associated with a small decrease in CMB diameter and a large increase of contrast (CMB signal relative to tissue signal) and thus an improved CMB detection (Figure 3),²⁸ due to a reduction of partial volume effects.

Reproducibility in longitudinal studies

No study was found to assess the reproducibility of any longitudinal measurement for CMB.

Brain volume

Definition and interest in SVD

Brain volumetry refers here to measures obtained through computational segmentation of high-resolution anatomical images. Global brain volume, intracranial volume (classically used for normalization purpose), and brain parenchymal fraction (BPF, brain volume divided by intracranial volume) were considered in this review as these are important markers in SVD.^80–85 For example, BPF has been associated with dementia rating scales and the modified Rankin score in CADASIL,^81,86 a monogenic SVD.² In sporadic forms, brain volume was notably associated with the presence and progression of lacunes.⁸⁷ Gray matter, white matter, or hippocampus volumes are sometimes specifically assessed and can be markers of interest in SVD studies, but were not included in this review as these structures depend more on post-processing stages. However, no study amongst any of the following has assessed the influence of SVD markers, particularly WMH, on the performance of any brain volume measurement methods.

Within-center reproducibility

Overall, several studies reported a satisfactory within-center reproducibility of brain volumetric measurements at 1.5T^{7,10,13–15,17,18} or 3T^10,14,15,18 with the main brain image-processing suites (FreeSurfer,^88,89 SPM,⁹⁰ FSL,⁹¹ BrainVisa⁹²). Table 1 presents per study the numbers of centers, subjects, rescans per subject, and main results for reproducibility. For example, in a large cohort of 671 subjects who underwent two successive 3D T1 weighted scans, the median of the relative absolute difference in brain volume was found to be less than 1%.¹⁷ However, the 90th percentile of the absolute value of the difference was reported at 5.11%, which is more than twice the value that would be expected for a Gaussian distribution. This highlights the importance of studying several statistical outcomes to infer the reproducibility beyond the median or mean value of the difference. Thus, little variability is added when measuring brain volume in a single center without changing the conditions of acquisition, at least regarding commonly reported disease-related changes. It is worth noting that most studies with a scan–rescan design included small samples (all n < 50 except Cover et al.¹⁷) and recruited mainly healthy subjects (two studies with mild cognitive impairment (MCI) and AD patients^14,17 and one study in multiple sclerosis¹⁶).

Between-center reproducibility (effect of various vendors, magnetic field strengths, coils)

Effects of combining several MRI vendors, static magnetic field strengths, or coil designs on brain volumetry can be assessed in studies where subjects are scanned several times in different centers (Table 1). As expected, compared to the within-center variability, the between-center volume variability was much higher at 1.5T^{7,8,10,11,14,15,18,65} and 3T.^{10,12,14,15,18} In one of the largest multicenter studies with a scan–rescan procedure (a subset of ADNI database in which 172 subjects were rescanned once in 59 different MRI sites on 17 different scanner types), the effects of scanner hardware and imaging protocol on volume quantification were analyzed.¹⁴ The most striking result was that the variance in volumetric measures was 10 times higher when subjects were imaged on different scanners compared to acquisitions on the same scanners. For example, for brain volume, the median (standard deviation) of the absolute within-subject variability was 0.3% (0.5%) versus 4.8% (4.6%) for repeated scans on the same scanner versus on different scanners. This was essentially explained by the scanner-dependent geometrical inaccuracies and by the differences in the gray to white matter tissue contrast from variation in the imaging sequences used and RF coils (while this contrast is obviously impacting gray matter and white matter volumes, it also influences the total brain volume computation¹⁴). Such geometric distortions originate from gradient field nonlinearity (in theory, a gradient system is supposed to produce an incremental magnetic field varying linearly with distance from isocenter) and the static field inhomogeneity, while the contrast between brain tissues is determined by a variety of sequence parameters. Both factors are dependent on the scanner hardware (a combination of vendor, magnetic field and coil systems), which explained 30 to 50% of the variance of any volumetric measure in that study. In another study, 15 subjects underwent four scan sessions at three sites with two different vendors.¹⁰ Interestingly, combining data of different vendors and/or field strength did not significantly change the standard deviation of the volume differences across conditions relative to the test–retest reproducibility within a fixed MRI system. However, the mean volume difference did change within one standard deviation from zero for intracranial volume.

Upgrades and repair

Only a few studies have examined the effects of MR systems upgrades and repairs on volumetric measurements. It appears that mixing volumes derived from data acquired across a major scanner upgrade (such as replacement of the magnet, gradient system and software) do not show significant changes, although volumes of some specific structures may show a slight scanner specific volume bias.¹⁰ Similarly, consistent measures for brain structure volumes have been found before and after gradient coil replacement in a 3T system.⁹³ However, inadvertent differences in sequence parameters for example may be introduced during the upgrade (human error)⁹⁴ which can cause major variance between pre- and post-upgrade scans if not detected quickly. Scanner upgrade or replacement are almost impossible to avoid and therefore centers should consider having longitudinal phantom and volunteer assessments for quality assurance and to calculate differences in brain volumetry across time that would allow conversion factors to be generated.

Sequence effect

Use of 3D inversion prepared T1-weighted fast gradient-echo sequences for brain tissue segmentation and volumetry is universal nowadays. It is widely used for its contrast properties and is a standard for most MRI vendors. It is therefore easy to implement in multicenter studies. Other sequences such as multi-echo fast low angle shot sequences have similar contrast properties and showed comparable volume reproducibility from Bland–Altman plot analysis than the MPRAGE sequence for example.¹⁰ However, comparison between volumes derived from both sequence types demonstrated significant biases in the mean volume difference, probably due to a differing T₂* sensitivity.¹⁰

Isotropic voxels are superior to non-isotropic voxels for accurate estimation of brain volume. In sequences with non-isotropic resolution, the choice of image orientation (sagittal, coronal or axial) naturally yields volumetric differences when comparing datasets acquired with different image orientations.⁹⁵ These differences may be due to partial volume averaging and susceptibility-induced geometric distortions. It is anticipated that these findings can be generalized to all MRI-derived metrics of brain volume.

Contrast effect

Variability of the tissue contrast, defined as the ratio between gray and white matter tissues image intensities on T1-weighted images, relates to the choice of MR sequence parameters such as echo time (TE), repetition time (TR), inversion time (TI), and flip angle (FA) along with the selection of field strength and coil. The contrast has been shown to have non-negligible influence on brain tissues segmentation and thus volumetry.^7,14 For example, a 6% change in the average contrast led to a 2% change in the computed brain volume,¹⁴ which is of a similar size as the annual change in brain volume seen in the early stages of AD so could easily be interpreted as a disease effect. Notably, datasets with a higher contrast have a lower variability, but here the effect of different burdens of WMH has not been tested.

Subject-related factors

Physiological variability within the subject being imaged is a source of variability beyond the disease-related variability. It has even been suggested in a study of BPF variability in multiple sclerosis that the variability due to physiological and pathological causes is important and likely larger in magnitude than scan–rescan repositioning effects.¹⁶ The authors found that the variance assigned to physiological fluctuations was nearly two times greater than the variance due to patient repositioning during scan–rescan. This implies that scan–rescan experiments might only provide a lower bound on the true error in repeated volumetric measurements from MRI exams. It is well known that the hydration status of an individual can affect the measurement of brain volume. Specifically, lack of fluid intake for 16 h has been found to result in a 0.55% decrease in cerebral volume and re-hydration an increase of 0.72% in brain volume.⁹⁶ In the ADNI dataset and a large cohort of multiple sclerosis patients, there was a significant effect of time-of-day on the BPF with a greater brain volume in the morning.⁹⁷ Similarly, brain volume varies with plasma sodium.⁹⁸ These results suggest potential acquisition time and hydration biases should be randomized or statistically controlled to account for the day-to-day brain volume fluctuations.

Factors with no or little effect

Correction for so-called B1 inhomogeneities (variations of image intensity due to the radiofrequency field) was not found to substantially change the reproducibility of whole-brain volumetric measurements,¹⁰ unlike its effect on WMH and focal infarcts.⁹⁹ Similarly, some studies found that greater acceleration factor in parallel imaging (this technique combines the signals of several coil elements in a phased array to reconstruct the image and finally improve the signal-to-noise ratio or reduce the scan time) had very little influence on test–retest accuracy.^12,100 However, these studies tested the influence of acceleration factor in healthy controls. Results might be different if older aged subjects or patients move during the acquisition of the reference lines part of the parallel imaging strategy. Finally, reproducibility of brain volumetric measures was also relatively unaffected by small differences in voxel geometry compared to the effects observed on measures of cortical thickness.¹²

Brain atrophy

Definition and interest in SVD

Brain atrophy is the difference of total brain volume between two time points computed by a dedicated image-processing software from high-resolution anatomical images. Quantitative measures derived from high-resolution anatomical images are considered over atrophy scales (not covered in the present review) as the quantitative measures have better sensitivity to change over time.¹⁰¹ Surprisingly, very few studies reported brain atrophy measurement in SVD cohorts.^102,103 Several methods are available, such as SIENA, which estimates the percentage brain volume change of the same subject between two time points, by registering the two brain images together and resampling into the space halfway between the two^104,105 and the boundary shift integral (BSI), that is an automated method that determines the total volume through which the boundaries of a given cerebral structure have moved.¹⁰⁶ One alternative method to assess brain atrophy is by performing tissue segmentation at both time points, but this approach suffers from a higher variability given the added variability of each measurement.²²

Within-center reproducibility

Within-center reproducibility of quantitative image-processing methods for atrophy assessment has been shown to be relatively high in several studies (Table 1).^17,9,19 For example, a study using the ADNI dataset reported a 0.35% mean difference in 385 subjects acquired in more than 50 sites (105 healthy controls, 195 subjects with MCI, and 85 with AD) undergoing two SIENA measurements 12 months apart.¹⁷ Further, the 90th percentile of the difference was much larger than expected (1.33%) for a Gaussian distribution. These results should be compared to the annualized mean percentage brain volume change obtained in that study: −0.65% ± 0.82% for healthy controls, −1.15% ± 1.21% for MCI, −1.84% ± 1.33% for AD.

Between-center reproducibility (effect of various vendors, magnetic field strengths, coils)

Some studies have examined the between-center reproducibility of atrophy measurements using SIENA^17,22,107 or other image-processing techniques.²² It has been shown that even with scanner of the exact same model, scanner drift (a shift in the magnet’s field) and inter-scanner variability may cancel out longitudinal brain volume change.⁴⁴ It is worth noting that compared to voxel-based morphometry approaches, SIENA, which corrects for differences in imaging geometry using the outer skull surface for both time points, reduced the effects of scanner drift and inter-scanner variability on longitudinal morphometric results.¹⁰⁷ Similarly, other registration-based algorithms such as BSI show better reproducibility compared to segmentation-based techniques such as Freesurfer or SIENAX.²² However, segmentation-based methods using atlases can be improved by using fine registration methods and age-relevant templates.¹⁰⁸

Evaluation of coil effects on volumetric measurements showed that a change from a 12- to a 32-channel coil was responsible for a shift of 0.5% in the whole-brain BSI,¹⁰⁰ which appears unacceptable as it corresponds to the annual atrophy rate observed in healthy subjects.¹⁰⁹

Finally, analyzing the back-to-back scans of the ADNI dataset in 200 pairs of images (118 subjects with 0–12 months pairs and 82 subjects with 6–24 months pairs), it has been reported than 3T was no more reproducible than 1.5T for the whole-brain atrophy measure using SIENA.¹¹⁰

Contrast effect

Atrophy measurement is strongly dependent on image contrast. Indeed, the error using BSI may exceed 100% if image contrast properties dramatically differ between the two scans in a measurement pair.¹¹¹ For example, in 10 volunteers, a mean brain volume change of −0.43% has been measured versus a +1.85% change when the second scan was acquired with a FA equal to 12° instead of 25° (the FA directly impacts the image contrast).

Noise effect

Noise is an unwanted random signal from various origins including: the subject’s body, coils, electronics, nearby machinery, etc. The signal-to-noise ratio is often computed as the ratio of the average signal intensity over the standard deviation of the noise. Signal-to-noise ratio directly impacts BSI values as, for example, the position of the brain surface is seen to retract as noise in the second time point scan increases.¹¹¹ Thus, the average percent volume change progressively diverges from zero with increasing noise mimicking increased brain atrophy.

Subject-related factors

Motion will have an impact on atrophy measurement. For example, it has been reported that, as the amplitude of the motion in the second scan increases, the mean absolute brain and ventricular BSI brain volume changes as well as the associated standard deviation increase progressively.¹¹¹ However, the different levels of motion have been simulated in this latter study and the severity may have been overestimated.

Subject’s head positioning in the magnet also influences brain atrophy measures. Indeed, nonlinear gradient distortions associated with shifts in the long magnet’s axis can significantly decrease the accuracy and precision of MRI-derived measures of brain atrophy assessed by SIENA.¹¹² These negative effects increase in magnitude with: (i) increases in the Z-distance (long magnet’s axis) between the brains to be compared at two time points; and, (ii) increases in the Z-distance from magnet isocenter to the center of the pair of brains to be compared.¹¹² Typical gradient distortion field for a T1-weighted acquisition is presented in Figure 4.

Figure 4.

Typical gradient distortion field in a T1-weighted acquisition (courtesy of Caramanos et al.¹¹²). Sagittal, coronal, and axial views of a phantom-based gradient distortion field that was determined based on a T1-weighted acquisition. The color scale represents the distance (in millimeters) that a voxel moves because of gradient distortion between its “real” location and its “apparent” location on such an MRI scan that has not been corrected for the gradient distortion.

Techniques to manage variability

We have highlighted a large number of reasons why quantitative MRI markers of SVD may be variable beyond subject-related or disease-related factors. These sources of variability introduce significant barriers, and potential biases, to multicenter imaging studies of SVD. Fortunately, while adding variability to the measurements of markers is inherent to imaging studies, some techniques have been designed to reduce it.

First, an optimized MRI protocol with high contrast-to-noise ratio, minimal artefacts, and good reproducibility must be determined. Hereto, a methodological approach has been proposed to target a protocol with the highest possible reproducibility based on the assessment of quality measures.²¹ The principle is to acquire several datasets including: within-subject within-center datasets, between-subjects between-center datasets and between-subject within-center datasets. By analyzing the summary data quality and quantitative measures extracted with dedicated and validated software, the desired optimal MRI protocol can be determined.

Second, to account for the problem of upgrades in longitudinal studies, Jovicich et al.¹⁰ advise planning a “system upgrade calibration study” as part of the design, with subjects scanned shortly prior to and immediately after the upgrade so as to derive a correct estimation for potential systematic biases.

Third, several corrective techniques based on the MR system characteristics^113,114 or using a dedicated phantom^112,115,116 have been developed to reduce the effects of gradient distortion on imaging reproducibility. The advantage of using a phantom travelling from one site to another is to apply a unified correction independently of the vendor’s own corrections. Correction for gradient nonlinearity and intensity non-uniformity reduces the variance of longitudinal changes in brain volumes and improves the accuracy for detecting subtle volumetric changes.¹¹⁷ For example, in 208 subjects, the raw mean percentage of brain volume change was −0.59% against −0.46% when images were corrected for gradient nonlinearity and intensity non-uniformity.¹¹⁷ However, if the expected group difference far exceeds brain volume change related to geometric distortions, the extra time and expense associated to the phantom-based correction need to be carefully evaluated.

Finally, the use of dedicated statistical methods is important in analyzing multicenter MRI data. A simple a posteriori method to account for scanner hardware variability (combination of different devices, coil types and field strengths) is to include the scanner hardware as a covariate in regression models.¹⁴ Moreover, it has been shown that T1-weighted regional anatomical brain volume data can be reliably combined across 1.5T and 3T centers with the application of an appropriate correction procedure (regression-based correction function when data varied in a linear and systematic fashion).¹¹⁸ More sophisticated statistical methods such as linear mixed-effect models including protocol as well as individual MRI acquisition parameters have also emerged to more accurately model longitudinal changes (BPF and volume of WMH).¹¹⁹ Another statistical approach is based on the statement that classical inference testing is inappropriate because it is designed to detect differences and not to prove similarity. Equivalence testing has then been applied to determine if MRI-based measurements obtained under variable conditions can be pooled.¹²⁰ This could be generalized to all MRI-derived measurements. Otherwise, Moorhead et al.¹²¹ used scanner specific priors (probability mappings of tissue occupancy) in the segmentation pipeline to reduce tissue classification differences between scanners to improve statistical power. Interestingly, they found that the between scanner differences were not reduced to the level of within scanner variability, the ideal for scanner harmonization.

Ultimately, central application of automatic quality assessment^94,122 and image processing on raw data, using consistent techniques, can reduce the variability introduced by a posteriori image processing across centers.

Suggestions for future research

A between-center variability study in a large group of subjects, representing a range of types and burdens of SVD lesions, to assess the effects of vendor and field strength (protocols as close as possible) on reproducibility of SVD lesions and brain volumes is missing. So are experiments examining the effects of changing coil, correcting for gradient distortion or field heterogeneities, upgrading the system, or varying classical parameters such as TE, TI or TR, on the final WMH volume. Finally, WMH appears to be a reliable marker in single center studies where all acquisition and processing parameters are kept constant, but additional studies are missing to infer the true reproducibility of WMH measures in a multicenter context.

As for WMH, a multicenter study assessing the effect of scanner hardware (same magnetic field but different vendors and models) on CMB detection with a given optimized protocol would greatly add to the knowledge on the reliability of CMB detection. New data on methods of automated detection of CMB throughout different brain regions are needed. Also, the analysis of the potential effects of coil, and voxel size would be particularly useful. Finally, CMB can be used as a reliable marker in single center studies but additional efforts are needed to determine the sources of variability in multicenter data.

For lacunes and PVS, more automated and validated image-processing algorithms are required, followed by assessment of reproducibility through both within and between-centers acquisitions. In view of the likely interactions between acquisition protocols and performance of analysis methods, the assessment of all SVD features should be included in any study that assesses reliability of WMH and/or CMB and/or atrophy quantification.

Ten key points for assessing quantitative SVD markers in multicenter studies

Based on our review of the current literature regarding reproducibility and acquisition-related sources of variability of MRI markers in SVD and on personal experiences, we propose the following key points should be considered when designing or analyzing a multicenter MRI study in SVD. Obviously, prior to acquisitions, the scientific question should guide which marker is the focus, although opportunities to assess all SVD markers should be obtained where possible. Optimizing acquisition for contrast, signal-to-noise ratio and spatial resolution is particularly important when imaging elderly or cognitively impaired patient populations who will not tolerate prolonged acquisition times. Importantly, although it is essential to have accurate and reproducible measurements, the nature of the questions will be guiding the level of precision that is needed. Requirements might be different between interventional clinical trials with relatively short-term duration and long-term population-based observational studies. For example, in some situations, the added noise due to multicenter variability will not alter a strong statistical association between one SVD-related MRI marker and one clinical outcome. Conversely, studying the quantitative evolution of one SVD-related marker during a short follow-up will require a very fine quantification that necessitates a variability caused by technological factors as small as possible. Hence, we organized our recommendations as being (a) advisable whatever the question guiding the study as they tackle effects being important and/or involve easy-to-follow techniques to manage variability, (b) advisable in some conditions, i.e. when the expected effect of interest is small relatively to the unwanted noise caused by the multicenter setting.

Strongly advisable and easy-to-follow recommendations

Automated quality checks for MRI sequence parameters are advisable (for each scan, comparison of the acquisition parameters with the intended protocol for each scanner). Also, images should be monitored for motion and other artefacts such as magnetic susceptibility artefacts, aliasing artefacts, or Gibbs artefacts. A minimum standard for quality control will have to be defined to exclude problematic data (for example, based on the severity of artefacts or from automatic measures of signal-to-noise ratio, background noise, etc.). Indeed, even if the computational method seems to run properly, artefacts can dramatically alter the data.

MRI gradient nonlinearities have to be corrected at least with corrections proposed in options by the manufacturer as they can affect dramatically measures of brain atrophy.

Subject’s positioning in the scanner is another key aspect and strict instructions for MR technicians may be particularly useful to reproduce confidently the position of the head, from one subject to another, and from the first time point to the following time points, in longitudinal studies.

Considering the hardware, the same coil has to be used during an MR protocol both in cross-sectional and longitudinal studies. When upgrades or repairs are needed during the study period, a strategy should be in place to scan some participants twice, before and after the upgrade, to estimate the potential bias for a posteriori analyses. Otherwise, the imaging protocol and hardware should be kept as constant as possible over the whole time of the study.

Recommendations to evaluate given the targeted trade-off between sensitivity and variability

Within-center reproducibility is much higher than between-center reproducibility. The need for a multicenter setup should then be carefully evaluated while taking into account specific aspects such as number of participants, subject transportation, data centralization, quality control, and cost. For multicenter studies, the use of only one vendor or only some models can be an option.

In multicenter studies, protocol harmonization is a mandatory step to obtain similar sequences with acquisition parameters as close as possible. There is no way to get a perfect harmonization of MRI protocols between different vendors and models. Therefore, it is a trade-off between a high reproducibility of markers (few centers, same vendors) and a sufficient statistical power through the recruitment of a large number of subjects, generally in several centers with different scanner vendors and models. Specifying acceptable ranges for different acquisition parameters may help obtain satisfactory harmonization for large-scale clinical studies.

Spatial resolution is a key parameter in assessing correctly the variation of any MRI marker such as brain volumetry, longitudinal atrophy, volume of WMH, or quantification of lacunes and PVS. Increasing spatial resolution decreases signal-to-noise ratio and increases acquisition duration. Sensitivity for quantifying small objects such as lacunes and PVS will be compromised if using voxels as big as the object itself or where there is a slice gap. Selection of optimal spatial resolution must therefore be targeted to the primary question.

When analyzing multicenter MRI data, the use of statistical models to take into account the induced variability is recommended (adjusting on center or using random effects models for example).

While some quantitative MRI markers of SVD are regularly assessed (brain atrophy, WMH volume), others are difficult to quantify given their small size in relation to conventional anatomical MRI (lacunes, PVS) or their strong dependence on MRI parameters (CMB). High-field MRI and detailed studies of these particular markers may be performed in studies nested within larger studies assessing more common markers in a clinical setting. For CMB characterization, SWI or quantitative susceptibility mapping are promising techniques. Since the total magnetic susceptibility of a CMB is an intrinsic physical property independent of imaging characteristics, mapping CMB by using quantitative susceptibility mapping may be an alternative approach more consistent over a wide spectrum of imaging parameters.⁷⁹

It is important for studies to acknowledge that variation is inevitable and they should detail the steps that have been made to minimize this variation.

Conclusion

There is a need to reduce image variability induced by technological choices (coil, MR sequence, MR parameters, and options) as much as possible in large multicenter research studies and clinical trials using MRI in SVD. Indeed, the added variance due to the multicenter setup can be deleterious in various applications and should be minimized using multiple approaches. Protocol harmonization and a posteriori corrections can help in minimizing biases; minimization on center or scanner type may also be advisable. Even if several sources of variability have been previously evaluated, most studies have focused on brain volume measurements while few have focused on WMH quantification. Further, less is known concerning the sources of variability when quantifying lacunes, CMB, and PVS. Additional studies are warranted to investigate the variability of these makers, which is a key aspect for future longitudinal or multicenter studies. Other potential SVD-related MRI markers derived from new and multiple techniques (such as diffusion tensor imaging, arterial spin labelling, magnetization transfer mapping, blood–brain barrier permeability imaging, BOLD functional MRI, cortical morphology, cortical microinfarcts, cortical superficial siderosis) will also have to address these important questions regarding their reproducibility to facilitate large-scale, multicentre research.

Footnotes

Funding

This study was supported by grants from Fondation Avenir and Fondation Leducq (Transatlantic Network of Excellence on the Pathogenesis of Small Vessel Disease of the Brain, ). This work follows from a project funded by the Centres of Excellence in Neurodegeneration (COEN), ref MRC MR/J006971/1, “Standards for determining the vascular contribution to neurodegeneration” in which many of the authors participated.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Authors’ contribution

FDG, EJ, and HC designed the review. FDG collected and analysed the literature. FDG, EJ, and HC wrote the initial draft of the review. JMW critically revised the manuscript. The manuscript was then sent to the imaging and small vessel disease experts involved in the COEN group (network of Centres of Excellence in Neurodegeneration, ). All authors of the present manuscript agreed to contribute and revised carefully the manuscript. Finally, all authors accepted the current version.

Supplementary material

Supplementary material for this paper can be found at

References

Román

Erkinjuntti

Wallin

. Subcortical ischaemic vascular dementia. Lancet Neurol 2002; 1: 426–436.

Chabriat

Joutel

Dichgans

. Cadasil. Lancet Neurol 2009; 8: 643–653.

Selnes

Vinters

. Vascular cognitive impairment. Nat Clin Pract Neurol 2006; 2: 538–547.

Pantoni

. Cerebral small vessel disease: from pathogenesis and clinical characteristics to therapeutic challenges. Lancet Neurol 2010; 9: 689–701.

Wardlaw

Smith

Biessels

. Neuroimaging standards for research into small vessel disease and its contribution to ageing and neurodegeneration. Lancet Neurol 2013; 12: 822–838.

Kessler

Barnhart

Buckler

. The emerging science of quantitative imaging biomarkers terminology and definitions for scientific studies and regulatory submissions. Stat Meth Med Res 2015; 24: 9–26.

Schnack

van Haren

NEM

Pol

HEH

. Reliability of brain volumes from multicenter MRI acquisition: a calibration study. Hum Brain Mapp 2004; 22: 312–320.

Ewers

Teipel

Dietrich

. Multicenter assessment of reliability of cranial MRI. Neurobiol Aging 2006; 27: 1051–1059.

Smith

Rao

De Stefano

. Longitudinal and cross-sectional analysis of atrophy in Alzheimer's disease: cross-validation of BSI, SIENA and SIENAX. Neuroimage 2007; 36: 1200–1206.

10.

Jovicich

Czanner

Han

. MRI-derived measurements of human subcortical, ventricular and intracranial brain volumes: reliability effects of scan sessions, acquisition sequences, data analyses, scanner upgrade, scanner vendors and field strengths. Neuroimage 2009; 46: 177–192.

11.

Reig

Sanchez-Gonzalez

Arango

. Assessment of the increase in variability when combining volumetric data from different scanners. HumBrain Mapp 2009; 30: 355–368.

12.

Wonderlick

Ziegler

Hosseini-Varnamkhasti

. Reliability of MRI-derived cortical and subcortical morphometric measures: effects of pulse sequence, voxel geometry, and parallel imaging. Neuroimage 2009; 44: 1324–1333.

13.

de Boer

Vrooman

Ikram

. Accuracy and reproducibility study of automatic MRI brain tissue segmentation methods. Neuroimage 2010; 51: 1047–1056.

14.

Kruggel

Turner

Muftuler

. Impact of scanner hardware and imaging protocol on image quality and compartment volume precision in the ADNI cohort. Neuroimage 2010; 49: 2123–2133.

15.

Huppertz

H-J

Kröll-Seger

Klöppel

. Intra- and interscanner variability of automated voxel-based volumetry based on a 3D probabilistic atlas of human cerebral structures. Neuroimage 2010; 49: 2216–2224.

16.

Sampat

Healy

Meier

. Disease modeling in multiple sclerosis: assessment and quantification of sources of variability in brain parenchymal fraction measurements. Neuroimage 2010; 52: 1367–1373.

17.

Cover

van Schijndel

van Dijk

. Assessing the reproducibility of the SienaX and Siena brain atrophy measures using the ADNI back-to-back MP-RAGE MRI scans. Psychiatry Res 2011; 193: 182–190.

18.

Shokouhi

Barnes

Suckling

. Assessment of the impact of the scanner-related factors on brain morphometry analysis with Brainvisa. BMC Med Imaging 2011; 11: 23.

19.

de Bresser

Portegies

Leemans

. A comparison of MR based segmentation methods for measuring brain atrophy progression. Neuroimage 2011; 54: 760–768.

20.

Landman

Huang

Gifford

. Multi-parametric neuroimaging reproducibility: a 3-T resource study. Neuroimage 2011; 54: 2854–2866.

21.

Chalavi

Simmons

Dijkstra

. Quantitative and qualitative assessment of structural magnetic resonance imaging data in a two-center study. BMC Med Imaging 2012; 12: 27.

22.

Durand-Dubief

Belaroussi

Armspach

. Reliability of longitudinal brain volume loss measurements between 2 sites in patients with multiple sclerosis: comparison of 7 quantification techniques. AJNR. Am J Neuroradiol 2012; 33: 1918–1924.

23.

Wei

Warfield

Zou

. Quantitative analysis of MRI signal abnormalities of brain white matter with high reproducibility and accuracy. J Magn Reson Imaging 2002; 15: 203–209.

24.

Sicotte

Voskuhl

Bouvier

. Comparison of multiple sclerosis lesions at 1.5 and 3.0 Tesla. Invest Radiol 2003; 38: 423–427.

25.

Theysohn

Kraff

Maderwald

. 7 Tesla MRI of microbleeds and white matter lesions as seen in vascular dementia. J Magn Reson Imaging 2011; 33: 782–791.

26.

Ramirez

Scott

Black

. A short-term scan-rescan reliability test measuring brain tissue and subcortical hyperintensity volumetrics obtained using the lesion explorer structural MRI processing pipeline. Brain Topogr 2013; 26: 35–38.

27.

Stehling

Wersching

Kloska

. Detection of asymptomatic cerebral microbleeds: a comparative study at 1.5 and 3.0 T. Acad Radiol 2008; 15: 895–900.

28.

Nandigam

RNK

Viswanathan

Delgado

. MR imaging detection of cerebral microbleeds: effect of susceptibility-weighted imaging, section thickness, and field strength. Am J Neuroradiol 2009; 30: 338–343.

29.

Conijn

MMA

Geerlings

Biessels

. Cerebral microbleeds on MR imaging: comparison between 1.5 and 7T. Am J Neuroradiol 2011; 32: 1043–1049.

30.

Potter

Marlborough

Wardlaw

. Wide variation in definition, detection, and description of lacunar lesions on imaging. Stroke 2011; 42: 359–366.

31.

Prins

Scheltens

. White matter hyperintensities, cognitive impairment and dementia: an update. Nat Rev Neurol 2015; 11: 157–165.

32.

Fazekas

Chawluk

Alavi

. MR signal abnormalities at 1.5 T in Alzheimer's dementia and normal aging. AJR Am J Roentgenol 1987; 149: 351–356.

33.

Schmidt

Fazekas

Kleinert

. Magnetic resonance imaging signal hyperintensities in the deep and subcortical white matter. A comparative study between stroke patients and normal volunteers. Arch Neurol 1992; 49: 825–827.

34.

Scheltens

Barkhof

Leys

. A semiquantitative rating-scale for the assessment of signal hyperintensities on magnetic-resonance-imaging. J Neuroll Sci 1993; 114: 7–12.

35.

Maillard

Delcroix

Crivello

. An automated procedure for the assessment of white matter hyperintensities by multispectral (T1, T2, PD) MRI and an evaluation of its between-centre reproducibility based on two large community databases. Neuroradiology 2008; 50: 31–42.

36.

Samaille

Fillon

Cuingnet

. Contrast-based fully automatic segmentation of white matter hyperintensities: method and validation. Plos One 2012; 7: e48953.

37.

Ramirez

Gibson

Quddus

. Lesion explorer: a comprehensive segmentation and parcellation package to obtain regional volumetrics for subcortical hyperintensities and intracranial tissue. Neuroimage 2011; 54: 963–973.

38.

Yoshita

Fletcher

DeCarli

. Current concepts of analysis of cerebral white matter hyperintensities on magnetic resonance imaging. Top Magn Reson Imaging 2005; 16: 399–407.

39.

Dyrby

Rostrup

Baare

. Segmentation of age-related white matter changes in a clinical multi-center study. Neuroimage 2008; 41: 335–345.

40.

van den Heuvel

ten Dam

de Craen

. Measuring longitudinal white matter changes: comparison of a visual rating scale with a volumetric measurement. AJNR Am J Neuroradiol 2006; 27: 875–878.

41.

Wardlaw

Brindle

Casado

. A systematic review of the utility of 1.5 versus 3 Tesla magnetic resonance brain imaging in clinical practice and research. Eur Radiol 2012; 22: 2295–2303.

42.

Polak

Magnano

Zivadinov

. 3D FLAIRED: 3D fluid attenuated inversion recovery for enhanced detection of lesions in multiple sclerosis. Magn Reson Med 2012; 68: 874–881.

43.

Ramirez

Berezuk

McNeely

. Visible Virchow-Robin spaces on magnetic resonance imaging of Alzheimer's disease patients and normal elderly from the Sunnybrook dementia study. J Alzheimers Dis 2015; 43: 415–424.

44.

Pettersen

Sathiyamoorthy

Gao

. Microbleed topography, leukoaraiosis, and cognition in probable Alzheimer disease from the Sunnybrook dementia study. Arch Neurol 2008; 65: 790–795.

45.

Garcia-Lorenzo

Francis

Narayanan

. Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging. Med Image Anal 2013; 17: 1–18.

46.

Filippi

Horsfield

Campi

. Resolution-dependent estimates of lesion volumes in magnetic-resonance-imaging studies of the brain in multiple-sclerosiS. Ann Neurol 1995; 38: 749–754.

47.

Valdes Hernandez MC, Ghandour D, Gonzales-Castro V, et al. On the computational assessment of white matter hyperintensity progression: difficulties in method selection and bias field correction performance on images with significant white matter pathology. Neuroradiology 2016; 58: 475–485.

48.

Wang

Catindig

Hilal

. Multi-stage segmentation of white matter hyperintensity, cortical and lacunar infarcts. Neuroimage 2012; 60: 2379–2388.

49.

Hervé

Godin

Dufouil

. Three-dimensional MRI analysis of individual volume of Lacunes in CADASIL. Stroke 2009; 40: 124–128.

50.

Viswanathan

Guichard

Gschwendtner

. Blood pressure and haemoglobin A1c are associated with microhaemorrhage in CADASIL: a two-centre cohort study. Brain 2006; 129(Pt 9): 2375–2383.

51.

Benjamin

Lawrence

Lambert

. Strategic lacunes and their relationship to cognitive impairment in cerebral small vessel disease. Neuroimage Clin 2014; 4: 828–837.

52.

Hernandez Mdel

Piper

Wang

. Towards the automatic computational assessment of enlarged perivascular spaces on brain magnetic resonance images: a systematic review. J Magn Reson Imaging 2013; 38: 774–785.

53.

Adams

HHH

Cavalieri

Verhaaren

BFJ

. Rating method for dilated virchow-robin spaces on magnetic resonance imaging. Stroke 2013; 44: 1732–1735.

54.

Potter

Chappell

Morris

. Cerebral perivascular spaces visible on magnetic resonance imaging: development of a qualitative rating scale and its observer reliability. Cerebrovasc Dis 2015; 39: 224–231.

55.

Wang

Hernandez

Doubal

. Development and initial evaluation of a semi-automatic approach to assess perivascular spaces on conventional magnetic resonance images. J Neurosci Meth 2015; 257: 34–44.

56.

Zong

Park

Shen

. Visualization of perivascular spaces in the human brain at 7T: sequence optimization and morphology characterization. Neuroimage 2016; 125: 895–902.

57.

Berezuk

Ramirez

Gao

. Virchow-robin spaces: correlations with polysomnography-derived sleep parameters. Sleep 2015; 38: 853–858.

58.

Greenberg

Vernooij

Cordonnier

. Cerebral microbleeds: a guide to detection and interpretation. Lancet Neurol 2009; 8: 165–174.

59.

de Laat

van den Berg

van Norden

. Microbleeds are independently related to gait disturbances in elderly individuals with cerebral small vessel disease. Stroke 2011; 42: 494–497.

60.

van Norden

van Uden

de Laat

. Cerebral microbleeds are related to subjective cognitive failures: the RUN DMC study. Neurobiol Aging 2013; 34: 2225–2230.

61.

Poels

Ikram

van der Lugt

. Cerebral microbleeds are associated with worse cognitive function: the rotterdam scan study. Neurology 2012; 78: 326–333.

62.

Gregoire

Scheffler

Jager

. Strictly lobar microbleeds are associated with executive impairment in patients with ischemic stroke or transient ischemic attack. Stroke 2013; 44: 1267–1272.

63.

Cordonnier

Potter

Jackson

. Improving interrater agreement about brain microbleeds development of the brain observer microbleed scale (BOMBS). Stroke 2009; 40: 94–99.

64.

Gregoire

Chaudhary

Brown

. The microbleed anatomical rating scale (MARS): reliability of a tool to map brain microbleeds. Neurology 2009; 73: 1759–1766.

65.

Barnes

SRS

Haacke

Ayaz

. Semiautomated detection of cerebral microbleeds in magnetic resonance images. Magn Reson Imaging 2011; 29: 844–852.

66.

Seghier

Kolanko

Leff

. Microbleed detection using automated segmentation (MIDAS): a new method applicable to standard clinical MR images. Plos One 2011; 6: e17547.

67.

Kuijf

de Bresser

Geerlings

. Efficient detection of cerebral microbleeds on 7.0 T MR images using the radial symmetry transform. Neuroimage 2012; 59: 2266–2273.

68.

Kuijf

Brundel

de Bresser

. Semi-automated detection of cerebral microbleeds on 3.0 T MR images. PLoS One 2013; 8: e66610.

69.

Fazlollahi

Meriaudeau

Giancardo

. Computer-aided detection of cerebral microbleeds in susceptibility-weighted imaging. Comput Med Imaging Graph 2015; 46: 269–276.

70.

Schrag

McAuley

Pomakian

. Correlation of hypointensities in susceptibility-weighted images to tissue histology in dementia patients with cerebral amyloid angiopathy: a postmortem MRI study. Acta Neuropathologica 2010; 119: 291–302.

71.

De Guio

Benoit-Cattin

Davenel

. Signal decay due to susceptibility-induced intravoxel dephasing on multiple air-filled cylinders: MRI simulations and experiments. MAGMA 2008; 21: 261–271.

72.

Vernooij

Ikram

Wielopolski

. Cerebral microbleeds: accelerated 3D T2*-weighted GRE MR imaging versus conventional 2D T2*-weighted GRE MR imaging for detection. Radiology 2008; 248: 272–277.

73.

Haacke

Cheng

YCN

. Susceptibility weighted imaging (SWI). Magn Reson Med 2004; 52: 612–618.

74.

Guo

Wang

Zhu

. Comparison of ESWAN, SWI-SPGR, and 2D T2*-weighted GRE sequence for depicting cerebral microbleeds. Clin Neuroradiol 2013; 23: 121–127.

75.

Cheng

Batool

McCreary

. Susceptibility-weighted imaging is more reliable than T2*-weighted gradient-recalled echo MRI for detecting microbleeds. Stroke 2013; 44: 2782–2786.

76.

Goos

JDC

van der Flier

Knol

. Clinical relevance of improved microbleed detection by susceptibility-weighted magnetic resonance imaging. Stroke 2011; 42: 1894–1900.

77.

Charidimou

Jager

Werring

. Cerebral microbleed detection and mapping: principles, methodological aspects and rationale in vascular dementia. Exp Gerontol 2012; 47: 843–852.

78.

Gregoire

Werring

Chaudhary

. Choice of echo time on GRE T2*-weighted MRI influences the classification of brain microbleeds. Clin Radiol 2010; 65: 391–394.

79.

Liu

Surapaneni

Lou

. Cerebral microbleeds: burden assessment by using quantitative susceptibility mapping. Radiology 2012; 262: 269–278.

80.

Peters

Holtmannspötter

Opherk

. Brain volume changes in CADASIL: a serial MRI study in pure subcortical ischemic vascular disease. Neurology 2006; 66: 1517–1522.

81.

Jouvent

Viswanathan

Mangin

J-F

. Brain atrophy is related to lacunar lesions and tissue microstructural changes in CADASIL. Stroke 2007; 38: 1786–1790.

82.

Viswanathan

Godin

Jouvent

. Impact of MRI markers in subcortical vascular dementia: a multi-modal analysis in CADASIL. Neurobiol Aging 2010; 31: 1629–1636.

83.

Lawrence

Patel

Morris

. Mechanisms of cognitive impairment in cerebral small vessel disease: multimodal MRI results from the St George's cognition and neuroimaging in stroke (SCANS) study. PloS one 2013; 8: e61014.

84.

Jokinen

Lipsanen

Schmidt

. Brain atrophy accelerates cognitive decline in cerebral small vessel disease: the LADIS study. Neurology 2012; 78: 1785–1792.

85.

Jouvent

Viswanathan

Chabriat

. Cerebral atrophy in cerebrovascular disorders. J Neuroimaging 2010; 20: 213–218.

86.

O'sullivan

Jouvent

Saemann

. Measurement of brain atrophy in subcortical vascular disease: a comparison of different approaches and the impact of ischaemic lesions. Neuroimage 2008; 43: 312–320.

87.

Kloppenborg

Nederkoorn

Grool

. Cerebral small-vessel disease and progression of brain atrophy: the SMART-MR study. Neurology 2012; 79: 2029–2036.

88.

Dale

Fischl

Sereno

. Cortical surface-based analysis – I. Segmentation and surface reconstruction. Neuroimage 1999; 9: 179–194.

89.

Fischl

Sereno

Dale

. Cortical surface-based analysis – II: Inflation, flattening, and a surface-based coordinate system. Neuroimage 1999; 9: 195–207.

90.

Ashburner

Friston

. Unified segmentation. Neuroimage 2005; 26: 839–851.

91.

Smith

Jenkinson

Woolrich

. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage 2004; 23: S208–S219.

92.

Mangin

J-F

Rivière

Cachia

. A framework to study the cortical folding patterns. Neuroimage 2004; 23: S129–S138.

93.

Huang

Wang

Baliki

. Reproducibility of structural, resting-state BOLD and DTI data between identical scanners. PLoS One 2012; 7: e47684.

94.

Gedamu

Collins

Arnold

. Automated quality control of brain MR images. J Magn Reson Imaging 2008; 28: 308–319.

95.

Patwardhan

Eliez

Warsofsky

. Effects of image orientation on the comparability of pediatric brain volumes using three-dimensional MR data. J Comput Assist Tomogr 2001; 25: 452–457.

96.

Duning

Kloska

Steinstrater

. Dehydration confounds the assessment of brain atrophy. Neurology 2005; 64: 548–550.

97.

Nakamura

Brown

Narayanan

. Diurnal fluctuations in brain volume: statistical analyses of MRI from large populations. Neuroimage 2015; 118: 126–132.

98.

Heye

Thrippleton

Chappell

. Blood pressure and sodium: association with MRI markers in cerebral small vessel disease. J Cereb Blood Flow Metabol 2016; 36: 264–274.

99.

Wardlaw

Valdes

HMC

Munoz-Maniega

. What are white matter hyperintensities made of ? Relevance to vascular cognitive impairment. J Am Heart Assoc 2015; 4: 001140.

100.

Krueger

Granziera

Jack

. Effects of MRI scan acceleration on brain volume measurement consistency. J Magn ResonImaging 2012; 36: 1234–1240.

101.

Ross

Ochs

DeSmit

. Man versus machine part 2: comparison of radiologists' interpretations and neuroquant measures of brain asymmetry and progressive atrophy in patients with traumatic brain injury. J Neuropsychiatry Clin Neurosci 2015; 27: 147–152.

102.

Ritchie

Dickie

Cox

. Brain volumetric changes and cognitive ageing during the eighth decade of life. Hum Brain Mapp 2015; 36: 4910–4925.

103.

Peters

Holtmannspotter

Opherk

. Brain volume changes in CADASIL: a serial MRI study in pure subcortical ischemic vascular disease. Neurology 2006; 66: 1517–1522.

104.

Smith

de Stefano

Jenkinson

. Normalized accurate measurement of longitudinal brain change. J Compute Assist Tomogr 2001; 25: 466–475.

105.

Smith

Zhang

Jenkinson

. Accurate, robust, and automated longitudinal and cross-sectional brain change analysis. Neuroimage 2002; 17: 479–489.

106.

Freeborough

Fox

. The boundary shift integral: An accurate and robust measure of cerebral volume changes from registered repeat MRI. IEEE Transac Med Imaging 1997; 16: 623–629.

107.

Takao

Hayashi

Ohtomo

. Effect of scanner in longitudinal studies of brain volume changes. J Magn Reson Imaging 2011; 34: 438–444.

108.

Dickie

Job

Gonzalez

. Use of brain MRI atlases to determine boundaries of age-related pathology: the importance of statistical method. PloS One 2015; 10: e0127939.

109.

O'Brien

Paling

Barber

. Progressive brain atrophy on serial MRI in dementia with Lewy bodies, AD, and vascular dementia. Neurology 2001; 56: 1386–1388.

110.

Cover

van Schijndel

Popescu

. The SIENA/FSL whole brain atrophy algorithm is no more reproducible at 3T than 1.5 T for Alzheimer's disease. Psychiatry Res 2014; 224: 14–21.

111.

Preboske

Gunter

Ward

. Common MRI acquisition non-idealities significantly impact the output of the boundary shift integral method of measuring brain atrophy on serial MRI. Neuroimage 2006; 30: 1196–1202.

112.

Caramanos

Fonov

Francis

. Gradient distortions in MRI: characterizing and correcting for their effects on SIENA-generated measures of brain volume change. Neuroimage 2010; 49: 1601–1611.

113.

Jovicich

Czanner

Greve

. Reliability in multi-site structural MRI studies: effects of gradient non-linearity correction on phantom and human data. Neuroimage 2006; 30: 436–443.

114.

Tadic

Jaffray

Stanescu

. Harmonic analysis for the characterization and correction of geometric distortion in MRI. Med Phys 2014; 41: 112303.

115.

Maikusa

Yamashita

Tanaka

. Improved volumetric measurement of brain structure with a distortion correction procedure using an ADNI phantom. Med Phys 2013; 40: 062303.

116.

Davids

Zollner

Ruttorf

. Fully-automated quality assurance in multi-center studies using MRI phantom measurements. Magn Reson Imaging 2014; 32: 771–780.

117.

Takao

Abe

Hayashi

. Effects of gradient non-linearity correction and intensity non-uniformity correction in longitudinal studies using structural image evaluation using normalization of atrophy (SIENA). J Magn Reson Imaging 2010; 32: 489–492.

118.

Pfefferbaum

Rohlfing

Rosenbloom

. Combining atlas-based parcellation of regional brain data acquired across scanners at 1.5 T and 3.0 T field strengths. Neuroimage 2012; 60: 940–951.

119.

Chua

Egorova

Anderson

. Handling changes in MRI acquisition parameters in modeling whole brain lesion volume and atrophy data in multiple sclerosis subjects: comparison of linear mixed-effect models. Neuroimage. Clin 2015; 8: 606–610.

120.

Pardoe

Cutter

Alter

. Pooling morphometric estimates: a statistical equivalence approach. J Neuroimaging 2016; 26: 109–115.

121.

Moorhead

Gountouna

Job

. Prospective multi-centre voxel based morphometry study employing scanner specific segmentations: procedure development using CaliBrain structural MRI data. BMC Med Imaging 2009; 9: 8.

122.

Mortamet

Bernstein

Jack

Jr. . Automatic quality assessment in structural brain magnetic resonance imaging. Magn Reson Med 2009; 62: 365–372.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB