Optimal cut-off criteria for duplex ultrasound compared with computed tomography angiography for the diagnosis of restenosis in stented carotid arteries in the international carotid stenting study

Abstract

Introduction

Previous studies that reported duplex-ultrasound cut-off criteria, based on blood velocity parameters, for the degree of stenosis in a stented carotid artery were either retrospective, or the reference test was carried out only when a patient was suspected of having restenosis at duplex ultrasound, which is likely to have resulted in verification bias. We performed a prospective study of diagnostic accuracy to find new blood velocity cut-offs in duplex ultrasound for in-stent restenosis.

Patients and methods

Stented patients within the international carotid stenting study were eligible. Patients had a carotid computed tomography angiography in addition to routine duplex ultrasound performed at a yearly follow-up. Duplex-ultrasound bloodflow velocity parameters were compared to the degree of stenosis on computed tomography angiography. The results were analysed using receiver-operating-characteristic curves.

Results

We included 103 patients in this study. On computed tomography angiography, 30 (29.1%) patients had a 30%–49% in-stent restenosis, 21 (20.4%) patients had 50%–69% in-stent restenosis and 5 (4.9%) patients a ≥70% in-stent restenosis. The cut-off value ≥50% stenosis was a peak systolic velocity of 125 cm/s (sensitivity: 63% (95% CI: 41–79), specificity: 83% (95% CI: 72–90)).

Discussion

This study provides a level 2b evidence for new cut-off values for in-stent restenosis. Unfortunately, we could not say anything about severe stenosis because of the low number of severe stenosis after one year.

Conclusions

The 125 cm/s cut-off value on duplex ultrasound is lower than found in previous studies and equal to unstented arteries. Duplex-ultrasound measurements made in stented carotid arteries should not be corrected for the presence of a stent when determining the degree of stenosis.

Keywords

Stroke computed tomography angiography duplex ultrasound stenting restenosis

Background

Carotid angioplasty with stenting (CAS) is an effective treatment for secondary prevention of stroke in patients with symptomatic carotid artery stenosis. Although CAS was associated with a higher rate of operative stroke than carotid endarterectomy (CEA) in trials such as the carotid revascularization endarterectomy versus stent trial¹ and the international carotid stenting study (ICSS),² this risk increase appears to be limited to elderly patients.³ In addition, CAS is still used in patients who are not suitable for CEA. Recently published data from ICSS show that CAS is as effective as CEA in preventing recurrent stroke after the procedure.⁴

An important factor in the follow-up of patients with stents is the occurrence of in-stent restenosis (ISR). Traditionally, the degree of stenosis in an untreated carotid artery was measured with conventional digital subtraction angiography (DSA). Because of a small but non-negligible risk of stroke or death, DSA has been replaced by non-invasive tests, such as duplex ultrasound (DUS), computed tomography angiography (CTA) or magnetic resonance angiography (MRA).^5–7 For routine evaluation of unstented carotid arteries, DUS is a well-validated diagnostic test with established cut-off criteria for different degrees of stenosis.^5,6 For measurements within stents, however, these criteria might not suffice.

In a stenosed artery, narrowing of the lumen results in higher blood flow velocities at that point. Estimating the degree of stenosis with DUS is based on this principle. The peak systolic velocity (PSV) threshold is one of the main criteria for grading internal carotid stenosis, together with diameter on B-mode image; (absence of) flow on Color Doppler image; the average PSV; poststenotic PSV and collateral flow.^8,9 However, it has been suggested that the use of PSV criteria validated in unstented arteries may overestimate the degree of (re)stenosis in stented arteries.¹⁰

We hypothesise that blood flow and blood turbulence behave differently in a stent than in a normal vessel because the stented carotid artery has a less elastic vessel wall than a non-stented one, because the metal struts of the stent will change the elastic properties of the artery. We therefore hypothesised that the PSV will be higher in a stented than in an unstented carotid artery with a similar degree of stenosis.¹¹ Previous studies reported higher PSV cut-offs for stented than for unstented arteries,^12–16 but these studies were often retrospective, or the reference test (DSA or CTA) was carried out only when a patient was suspected of having restenosis at DUS, which is likely to have resulted in verification bias.¹⁷ Furthermore, different cut-off points have been suggested for different types of stent cell design (open vs. closed).¹⁸ We report the results of the ICSS-CTA-DUS substudy, a prospective diagnostic study in which DUS parameters in patients included in ICSS who were allocated and treated with a carotid stent were compared to the ipsilateral degree of carotid stenosis on CTA.

Patients and methods

The ICSS is a randomised multicentre international trial in which patients aged older than 40 years with symptomatic atherosclerotic carotid artery stenosis measuring at least 50% in diameter were randomly allocated treatment of the stenosis by CAS or CEA.² Exclusion criteria for the trial included contraindications to stenting or surgery.

Participating centres

The participating centres were the National Hospital for Neurology and Neurosurgery, London, United Kingdom; Academic Medical Center, Amsterdam, the Netherlands; University Medical Centre, Utrecht, the Netherlands; Erasmus MC, University Medical Center, Rotterdam, the Netherlands.

Patients

The diagnostic tests were performed during routine follow-up of the ICSS trial.² In addition to the general ICSS criteria, patients were excluded for the substudy if they had a contraindication to the contrast agent used for the CTA, such as renal failure. We asked patients who received a stent to participate in this diagnostic study at one of the yearly follow-up visits, at least one year after treatment.

Ethics, consent and permission

The substudy received approval from the Multicentre Research Ethics Committee in the UK and from the Ethics Committees of the participating centres in the Netherlands. Written informed consent was obtained from all participating patients.

Stent type

In ICSS, stents and other devices used for carotid stenting were chosen at the discretion of the interventionist but had to have a CE mark. Open type stents were used as well as closed type stents. In this study, four different open type stents were used, with an open space surface (OSS) > 5 mm³: Smart stent (Cordis, Cashel, Ireland); Precise stents (Cordis, Cashel, Ireland); Protegé EV3 stents (Covidien, Plymouth, USA) and Acculink stents (Guidant, Indianapolis, USA). One closed type stent was used (OSS < 5 mm³): Carotid Wallstents (Boston scientific, Marlborough, USA).

Duplex ultrasound

DUS of the treated carotid arteries was performed by experienced vascular technician at the participating centres. They were blinded for CTA results. The PSV was recorded within the stent at the point of stenosis and in the unstented common carotid artery (CCA). The internal carotid artery (ICA)/CCA ratio was calculated with the analysis of the results.

Reference test

All CTAs were done with 16 slice CT scanner (MX8000 IDT, Philips), 40 slice (Brilliance-40, Philips), 64 slice (Brilliance-64, Philips) or 128 slice (SOMATOM Definition flash, Siemens), depending on the availability and in what participating centre the scan was performed. All CTAs were evaluated centrally by an experienced neuroradiologist at the UCL Institute of Neurology, blinded to all clinical data and the results of other diagnostic tests, at a workstation with reconstructions in the axial, sagittal and coronal direction. Sagittal and coronal reconstructions were evaluated for the presence of in-stent stenosis. Measurements to assess the severity of ISR were performed at the location with the most severe in-stent lumen reduction, either on the sagittal or coronal reconstruction according to method of the North American Symptomatic Carotid Endarterectomy Trial (NASCET).¹⁹ Wall-to-wall diameter measurements were performed by drawing a measurement line perpendicular to both vessel walls. A distal reference diameter was performed in the internal carotid artery distal to the stent where the lumen of the carotid artery has a constant diameter. These two measurements were used for calculating the degree of stenosis.

Analyses

Receiver-operating-characteristic (ROC) curves were made to find the highest sensitivity and specificity (Q* point) for the threshold of ≥30%, ≥50% and ≥70% ISR, these Q* points were defined as the optimal cut-off criteria. In these curves, the optimal cut-off criteria for different degrees of stenosis were determined. A different analysis will be made in optimal cut-off values for open versus closed type stents. To assess the effects of verification bias, a subgroup analysis was performed in patients with ≥50% stenosis based on established DUS criteria.

For all cut-off values, sensitivity, specificity, negative predictive value, positive predictive value and 95% CI were calculated. All data were analysed using IBM SPSS statistics, version 21.

Results

In sum, 103 patients in this study were included in ICSS between September 2003 and November 2008. The CTA and follow-up DUS were performed between November 2004 and July 2010. A flow chart of the inclusion is presented in Figure 1. The baseline characteristics from the time of randomisation are presented in Table 1. On CTA, 47 (45.6%) patients had ISR of 0%–30%; 30 (29.1%) of 30%–49%; 21 (20.4%) of 50%–69% and 5 (4.9%) of ≥70%. Scatter plot visualisation with linear regression for the degree of stenosis on CTA and two different ultrasound characteristics (PSV and ICA/CCA ratio) are presented in Figure 2.

Figure 1.

Flow chart.

Table 1.

Baseline characteristics.

	No. of patients (%) n = 103
Age (SD)	67 (9)
Age <70	67 (64.4)
Male	75 (72.1)
Median days between DUS and CTA	0 (0–29)
Median years between CAS and CTA	2 (1–5)
Ipsilateral carotid stenosis^a
50%–69%	94 (89.4)
70%–99%	10 (9.7)
Contralateral carotid stenosis^b
0%–49%	67 (62.2)
50%–69%	15 (14.5)
70%–99%	15 (13.5)
Occluded	5 (4.9)
Prior ipsilateral stroke^a	23 (22.1)
Treated hypertension^a	63 (60.6)
Cardiac failure^a	3 (2.9)
Angina^a	12 (11.5)
Previous myocardial infarction^a	12 (11.5)
Atrial fibrillation^a	5 (4.8)
Cardio-embolic source of stroke^a	4 (3.8)
Diabetes type 2	17 (16.3)
Insulin-dependent diabetes^a	5 (4.8)
Non-insulin-dependent diabetes^a	12 (11.5)
Current smoker^a	35 (33.7)
Ex-smoker^a	42 (40.4)
Previous CABG^a	13 (12.5)
Peripheral arterial disease^a	12 (11.5)

SD: standard deviation; DUS: duplex ultrasound; CAS: carotid angioplasty with stenting; CTA: computed tomography angiography; CABG: coronary artery bypass grafting.

Data of one patient missing.

Data of two patients missing.

Figure 2.

Scatter plot of the degree of stenosis found on CTA (x-axis) correlated with the PSV in the ICA (a) and the PSV ICA/CCA ratio (b) (y-axis).

On the ROC curves, as shown in Figure 3, the best cut-off values for a ≥50% stenosis on CTA were a PSV of 125 cm/s (sensitivity: 62%, specificity: 83%) and an ICA/CCA ratio of 1.5 (sensitivity: 76%, specificity: 74%) (Table 2).

Table 2.

Cut-off values for blood velocity parameters on DUS for ≥30% and ≥50% stenosis in a stented carotid artery.

	Cut-off value	Sensitivity (%)	95% CI	Specificity (%)	95% CI	PPV	NPV
≥30%
PSV	92 cm/s	74	59–86	71	58–82	76	69
ICA/CCA	1.2	69	54–80	67	51–79	70	65
≥50%
PSV	125 cm/s	62	41–79	83	72–90	55	86
ICA/CCA	1.5	76	54–90	76	65–85	51	91

PPV: positive predictive value; NPV: negative predictive value; ICA: internal carotid artery; CCA: common carotid artery; PSV: peak systolic velocity.

Of the five patients with ISR of >70% on CTA, two had a stenosis of 71% (PSV: 183 and 185 cm/s), one of 73% (PSV: 135 cm/s), one of 75% (PSV: 170 cm/s) and one of 76% (PSV: 220 cm/s). There were no adverse events reported in this study.

Open cell versus closed cell design stents

Five different stents were used. In six patients, the type of stent used was unknown; 77 open cell stents were used, with an OSS > 5 mm³; 20 closed cell stents were used, with an OSS < 5 mm³. There was a significant difference in >50% ISR measured with CTA in open vs. closed cell stents; 14 out of 77 (18%) patients with an open cell stent had a >50% ISR on CTA versus 10 out of 20 (50%) in closed cell stents (p = 0.003). The PSV in open cell stents ranged from 40 to 220 cm/s versus 78 to 220 cm/s in closed cell stents. ROC curves for open and close stents were separately made. The cut-off value for >50% ISR in open cell stents was 118 cm/s, sensitivity: 65% (95% CI: 35–87), specificity: 78% (95% CI: 66–87). The cut-off value for >50% ISR in closed cell stents was 128 cm/s, sensitivity: 80% (95% CI: 44–97), specificity: 70% (95% CI: 35–93).

Subgroup analysis

In an analysis limited to the 29 ICA’s with a PSV >125 cm/s on DUS, for a >50% ISR, the optimal cut-off PSV was 159 cm/s, sensitivity: 56% (95% CI: 25–81), specificity 56% (95% CI: 30–80) and an ICA/CCA ratio of 1.9, sensitivity: 63% (95% CI: 35–85), specificity: 62% (95% CI: 32–86).

Discussion

In this diagnostic substudy of the ICSS, we found substantially lower cut-off values for flow velocity parameters on DUS in a stented ICA than previously reported, with cut-off values for a ≥50% stenosis of 125 cm/s for the PSV and 1.5 for the ICA/CCA ratio.

We think that the main reason for this big difference with previous studies would be fact that there was no possibility of verification bias in our study. We did not include patients by the results of DUS, we included patients regardless of PSV and stenosis on DUS. Therefore, we could have lower PSV’s and find more >50% stenosis on CTA with lower blood flow velocities, which we otherwise would have missed, therefore giving a lower cut-off value. Another possibility is that the quality of CTA is improved over the last years, which gives a more clear image of the lumen which makes a more precise measurement possible. In this study, we used the NASCET measurement to determine ISR. With the more distal and (usually more narrow) ICA in the NASCET measurement – compared to the more proximal comparison of the European Carotid Surgery Trial (ECST) or Common Carotid (CC) measurement – you might expect a higher velocity with the same degree of ISR. Despite this, we found a lower cut-off value for ISR than found in previous studies. According to this idea, the cut-off value with ECST measurement in our study might even have been lower, but unfortunately we cannot say this with the results we have because we did not perform ECST or CC measurements in our study.

Over the period of one to five years after CAS, we only had five patients (4.9%) with a severe stenosis (>70%). There was no correlation in ISR and amount of years after procedure. In the whole ICSS trial, using commonly defined ultrasound criteria (not adjusted for stents), the cumulative incidence of severe restenosis five years after completed treatment was 10.8% in the CAS group and 8.6% in the CEA group, a difference which was not statistically significant.⁴ Therefore, we were unable to determine a threshold PSV for severe stenosis.

As a reference test, we chose CTA to compare to the velocity parameters measured with DUS. A diagnostic test that provides clear images of the lumen of the internal carotid artery is crucial, because a NASCET-like stenosis measurement is necessary as a reference to estimate the optimal PSV cut-offs for DUS. CTA offers high spatial resolution and contrast resolution, and it is a fast technique. The diagnostic accuracy of CTA compared to DSA, to diagnose a 70%–99% stenosis for unstented arteries, was calculated in several studies.^5,7 A systematic review reported a sensitivity and specificity of 77% (95% CI: 68%–84%) and 95% (95%CI: 91%–97%), respectively,⁵ but these figures are likely to underestimate the accuracy of CTA in the present study because the quality of CTA has improved substantially since the studies included in the review. Nonetheless, CTA in stented arteries gives a bigger challenge then unstented arteries. Artificial lumen narrowing because of the material of the stent is a known earlier described phenomenon.²⁰ The lumen narrowing differs between different kinds of stents, different materials and different methods of measuring. In our study, there were two different materials used for the stents: nitinol (an alloy of nickel and titanium) and a cobalt alloy (cobalt-chromium-iron-nickel-molybdenum). Lettau et al.²⁰ compared, in a small study, in vitro different types of carotid artery stents in CTA and 1.5 T and 3 T MRA and found a mean artificial narrowing of 28.1% and 29.2% for nitinol and cobalt alloy, respectively, on CTA. The article, however, does not mention the kind of CT-scanner used. The same author compared in a different study 16 carotid stents in vitro and found artificial lumen narrowing ranging from 18.8% to 59.9% with significant differences in stent type and diameter, but no significant difference for the use of 16 -, 64 - or 320-row CT scanners.²¹ These were small studies, and there are no studies or reviews comparing CTA with DSA, the golden standard, in stented carotid arteries. A 2007 review compared CTA with DSA in stented coronary arteries and found a sensitivity and specificity of 85% (95% CI 78%–90%) and 97% (95% CI: 95%–98%) for 16 - to 64-detector row CT scanners,²² which are similar when compared with CTA and DSA in unstented carotid arteries. There was no difference between the different types of CT scanners. We recognise the shortcomings of CTA in stented carotid arteries but we think, when looking at other studies in different stented arteries, it is a feasible method for detecting ISR.

One of the advantages of DUS for examining CAS patients is the ability to perform as much examination is wanted, since there is no risk for the patients. The comparison between DUS and CTA or DSA in a stented ICA has been done before and higher cut-off values were found in all degrees of stenosis, but previous studies often were retrospective, or the reference test (DSA or CTA) was carried out only when a patient was suspected of having restenosis at DUS, which may result in verification bias.^12–16 Verification bias is introduced if the decision to perform the reference standard procedure depends on the results of the test under investigation, precluding a reliable estimate of the diagnostic accuracy of the latter.¹⁷ To our knowledge, this is so far the only prospective study without any possibility of verification bias that compares DUS to a reference test for ISR in a stented ICA.

We identified five unique diagnostic series on in-stent stenosis measurements with DUS compared to a reference test (CTA or DSA), where ISR was measured according to NASCET, whom all proposed new criteria.^12–16 Table 3 summarises the five series in this study. In general, the cut-off values are higher than those reported for unstented arteries. But these studies where either retrospective or included the patients if they had a >50 stenosis on DUS. Only Kwon et al.²³ reported a series of patients all undergoing both DUS and the reference test, CTA. This study, however, was too small (n = 27) to provide new in-stent cut-off criteria.

Table 3.

Previously reported DUS cut-off values for stenosis measurements within a stent.

First author	Year	Population size (n)	Prospective	Subgroup selection^a	Subgroup (n)	Reference test	PSV^b ≥30%	PSV ≥50%	PSV ≥70%	PSV ≥80%
Stanziale¹⁶	2005	605	No	Yes	118	DSA		225	350
Chi¹⁵	2007	260	Yes	Yes	13	DSA		240	450
Aburhama¹²	2008	144	Yes	Yes	144	CTA	178	278		403
Lal¹³	2008	225	No	Yes	99	CTA/DSA		220		340
Zhou¹⁴	2008	256	No	Yes	22	DSA			300
Bosch	2016	103	Yes	No	–	CTA	92	125

DSA: digital subtraction angiography; CTA: computed tomography angiography.

Selection of a subgroup based on DUS results (the test under evaluation) indicates possible verification bias. These subgroups are compared to the reference test in order to obtain the listed DUS cut-off values.

Suggested cut-offs for in-stent restenosis measurements based on the peak systolic velocity (PSV); in cm/s.

To emphasise the importance of avoiding verification bias, we did a subgroup analysis with only patients with a PSV >125 cm/s, to mimic a study with DUS as a selection criterion. This shows a difference in optimal cut-off value of 34 cm/s only by using different selection criteria (Table 4). This could be one explanation for the higher cut-off values for the same degree of stenosis observed in previous studies.

Figure 3.

ROC curves for the cut-off values for PSV in, respectively, (a) ≥30% stenosis (AUC 0.78; p-value ˂0.001; 95% CI 0.69–0.87); (b) ≥50% stenosis (AUC 0.73; p-value ˂0.001; 95% CI 0.61–0.85); and for the ICA/CCA ratio in (c) ≥30% stenosis (AUC 0.73; p-value ˂0.001; 0.63–0.83) and (d) ≥50 stenosis (AUC 0.73; p-value 0.001; 95% CI 0.61–0.86).

Table 4.

Optimal cut-off criteria with and without verification bias.

No verification bias			Verification bias
PSV (cm/s)	Sensitivity	Specificity	PSV (cm/s)	Sensitivity	Specificity
38	100	0	128	100	0
49	100	8	129	94	0
56	94	12	131	94	23
77	85	31	133	88	23
81	85	47	138	69	23
97	77	62	146	63	23
109	65	75	154	56	46
125 ^a	62	83	159 ^a	46	63
131	58	87	168	39	63
145	39	87	177	39	75
158	35	92	181	39	61
177	15	94	184	13	61
187	4	94	192	4	69
197	4	96	197	4	84
221	0	100	221	0	100

Left: Optimal cut-off criteria for a ≥50% stenosis in this study. Right: hypothetical optimal cut-off criteria if only patients with a PSV >125 cm/s on DUS would have been included in this study.

PSV: peak systolic velocity.

Optimal cut-off criteria for a >50 stenosis with (right) and without (left) verification bias.

Although differences in PSV’s in open cell stent designs versus closed cell stent designs are well reported, in this study, the difference between these two groups remained small, with an optimal cut-off value for a ≥50% of 118 cm/s for open cell stents versus 128 cm/s in closed cell stents.

A limitation of this study is that we had very few severe ISR (≥70%). Therefore, we could not reliably estimate cut-off values for severe ISR. In general, in the entire ICSS study, ISR occurred less frequently, later in time, and with lower degree of stenosis, than expected.⁴ Also the big spread in PSV values (35–220 cm/s) is one of the difficulties. It shows a lower cut-off all together but comes with a lower sensitivity and specificity. We can partially explain this by not including just cases with severe stenosis on CTA or ultrasound, which are more likely to have higher velocities. By including everybody, one would expect a more spread scatter plot. Another point to discuss is that we do not know much about the change a proximal ICA undergoes after placement of the stent. Some stents unfold incomplete giving different velocities in the ICA. A good way to get to know more about velocity change in stents would be to monitor changes in stent properties and blood flow velocities shortly after the procedure and after one year. Unfortunately, we only have one measurement to compare with.

The optimum cut-off value of 125 cm/s we report for stented arteries is identical to the ‘strandness’ velocity criteria widely used in ultrasound laboratories to identify ≥50% stenosis in unstented arteries. Our data therefore suggest that DUS measurements made in stented carotid arteries should not be corrected for the presence of a stent when determining the degree of carotid stenosis.

Conclusion

This study showed an optimal cut-off value of a PSV of 125 cm/s on DUS for the diagnosis of a 50% ISR and 92 cm/s for a 30% stenosis, which is lower than found in previous studies and equal to unstented arteries. The difference in results with the previous studies may be explained by avoiding verification bias. Ultrasound measurements made in stented carotid arteries should not be corrected for the presence of a stent when determining the degree of carotid stenosis.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: ICSS and this substudy have been funded by grants from the Medical Research Council, The Stroke Association, Sanofi Synthelabo and the European Commission. LHB was supported by grants from the Swiss National Science Foundation (PBBSB-116873) and the University of Basel. MMB’s Chair in Stroke Medicine at University College London is supported by the Reta Lila Weston Trust for Medical Research. Part of this work was undertaken at UCLH/UCL who received a proportion of funding from the Department of Health’s NIHR Biomedical Research Centres funding scheme. HBvdW is supported by a grant from the Dutch Heart Foundation (2010T075).

Informed consent

Written informed consent was obtained from all participating patients.

Ethical approval

The substudy received approval from the Multicentre Research Ethics Committee in the UK and from the Ethics Committees of the participating centres in the Netherlands.

Guarantor

PJN.

Contributorship

This study was designed by PJN and MMB. The data were analysed by ID and JH. FTMB compared the data and wrote the manuscript. The manuscript was reviewed and approved by all co-authors. MMB and PJN contributed equally to this article.

Acknowledgements

None.

References

Hobson

. Update on the Carotid Revascularization Endarterectomy vs. Stent Trial (CREST) protocol. J Am Coll Surg 2002; 194: S9–S14.

Featherstone

Brown

Coward

. ICSS Investigators. International carotid stenting study: protocol for a randomised clinical trial comparing carotid stenting with endarterectomy in symptomatic carotid artery stenosis. Cerebrovasc Dis 2004; 18: 69–74.

International Carotid Stenting Study Investigators. Carotid artery stenting compared with endarterectomy in patients with symptomatic carotid stenosis (International Carotid Stenting Study): an interim analysis of a randomised controlled trial. Lancet 2010; 375: 985–997.

Bonati

Dobson

Featherstone

et al.

Long-term outcomes after stenting versus endarterectomy for treatment of symptomatic carotid stenosis: the International Carotid Stenting Study (ICSS) randomised trial. Lancet 2015; 385: 529–538.

Wardlaw

Chappell

Best

et al.

NHS Research and Development Health Technology Assessment Carotid Stenosis Imaging Group. Non-invasive imaging compared with intra-arterial angiography in the diagnosis of symptomatic carotid stenosis: a meta-analysis. Lancet 2006; 367: 1503–1512.

Nederkoorn

van der Graaf

Hunink

. Duplex ultrasound and magnetic resonance angiography compared with digital subtraction angiography in carotid artery stenosis: a systematic review. Stroke 2003; 34: 1324–1331.

Koelemay

Nederkoorn

Reitsma

et al.

Systematic review of computed tomographic angiography for assessment of carotid artery disease. Stroke 2004; 35: 2306–2312.

Hunink

Polak

Barlan

et al.

Detection and quantification of carotid artery stenosis: efficacy of various doppler velocity parameters. AJR Am J Roentgenol 1993; 160: 619–625.

von Reutern

Goertler

Bornstein

et al.

Grading carotid stenosis using ultrasonic methods. Stroke 2012; 43: 916–921.

10.

Lal

Hobson

Goldstein

et al.

Carotid artery stenting: is there a need to revise ultrasound velocity criteria?

J Vasc Surg 2004; 39: 58–66.

11.

Nederkoorn

Brown

. Optimal cut-off criteria for duplex ultrasound for the diagnosis of restenosis in stented carotid arteries: review and protocol for diagnostic study. BMC Neurol 2009; 9: 36.

12.

Aburahma

Abu-Halimah

Bensenhaver

et al.

Optimal carotid duplex velocity criteria for defining the severity of carotid in-stent restenosis. J Vasc Surg 2008; 48: 589–594.

13.

Lal

Hobson

Tofighi

et al.

Duplex ultrasound velocity criteria for the stented carotid artery. J Vasc Surg 2008; 47: 63–73.

14.

Zhou

Felkai

Evans

et al.

Ultrasound criteria for severe in-stent restenosis following carotid artery stenting. J Vasc Surg 2008; 47: 74–80.

15.

Chi

White

Woods

et al.

Ultrasound velocity criteria for carotid in-stent restenosis. Catheter Cardiovasc Interv 2007; 69: 349–354.

16.

Stanziale

Wholey

Boules

et al.

Determining in-stent stenosis of carotid arteries by duplex ultrasound criteria. J Endovasc Ther 2005; 12: 346–353.

17.

Begg

Greenes

. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics 1983; 39: 207–215.

18.

de Borst

Meijer

et al.

Effect of carotid angioplasty and stenting on duplex velocity measurements in a porcine model. J Endovasc Ther 2008; 15: 672–679.

19.

North American Symptomatic Carotid Endarterectomy Trial Collaborators. Beneficial effect of carotid endarterectomy in symptomatic patients with high grade carotid stenosis. N Engl J Med 1991; 325: 445–453.

20.

Lettau

Sauer

Heiland

et al.

Carotid artery stents: in vitro comparison of different stent designs and sizes using CT angiography and contrast-enhanced MR angiography at 1.5T and 3T. AJNR 2009; 30: 1993–1997.

21.

Lettau

Kotter

Bendszus

et al.

Carotid artery stents on CT angiography: in vitro comparison of different stent designs and sizes using 16-, 64- and 320-row CT scanners. J Neuroradiol 2014; 41: 259–268.

22.

Sun

Davidson

Lin

. Multi-detector row CT angiography in the assessment of coronary in-stent restenosis: a systematic review. Eur J Radiol 2009; 69: 489–495.

23.

Kwon

Jung

Sheen

et al.

CT angiography of stented carotid arteries: comparison with Doppler ultrasonography. J Endovasc Ther 2007; 14: 489–497.