Abstract
This study aims to compare the evidence base and practical results of the Joint Recommendations for Reporting Carotid Ultrasound Investigations in UK, published in 2009, and existing carotid scan protocols based on the Society of Radiologists in Ultrasound 2003 Consensus. A prospective sequential evaluation of the 2009 recommendations was performed at the University Hospital of Wales, Cardiff. Additional measurements in line with the recommendations were made during carotid scans. The grading of internal carotid artery stenosis using the 2003 and 2009 UK recommendations, and recommended measures of PSV, PSV ratio and St Mary’s ratio were compared. In comparison to PSV classification, PSV ratio produced lower stenosis classification in 29% and 24% of cases in the 50–69% and 70–89% stenosis bands respectively. St Mary’s ratio produced poor classification agreement across all bands, particularly the 50%–69% stenosis band. Agreement of two measures is recommended for diagnostic confidence; however, in the 50%–69% and 70%–89% stenosis bands, agreement of two measures only occurred in 70% of scans. This evaluation suggests that the use of three measurements in the 2009 recommendations complicates rather than aids diagnosis, especially in the 50%–69% and 70%–89% stenosis bands, and does not provide significant improvement over the 2003 guidelines. No evidence was found to support the combined use of the three measures.
Introduction
Duplex ultrasound is a well-established technique in the investigation of carotid stenosis. In centres where carotid duplex is performed according to a well-defined protocol, the result agrees well with other imaging modalities, with reported sensitivities and specificities of 90%–95%.1,2 Consequently, many surgeons will base their management decisions on the carotid duplex result alone.
The problem is that there is variation in the methods and criteria that different centres use for assessing and reporting carotid stenosis. 3 Carotid ultrasound is performed by people with different expertise, experience and qualifications. There is also a range of measurements currently adopted to grade the degree of internal carotid artery (ICA) stenosis: ICA peak systolic velocity (PSV), ICA end diastolic velocity (EDVICA), PSV ratio (PSVICA/PSVCCA), EDV ratio (EDVICA/EDVCCA), St Mary’s ratio (PSVICA/EDVCCA) and direct residual lumen measurement. There are many hybrid measurement algorithms, usually validated ‘in-house’, producing local reporting criteria, each with different cut-off values and derived indices for the same percentage stenosis. Often, practitioners are not sure where their own criteria came from or how they were derived. 4
The main factors contributing to this confusion are historical. First, the decision to treat a symptomatic patient with a transient ischemic attack or stroke is currently based on the results of the North American Symptomatic Carotid Endarterectomy Trial (NASCET) and the European Carotid Surgery Trial (ECST) trial.3,5–7 The velocities representing a significant stenosis on Doppler ultrasound depend on whether NASCET or ECST measurements are used. Historically, this has complicated the validation of reliable duplex criteria for the assessment of stenosis. Second, many of the early comparative studies comparing duplex criteria with angiography, prior to these trials, used different stratification of stenosis bands, compared with NASCET and ECST. 8 In 2003, Rothwell et al. 9 reanalysed ECST trial data using NASCET measurement criteria and confirmed maximum surgical benefit with >70% NASCET stenosis. The question is how this relates to ultrasound-derived velocities. Even when the stenoses are stratified according to NASCET criteria, the velocities and derived indices differ for reportedly the same degree of stenosis. 10
Recommended Diagnostic criteria of the SRU 2003 consensus, and the UK 2009 recommendations
An audit of UK practice in 2006 showed considerable uncertainty regarding what constitutes a >70% stenosis and which method is used to grade stenosis in day-to-day practice. 4 In addition, there is also some debate as to the stenosis level for which carotid endarterectomy is recommended, ranging from >50% to >70% stenosis.18–20 The UK audit indicated that 26% used NASCET and 31% used ECST, while 43% did not know which criteria were used. However, when all velocity criteria were compared, an almost equal number of ESCT, NASCET and ‘don’t know’ respondents used a PSV of >230 cm s−1 as being diagnostic of a >70% stenosis. This corresponds with the SRU consensus for NASCET stenosis of >70%. 19 Differences between centres still existed, with different centres using a range of different measurements, including PSV, EDV, PSV ratio, EDV ratio, St Mary’s ratio, direct residual lumen measurement and other hybrid measurement algorithms, with no standardisation of protocol, cut-off values or reporting.
A current difficulty in evaluating and developing standardised Doppler protocols is that the gold standard, intra-arterial angiography, used to produce the NASCET and ECST criteria, is no longer commonly used. Guidelines recommend carotid imaging to be performed twice prior to endarterectomy, preferably by two different operators, but do not stipulate different imaging modalities.21,22 In the UK, Doppler ultrasound imaging is most widely available and, therefore, usually performed for diagnosis and again prior to surgery. CT and MRI-based angiography techniques are normally only used in addition to Doppler ultrasound imaging for difficult cases; not all patients undergo a CT or MRI angiography prior to surgery. Therefore, it is currently not possible to routinely compare carotid Doppler ultrasound imaging criteria to a gold standard. Modern management places more emphasis on rapid carotid assessment. Whilst there is some debate between centres as to the level of stenosis considered appropriate for carotid endarterectomy, it is not possible to standardise management until scanning criteria, to reliably measure carotid stenosis levels, have been established.
Cautions in UK 2009 recommendations
Protocols for carotid scans in the Doppler Ultrasound Department at University Hospital of Wales (UHW) in Cardiff have been based on the SRU recommendations since 2003, with grading of ICA stenosis based on the primary recommended parameter of ICA PSV. 10 Following publication of the 2009 UK recommendations, additional measurements were made during each scan, so that the value of the recommendations could be evaluated. In this work, we report on our data and experience in clinical evaluation of the Joint Recommendations for Reporting Carotid Ultrasound Investigations in the UK published in 2009 and the Society of Radiologists in Ultrasound 2003 Consensus Guidelines for Carotid Scanning.
Methods
All carotid scans performed between October 2009 and February 2010 at the UHW in Cardiff were analysed. All scans were performed following an existing protocol based on the 2003 SRU recommendations, and additional measurements were recorded in line with the 2009 joint UK recommendations. The analysis included all carotid scan referrals, excluding repeat referrals: 1074 carotid arteries in total. Ethics committee approval was not sought because all scans were routine and were performed following recommended protocols. The analysis was performed as a registered audit within the Clinical Governance Unit of Cardiff and Vale NHS Trust (reference number 6664). No additional information other than that required for a routine scan was collected, and all analysis was performed on anonymised data.
All scans were performed by one of five experienced clinical scientists, using Toshiba Aplio XG ultrasound machines (Toshiba Medical Systems, Crawley, UK), with a linear PLT 704BT probe on the standard carotid preset used in UHW. All measurements were taken in accordance with the recommendations, with the Doppler angle between 45° and 60°, Doppler gate cursor placement and alignment and velocity cursor placement as described in the recommendations.
For each carotid scan, the ICA PSV, PSV ratio and St Mary’s ratio values were calculated and stratified into stenosis bands <50%, 50%–69%, 70%–89% and >90%, in line with the SRU and UK recommendations, using the threshold values as given in Table 1.
Both the 2003 and 2009 guidelines recommend measurement of ICA PSV, and the value common to the PSV ratio and St Mary’s ratio is the ICA PSV. In addition, the majority of comparative studies using angiography have been against ICA PSV data.11–17,23–27 Therefore, a logical starting point for a general comparison was to compare classification by the ratios against classification by ICA PSV alone. This provided both a comparison of the three individual measurement methods and a direct comparison of the 2003 and 2009 guidelines. The ICA PSV classification values used to grade stenosis were those recommended by both the 2003 and 2009 guidelines, given in Table 1. The banding outcomes from the three classification criteria: ICA PSV, PSV ratio and St Mary’s ratio were intercompared within each stenosis band.
The stratified data were analysed to determine whether or not agreement of two-measurement criteria, as recommended by the working group, aids diagnostic confidence. 3 The total number of scans in which two or more classification criteria indicated that the same stenosis band was calculated. For each stenosis band, the number of scans with agreement in banding by two or more measurement criteria was compared against the number of scans placed into that band by ICA PSV classification.
Additional analysis of the ICA and CCA velocity data was then performed in order to try to explain the results of the intercomparisons. Plots of the CCA PSV and ICA PSV values, and the mean EDV and mean PSV values for each stenosis band, were examined.
Results
Classification results
Under PSV classification, 86.5% of scans (n = 929) were placed into the <50% stenosis band, 7.4% (n = 79) were placed into the 50%–69% stenosis band, 4.3% (n = 46) were placed into the 70%–89% stenosis band and 1.9% (n = 20) were placed into the >90% stenosis band.
Figures 1–4 show the disease classification by PSV ratio and by St Mary’s ratio for the carotid scans corresponding to each of the PSV classification bands. In analysis of the 929 scans classified by PSV in the <50% stenosis band, Figure 1 shows that under PSV ratio classification, 97.8% (n = 909) were placed into the <50% stenosis band and 2.2% (n = 20) were placed into the 50%–69% stenosis band. Under St Mary’s ratio classification, 91.6% (n = 851) were placed into the <50% stenosis band, 7.5% (n = 70) were placed into the 50%–69% stenosis band and 0.9% (n = 8) were placed into the 70%–89% stenosis band.
Classification by PSV ratio and St Mary’s ratio for scans placed into the <50% stenosis band by PSV Classification by PSV ratio and St Mary’s ratio for scans placed into the 50–69% stenosis band by PSV Classification by PSV ratio and St Mary’s ratio for scans placed into the 70–89% stenosis band by PSV Classification by PSV ratio and St Mary’s ratio for scans placed into the >90% stenosis band by PSV



In analysis of the 79 scans classified by PSV in the 50%–69% stenosis band, Figure 2 shows that under PSV ratio classification, 29.1% (n = 23) were placed into the <50% stenosis band, 69.6% (n = 55) were placed into the 50%–69% stenosis band and 1.3% (n = 1) was placed into the 70%–89% stenosis band. Under St Mary’s ratio classification, 29.1% (n = 23) were placed into the <50% stenosis band, 44.3% (n = 35) were placed into the 50%–69% stenosis band, 20.3% (n = 16) were placed into the 70%–89% stenosis band and 6.3% (n = 5) were placed into the >90% stenosis band.
In analysis of the 46 scans classified by PSV in the 70%–89% stenosis band, Figure 3 shows that under PSV ratio classification, 23.9% (n = 11) were placed into the 50%–69% stenosis band, 17.4% (n = 8) were placed into the 70%–89% stenosis band and 58.7% (n = 27) were placed into the >90% stenosis band. Under St Mary’s ratio classification, 10.9% (n = 5) were placed into the 50%–69% stenosis band, 69.6% (n = 32) were placed into the 70%–89% stenosis band and 19.6% (n = 9) were placed into the >90% stenosis band.
In analysis of the 20 scans classified by PSV in the >90% stenosis band, Figure 4 shows that under PSV ratio classification, 100% (n = 20) were placed into the >90% stenosis band. Under St Mary’s ratio classification, 25% (n = 5) were placed into the 70%–89% stenosis band and 75% (n = 15) were placed into the >90% stenosis band.
Matching criteria results
Number and percentage of scans diagnosed into each stenosis band using a match of ≥2 measures compared to the number diagnosed into the band using PSV classification
Number of scans out of the total 1074, for which 2 or more measurement criteria indicated the same stenosis band
Discussion
Classification results
Figures 1–4 demonstrate the variability in stenosis banding outcomes between the three measures. This variation in outcomes, especially around the 50%–69% and 70%–89% stenosis levels, is problematic for decision making regarding surgical management. In the comparison of the PSV and PSV ratio classifications, There is good agreement between PSV ratio and PSV classification within the <50% stenosis band with 97.8% agreement and in the >90% stenosis band with 100% agreement. However, in the 50%–69% stenosis PSV band, the PSV ratio places the stenosis in a lower stenosis band in 29% of cases. More crucially, in the 70%–89% stenosis PSV band, the PSV ratio places the stenosis into a lower stenosis band in 23.9% of cases, potentially placing them in the non-surgical management group. PSV ratio also places the stenosis in a higher band in 58.7% of cases in this group.
St Mary’s ratio is believed to be the most robust index, enabling deciles to be given and is the index recommended by the working group of the 2009 recommendations. 3 However, in comparison of the PSV and St Mary’s ratio classification results, Figures 1–4 show that there is poor agreement between all bands. St Mary’s ratio classified results differently in every band, particularly the 50%–69% PSV band. St Mary’s ratio is reported to help stratify the disease in the 50%–69% stenosis group; however, Figure 2 shows that agreement within the 50%–69% stenosis PSV band is particularly poor, with placement in lower disease bands in 29.1% of cases and higher bands in 26.6% of cases.
Matching criteria results
The results in Table 3 indicate diagnosis by at least two matches, in line with the 2009 recommendations, produced a diagnosis for 94.6% (1016/1074) of scans overall, and performed well in the <50% and >90% stenosis bands. In the 50%–69% and 70%–89% stenosis bands, where correct diagnosis is more critical for decisions regarding surgical or non-surgical management, agreement of two measures only occurred in approximately 70% of scans. In 30% of these scans, the 2009 recommendations did not produce a diagnosis because all three measures indicated a different stenosis band. If all three measures indicate a different stenosis band, then which stenosis band should be diagnosed?
Another problem is that different combinations of the three measures produced the best performance for a match of two for different stenosis bands, as highlighted in Table 4. If this is the case, which two measures should be used? In the <50%, 50%–69% and >90% bands agreement by PSV and PSV ratio achieved the best performance. In the 70%–89% band, the best performance was achieved with agreement by PSV and St Mary’s ratio. Is there any scientific evidence behind choosing the two matching measures? It may be the case, because of haemodynamic changes with the development of stenosis that different measures work best at different stenosis grades. However, there needs to be more scientific evidence to determine which measures these are, at which stenosis grades. If different stenosis bands require agreement of different measures, this is acceptable, so long as there is scientific evidence to back this; currently, there is not.
The 2009 recommendations state that diagnostic confidence is gained if two or more measures are in agreement, however, there is no evidence given to support this.
ICA PSV, PSV ratio and St Mary’s ratio have all been validated against the gold standard arteriography.11–17,23–27 However, these original comparative studies, on which both sets of recommendations are based, were performed over a decade ago.
There is no evidence to support the combined use of three measures or that diagnostic confidence is gained if two or more measures are in agreement.
Further discussion of our findings
After analysing the results, several questions were posed following our initial experience. We found that in a significant number of cases, the new recommendations complicated assessments because all three measurement methods produced different diagnostic results, each placing the patient into different stenosis bands. Therefore, if all three measurements are different, which should we use? If all three measurements give the same result, why measure all three? If two measurements correlate, does this improve diagnostic accuracy? Is the PSV ratio useful? Is the St Mary’s ratio useful? Is using PSV with B-mode appearance better or is PSV too prone to error?
In order to try to explain why the PSV ratio and St Mary’s ratio classifications differ from PSV classification, further analysis was performed on the ICA and CCA velocity data. Figure 5 shows the mean CCA EDV and mean CCA PSV in each stenosis band. We can see that there is a significant decrease in CCA PSV (p = 0.003) and EDV (p = 0.034) for ‘haemodynamically significant’ (>70%) stenosis. In the PSV ratio calculation, because the CCA PSV is the denominator, a reduction in PSV will increase the ratio, which may explain why it places some stenoses in higher bands. However, this does not explain why some are also placed in lower non-surgical management bands, when PSV would indicate a higher band. This may be because a PSV ratio of >4 is used to represent a 70% stenosis. If we assume that 230 cm s−1 is a fair representation of the PSV in the ICA of a 70% stenosis, this ratio of four implies that the mean CCA PSV must be 57.5 cm s−1. Any velocity greater than this with an ICA velocity of 230 cm s−1 would reduce the PSV ratio. Figure 6 demonstrates that there is a wide variability in CCA and ICA PSVs across both normal and diseased groups. The mean CCA velocity in our normal group is 70.5 cm s−1, well above 57.5 cm s−1 consistent with a ratio of four. Our results suggest that a PSV ratio of 3.3 may be more consistent with an ICA PSV of 230 cm s−1, assuming an average CCA velocity of 70.5 cm s−1. We also suggest that in the St Mary’s ratio calculation, with EDV as denominator, the decrease in EDV with >70% stenosis may also explain why some cases are placed in higher stenosis bands. The wide variation of PSV values across normal and disease states also indicates that diseased states will not always conform to set cut-off values for PSV.
Mean CCA PSV and CCA EDV plotted against stenosis band CCA PSV plotted against ICA PSV. Line at ICA PSV = 230 cm s−1 corresponds to recommended PSV for 70% stenosis. Line at CCA PSV = 57.5 cm s−1 represents CCA PSV at PSV ratio = 4 for 70% stenosis. Line at 70.5 cm s−1 represents mean CCA velocity within normal group

The 2009 guidelines do provide cautions for the interpretation of results, given in Table 2. However, these are subjective and therefore could not be applied systematically to this data. Their effect is likely to have provided fewer matches between the classifications. Our review of the recommendations suggests that using the three measurements can complicate rather than aid diagnosis, especially in the 50%–69% and 70%–89% stenosis bands. The three do not, indeed cannot, agree in all cases and, therefore, will always produce uncertainty about the classification of stenoses. If we wish to produce recommendations that do not produce contradictory outcomes, then the question is which measurement methods should be used? PSV with B-mode appearance alone could be used, but this is open to criticism that PSV is too prone to error to be the sole classifier. However, if a new method is to be established that takes account of these ‘errors’, then those errors need to be assessed and characterised in an objective manner, which has not yet taken place. Simply adding ratio measurements to PSV does not reduce error, but it only adds confusion. If two or more measures always agree, there is no reason to use both. If they do not always agree, then there is no way of deciding between the two. Either ratio alone could be used, but they have not been as well validated historically, and St Mary’s ratio in particular shows great variability in comparison with the other two methods.
Since the original comparative studies, performed over a decade ago,11–17,23–27 B-mode and Doppler imaging capabilities have changed. There have been significant advances in the capabilities and technologies of ultrasound scanners since this time. Harmonic imaging, spatial and frequency compound imaging, broadband Doppler and improved post-processing techniques now enable a much improved visualisation of plaque and stenosis. Visual assessment of greyscale, colour and broadband Doppler information can now often be a good guide to degree of stenosis. In addition to stratifying disease severity, B-mode ultrasound can also provide additional information regarding stroke risk, for example plaque composition, which was not considered in the original angiographic imaging trials.
The 2003 SRU consensus recommending PSV along with complete visual assessment of plaque and diameter reduction appears increasingly sensible. There is no clear evidence to show that the ratios are more reliable.
Conclusion
This evaluation suggests that the use of three measurements in the 2009 recommendations complicates rather than aids diagnosis, especially in the 50%–69% and 70%–89% stenosis bands, and does not provide significant improvement over the 2003 guidelines. We suggest that a multicentre trial is now necessary to re-evaluate stenosis measurement, taking advantage of current ultrasound technologies, including direct stenosis measurement by B-mode and Doppler imaging and Doppler velocity measurements. This could produce more robust measurement methods and criteria that are diagnostically accurate, backed by strong scientific evidence and give clinical clarity.
Footnotes
DECLARATIONS
The authors have no conflicts of interest to declare
CB: Study design, Data collection, Data analysis, writing
NDP: Study design, data collection, data analysis
KNH: Study design, writing
DPC: Data collection, data analysis
RJM: Data collection, writing
PTW: Data collection
