Software-based on-site estimation of fractional flow reserve using standard coronary CT angiography data

Abstract

Background

The significance of a coronary stenosis can be determined by measuring the fractional flow reserve (FFR) during invasive coronary angiography. Recently, methods have been developed which claim to be able to estimate FFR using image data from standard coronary computed tomography angiography (CCTA) exams.

Purpose

To evaluate the accuracy of non-invasively computed fractional flow reserve (cFFR) from CCTA.

Material and Methods

A total of 23 vessels in 21 patients who had undergone both CCTA and invasive angiography with FFR measurement were evaluated using a cFFR software prototype. The cFFR results were compared to the invasively obtained FFR values. Correlation was calculated using Spearman’s rank correlation, and agreement using intraclass correlation coefficient (ICC). Sensitivity, specificity, accuracy, negative predictive value, and positive predictive value for significant stenosis (defined as both FFR ≤0.80 and FFR ≤0.75) were calculated.

Results

The mean cFFR value for the whole group was 0.81 and the corresponding mean invFFR value was 0.84. The cFFR sensitivity for significant stenosis (FFR ≤0.80/0.75) on a per-lesion basis was 0.83/0.80, specificity was 0.76/0.89, and accuracy 0.78/0.87. The positive predictive value was 0.56/0.67 and the negative predictive value was 0.93/0.94. The Spearman rank correlation coefficient was ρ = 0.77 (P < 0.001) and ICC = 0.73 (P < 0.001).

Conclusion

This particular CCTA-based cFFR software prototype allows for a rapid, non-invasive on-site evaluation of cFFR. The results are encouraging and cFFR may in the future be of help in the triage to invasive coronary angiography.

Keywords

Cardiac computed tomography angiography (CTA)heart arteries adults computer applications – detection/diagnosis

Introduction

Invasive coronary angiography has for a long time been the method of choice for visualizing the lumen of the coronary arteries, in order to assess the presence or absence of atherosclerotic obstruction and to determine the need for surgical or intravascular intervention.

However, research has shown that visual assessment of coronary stenoses of intermediate severity during the coronary angiography procedure may not always be accurate and the concordance between different interventionists may also be poor (1,2). Data from the FAME study also showed that the functional severity of a stenosis was difficult to judge from angiographic images alone (3). Instead, the determination of the fractional flow reserve (FFR) during invasive coronary angiography has been established as the gold standard in determining the significance of a stenosis. FFR expresses the relative decrease in coronary artery flow caused by a stenosis during maximal coronary microvascular dilation, and is determined by measuring the pressure upstream and downstream of the stenosis. The use of FFR was first described in 1993 (4) and previous studies have shown that a stenosis with FFR ≤0.75 almost always induces ischemia, while a stenosis with FFR 0.75–0.80 is in a “gray zone” area in terms of significance (5 –7). However, FFR is an invasive and thus expensive method which requires an additional catheter passage with a small but significant procedural risk for the patient, and requires the use of radiation and of a contrast agent. Thus, the development of non-invasive alternatives is essential in order to decrease both the cost and the risk of unnecessary invasive procedures.

Recently, methods have been developed which, through the utilization of computational fluid dynamics, claim to be able to estimate FFR using image data from standard coronary computed tomography angiography (CCTA) exams (8,9). Previously described software has required CT datasets to be transferred to the vendor for post-processing before being returned to the referring physician (10). In this pilot study we investigate a new workstation-based software which allows for rapid on-site post-processing of the CCTA image data, followed by an FFR computation. The aim of this study was to evaluate the accuracy of CCTA-based FFR (cFFR) obtained through the use of a prototype software, compared with invasively obtained FFR-values (invFFR) from the same patients.

Material and Methods

The study was conducted according to principles set forward in the Declaration of Helsinki and according to good clinical practice. Permission was obtained from the regional ethical review board in Linköping, Sweden. The principle of informed consent was waived, due to the retrospective nature of the study.

Patients

All CCTA examinations performed at our center between September 2009 and March 2012 (approximately 1400 exams) were screened. Eligible patients were those who had been referred to invasive angiography within 120 days of the CCTA where invFFR had been measured in the right coronary artery (RCA), the left anterior descending artery (LAD) or the circumflex artery (Cx). In all, 63 patients with invasive FFR measurements in 72 vessels were identified.

Lack of images documenting the position of the pressure guidewire during the invasive angiographic procedure was the most frequent reason for exclusion, in total 45 patients (71%) representing 49 vessels (68%). Additional reasons for exclusion are shown in Table 1.

Table 1.

Vessel inclusion and exclusion.

	Vessels (n)	Patients (n)
Total number included	71	63
Excluded
No pressure guidewire image	22	21
Suboptimal image quality - Motion artifacts (n = 2/1) - Excessive image noise (n = 1/1) - Insufficient contrast filling (n = 3/2) - Misaligned field of view (n = 1/1)	7	5
Previous surgery/vascular intervention	9	7
Ostial stenosis	2	2
Rare findings - Floating thrombus in RCA (n = 2/1) - Myxoma in left atrium (n = 1/1)	3	2
Missing thin section data	4	4
Obvious discrepancy invFFR angiography	1	1
Remaining - LAD (n = 16) - Cx (n = 3) - RCA (n = 4)	23	21

The final sample consisted of 23 FFR measurements in 21 patients (11 men, 12 women; three diabetics, 10 ex-smokers, and two current smokers). Further demographics are described in Table 2.

Table 2.

Patient demographics.

	Mean	Median	Range
Age* (years)	60	59	40–74
Body mass index (BMI)	27.6	26.3	17.6–37.9
Systolic blood pressure (mmHg)	146	147	103–193
Diastolic blood pressure (mmHg)	78	79	56–94
Heart rate CCTA (beats per minute)	61	61	53–84
Time from CCTA to invasive angiography (days)	49	49	4–106
Dose CCTA^† (mSv)	3.9	2.55	0.7–14.1
Dose invasive angiography^‡ (mSv)	9.7	7.08	1.4–18.7
Diabetes mellitus (DM)	None: 18 Insulin dependent DM: 2 Non-insulin dependent DM: 1
Smoking	None: 9 Ex-smokers: 10 Current smokers: 2

At the time of cardiac computed tomography angiography (CCTA).

†

Based on conversion factor E_DLP = 0.017.

‡

Based on conversion factor E_DAP = 0.18.

Image acquisition and reading

All CT images were acquired according to a standard CCTA protocol using a dual source/dual energy CT scanner (Definition FLASH, Siemens Healthcare, Forchheim, Germany) in mono-energetic CCTA mode. In the standard setting, the exam would be performed with prospective gating, with tube voltage set at 100 kV, ref. mAs at 185 A, and the rotation time at 0.285 s. Depending on body weight and heart rate, sequential imaging or a high pitch FLASH protocol could be selected by the technician.

The CCTA datasets were anonymized and diastolic thin section 0.75 mm datasets were transferred to a standalone workstation equipped with a software research prototype (Siemens cFFR, versions 1.4 and 1.7; Siemens Healthcare, currently not commercially available). Vessel centerlines and luminal contours were automatically generated by the software and manually adjusted by two radiologists (JDG and MS, both with more than 5 years of experience as specialists) working in consensus. Atherosclerotic vascular segments were identified and their extent manually defined. Thus a specific three-dimensional (3D) mesh was constructed for each patient (Fig. 1).

Fig. 1.

Overview of segmentation process, with the left anterior descending artery (LAD) in cross-sectional and curved MPR view, and the coronary tree mesh. Stenotic segments are manually defined. Vessel contours are automatically outlined but can be manually adjusted.

The calculation of non-invasive FFR was based on lumped models of the heart and of the coronary circulation. The model of the coronary stenosis included effects of viscosity, stenosis length and coefficients of turbulence, fluid pressure, and viscosity (11,12). The resting state boundaries were determined by physiological data in the form of left ventricular mass, heart rate, and systolic and diastolic blood pressure, with left ventricular mass and heart rate being automatically estimated from the CT scan data. The blood pressure values recorded at the time of the invasive angiography were used, as blood pressure is not routinely recorded in conjunction with CCTA at our institution. The boundary conditions at hyperemia were then derived from the resting state values via an algorithm simulating the vasodilation and decrease in microvascular resistance usually induced by adenosine (12,13). A full order cFFR algorithm was applied to the atherosclerotic/stenotic vascular segments while a reduced order algorithm, requiring far less computer power, was used for the normal segments. Coronary flow for the whole coronary tree was estimated by coupling the reduced and full order models. The results were presented as a color-coded, 3D cFFR-map of the coronary vascular tree, with the point of interest being freely adjustable (Fig. 2).

Fig. 2.

CT-based fractional flow reserve (cFFR) color map in the same patient as in Fig. 1, with cFFR = 0.82 in the LAD. The point of cFFR evaluation is freely adjustable.

The cFFR-map was compared to the corresponding invasive coronary angiography images and the exact point of cFFR estimation was deduced from the position of the pressure guide wire by an expert cardiac interventionist (>5000 interventions), who was blinded to both the CCTA and the cFFR segmentation process (Fig. 3).

Fig. 3.

Invasive angiography from the same patient as in Figs. 1 and 2, with pressure guidewire in the left anterior descending artery (LAD). FFR = 0.81.

Because of the learning curve involved in the use of new software, we did not measure the time needed for performing the cFFR post-processing in the cases included in this study. However, after the data collection was completed, a further 11 similar cases were separately processed in order to evaluate the time required for cFFR post-processing and subsequent cFFR calculation. The processing in these 11 cases was performed exclusively on a standalone, state-of-the-art personal computer (Fujitsu Celsius, 2 × CPU Intel Xeon E5-2670 v2 @ 2.50 GHz and 32 GB RAM), using the Siemens cFFR prototype v1.7. Data on smoking and diabetes was retrieved from the Swedish Swedeheart register.

Statistical analysis

The agreement between cFFR and invFFR was expressed as an intraclass correlation coefficient (ICC). A Bland–Altman plot displaying bias and limits of agreement was constructed. The correlation between invFFR and cFFR was expressed as a Spearman’s Rank correlation coefficient. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and the accuracy of cFFR in predicting significant stenosis (defined from the invasive procedure as both FFR ≤0.80 and FFR ≤0.75) were calculated per vessel as well as per patient using invFFR as the reference. Since the result is dependent on the segmentation of the coronary stenosis, we also performed inter- and intra-observer reproducibility measurements on the severity of the stenosis in terms of the smallest diameter in a total of 10 cases. The results were expressed as an ICC. All statistical calculations were performed using IBM SPSS v.22 (IBM SPSS, Chicago, IL, USA).

Results

The agreement between invFFR and cFFR expressed as an ICC was 0.73 (P < 0.001). The Bland–Altman analysis, as presented in Fig. 4, showed a low systemic bias and tight limits of agreement. The Spearman’s correlation coefficient was ρ = 0.77 (P < 0.001).

Fig. 4.

Bland–Altman plot comparing invasively obtained fractional flow reserve (invFFR) and CT-based fractional flow reserve (cFFR) on a per-lesion basis. The systematic bias is low (95% limits of agreement are −0.18 to 0.12).

The mean cFFR value for the whole group was 0.81, and the corresponding mean invFFR value was 0.84. The cFFR sensitivity, specificity, PPV, and NPV for significant stenosis as defined by invFFR ≤0.80 and invFFR ≤0.75 are presented in Table 3. The distribution of the results from cFFR and the corresponding invasive angiography with FFR are presented in Tables 4 and 5.

Table 3.

Results for CT-based fractional flow reserve (cFFR) in detecting significant stenosis, defined as both FFR ≤0.80 and FFR ≤0.75.

	FFR ≤0.80 per lesion (n = 23)	FFR ≤0.80 per patient (n = 21)	FFR ≤0.75 per lesion (n = 23)	FFR ≤0.75 per patient (n = 21)
Sensitivity	0.83	0.83	0.80	0.80
Specificity	0.76	0.80	0.89	0.88
Positive predictive value (PPV)	0.56	0.63	0.67	0.67
Negative predictive value (NPV)	0.93	0.93	0.94	0.93
Accuracy	0.78	0.81	0.81	0.86

Table 4.

Two-by-two frequency table for FFR ≤0.80.

	InvFFR ≤0.80 (ref.)	InvFFR >0.80 (ref.)
cFFR ≤0.80 (lesions/pat)	5/5	4/3
cFFR >0.80 (lesions/pat)	1/1	13/12

Table 5.

Two-by-two frequency table for FFR ≤0.75.

	InvFFR ≤0.75 (ref.)	InvFFR >0.75 (ref.)
cFFR ≤0.75 (lesions/pat)	4/4	2/2
cFFR >0.75 (lesions/pat)	1/1	16/14

The mean and median time required for the complete cFFR post-processing procedure was 45 and 46 min, respectively (range, 34–61 min). The mean and median time required for the computation of cFFR once the semi-automated segmentation process was completed was 4 min 16 s, and 3 min 28 s, respectively (range, 2 min 10 s – 12 min 30 s). The intra-observer variability (for stenosis diameter) was 0.994 for observer 1 (JDG) and 0.996 for observer 2 (MS). The inter-observer variability was 0.966.

Discussion

Ever since its conception in the 1960s, invasive coronary angiography has been the method of choice for visualizing the coronary arteries and for stenosis grading. Unfortunately, several studies have shown that visual evaluation alone may not be good enough to determine the significance of a stenosis (1,2,14). Visual estimation is in itself difficult, especially in the borderline region between significant and non-significant stenosis. This is partly due to the fact that the angiographic images, although obtained from a number of different angles, are always two-dimensional (2D) and the information needs to be transformed into a 3D volume by the interventionist. This also holds for quantitative coronary angiography, which may add objective measurement but is limited by the 2D representation of a 3D structure (15). When the concept of FFR was introduced, it provided the interventionist with an objective and probably more exact method for determining the hemodynamic significance of a given stenosis. However, as previously stated, invasive coronary angiography with measurement of FFR may have some downsides. Any invasive method carries a risk for complications, in this case primarily arrhythmias, vascular injury, and ischemia (16,17). The inevitable use of iodinated contrast agent also carries a risk of anaphylactic reactions as well as renal impairment. Radiation is also a consideration, with a described average dose of approximately 7 (range, 2–15) mSv for diagnostic coronary angiography (18). Last, but not least, the high cost of invasive procedures also needs to be taken into consideration. Thus, a non-invasive method for predicting who will benefit from an invasive procedure is essential (19). The methods of choice have been exercise testing, SPECT, and to some extent stress echocardiography, with the addition of cardiac MR (cMR) during the last decade. Different kinds of CT-based perfusion methods are also being evaluated (20). CCTA has previously been most successful as a screening tool in patients with a low to intermediate pre-test risk but when a stenosis has been detected additional, sometimes invasive, tests have frequently been applied in order to determine the degree of stenosis and thus its significance (21). However, this could change if CCTA-based cFFR proves to be reliable, possibly reducing the need for further investigation in a number of patients.

The software used in this study is designed for on-site evaluation of cFFR. This allows for an efficient patient flow, as the results from the cFFR evaluation can be available within 1 h of the CCTA being performed. This is in sharp contrast to other studies, where the CCTA dataset has been transferred to an external laboratory for evaluation (10,22).

Although still only a prototype, the results obtained are encouraging and in line with the results obtained by other authors using the same software. In the studies by Coenen et al. (23) and Renker et al. (24), the sensitivity for significant stenosis (defined as FFR ≤0.80) was 0.81 and 0.85, respectively, and the Spearman correlation coefficient was 0.59 and 0.66, as compared to a sensitivity of 0.83 and a Spearman correlation coefficient 0.77 in this study. The results are also in the same range as those obtained in the DeFACTO and NXT studies which also used a flow computation model for determining cFFR (22,25). In particular, the high negative predictive value of >0.90 has the potential to be useful in the triage to invasive angiography. We did get a number of false negative and false positive results but in the case of the false negative result (which could potentially have the worst consequences), the difference between the invFFR and the cFFR was only 0.01 (for FFR ≤0.80). With the false positive cases, the difference varied between 0.03 and 0.14. The cause of these discrepancies is difficult to explain but they could be the result of inadequate segmentation or due to the lack of accurate blood pressure data. However, it is clear that the accuracy is best when analyzing uncomplicated stenoses. Even though there seems to be a slight trend towards an overestimation in complex lesions where the stenosis is long or even made up of several stenoses in series, the lack of data at this stage implies that it might still be wise to treat these as potentially significant, regardless of the cFFR. The software at its current level of development does not support evaluation of complex lesions. It may also be prudent to use the higher cutoff of cFFR ≤0.80 in a triage situation, in order to avoid under-treatment of borderline lesions.

The fact that the cFFR calculation is based on simulated hyperemia also warrants some caution, as the same systematic reduction of resistance may not be applicable across all patient sub-groups (11 –13).

The average time of approximately 45 min required for the analysis of cFFR is in line with the results obtained by Coenen et al. (23). It is likely that in the near future, the time required for evaluation will be even shorter as the automated segmentation process is further improved. The already short cFFR computation time of approximately 3–5 min on a standalone personal computer can probably be expected to become a near real-time process as the software is integrated with the CT scanner. In terms of patient flow and cost, this compares very favorably with other solutions where the CCTA dataset has to be uploaded to an external site for analysis, with a turnaround time of 24 h or more. However, it remains to be determined whether the accuracy of this particular software is superior to available competing methods, and whether this potential difference would affect the overall risk and treatment cost for cardiac patients.

There are several limitations to this study. The most important is the low number of included patients, which was due to a desire to use cases from one particular CT scanner only, thus avoiding variation between studies due to vendor specific scanner limitations. Since this was a retrospective study on clinical exams acquired without the requirements displayed in a pre-specified research protocol, a relatively large number of potential cases had to be excluded due to a lack of image documentation of the pressure guide wire position at the point of measurement, as we felt that this kind of documentation was necessary in order to ensure the most accurate comparison between invasive FFR and cFFR. A comparison with the diagnostic performance of “plain” CCTA would have been interesting but could have been affected by “verification bias”, since invasive coronary angiography was performed to assess the severity of suspected coronary lesions on the CCTA (26). Additionally, the blood pressure entered into the calculation of cFFR was taken from a recording at the invasive study since blood pressure is not routinely measured at CCTA in our center. In this study we primarily investigated stenoses in the RCA, LAD and Cx, avoiding the left main artery (LM), in line with recommendations from the developer of the software. Only one potential case with a stenosis being present in the LM was found in the material.

Most of the above mentioned limitations can be solved with a prospective study design.

In conclusion, this study clearly shows the potential of predicting the invasive FFR using CCTA image data only. Especially the power to exclude the presence of a significant stenosis might be useful in avoiding unnecessary invasive procedures. The software has the potential to be included in workstations presently used for on-site CCTA evaluation and thus offer a much improved workflow compared to competing solutions on the market.

Footnotes

Acknowledgements

The authors thank Petter Quick (RN), CMIV for scanning the patients and the Seldinger unit of the Department of Cardiology, Linköping University Hospital for providing invasive FFR data. Max Schöbinger and Chris Schwemmer from Siemens Healthcare provided background information and valuable support regarding the cFFR software. Equipment was provided by CMIV, Linköping University, and by Siemens Healthcare.

Declaration of conflicting interests

The The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Grant support was obtained from the Department of Radiology, Region Östergötland and the Swedish Heart-Lung-foundation (grant no. 20120449).

References

Fischer

Samady

McPherson

. Comparison between visual assessment and quantitative angiography versus fractional flow reserve for native coronary narrowings of moderate severity. Am J Cardiol 2002; 90: 210–215.

Fisher

Judkins

Lesperance

. Reproducibility of coronary arteriographic reading in the coronary artery surgery study (CASS). Cathet Cardiovasc Diagn 1982; 8: 565–575.

Tonino

Fearon

De Bruyne

. Angiographic versus functional severity of coronary artery stenoses in the FAME study fractional flow reserve versus angiography in multivessel evaluation. J Am Coll Cardiol 2010; 55: 2816–2821.

Pijls

van Son

Kirkeeide

. Experimental basis of determining maximum coronary, myocardial, and collateral blood flow by pressure measurements for assessing functional stenosis severity before and after percutaneous transluminal coronary angioplasty. Circulation 1993; 87: 1354–1367.

De Bruyne

Pijls

NHJ

Bartunek

. Fractional flow reserve in patients with prior myocardial infarction. Circulation 2001; 104: 157–162.

Pijls

Van Gelder

Van der Voort

. Fractional flow reserve a useful index to evaluate the influence of an epicardial coronary stenosis on myocardial blood flow. Circulation 1995; 92: 3183–1393.

Pijls

NHJ

de Bruyne

Peels

. Measurement of fractional flow reserve to assess the functional severity of coronary-artery stenoses. N Engl J Med 1996; 334: 1703–1708.

Taylor

Fonte

Min

. Computational fluid dynamics applied to cardiac computed tomography for noninvasive quantification of fractional flow reserve: scientific basis. J Am Coll Cardiol 2013; 61: 2233–2241.

Zarins

Taylor

Min

. Computed fractional flow reserve (FFTCT) derived from coronary CT angiography. J Cardiovasc Tranl Res 2013; 6: 708–714.

10.

Koo

Erglis

Doh

. Diagnosis of ischemia-causing coronary stenoses by noninvasive fractional flow reserve computed from coronary computed tomographic angiograms. Results from the prospective multicenter DISCOVER-FLOW (Diagnosis of Ischemia-Causing Stenoses Obtained Via Noninvasive Fraction al Flow Reserve) study. J Am Coll Cardiol 2011; 58: 1989–1997.

11.

Itu

Sharma

Mihalef

. A patient-specific reduced-order model for coronary circulation. Proc IEEE Int Symp Biomed Imaging 2012, pp. 832–835.

12.

Sharma P, Itu L, Zheng X, et al. (eds). A feed-back control framework for personalization of coronary flow simulations during rest and hyperemia. Conf Proc IEEE Eng Med Biol Soc 2012;2012:6665–6668.

13.

Wilson

Wyche

Christensen

. Effects of adenosine on human coronary arterial circulation. Circulation 1990; 82: 1595–1606.

14.

Tonino

De Bruyne

Pijls

. Fractional flow reserve versus angiography for guiding percutaneous coronary intervention. N Engl J Med 2009; 360: 213–224.

15.

Leber

Knez

von Ziegler

. Quantification of obstructive and nonobstructive coronary lesions by 64-slice computed tomography. J Am Coll Cardiol 2005; 46: 147–154.

16.

Dehmer

Weaver

Roe

. A contemporary view of diagnostic cardiac catheterization and percutaneous coronary intervention in the United States: a report from the CathPCI Registry of the National Cardiovascular Data Registry, 2010 through June 2011. J Am Coll Cardiol 2012; 60: 2017–2031.

17.

West

Ellis

Brooks

. Joint Audit Committee of the British Cardiac Society and Royal College of Physicians of London. Complications of diagnostic cardiac catheterisation: results from a confidential inquiry into cardiac catheter complications. Heart 2006; 92: 810–814.

18.

Mettler

Jr Huda

Yoshizumi

. Effective doses in radiology and diagnostic nuclear medicine: a catalog. Radiology 2008; 248: 254–263.

19.

Hachamovitch

Berman

Kiat

. Exercise myocardial perfusion SPECT in patients without known coronary artery disease: incremental prognostic value and use in risk stratification. Circulation 1996; 93: 905–914.

20.

Rossi

Merkus

Klotz

. Stress myocardial perfusion: imaging with multidetector CT. Radiology 2014; 270: 25–46.

21.

Taylor

Cerqueira

Hodgson

. ACCF/SCCT/ACR/AHA/ASE/ASNC/NASCI/SCAI/SCMR 2010 Appropriate Use Criteria for Cardiac Computed Tomography: A Report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, the Society of Cardiovascular Computed Tomography, the American College of Radiology, the American Heart Association, the American Society of Echocardiography, the American Society of Nuclear Cardiology, the North American Society for Cardiovascular Imaging, the Society for Cardiovascular Angiography and Interventions, and the Society for Cardiovascular Magnetic Resonance. Circulation 2010; 122: e525–e555.

22.

Norgaard

Leipsic

Gaur

. Diagnostic performance of noninvasive fractional flow reserve derived from coronary computed tomography angiography in suspected coronary artery disease: the NXT trial (Analysis of Coronary Blood Flow Using CT Angiography: Next Steps). J Am Coll Cardiol 2014; 63: 1145–1155.

23.

Coenen

Lubbers

Kurata

. Fractional flow reserve computed from noninvasive CT angiography data: diagnostic performance of an on-site clinician-operated computational fluid dynamics algorithm. Radiology 2015; 274: 674–683.

24.

Renker

Schoepf

Wang

. Comparison of diagnostic value of a novel noninvasive coronary computed tomography angiography method versus standard coronary angiography for assessing fractional flow reserve. Am J Cardiol 2014; 114: 1303–1308.

25.

Min

Leipsic

Pencina

. Diagnostic accuracy of fractional flow reserve from anatomic CT angiography. JAMA 2012; 308: 1237–1245.

26.

McDermott

. A robust method using propensity score stratification for correcting verification bias for binary tests. Biostatistics 2012; 13: 32–47.