A computer-assisted systematic quality monitoring method for cervical hip fracture radiography

Abstract

Background

A thorough quality analysis of radiologic performance is cumbersome. Instead, the prevalence of missed cervical hip fractures might be used as a quality indicator.

Purpose

To validate a computer-based quality study of cervical hip fracture radiography.

Material and Methods

True and false negative and positive hip trauma radiography during 6 years was assessed manually. Patients with two or more radiologic hip examinations before surgery were selected by computer analysis of the databases. The first of two preoperative examinations might constitute a missed fracture. These cases were reviewed.

Results

Out of 1621 cervical hip fractures, manual perusal found 51 (3.1%) false negative radiographic diagnoses. Among approximately 14,000 radiographic hip examinations, there were 27 (0.2%) false positive diagnoses. Fifty-seven percent of false negative reports were occult fractures, the other diagnostic mistakes. There were no significant differences over the years. Diagnostic sensitivity was 96.9% and specificity 99.8%. Computer-assisted analysis with a time interval of at least 120 days between the first and the second radiographic examination discovered 39 of the 51 false negative reports.

Conclusion

Cervical hip trauma radiography has high sensitivity and specificity. With computer-assisted analysis, 76% of false negative reports were found.

Keywords

Femoral neck fractures healthcare quality assurance radiography computer-assisted diagnosis incidence

Introduction

Systematic quality monitoring programs that evaluate the outcome of the entire chain from imaging to reporting are rarely undertaken in radiology since manual evaluations are time-consuming and expensive. One report (1) described a semi-automated way to evaluate several imaging chains in radiology, which were much easier to perform than purely manual analyses.

Radiographic diagnosis of cervical hip fracture is suitable for systematic quality monitoring (1) for several reasons: Cervical hip fracture is an important and common fracture with high co-morbidity (2) and increased mortality (3), with severe implications on healthcare economy and efficiency (4 –6). A missed fracture means unnecessary patient suffering, with the risk of a non-displaced fracture progressing into a displaced fracture. Usually, the radiographic diagnosis of hip fracture is straightforward (5), but in a small percentage of examinations radiography is false negative (7 –10).

To facilitate systematic quality monitoring of cervical hip fracture radiography, a computer-assisted analysis method was developed, called the rule of two (11), based on the fact that patients with hip fractures who are not operated on usually have symptoms for an extended period of time. Consequently, almost all patients with a missed fracture will undergo repeat imaging with radiography, magnetic resonance imaging (MRI) or computed tomography (CT) until the correct diagnosis is determined.

The aim of the current study was to evaluate the performance of a computer-assisted quality monitoring tool in cervical hip fracture.

Material and Methods

The current report is a validation of a computerized study. This, in turn, is based on the retrospective analysis of the outcome of all hip radiography performed for suspicion of cervical hip fracture in 2003 and during the years 2007–2011; a quality control project without the need for informed consent. The setting was a university hospital with a catchment area for primary hip fracture surgery for about 320,000 people.

Imaging

All hip trauma radiography was done with an anterior-posterior (AP) and a cross-table lateral radiograph, and an AP pelvic radiograph. In cases without obvious hip fracture, the examination was often supplemented by two anterior oblique pelvic radiographs, with a rotation of about 20–30°. Supplemental CT imaging was done on various scanners, with multiplanar reformations in the axial, coronal, and sagittal planes. Supplemental MRI was mostly done on 1.5 T scanners, using a hip fracture protocol consisting of coronal T1-weighted (T1W) and short tau inversion recovery (STIR) sequences, often combined with axial T1W and STIR or fat-suppressed T2-weighted (T2W) sequences. All imaging was archived in the regional digital radiology archive, where all imaging for this and the surrounding hospitals within a 100 km radius was available.

Data

Data were retrieved from both the surgery and the radiology databases to find all patients with a cervical hip fracture. The orthopedic surgery database provided a list of all patients operated on for cervical hip fracture. From the radiology information system (RIS), data on all hip radiography examinations were retrieved. All reports on radiography, MRI, and CT for suspicion of hip trauma or fracture, as well as those lacking a clearly stated indication for the examination, were selected for study.

Manual analysis

All dates for radiologic examinations were manually perused in the RIS to find true and false negative and positive hip radiography reports. For each hip radiography the RIS was perused for at least 6 months backwards to find potential prior trauma episodes with radiographic examinations, and followed for up to 1 year to find missed fractures. Hip radiography with a negative report and an uneventful follow-up in the RIS without evidence of a later fracture was scored as true negative. Cases without follow-up imaging were also scored as true negative since no further complications from the index trauma required repeat imaging, i.e. no hip fracture. Cases where a cervical hip fracture became apparent on subsequent imaging with radiography, MRI, or CT within 6 months without interval trauma were scored as false negative. Cases where a cervical hip fracture could be detected on the radiographs, and the report was definite or suspicious for cervical hip fracture, were scored as true positive. Cases where a fracture or a suspicious fracture was reported and was not surgically treated, could not be seen at review or on follow-up imaging, and was without follow-up complications were scored as false positive.

Scoring was based on the archived original reports on the studies and not on a re-evaluation of the radiographs. A re-evaluation of the radiographs was only done to classify undiagnosed fractures into occult (invisible) (12,13) and missed (in retrospect visible) fractures. To reduce subjectivity two observers re-evaluated equivocal cases.

Computer-assisted analysis (rule of two)

Surgery of cervical hip fracture requires prior radiologic diagnosis. If more than one radiologic examination is performed before surgery, the first of these examinations might be false negative. Among the patients who had been operated on for cervical hip fracture a subgroup was identified with more than one preoperative radiologic study. This subgroup was thought to contain most of the patients with false negative reports and was reviewed.

For the computer-assisted analysis using the rule of two, all radiologic hip examinations (radiography, CT, and MRI), whether trauma-related or not, were merged into one list which was then, by computer, compared with the list of patients operated on for cervical hip fracture. Thus, by applying this rule of two on the RIS data, all patients operated on for cervical hip fracture with more than one preoperative radiologic hip examination were identified. All these cases were manually re-evaluated in the RIS and PACS. Four different maximum time intervals between the two preoperative radiologic examinations (14, 30, 60, and 120 days) were evaluated.

Statistical analysis

Descriptive statistics were used to characterize the data. Sensitivity and specificity were calculated based on frequencies for true and false positive and negative radiography reports. A χ² analysis was performed to test for significance of differences of false negative reports between years, with the level of significance set to P < 0.05.

Results

Patients

For 13,996 patients referred to radiography for suspicion of cervical hip fracture, the female/male ratio was 1.94. The mean age was 73 years (age range, 0–106 years); for girls/women, the mean age was 76 years (age range, 0–106 years); for boys/men, the mean age was 68 years (age range, 0–103 years).

Manual review

Out of more than 31,000 hip radiographs, including CT and MRI, about 14,000 were performed because of trauma or suspected fracture, and among these, 1570 were true positive for cervical hip fracture. There were 51 false negative reports (3.1% of all fractures) and 27 false positive reports (0.2% of all non-fractured hips) (Table 1). Among 47 patients with cervical hip fracture, who for various reasons were not operated, there were five false negative radiography reports. Twelve cases with a preliminary report on call of “no fracture” were corrected within 24 h in the final report. These were not counted as diagnostic mistakes.

Table 1.

Diagnostic accuracy of radiography of cervical hip fracture. Results of manual perusal from 2003 and 2007–2011.

	2003	2007	2008	2009	2010	2011	Total	Annual average
Radiography, CT, and MRI of the hip	4816	6485	5104	4832	5824	4348	31,409	5235
Hip radiography with trauma referral	2096	2385	2379	2172	2399	2565	13,996	2333
No cervical hip fracture (true negative)	1834	2098	2075	1913	2125	2303	12,348	2058
Cervical hip fracture (true positive)	253	266	291	249	259	252	1570	262
Fracture not diagnosed (false negative)	7 (2.7%)	12 (4.3%)	10 (3.3%)	4 (1.6%)	9 (3.4%)	9 (3.4%)	51 (3.1%)	9 (3.1%)
Fracture diagnosis wrong (false positive)	2 (0.1%)	9 (0.4%)	3 (0.1%)	6 (0.3%)	6 (0.3%)	1 (0.0%)	27 (0.2%)	5 (0.2%)

In 29 of 51 false negative studies, there was an occult (invisible) fracture, even at review. In 22 false negative studies, the fracture could be seen at review, constituting diagnostic mistakes (Table 2). In 10 of these 22 cases, the fracture was identified solely on the pelvic radiographs.

Table 2.

False negative radiographic diagnosis of cervical hip fracture with occult and non-occult fractures at review. Results of manual perusal from 2003 and 2007–2011.

	2003	2007	2008	2009	2010	2011	Total
All cervical hip fractures	260	278	301	253	268	261	1621
False negative	7 (2.7%)	12 (4.3%)	10 (3.3%)	4 (1.6%)	9 (3.4%)	9 (3.4%)	51 (3.1%)
Occult fractures	4 (1.5%)	6 (2.2%)	6 (2.0%)	2 (0.8%)	6 (2.2%)	5 (1.9%)	29 (1.8%)
Non-occult fractures at review	3 (1.2%)	6 (2.2%)	4 (1.3%)	2 (0.8%)	3 (1.1%)	4 (1.5%)	22 (1.4%)
Subgroup of non-occult fractures where the fracture was visible only on pelvis radiograph	0	2	3	2	2	1	10

There were no significant differences in diagnostic accuracy between years. Overall diagnostic sensitivity of radiography of cervical hip fracture was 96.9% and specificity was 99.8%.

Computer-assisted analysis using the rule of two

Four different maximum time intervals between the two preoperative hip radiographs were tested. The interval of 14 days identified 126 cases where two radiologic studies had been done before surgery. Twenty-four cases were false negative. The interval of 30 days identified 30 false negative reports among 147 cases. The interval of 60 days found 38 false negative reports among 169 cases. Increasing the interval to 120 days identified 39 false negative reports among 203 cases (Table 3). Further increase of the interval did not disclose additional false negative cases. The rule of two failed to identify 12 false negative cases. The reasons were: no surgery (n = 5); surgery at another hospital (n = 1); different surgical registration code (n = 3); and wrong radiologic examination code (n = 3).

Table 3.

Manual perusal compared with computer-assisted analysis using the rule of two for radiographic diagnosis of cervical hip fracture during 2003 and 2007–2011. Number of false negative results for four different time intervals between the first and the second radiologic study, and number of reports to evaluate at each interval.

	2003	2007	2008	2009	2010	2011	Total	Reports toprocess manually (n)
False negative reports found by manual perusal	7	12	10	4	9	9	51
False negative reports found by computer-assisted analysis using the rule of two
14-day interval	1	5	6	1	4	7	24	126
30-day interval	6	9	8	2	4	7	36	147
60-day interval	6	9	8	3	5	7	38	169
120-day interval	6	10	8	3	5	7	39	203

Discussion

The current study confirmed that radiography has high sensitivity and specificity for cervical hip fracture. It showed that a computer-assisted analysis of cervical hip fracture radiography can be used for quality monitoring, especially if the digital coding is optimal. Of all missed cervical hip fractures, about half were radiographically occult and the other half were diagnostic mistakes.

The current report on manual perusal of all hip and pelvis radiography, CT, and MRI with trauma indication during 6 years identified all false negative and false positive reports, enabling the calculation of sensitivity and specificity for hip radiography in suspect cervical hip fracture.

The findings in the current study are in agreement with previous reports which showed similar high sensitivity for radiography of cervical hip fracture (Table 4) with a rate of false negative radiographs in the range of 0.2–4.9%. The rate of disagreement in cervical hip fracture diagnosis is thus similar to but somewhat lower than published data on general discrepancy rates between radiologists, which are in the range of 3–6% (14 –16).

Table 4.

Studies reporting false negative and false positive hip radiography in cervical hip fracture.

Author, year	Patients (n)	Type of patient	Patients with cervical hip fracture (n)	Further evaluated patients (n)	Outcome reference	FN (%)	Occult FN (%)	Missed FN (%)	FP (%)
Pathak, 1997 (7)	1108	Cervical hip fracture	1108		Surgery	2.7	0.8	1.9
Pandey, 1998 (22)	770	Suspect hip fracture	N/A	33	MRI	0.2
Lim, 2002 (37)	422	Definite or suspect hip fracture	N/A	57	MRI	0.2
Dominguez, 2005 (9)	764		301	62	MRI	4.9
Current study	13,996	Suspect hip fracture	1621		Surgery, radiography, CT, MRI	3.1	1.8	1.4	0.2

FN, false negative; FP, false positive.

A little more than half of the false negative reports in the current study were caused by truly occult fractures, not visible even in retrospect. A truly occult fracture is defined as clinical symptoms or signs of fracture without radiographic evidence (12,13). Occult and suspect hip fractures may be further investigated by MRI (17 –23) or by CT (5,10,24 –29). Modern CT, especially when viewed with a soft tissue window width/level setting of about 400/40 (26), has the ability to evaluate bone marrow edema and lipohemarthrosis and has a diagnostic capability approaching that of MRI (26,28,29). CT is a valuable alternative when MRI is unavailable or contraindicated (25). Bone scintigraphy is not recommended in acute trauma. MRI is quicker and more specific (30,31) and there is a risk of false negative outcome when bone scanning is performed too early, especially in elderly patients and in patients with metabolic diseases (32 –34). However, in stress fracture bone scintigraphy is a highly sensitive imaging modality.

In ten of the false negative reports the only finding suggesting a fracture was a minimal cortical break in the femoral neck, visible on one or more of the pelvic radiographs but not on the hip radiographs. No other systematic cause was found for false negative reports due to diagnostic mistakes. Several factors seemed to add to the diagnostic difficulties, e.g. low bone mineral content, underexposed images, obesity, subcapital location of the fracture, and presence of only minute cortical disruptions. Another reason was failure to note an asymmetric projection of the hips where the fracture was suggested by unilateral shortening and external rotation of the fractured hip. In some cases, calcifications of the femoral artery masked the fracture and in some cases severe osteoarthritis with large osteophytes interfered with diagnosis (35). The false positive diagnoses were caused by similar difficulties. Further sources of error were image artefacts mimicking fractures and physeal vestiges which were misinterpreted as fractures.

A quality analysis by a manual perusal of hip radiography (11) is extremely time-consuming. At the authors’ institution between 2000 and 3000 radiography reports annually would have to be reviewed. The computer-assisted evaluation requires much less manual labor. After creation of patient lists and computerized selection of the study group, far fewer reports need to be checked. During the 6 years of the study, more than 31,000 hip radiographs were performed. In 1621 cervical hip fractures, 1570 radiographic examinations were true positive (Table 1). In total, 1574 of 1621 fractures were operated. Of these, at most 203 cases (12.9% of the operated fractures; Table 3) were flagged in the computerized analysis as having two or more radiographic examinations before surgery and would require a manual evaluation. This means that 87% of the operated fractures would not have to be evaluated in the RIS or PACS if the computerized analysis were used.

A disadvantage of the computerized method was that for several reasons only about 76% of the false negative reports were identified (Table 3). Another disadvantage was that the rule of two cannot identify false positive reports, and thus it is not possible to calculate both the sensitivity and specificity of hip fracture radiography. However, for a year-by-year comparison of diagnostic quality, the ratio between false negative reports and total number of fractures might well be sufficient. A computer-assisted analysis using the rule of two is a practical and quick way to systematically evaluate the diagnostic quality of cervical hip fracture radiography. The method is not exact, but should be good enough to show systematic changes or trends in the diagnostic quality, especially if the false negative cases are reviewed to decide if the misses are due to occult fractures or to diagnostic mistakes.

In the current study, only cervical hip fracture radiography was used as a quality assurance model. Cervical hip fracture has the advantage of being binary in diagnosis (there are almost no incomplete fractures) as well as in treatment (almost all fractures are treated surgically) (36) with very few dropouts from the inclusion process. Other types of fractures may be less well suited for computerized quality analysis, lacking a clear surgical outcome reference.

Quality control programs, similar to the current study, have been undertaken elsewhere where other aspects of radiology have been analyzed. In a report by Liu et al. (1), eight subspecialties were requested to select a specific component of its imaging reports or procedures that could be measured and compared with a readily available reference standard database such as in the current report. Also, reference accuracy levels had to be available from, for example, published meta-analyses or multi-center studies. The manual analysis was mostly performed by staff, minimizing additional workload for radiologists. Compared with more commonly instituted peer-review programs, reviewer bias and erroneous second reviewer interpretations are avoided by such database analysis.

Among the limitations of the study are the fact that study cases might be missed due to radiography or surgery at different hospitals. One such case was identified, where surgery was performed at another location and the patient thus did not appear in the local surgery database. Regarding radiologic studies performed at other locations, the risk of missing one in the current study was much less since the regional database contains all imaging from surrounding hospitals within a 100 km radius. Further studies are needed to demonstrate whether the fact that only 76% of false negative reports were identified by this computerized method is an acceptable level for the purpose of rapid and economic quality evaluation of hip fracture radiography.

In conclusion, the computerized analysis detected 76% of the false negative radiography reports and could be used for diagnostic quality evaluation of cervical hip fracture. About 3% of all cervical hip fractures were missed at initial radiography.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Liu

Johnson

Miranda

. A reference standard-based quality assurance program for radiology. J Am Coll Radiol 2010; 7: 61–66.

Sircar

Godkar

Mahgerefteh

. Morbidity and mortality among patients with hip fractures surgically repaired within and after 48 hours. Am J Ther 2007; 14: 508–513.

Panula

Pihlajamäki

Mattila

. Mortality and cause of death in hip fracture patients aged 65 or older: a population-based study. BMC Musculoskelet Disord 2011; 12: 105–105.

Borgström

Sobocki

Ström

. The societal burden of osteoporosis in Sweden. Bone 2007; 40: 1602–1609.

Cannon

Silvestri

Munro

. Imaging choices in occult hip fracture. J Emerg Med 2009; 37: 144–152.

Kanis

Johnell

De Laet

. International variations in hip fracture probabilities: implications for risk assessment. J Bone Miner Res 2002; 17: 1237–1244.

Pathak

Parker

Pryor

. Delayed diagnosis of femoral neck fractures. Injury 1997; 28: 299–301.

Parker

. Missed hip fractures. Arch Emerg Med 1992; 9: 23–27.

Dominguez

Liu

Roberts

. Prevalence of traumatic hip and pelvic fractures in patients with suspected hip fracture and negative initial standard radiographs–a study of emergency department patients. Acad Emerg Med 2005; 12: 366–369.

10.

Dunker

Collin

Göthlin

. High clinical utility of computed tomography compared to radiography in elderly patients with occult hip fracture after low-energy trauma. Emerg Radiol 2011; 19: 135–139.

11.

Laurin

Jonsson

. Low frequency of missed or invisible hip fracture in X-ray examination. The rule-of-2–a simple method for quality assessment [Låg frekvens av missad eller osynlig höftfraktur på röntgen. 2-regeln–enkel metod för kvalitetsmätning. In Swedish]. Lakartidningen 2004, pp. 101–24232425.

12.

Berger

Ofstein

Jackson

. MRI demonstration of radiographically occult fractures: what have we been missing? Radiographics 1989; 9: 407–436.

13.

Weishaupt

Schweitzer

. MR imaging of the foot and ankle: Patterns of bone marrow signal abnormalities. Eur Radiol 2002; 12: 416–426.

14.

Jackson

Cushing

Abujudeh

. RADPEER scoring white paper. J Am Coll Radiol 2009; 6: 21–25.

15.

Soffa

Lewis

Sunshine

. Disagreement in interpretation: a method for the development of benchmarks for quality assurance in imaging. J Am Coll Radiol 2004; 1: 212–217.

16.

Siegle

Baram

Reuter

. Rates of disagreement in imaging interpretation in a group of community hospitals. Acad Radiol 1998; 5: 148–154.

17.

Chana

Noorani

Ashwood

. The role of MRI in the diagnosis of proximal femoral fractures in the elderly. Injury 2006; 37: 185–189.

18.

Evans

Wilson

Lyons

. Comparison of MRI with bone scanning for suspected hip fracture in elderly patients. J Bone Jt Surg Br 1994; 76B: 158–159.

19.

Frihagen

Nordsletten

Tariq

. MRI diagnosis of occult hip fractures. Acta Orthop 2005; 76: 524–530.

20.

Hossain

Barwick

Sinha

. Is magnetic resonance imaging (MRI) necessary to exclude occult hip fracture? Injury 2007; 38: 1204–1208.

21.

Oka

Monu

. Prevalence and patterns of occult hip fractures and mimics revealed by MRI. Am J Roentgenol 2004; 182: 283–288.

22.

Pandey

McNally

Ali

. The role of MRI in the diagnosis of occult hip fractures. Injury 1998; 29: 61–63.

23.

Verbeeten

Hermann

Hasselqvist

. The advantages of MRI in the detection of occult hip fractures. Eur Radiol 2005; 15: 165–169.

24.

Jordan

Dickenson

Westacott

. A vast increase in the use of CT scans for investigating occult hip fractures. Eur J Radiol 2013; 82: e356–e359.

25.

Heikal

Riou

Jones

. The use of computed tomography in identifying radiologically occult hip fractures in the elderly. Ann R Coll Surg Engl 2014; 96: 234–237.

26.

Geijer

Collin

Dunker

. Bone bruise, lipohemarthrosis, and joint effusion in CT of non-displaced hip fracture. Acta Radiol 2012; 53: 197–202.

27.

Gill

Smith

Fox

. Investigation of occult hip fractures: the use of CT and MRI. Sci World J 2013; 2013: 1–4.

28.

Haubro

Stougaard

Torfing

. Sensitivity and specificity of CT- and MRI-scanning in evaluation of occult fracture of the proximal femur. Injury 2015; 46: 1557–1561.

29.

Thomas

Williams

HLM

Carpenter

. The validity of investigating occult hip fractures using multidetector CT. Br J Radiol 2016; 89: 20150250–20150250.

30.

Rizzo

Gould

Lyden

. Diagnosis of occult fractures about the hip. Magnetic resonance imaging compared with bone-scanning. J Bone Jt Surg Am 1993; 75: 395–401.

31.

Rubin

Marquardt

Gottlieb

. Magnetic resonance imaging: a cost-effective alternative to bone scintigraphy in the evaluation of patients with suspected hip fractures. Skelet Radiol 1998; 27: 199–204.

32.

Lewis

Rees

Thomas

. Pitfalls of bone scintigraphy in suspected hip fractures. Br J Radiol 1991; 64: 403–408.

33.

Mulcahy

O’Malley

. Negative radioisotope bone scan in a patient with a fractured neck of femur. Ir J Med Sci 1995; 164: 42–44.

34.

Scott

Alazraki

Manaster

. Failure of bone scanning to detect fractures in a woman on chronic steroid therapy. Skeletal Radiol 1984; 12: 204–207.

35.

Colhoun

Johnson

Fairclough

. Bone scanning for hip fracture in patients with osteoarthritis: brief report. J Bone Joint Surg Br 1987; 69: 848–848.

36.

Palm

Teixidor

. Proximal femoral fractures: Can we improve further surgical treatment pathways? Injury 2015; 46: S47–S51.

37.

Lim

KBL

Eng

AKH

Chng

. Limited magnetic resonance imaging (MRI) and the occult hip fracture. Ann Acad Med Singapore 2002; 31: 607–610.