Reliability in 2-Dimensional On-track/Off-track Measurements: Analysis of Key Values for Glenohumeral Bone Loss

Abstract

Background:

The relationship between glenoid bone loss and Hill-Sachs lesions (HSLs), which is known as the glenoid track, has been well described in the literature. Off-track lesions have been found to be associated with recurrent shoulder instability.

Purpose:

To assess the intraobserver and interobserver reliability in glenoid track measurement using 2-dimensional (2D) computed tomography (CT).

Study Design:

Cohort study (diagnosis); Level of evidence, 3.

Methods:

Electronic medical records between 2009 and 2019 were reviewed for patients with known shoulder instability and bone loss. Using 2D CT, fellowship-trained orthopaedic surgeons and orthopaedic residents measured glenoid loss and humeral head bone loss to calculate the glenoid track and to assess for on-track and off-track lesions.

Results:

A total of 38 patients met the inclusion criteria. For fellowship-trained surgeons, there was moderate intraobserver reliability when assessing the glenoid diameter (intraclass correlation coefficient [ICC], 0.704) and HSI (ICC, 0.720). There was good reliability when evaluating glenoid defects (ICC, 0.761) and glenoid track (ICC, 0.825). Intraobserver assessment of on-track and off-track lesions according to kappa criteria was moderate (κ = 0.531; P < .001). The interobserver reliability among fellowship-trained surgeons for the glenoid track was good (ICC, 0.762); the reliability measurement for the glenoid defect (ICC, 0.672), glenoid diameter (ICC, 0.627), and HSI (ICC, 0.520) were moderate; and the on-track and off-track assessments were moderate (κ = 0.405; P < .001). For the residents, there was moderate intraobserver reliability when assessing the glenoid diameter (ICC, 0.633), glenoid defect (ICC, 0.709), HSI (ICC, 0.536), and glenoid track (ICC, 0.708). Interobserver reliability within the residents was moderate for the glenoid diameter (ICC, 0.542), glenoid defect (ICC, 0.574), and glenoid track (ICC, 0.629) and poor for the HSI (ICC, 0.292). Determination of on- and off-track lesions among residents was fair (κ = 0.234; P < .001).

Conclusion:

Among fellowship-trained surgeons, both the interobserver and intraobserver reliability of measuring the glenoid track on 2D CT were good; however, agreement on whether lesions were on-track or off-track was only moderate. The reliability among residents was moderate; however, their agreement on whether lesions were on-track or off-track was poor.

Keywords

glenoid track Hill-Sachs shoulder,instability shoulder,glenoid labrum imaging and radiology imaging,computed tomography

Traumatic anterior shoulder instability is one of the most common shoulder injuries among athletes in the United States.⁴ Arthroscopic stabilization has demonstrated high rates of success; however, management of shoulder instability in the young, active patient population often poses many challenges because the risk of recurrent instability is multifactorial.^1,9-11

One of the many challenges in treating anterior glenohumeral instability is management of bone loss, which is a risk factor for recurrent instability after arthroscopic stabilization.^1,2,9,17 Historically, glenoid bone loss of 25% was defined as critical bone loss. In patients with critical glenoid bone loss, an arthroscopic Bankart repair alone has high rates of recurrent instability; therefore, augmentative procedures are indicated.^17,22 Recent studies have challenged the threshold of glenoid bone loss. Shin et al¹⁹ showed that patients undergoing arthroscopic Bankart repair with a glenoid bone loss of 17.3% had a 28.6% risk of recurrent shoulder instability. Dekker et al⁶ demonstrated that patients with >15% glenoid bone loss had nearly 3 times greater odds of recurrent instability. Shaha et al¹⁸ looked at subcritical glenoid bone loss in the active-duty military population and noted that as little as 13.5% of glenoid bone loss resulted in poor functional outcome scores.

Another variable in the management of anterior shoulder instability is the Hill-Sachs lesion (HSL). Yamamoto et al²⁰ described the relationship between glenoid bone loss and HSLs, which is known as the glenoid track. Subsequent studies have further validated the concept of the glenoid track and engagement of the HSL on the glenoid rim.^3,14,17

Measurements of glenoid bone loss and the HSL are used to determine whether a lesion is on-track versus off-track, and the amount of bone loss is frequently used to guide treatment. In the study by Yamamoto and colleagues,²⁰ their initial description and measurement of the glenoid track was in a cadaveric model. Subsequent studies have assessed the intraobserver and interobserver reliability of measuring the glenoid track in 3-dimensional reconstruction models¹⁶; however, these measurement techniques may not be true to clinical practice. In clinical practice, it is more common to use 2-dimensional (2D) image sequences to measure bone loss. The reliability of measuring the glenoid track with 2D imaging has not been described. There is a lack of literature examining the accuracy of glenoid track measurement in 2D imaging, and thereby its reliability in aiding clinical decision-making remains unclear.

The purpose of this study was to assess the intraobserver and interobserver reliability in glenoid track measurement using 2D computed tomography (CT). Our hypothesis was that on-track and off-track measurements using 2D imaging would demonstrate good intraobserver and interobserver reliability.

Methods

We retrospectively reviewed the medical records of consecutive patients between 2009 and 2019 at a single institution who underwent a glenoid bony augmentation procedure for recurrent anterior shoulder instability. The decision to undergo a glenoid augmentation procedure was at the discretion of the treating surgeon and commonly included recurrent instability after a prior anterior stabilization procedure or a history of recurrent instability with evidence of glenoid and/or humeral head bone loss on advanced imaging. Data collection was performed via an electronic medical records search.

Preprocedure CT scans were retrospectively reviewed for glenoid and humeral head bone loss. Inclusion criteria were patients >17 years of age with both glenoid and humeral head bone loss demonstrated on CT scans, and patients undergoing a Latarjet procedure. Exclusion criteria were patients without adequate advanced imaging (Figure 1).

Figure 1.

Flowchart of inclusion and exclusion criteria.

Having a prior shoulder surgery or diagnosis of multidirectional instability was not considered an exclusion criterion, although these data were not specifically gathered for this study.

Two fellowship-trained shoulder and elbow surgeons (K.M., W.D.), 1 fellowship-trained sports surgeon (L.Z.), and 4 orthopaedic surgery residents (postgraduate years 2-5; A.M.) independently reviewed 38 CT series to assess glenoid bone loss and HSL size, using our Synapse 5 PACS. All surgeons performed 2 separate reviews, with the second review performed >2 weeks after the first review. In an effort to standardize the measurement technique, a group measurement training session was conducted before individual measurements.

Measurement Technique

Measurement of the HSL was standardized before the start of the study and measured as described by Di Giacomo et al.^7,8 True to practice, the measuring surgeon had the entire imaging series to select, in their interpretation, the best 2D images to be measured.

A perfect circle method was used to measure the estimated diameter of the glenoid (D), and the amount of anterior glenoid bone loss (d) was measured. Calculation of the glenoid track was determined by the equation 0.83D – d. The Hill-Sachs interval (HSI) was measured on axial images; the HSI was determined by the width of the HSL plus the width of the bone bridge (BB). The BB was determined from the lateral border of the HSL to the medial border of the infraspinatus footprint as seen on axial imaging. All CT scans were obtained under our institution’s shoulder instability protocol, known as the “Moose protocol,” where each CT was calibrated to acquire images in the plane of the scapula to be able to more accurately assess the glenoid face. The Moose protocol used fine CT cuts of 2.5 mm so the glenoid face could be accurately assessed. If the HSI was greater than the glenoid track, then the lesion was deemed off-track; if the HSI was less than the glenoid track, the lesion was deemed on-track (Figures 2 and 3).

Figure 2.

Measurements of the bone bridge and Hill-Sachs lesion, used to calculate the Hill-Sachs interval.

Figure 3.

Measurement of the glenoid track using the best-fit circle method; with D being the diameter of the glenoid and d representing the amount of glenoid bone loss. These measurements are made using sagittal computed tomography images.

Statistical Analysis

The interpretations for reliability estimates were based on Portney and Watkins¹⁵ for intraclass correlation coefficient (ICC) and Landis and Koch¹² for kappa statistic. Interobserver and intraobserver rater reliability for continuous measures such as diameter, defect, HSI, and track were assessed by ICC (95% confidence interval). The value of an ICC can range from 0 to 1, with 0 indicating no reliability among raters and 1 indicating perfect reliability among raters. ICC values <0.5 are indicative of poor reliability, values between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.90 indicate good reliability, and values >0.90 indicate excellent reliability.¹⁵

For on-track and off-track determination, we used the Cohen kappa for 2 raters and Fleiss kappa for multiraters, with the P value. Kappa measures can be interpreted as follows: values ≤0, no agreement; 0.01 to 0.20, no to slight agreement; 0.21 to 0.40, fair agreement; 0.41 to 0.60, moderate agreement; 0.61 to 0.80, substantial agreement; and 0.81 to 1.00, near-perfect agreement.

Results

Patient Characteristics

A total of 38 patients met the inclusion criteria. There were 37 male patients and 1 female patient. The mean age was 26.3 years (range, 18-43 years).

Intraobserver Reliability

The intraobserver reliability among all reviewers as indicated by ICC values was good for the glenoid track (ICC, 0.750). It was moderate for the diameter (ICC, 0.656), glenoid defect (ICC, 0.727), and HSI (ICC, 0.613) among all reviewers. Kappa assessment of intraobserver reliability for on-track or off-track assessment was moderate among all reviewers (κ = 0.459; P < .001) (Table 1).

Table 1

Intraobserver Reliability^a

	Diameter	Defect	HSI	Track	On/Off, Kappa
All	0.656 (0.581-0.721)	0.727 (0.661-0.782)	0.613 (0.531-0.684)	0.750 (0.691-0.799)	0.459 (P < .001)
Attendings	0.704 (0.597-0.787)	0.761 (0.658-0.834)	0.720 (0.617-0.799)	0.825 (0.753-0.877)	0.531 (P < .001)
Residents	0.633 (0.525-0.72)	0.709 (0.619-0.781)	0.536 (0.41-0.642)	0.708 (0.618-0.78)	0.401 (P < .001)

Data are presented as intraclass correlation coefficient (ICC) (95% CI) unless otherwise indicated. ICC values <0.5 indicate poor reliability; 0.5 to 0.75, moderate reliability; 0.75 to 0.90, good reliability; >0.90, excellent reliability. HSI, Hills-Sachs interval.

The intraobserver reliability was observed to be greater across all categories among fellowship-trained surgeons when compared with residents in training. For fellowship-trained surgeons, there was moderate reliability when assessing the diameter (ICC, 0.704) and HSI (ICC, 0.720). There was good reliability when evaluating defects (ICC, 0.761) and the glenoid track (ICC, 0.825). The intraobserver assessment of on-track and off-track lesions according to kappa criteria was moderate (κ = 0.531; P < .001).

Interobserver Reliability

The interobserver reliability among all reviewers was moderate for the glenoid diameter (ICC, 0.567), moderate for the glenoid defect (ICC, 0.610), poor for the HSI (ICC, 0.328), moderate for the glenoid track (ICC, 0.668), and fair for on-track and off-track assessment (κ = 0.267; P < .001).

The interobserver reliability among fellowship-trained surgeons for the glenoid track was good (ICC, 0.762); measurements for the glenoid defect (ICC, 0.672), glenoid diameter (ICC, 0.627), and HSI (ICC, 0.520) were moderate; and on-track and off-track assessment was moderate (κ = 0.405; P < .001) (Table 2).

Table 2

Interobserver Reliability^a

	Diameter	Defect	HSI	Track	On/Off, Kappa
All	0.567 (0.474-0.663)	0.610 (0.517-0.702)	0.328 (0.232-0.441)	0.668 (0.584-0.75)	0.267 (P < .001)
Attendings	0.627 (0.505-0732)	0.672 (0.558-0.768)	0.520 (0.37-0.651)	0.762 (0.674-0.833)	0.405 (P < .001)
Residents	0.542 (0.43-0.653)	0.574 (0.455-0.684)	0.292 (0.17-0.428)	0.629 (0.519-0.728)	0.234 (P < .001)

Discussion

Recurrent glenohumeral dislocations often result in attritional bone loss of both the glenoid and humeral head and can lead to off-track glenohumeral lesions. An off-track lesion is a risk factor of recurrent instability¹⁷; however, the true-to-practice application of the glenoid track measurement in 2D imaging has not been studied. Our study demonstrated that among fellowship-trained orthopaedic surgeons, both the interobserver and intraobserver reliability of measuring the glenoid track are good, and agreement on whether the lesion is on-track or off-track is moderate. In contrast, for residents, the intraobserver and interobserver reliability of measuring the glenoid track are moderate, and agreement on whether the lesion is on-track or off-track is fair. Shaha et al¹⁸ found that assessing the glenoid track preoperatively may also be useful in determining which patients would be at increased risk of recurrent instability after treatment with an arthroscopic Bankart repair alone. In their study, they found that 75% of patients with off-track lesions had recurrent instability, which comprised 60% of the total recurrent instability cohort. They highlight the importance of identifying off-track lesions preoperatively to appropriately indicate patients for potential augmentation procedures.

In patients with glenoid and/or humeral head bone loss, the arthroscopic Bankart repair may be insufficient to restore stability. MacDonald et al¹³ demonstrated the effectiveness of the arthroscopic Bankart repair with remplissage. They showed that there is a significantly greater risk of postoperative recurrent instability in patients who did not have a remplissage performed in conjunction with an arthroscopic Bankart repair for the treatment of traumatic recurrent anterior shoulder instability with HSLs of any size and glenoid bone loss <15%.¹³ In contrast, Yang et al²¹ demonstrated that for collision athletes, there was a higher re-dislocation rate in patients who were treated with a Bankart repair and remplissage, compared with patients treated with a Latarjet procedure. They found superior outcomes in patients treated with the Latarjet if there was >10% glenoid bone loss. In the military population, Cruz et al⁵ showed high return-to-duty rates in patients treated with the open Latarjet; however, the effectiveness of the Latarjet procedure in treating off-track lesions is unknown.

In its original description, the glenoid track was measured on a cadaveric specimen. Although the concept of track is well accepted, the reproducibility of the measurement can vary. Schneider et al¹⁶ assessed the interobserver and intraobserver reliability of glenoid track measurement using static 3D imaging. In their study, they used preselected static 3D images for measurement. They concluded that the evaluation of HSLs demonstrates a high level of variability and poor interobserver reliability when measured between 3 orthopaedic fellowship-trained shoulder specialists and 1 musculoskeletal radiologist. This contrasts with our study, in which we demonstrated moderate intraobserver and interobserver reliability of determining the glenoid track among fellowship-trained orthopaedic surgeons. Although this study was the first to assess the intraobserver and interobserver reliability among glenoid track measurements, using preselected 3D images makes the applicability of their results to clinical practice difficult. In the clinical setting, 2D imaging is more commonly used, and the surgeon selects the image to measure the defects.

To our knowledge, this is the first study to evaluate the intraobserver and interobserver reliability of measuring on-track and off-track lesions using 2D imaging. Among all reviewers, we found good intraobserver reliability for the glenoid track, moderate reliability for the HSI, and moderate on-/off-track assessment. Among fellowship-trained surgeons, the interobserver reliability for the glenoid track was good; the measurements for the glenoid defect, glenoid diameter, and HSI were moderate; and on-track and off-track assessments were moderate. However, interobserver reliability among residents for the HSI was poor, and on-track and off-track assessments were fair. Overall, the reliability was observed to be greater across all categories among fellowship-trained surgeons when compared with orthopaedic surgery residents in training.

We believe that there was an overall poor interobserver reliability when compared with intraobserver reliability because of a wider range of variability between resident measurements when compared with attending measurements. There was also an overall low reliability of measurements between the residents, which would adversely affect interobserver reliability, but resulted in no statistically significant differences, likely given to our lower sample size. Based on our results, surgeons should use caution when basing their treatment decisions solely on the 2D CT imaging when evaluating patients with glenoid bone loss.

Limitations

There are several limitations to this study. It is a radiographic review without patient-reported outcomes. Furthermore, we only conducted measurements on CT scans. In the general population, it is more common to undergo magnetic resonance imaging (MRI) for advanced imaging; however, in our institution shoulder instability is common and we often obtain CT to better assess bone defects. There is further research to be done comparing the reliability of MRI scans versus CT scans, as well as the reliability of measurements when both CT and MRI are used in conjunction. We also only looked at 2D images and did not compare the accuracy of 2D measurements with 3D measurements.

Furthermore, measuring the HSI, glenoid diameter, and track is dependent on the gantry of the CT and the positioning of the patient’s scapula and humerus. This variability in patient positioning and gantry of the scanner likely contributes to the variability in measurement, as well as measuring the HSL on the soft tissue window, instead of the bone window, to better assess the soft tissue attachment of the infraspinatus, which was not done in this study. Another limitation is our small sample size. However, we specifically included patients who only underwent a Latarjet procedure, as there is another follow-up study ongoing at our institution assessing the outcomes of treating on- and off-track lesions with a Latarjet procedure.

Conclusion

Preoperative measurement of the glenoid track to identify off-track lesions is another tool that the orthopaedic surgeon can use to guide surgical treatment of shoulder instability. Our study shows that the assessment of the glenoid track on 2D CT for on-track and off-track lesions is more reliable among fellowship-trained surgeons. Among fellowship-trained surgeons, both the interobserver and intraobserver reliability of measuring the glenoid track on 2D CT were good, and agreement on whether lesions were on-track or off-track was moderate.

Footnotes

Acknowledgements

The authors thank Austin T. McCadden, MD, Ryan J. Bickley, MD, Jeffrey L. Wake, DO, Alexys M. Bermudez, MD, and Craig R. Bottoni, MD, for their contributions to the study.

Final revision submitted August 25, 2024; accepted August 30, 2024.

One or more of the authors has declared the following potential conflict of interest or source of funding: A.J.M. has received education payments from ImpactOrtho. L.S.Z. has received hospitality payments from Linvatec and Stryker, education payments from Elevate Surgical, and a grant from Arthrex. W.E.D. has received hospitality payments from Arthrex, Elevate Surgical, and Stryker; and education payments from ImpactOrtho and Stryker. K.S.M. has received education payments from ImpactOrtho and Arthrex. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.

Ethical approval for this study was obtained from the Department of the Army, Regional Health Command-Pacific (220007).

ORCID iD

Anthony J. Magee

Kyong S. Min

References

Boileau

Villalba

Hery

Balg

Ahrens

Neyton

Risk factors for recurrence of shoulder instability after arthroscopic Bankart repair. J Bone Joint Surg Am. 2006;88(8):1755-1763.

Bottoni

Johnson

Zhou

, et al. Arthroscopic versus open anterior shoulder stabilization: a prospective randomized clinical trial with 15-year follow-up with an assessment of the glenoid being “on-track” and “off-track” as a predictor of failure. Am J Sports Med. 2021;49(8):1999-2005.

Burns

Chahal

Shahrokhi

, et al. Diagnosis of engaging bipolar bone defects in the shoulder using 2-dimensional computed tomography: a cadaveric study. Am J Sports Med. 2016;44(11):2771-2777.

Cameron

Mauntel

Owens

BD.

The epidemiology of glenohumeral joint instability: incidence, burden, and long-term consequences. Sports Med Arthrosc Rev. 2017;25(3):144-149.

Cruz

Miles

Bottoni

Min

KS.

Surgical treatment of anterior shoulder instability with glenoid bone loss with the Latarjet procedure in active-duty military service members. J Shoulder Elbow Surg. 2022;31(3):629-633.

Dekker

Goldenberg

Lacheta

Horan

Millett

PJ.

Anterior shoulder instability in the professional athlete: return to competition, time to return, and career length. Orthop J Sports Med. 2020;8(11):2325967120959728.

Di Giacomo

de Gasperis

Scarso

. Bipolar bone defect in the shoulder anterior dislocation. Knee Surg Sports Traumatol Arthrosc. 2016;24(2):479-488.

Di Giacomo

Itoi

Burkhart

. Evolving concept of bipolar bone loss and the Hill-Sachs lesion: from “engaging/non-engaging” lesion to “on-track/off-track” lesion. Arthroscopy. 2014;30(1):90-98.

Egger

Willimon

Busch

Broida

Perkins

CA.

Arthroscopic Bankart repair for adolescent anterior shoulder instability: clinical and imaging predictors of revision surgery and recurrent subjective instability. Am J Sports Med. 2023;51(4):877-884.

10.

Elsenbeck

Dickens

JF.

Return to sports after shoulder stabilization surgery for anterior shoulder instability. Curr Rev Musculoskelet Med. 2017;10(4):491-498.

11.

Hovelius

Olofsson

Sandstrom

, et al. Nonoperative treatment of primary anterior shoulder dislocation in patients forty years of age and younger. A prospective twenty-five-year follow-up. J Bone Joint Surg Am. 2008;90(5):945-952.

12.

Landis

Koch

. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1): 159-74.

13.

MacDonald

McRae

Old

, et al. Arthroscopic Bankart repair with and without arthroscopic infraspinatus remplissage in anterior shoulder instability with a Hill-Sachs defect: a randomized controlled trial. J Shoulder Elbow Surg. 2021;30(6):1288-1298.

14.

Metzger

Barlow

Leonardelli

, et al. Clinical application of the “glenoid track” concept for defining humeral head engagement in anterior shoulder instability: a preliminary report. Orthop J Sports Med. 2013;1(2):2325967113496213.

15.

Portney

Watkins

MP.

Foundations of Clinical Research: Applications to Practice. 2nd ed. Prentice Hall; 2000.

16.

Schneider

Hoy

, et al. Interobserver and intraobserver variability of glenoid track measurements. J Shoulder Elbow Surg. 2017;26(4):573-579.

17.

Shaha

Cook

Rowles

, et al. Clinical validation of the glenoid track concept in anterior glenohumeral instability. J Bone Joint Surg Am. 2016;98(22):1918-1923.

18.

Shaha

Cook

Song

, et al. Redefining “critical” bone loss in shoulder instability: functional outcomes worsen with “subcritical” bone loss. Am J Sports Med. 2015;43(7):1719-1725.

19.

Shin

Kim

Jeon

Kwon

TH.

Critical value of anterior glenoid bone loss that leads to recurrent glenohumeral instability after arthroscopic Bankart repair. Am J Sports Med. 2017;45(9):1975-1981.

20.

Yamamoto

Itoi

Abe

, et al. Contact between the glenoid and the humeral head in abduction, external rotation, and horizontal extension: a new concept of glenoid track. J Shoulder Elbow Surg. 2007;16(5):649-656.

21.

Yang

Mehran

Mazzocca

, et al. Remplissage versus modified Latarjet for off-track Hill-Sachs lesions with subcritical glenoid bone loss. Am J Sports Med. 2018;46(8):1885-1891.

22.

Yian

Weathers

Knott

Sodl

Spencer

HT.

Predicting failure after primary arthroscopic Bankart repair: analysis of a statistical model using anatomic risk factors. Arthroscopy. 2020;36(4):964-970.