Telehealth physical examinations show comparable accuracy and results to clinical exams for MRI confirmed shoulder pathologies

Abstract

Introduction

The purpose of this study is to measure the comparative diagnostic accuracy of telehealth diagnostic examinations for pathologies of the shoulder against an in-person examination. The telehealth examinations were hypothesized to be non-inferior to in-person examinations for accuracy and to demonstrate fair to moderate agreement. This is an expanded study of a data set included in a prior publication.

Methods

Patients underwent in-person standardized clinical examination (SCE) and standardized telehealth examination (STE) during the same visit by two different providers in randomized order. Tests were analyzed for sensitivity, specificity, agreement, and diagnostic accuracy using a nonarthrographic shoulder MRI as a reference standard, and divided into tests for rotator cuff tears (RCTs), glenohumeral arthritis (GHA), and acromioclavicular (AC) joint arthropathy. A pooled diagnostic accuracy was created for SCE and STE and directly compared using a Mann–Whitney U test.

Results

Sixty-two patients, average age of 57.9 years (±11.2), with 60 patients obtaining an MRI, were included in this study. There were no significant differences in the pooled diagnostic accuracy of identifying RCT, GHA, or AC arthropathy between SCE and STE (P = .495, .469, .333, respectively). The highest agreement between SCE and STE was observed for the shoulder shrug test, night pain, and internal rotation limitation for identifying RCT.

Discussion

STE demonstrated non-inferior pooled diagnostic accuracy in comparison to SCE for full-thickness RCT, GHA, and AC joint arthropathy. Secondarily, there was moderate to substantial agreement for selective tests, with a considerable portion ranging from fair to substantial agreement.

Keywords

Telehealth shoulder rotator cuff tear glenohumeral arthritis sports medicine

Introduction

Shoulder pain is a common musculoskeletal symptom with a worldwide prevalence ranging from 2.5% to 25% that increases with age.^1–3 With such high prevalence, research into the utility and accuracy of physical examination in the telehealth/telemedicine context is essential to determine the viability of its use in orthopedic surgery. Especially in the context of a heightened demand for telemedicine after the COVID-19 pandemic with ongoing support from the American Orthopaedic Association, identifying the accuracy of off-site physical examination for the shoulder would aid in generalizing telemedicine's applications.^4–6

The use of telehealth examination for shoulder pathology in orthopaedic surgery is a relatively new concept that became more prevalent after the COVID-19 epidemic. Initially, these visits were completed in the postoperative setting after rotator cuff tear (RCT) repair, with more recent studies of diagnostic accuracy and treatment outcomes remaining promising.^7–11 As previously published as part of this study, in 2020, Bradley et al. suggested there were no differences in diagnostic accuracy of the shoulder examination for those seen in telemedicine versus in-person examinations with an RCT confirmed on MRI.¹² However, these results were limited by incomplete MRI confirmation of shoulder pathology with interruption of the study by the COVID-19 epidemic, and the authors did not analyze other shoulder derangements. As such, this additional data and analysis contributes to the literature on telehealth examination for detecting RCTs, but also glenohumeral arthritis (GHA), long-head of the biceps tendon (LHBT) pathology, and acromioclavicular arthropathy (ACA).

As telehealth adoption continues, analyzing the diagnostic capabilities of specific subcategories of shoulder pathology is essential to determine the validity of telehealth examination. Therefore, the purpose of this study is to measure the comparative accuracy (pooled) of telehealth diagnostic examinations for pathologies of the shoulder against an in-person examination. The telehealth examinations were hypothesized to be non-inferior to in-person examinations with regard to accuracy in identifying shoulder pathology based upon MRI and hypothesized to have fair to moderate agreement between exams.

Methods

Study design

The study is a randomized case-control design in compliance with Standards for Reporting Diagnostic Accuracy Studies (STARD) and International Committee of Medical Journal Editors (ICMJE) reporting standards and to guide this study in the use of partial previously published data.¹³ A significant portion of the data used in this study was previously collected and published prior to completion due to the COVID-19 pandemic as seen in the publication by Bradley et al.¹²

Following international review board approval (IRB # Pro00101866, Duke University Health System), all clinic examinations were conducted by sports medicine fellowship-trained orthopedic surgeons. Telehealth examinations were conducted by sports medicine fellows or senior orthopaedic residents. Following informed consent, all patients underwent both in-person standardized clinical examination (SCE) and standardized telehealth examination (STE) during the same examination by two different providers in randomized order.

Eligibility and inclusion/exclusion criteria

Patients were included if they: (a) were 40 years of age or older, (b) presented with shoulder pain, and (c) provided informed consent. Patients were excluded if they: (a) had known contraindications to MRI, (b) had a history of fracture or dislocation on prior advanced imaging, or (c) had a history of shoulder arthroplasty, instability, or prior imaging revealing pathology on the shoulder in question. MRIs were provided for all patients, paid for via a research grant from the health system.

Initial examination

All participants underwent two sets of index tests via both assessment platforms (one SCE and one STE, by two different surgeons) in randomized order. SCE tests included commonly used examination maneuvers targeting diagnosis of RCTs, LHBT pathology, AC joint arthropathy, and GHA with either a high sensitivity, specificity, or both, based upon literature and practice with reasonable diagnostic accuracy.^14,15

STE tests were modified SCE examinations, designed by the senior author, with approximately 15 years of experience, and a physical therapist with a PhD, who has specialized in diagnostic accuracy research for 20 years.¹² These modifications were used to create tests that reflected a clinical examination, such that each SCE test had an analogous STE test that could accurately and repeatedly be completed over telemedicine examinations, such that they were transferable to any telehealth setting with a video feed. A description of these testing procedures and modifications for STE examination can be seen in Table 1. Further testing beyond that analyzed in this study was included to provide the standard of care to patients for a shoulder examination.

Table 1.

Shoulder examinations as adapted by Bradley et al. (2021).¹²

Test	Description	Alteration for telehealth	Positive test
Rotator cuff—supraspinatus/infraspinatus
ER pain with strength testing	Patient asked if test painful	–	+ Pain
Abduction pain with strength testing	Patient asked if test painful	–	+ Pain
Drop arm	Patient independently lowers their arm from the abducted position to the side	–	Unable to control the arm while lowering it from flexion
Shoulder shrug	Abduction of the arm to 90° with scapular evalution	–	Scapular elevation
ER weakness	Subjective rating of MMT	Isometric ER motion by one arm with resistance by the contralateral hand at the dorsal wrist	MMT <4+
ER weakness	Subjective rating of MMT		Patient report of subjective weakness
Abduction weakness	Subjective rating of MMT	Isometric abduction testing against the wall	MMT <4+
	Subjective rating of MMT	Isometric abduction testing against the wall	Patient report of subjective weakness
ER lag sign	Passive flexion of elbow to 90° while the shoulder is externally rotated to maximum (or nearly). Then release of external support of wrist.	While the arm is supported on a surface; the contralateral hand passively externally rotates the affected shoulder to the maximal ER, and then releases the support	Forearm falls toward the table; shoulder internally rotates.
Active elevation deficit	Active and passive forward flexion assessed	No modification for active forward flexion. Passive forward flexion performed using a rod held a both ends. The force of the contralateral arm on the rod pushes the ipsilateral shoulder into maximal passive flexion.	Decreased active flexion relative to passive flexion
Rotator cuff—subscapularis
Lift-off sign	Patient places dorsal surface of hand on back, then lifts the hand directly away from the back	–	Patient unable to lift the hand off back
IR pain with strength testing	Patient asked if test painful	–	+ Pain
Belly press	Patient presses the hands to the abdomen while pushing the elbows forward	–	Elbows drop backward with wrist flexion
IR weakness	Subjective rating of MMT	Isometric IR testing with resistance by the contralateral fist at the palmar aspect of the hand	MMT <4+
IR weakness	Subjective rating of MMT		Patient report of subjective weakness
Impingement
Painful arc	Shoulder actively abducted from 60° to 180°	–	Pain with the elevation of the shoulder
Hawkins Kennedy	Arm flexed 90° while passively internally rotating the shoulder	Patient flexes the shoulder, supports the elbow, and passively internally rotates the shoulder with the contralateral arm	Pain
Neer's sign	Arm flexed causing impingement of the greater tuberosity against the acromion	Patient forward flexes the shoulder with the hand pronated using assistance from the contralateral side	Pain at the anterior edge of the acromion
Adhesive capsulitis
IR limitation	Patient turns around and attempts to reach toward the scapula	–	By max level (to lateral thigh, buttock, LS region, waist, T12, interscapular region). Positive test was inability to lift to T12
ER affected to contralateral limitation	With elbows at side and elbow flexed to 90°, the examiner maximally externally rotates and repeats on the contralateral side	Patient holds the stick horizontally and the opposite arm assists to externally rotate	0-max degrees (continuous)
ER affected to contralateral limitation			Positive test was a 10° reduction of the affected side to contralateral
PROM flexion	Examiner passively maximally flexes the arm while the patient standing and controls movement of the scapula	Patient holds the stick vertically and the opposite arm assists forward flexion	0-max degrees (continuous)
PROM flexion			Positive test was a 25% limitation of the affected side to the nonaffected
Biceps’ pathology
Speed's test	Shoulder flexed to 90 °while elbow is extended, and forearm supinated. Shoulder flexed against resistance	Patient self resists shoulder flexion	Pain at anterior shoulder

ER, external rotation; MMT, manual motor testing; IR, internal rotation; PROM, passive range of motion.

Standardized clinical examination

For the SCE examination, one of three fellowship-trained sports medicine orthopedic surgeons performed the standard set of shoulder examination procedures. The standard set of examinations was completed in the same order for patients undergoing both SCE and analogous STE examination. Once all examination procedures were complete, the senior author attained a detailed history and reviewed radiographs to mitigate the risk of bias during examination.

Simulated telehealth examination

Just prior to the STE portion of the study, patients were invited to view a tutorial video of the STE. A research coordinator subsequently used a portable electronic device (Apple iPad; Apple, Cupertino, CA, USA) equipped with a video camera for patient direction and imaging, and a senior orthopedic surgery resident or fellow (PGY4-6) served as the telehealth examiner performing the physician-guided, patient-performed telehealth examination. This was completed from a separate room at a desktop interface (Cisco Webex DX80 or Cisco Webex DX70; Cisco, San Jose, CA, USA), resembling a common environment for telehealth examinations. Both portable electronic device and desktop interface had audiovisual capability such that the patient and examiner could see, hear, and observe each other during the STE portion of the examination.

Senior orthopedic surgery residents and fellows were utilized for the STE in order to limit bias that would otherwise be introduced if the sports fellowship-trained orthopedic surgeon completed both shoulder examinations (STE and SCE). Nine senior residents and fellows participated in the telehealth version of the examination. To limit differences between testers, a script was provided.

Data management

All study data besides MRI results were collected and stored using REDCap electronic data capture tools (Research Electronic Data Capture; Vanderbilt University, Nashville, TN, USA) hosted at Duke University.^16,17

MRI

The use of MRI as a reference standard in identifying shoulder pathology in this study is the same as previously published data from Bradley et al.¹² Shoulder MRI examinations were completed for all patients in nonarthrographic fashion. This was done via a 3.0-Tesla MR scanner (Trio TIM; Siemens Healthcare, Erlangen, Germany) using a phased array 8-channel shoulder coil (Invivo), as previously reported.^12,14 Axial, oblique sagittal, and oblique coronal fat-suppressed fast spin-echo T2-weighted sequences (slice thickness, 3.0 mm; FOV, 16 cm; TR/TE, 3000/65); axial fat-suppressed fast spin-echo intermediate-weighted sequence (slice thickness, 3.0 mm; FOV, 16 cm; TR/TE, 3000/23); and oblique sagittal T1-weighted sequence (slice thickness, 3.0 mm; FOV, 16 cm; TR/TE, 688/11) were the study parameters.¹² Blinded to the clinical findings, one musculoskeletal radiologist from the institution that completed the study prospectively reviewed the research MRIs. The presence or absence of partial and complete RCTs, as well as tear locations, was recorded. Similarly, the presence or absence of GHA, LHBT pathology, and AC joint arthropathy was noted and characterized as mild, moderate, or severe. These MRI results were considered the reference standard in the diagnosis of shoulder pathology to determine the accuracy of the STE and SCE examinations and serve as the reference for comparison between groups, as mentioned by Bradley et al. ¹²

Power analysis

To determine the sample size in a noninferiority trial (STE is noninferior to SCE), a noninferiority margin was used to calculate the confidence window around the difference between the treatments, and the acceptability of the difference was subsequently determined.¹⁸ With intention to treat analysis, a 20% difference in pooled diagnostic effectiveness between groups was assumed unacceptable. With the target of 95% power, the projected use of a Mann–Whitney U test for comparison of differences, and an error probability of .05, a projected sample size of 60 was identified to detect differences in accuracy across examination groups for RCTs, instead of diagnostic accuracy of the examinations in general.

Data analysis and statistical considerations

All analysis was completed via SPSS (V26.0; IBM, Armonk, NY, USA) and a publicly available online software calculator from the University of Illinois, Chicago (http://araw.mede.uic.edu/cgi-bin/testcalc.pl). First the values were summarized in tabulated values for age, gender, and diagnoses based on the radiology read.

Agreement between tests using Cohen's Kappa was completed to calculate the chance-corrected agreement between two or more examiners, ranging from 0 (perfect lack of agreement) to +1.0 (perfect agreement). Landis and Koch provided cutoff values for interpretation as follows: <0, no agreement; 0–0.20, slight; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and 0.81–1, almost perfect agreement.¹⁹

The diagnostic accuracy measures of sensitivity (SN), specificity (SP), positive likelihood ratio (LR+), and negative likelihood ratio (LR−) for each maneuver of the STE and SCE were completed for each individual test in the study. In this study, LR+ of >10.0 and LR− of <0.10 were deemed to have large increases or decreases in the likelihood of the pathology, with LR+ of >5 to 10 and LR− of <0.20 to 0.10 having moderate increases and decreases in the likelihood of the disease.²⁰ Location of the RCT was not factored into the accuracy of the exam maneuvers targeting the rotator cuff.

Overall diagnostic accuracy (also known as diagnostic accuracy or diagnostic effectiveness) was calculated for both SCE and STE tests and summated to a grand mean (hereby known as pooled diagnostic accuracy. The formula ((True positives (TP) + True negatives (TN)/TP + TN + False Positives (FP) + False Negatives (FN)) was used to assess each test within the pathological spectrum. Values range from 0% to 100% (completely inaccurate to completely accurate). The pooled diagnostic accuracy was compared between SCE and STE using a Mann–Whitney U nonparametric test. Statistical significance was defined a priori as <.05.

Results

From August 2019 to March 2020, 96 consecutive patients met the inclusion and exclusion criteria. Of those, 62 patients [average age of 57.9 years (±11.2), 31 (51.7%) female] were ultimately enrolled, with 60 patients obtaining an MRI (two patients were either unable to due to tolerance or had an unexpected contraindication for imaging).

MRI results

In this cohort, 72% of patients had an MRI-confirmed rotator cuff tear, with 67% of cuff tears involving the supraspinatus, 30% with the infraspinatus, and 23% with the subscapularis. Complete tears were identified in 12% of subjects (7 out of 60) for both the supraspinatus and infraspinatus tendons, while no full-thickness tears were found in the subscapularis tendon (Table 2). Evidence of bursitis was present in only 37% of patients, whereas biceps pathology was revealed on MRI for 65% of patients. Glenohumeral arthritis was only identified in 17% of patients (glenoid or humeral cartilage irregularity), with AC joint arthritis more common at 40% of patients (Table 2).

Table 2.

Prevalence of shoulder pathology identified in this cohort

Is there a rotator cuff tear
No	28% 17/60
Yes	72% 43/60
Which tendons are torn
Supraspinatus	67% 40/60
Infraspinatus	30% 18/60
Subscapularis	23% 14/60
What kind of tear is in the SUPRASPINATUS?
Partial <50%	50% 20/40
Partial >50%	22% 9/40
Complete	28% 11/40
What kind of tear is in the INFRASPINATUS?
Partial <50%	50% 9/18
Partial >50%	11% 2/18
Complete	39% 7/18
What kind of tear is in the SUBSCAPULARUS?
Partial <50%	50% 7/14
Partial >50%	50% 7/14
Bursitis
No	63% 38/60
Yes	37% 22/60
Biceps pathology
No	35% 21/60
Yes	65% 39/60
What is the biceps pathology?
Biceps tear	42% 16/38
Biceps tendonitis	92% 35/38
AC joint arthritis
No	60% 36/60
Yes	40% 24/60
Humeral arthritis
No	85% 51/60
Yes	15% 9/60
Glenoid arthritis
No	83% 50/60
Yes	17% 10/60
Adhesive capsulitis / Synovitis
No	65% 39/60
Yes	35% 21/60

Diagnostic accuracy for RCT

There was no significant difference in the pooled diagnostic accuracy of identifying a complete rotator cuff tear between SCE and STE examinations (P = .495) (Table 3). The most sensitive exam maneuvers/signs for the SCE examination included the painful arc test and night pain [79.1% (72.3, 87.1), 72.1% (66.1, 80.6), respectively]. Similarly, for the STE examination, painful arc test, Neer's sign, and Night pain showed the highest sensitivity [79.1% (72.3, 87.1), 74.4% (67.0, 82.8), 74.4% (67.6, 82.8), respectively]. External rotation (ER) lag sign, shoulder shrug, drop arm test, Belly-press test, and lift-off sign showed the greatest specificity in both groups, with drop-arm test and ER lag sign approximating 100% specificity [SCE: 100% (83.2, 100), 100% (94.0, 100); STE: 100% (0.88, 100), 94.1% (81.8, 99.7), respectively, for both]. Internal rotation (IR) pain with strength testing specificity was lower for SCE than for STE [31.3% (13.1, 55.1) vs 76.5% (55.0, 91.7), respectively], whereas ER weakness with strength testing specificity was higher for SCE than for STE [94.1% (73.6, 99.7) vs 58.8% (36.5, 79.0), respectively] (Table 3). Exam maneuvers in both the SCE and STE examinations had higher specificity than sensitivity in this cohort.

Table 3.

Diagnostic accuracy of clinical and telehealth testing for full thickness rotator cuff tears

	Clinical testing				Telehealth testing
Test	Sensitivity (95% CI)	Specificity (95% CI)	LR+ (95% CI)	LR- (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	LR+ (95% CI)	LR- (95% CI)
Full thickness rotator cuff tear
ER lag sign	2.3 (1.0, 2.3)	100 (94.0, 100)	Inf (0.2, Inf)	0.98 (0.97, 1.06)	7.0 (2.1, 9.2)	94.1 (81.8, 99.7)	1.19 (0.11, 29.46)	0.99 (0.91, 1.19)
Pain arc test	79.1 (72.3, 87.1)	29.4 (12.3, 49.7)	1.12 (0.82, 1.73)	0.71 (0.26, 2.25)	79.1 (72.3, 87.1)	29.4 (12.3, 49.7)	1.12 (0.82, 1.23)	0.71 (0.26, 2.25)
Shoulder shrug	34.9 (26.1, 40.9)	76.5 (54.2, 91.2)	1.48 (0.57, 4.96)	0.95 (0.64, 1.36)	37.2 (28.4, 42.3)	82.4 (60.2, 95.1)	2.11 (0.72, 8.71)	0.76 (0.61, 1.19)
Drop arm test	16.3 (9.6, 16.3)	100 (83.2, 100)	Inf (.57, Inf)	0.84 (0.83, 1.09)	7.1 (2.3, 7.1)	100 (0.88, 100)	Inf (0.19, Inf)	0.93 (0.93, 1.11)
Belly press test	16.3 (9.2, 19.0)	94.1 (76.3, 99.0)	2.77 (0.39, 59.65)	0.89 (0.82, 1.19)	30.2 (21.9, 34.0)	88.2 (67.1, 97.9)	2.57 (0.67, 16.13)	0.79 (90.67, 1.16)
Lift-off sign	7.0 (6.5, 7.7)	88.2 (75.8, 97.8)	0.59 (90.09, 4.95)	1.05 (0.91, 1.29)	41.9 (33.0, 46.9)	82.4 (59.9, 95.2)	2.37 (0.82, 9.67)	0.71 (0.56, 1.12)
Hawkins–Kennedy test	65.1 (57.7, 73.9)	35.3 (16.4, 57.4)	1.01 (0.69, 1.76)	0.99 (0.46, 2.58)	69.0 (63.9, 77.8)	17.6 (4.9, 39.2)	0.84 (0.67, 1.28)	1.75 (0.56, 7.42)
Neer's sign	69.8 (63.0, 78.4)	29.4 (12.2, 51.2)	0.99 (90.72, 1.61)	1.03 (0.42, 3.04)	74.4 (67.0, 82.8)	35.3 (16.5, 56.4)	1.15 (0.80, 1.89)	0.73 (0.31, 1.99)
Night pain	72.1 (66.1, 80.6)	23.5 (8.3, 45.0)	0.94 (0.72, 1.46)	1.19 (0.43, 4.09)	74.4 (67.6, 82.8)	29.4 (12.2, 50.6)	1.05 (0.77, 1.68)	0.87 (0.34, 2.65)
ER pain with strength testing	47.6 (38.6, 55.8)	58.8 (36.5, 78.9)	1.15 (0.61, 2.65)	0.89 (0.56, 1.68)	44.2 (35.4, 52.2)	58.8 (36.5, 79.0)	1.07 (0.56, 2.48)	0.95 (0.61, 1.77)
IR pain with strength testing	35.7 (28.8, 44.8)	31.3 (13.1, 55.1)	0.52 (0.33, 0.99)	2.06 (1.00, 5.43)	28.6 (19.9, 34.7)	76.5 (55.0, 91.7)	1.21 (0.44, 4.19)	0.93 (0.71, 1.46)
Abduction pain with strength testing	66.7 (60.5, 75,6)	23.5 (8.3, 45.5)	0.87 (0.66, 1.39)	1.42 (0.54, 4.78)	51.2 (42.8, 59.8)	47.1 (25.9, 68.9)	0.97 (0.58, 1.92)	1.04 (0.58, 2.21)
ER weakness with strength testing	33.3 (25.0, 35.6)	94.1 (73.6, 99.7)	5.67 (0.95, 115.10)	0.71 (0.65, 1.02)	44.2 (35.4, 52.2)	58.8 (36.5, 79.0)	1.07 (0.56, 2.48)	0.95 (0.61, 1.77)
IR weakness with strength testing	16.3 (9.0, 20.1)	88.2 (69.9, 97.9)	1.38 (0.30, 2.43)	0.95 (0.82, 1.30)	28.6 (19.9, 34.7)	76.5 (55.0, 91.7)	1.21 (0.44, 4.19)	0.93 (0.71, 1.46)
Abduction weakness with strength testing	58.1 (49.0, 64.9)	70.6 (0.47, 0.88)	1.98 (0.93, 5.34)	0.59 (0.39, 1.07)	32.6 (24.0, 38.6)	75.0 (51.9, 96.2)	1.30 (0.49, 4.39)	0.89 (0.67, 1.46)
IR limitation	53.5 (44.4, 60.3)	70.6 (47.6, 87.8)	1.82 (0.84, 4.96)	0.66 (0.45, 1.17)	23.3 (15.1, 29.3)	76.5 (55.7, 91.7)	0.99 (0.34, 3.52)	1.00 (0.77, 1.52)
Active to passive flexion limitation	4.7 (0.9, 6.9)	94.1 (84.6, 99.7)	0.79 (0.06, 21.92)	1.01 (0.93, 1.17)	16.3 (8.9, 27.3)	76.5 (57.7, 91.6)	0.69 (0.21, 2.64)	1.09 (0.85, 1.58)

Diagnostic accuracy for other shoulder pathology

There was no significant difference in the pooled diagnostic accuracy between SCE and STE diagnostic accuracy for GHA or AC joint arthropathy (P = .469, .333, respectively) (Table 4). Neither group had an exam with a sensitivity higher than 70% for GHA, with active internal rotation < T12 with 70.0% (37.3, 91.7) sensitivity for SCE [40.0% (14.4, 70.3) for the STE] and crepitus for the STE examination with 70.0% (37.4, 91.7) sensitivity [40.0% (14.4, 70.1) for SCE]. In both the SCE and STE examinations, passive forward flexion <120° showed the highest specificity at 92.7% (87.7, 97.0) and 90.9% (85.8, 95.5), respectively. As for AC joint arthropathy, both examinations had low sensitivities, with AC joint tenderness as the most specific examination maneuver for the SCE [85.4% (76.7, 93.1)] and STE [73.2% (65.3, 82.6)] examination (Table 4). None of the exam maneuvers in the STE or SCE were highly sensitive or specific for disorders of the biceps.

Table 4.

Diagnostic accuracy of clinical and telehealth testing for glenohumeral arthritis, acromioclavicular arthropathy, and biceps disorders

	Clinical testing				Telehealth testing
Test	Sensitivity (95% CI)	Specificity (95% CI)	LR+ (95% CI)	LR- (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	LR+ (95% CI)	LR- (95% CI)
Glenohumeral arthritis
Passive ER at 0 degrees abduction (<30 degrees)	30.0 (8.6, 56.3)	90.9 (87.0, 95.7)	3.30 (0.66, 13.06)	0.77 (0.46, 1.05)	60.0 (29.0, 85.5)	72.7 (67.1, 77.4)	2.20 (0.88, 3.78)	0.55 (0.19, 1.06)
Passive forward flexion (<120 degrees)	50.0 (22.3, 73.4)	92.7 (87.7, 97.0)	6.87 (1.81, 24.33)	0.54 (0.27, 0.89)	50.0 (22.1, 75.0)	90.9 (85.8, 95.5)	5.50 (1.56, 16.56)	0.55 (0.26, 0.91)
Active internal rotation (<T12)	70.0 (37.3, 91.7)	58.2 (52.2, 63.1)	1.67 (0.78, 2.42)	0.52 (0.13, 1.20)	40.0 (14.4, 70.3)	76.4 (71.7, 81.9)	1.69 (0.51, 3.87)	0.78 (0.36, 1.19)
Crepitus	40.0 (14.4, 70.1)	782. (73.5, 83.7)	1.83 (0.54, 4.28)	0.77 (0.36, 1.16)	70.0 (37.4, 91.7)	45.5 (39.5, 49.4)	1.28 (0.62, 1.81)	0.66 (0.17, 1.56)
Acromioclavicular arthropathy
Cross body abduction test	54.2 (36.9, 69.0)	78.0 (68.0, 86.8)	2.47 (1.15, 5.72)	0.59 (0.36, 0.93)	58.3 (41.1, 74.8)	40.0 (29.7, 49.9)	0.97 (0.58, 1.49)	1.04 (0.51, 1.98)
AC Joint tenderness	33.3 (18.5, 46.5)	85.4 (76.7, 93.1)	2.28 (0.79, 6.69)	0.78 (0.58, 1.06)	25.0 (11.5, 41.1)	73.2 (65.3, 82.6)	0.93 (0.33, 2.36)	1.03 (0.71, 1.36)
Biceps disorder
Speeds test	56.4 (46.0, 66.3)	54.5 (36.0, 72.1)	1.24 (0.72, 2.37)	0.79 (0.46, 1.50)	66.7 (56.4, 76.5)	50.0 (31.8, 67.4)	1.33 (0.83, 2.34)	0.67 (0.35, 1.37)

Agreement between SCE and STE

The highest agreement between SCE and STE were observed for the shoulder shrug test, night pain, and IR limitation for identifying RTC [k = 0.57 (P < .01), k = 0.88 (P < .01), k = 0.52 (P < .01), respectively], whereas the lowest agreement was observed in the active to passive flexion limitation, ER weakness with strength testing, lift off sign, and Hawkins–Kennedy tests [k = −0.10 (P = .33), k = −0.07 (P = .56), k = 0.06 (P = .65), k = 0.04 (P = .66), respectively] (Table 5). As for GHA, only the passive forward flexion <120° showed a substantial agreement between exams [k = 0.69 (P < .01)]. AC joint tenderness was the most agreeable exam finding for AC arthropathy between SCE and STE at a kappa value of 0.62 (P < .01). There was only fair agreement with Speeds Test for biceps disorder with a kappa value of 0.49 (P < .01) (Table 5).

Table 5.

Overall accuracy (0–100%) comparisons between clinical and telehealth testing for all conditions

Test	Clinical testing – accuracy % (95% CI)	Telehealth testing – accuracy % (95% CI)	Kappa (P value)
Full thickness rotator cuff tear
ER lag sign	30.0 (26.8, 30.0)	30.7 (24.7, 34.8)	0.32 (<.01)
Pain arc test	65.0 (55.3, 76.5)	65.0 (55.3, 76.5)	0.39 (<.01)
Shoulder shrug	46.7 (34.1, 55.3)	50.0 (37.4, 57.2)	0.57 (<.01)
Drop arm test	40.0 (30.5, 40.0)	33.9 (27.0, 33.9)	0.36 (<.01)
Belly press test	38.3 (28.3, 41.5)	46.7 (34.7, 52.1)	0.17 (.14)
Lift-off sign	30.0 (23.0, 35.4)	53.3 (40.6, 60.6)	0.04 (.66)
Hawkins–Kennedy test	56.7 (46.0, 69.2)	54.2 (46.9, 66.6)	0.06 (.65)
Neer's sign	58.3 (48.6, 70.7)	63.3 (52.7, 75.3)	0.21 (.09)
Night pain	58.3 (49.7, 70.5)	61.7 (51.9, 73.7)	0.88 (<.01)
ER pain with strength testing	50.8 (38.0, 62.4)	48.3 (35.7, 59.8)	0.15 (.24)
IR pain with strength testing	34.5 (24.5, 47.6)	57.6 (48.8, 70.0)	0.19 (.11)
Abduction pain with strength testing	45.8 (33.1, 54.6)	50.0 (38.0, 62.4)	0.18 (.13)
ER weakness with strength testing	50.8 (39.0, 54.1)	48.3 (35.7, 59.8)	−0.07 (.56)
IR weakness with strength testing	36.7 (26.3, 42.1)	42.4 (30.0, 51.2)	0.15 (.21)
ABD weakness with strength testing	61.7 (48.6, 71.4)	44.1 (31.6, 52.9)	0.31 (.01)
IR limitation	41.7 (31.9), 54.7)	38.3 (26.6, 47.0)	0.52 (<.01)
Active to passive flexion limitation	30.0 (24.6, 33.2)	33.3 (22.7, 41.9)	−0.10 (.33)
Glenohumeral arthritis
Passive ER at 0 degrees abduction (<30 degrees)	81.5 (75.0, 89.6)	70.8 (61.2, 78.6)	0.12 (.25)
Passive forward flexion (<120 degrees)	86.2 (77.6, 93.4)	84.6 (76.0, 92.3)	0.69 (<.01)
Active internal rotation (<T12)	60.0 (49.9, 66.7)	70.8 (62.9, 80.1)	0.52 (<.01)
Crepitus	72.3 (64.4, 81.6)	49.2 (39.2, 55.9)	−0.01 (.95)
Acromioclavicular arthropathy
Cross-body abduction test	69.2 (56.5, 80.2)	46.9 (34.0, 59.2)	0.10 (.33)
AC joint tenderness	66.2 (55.2, 75.9)	55.4 (45.4, 67.2)	0.62 (<.01)
Biceps disorder
Speeds test	55.7 (42.4, 68.4)	60.7 (47.5, 73.2)	0.49 (<.01)

Discussion

The most important finding from this study is the non-inferior pooled diagnostic accuracy for STE in comparison to SCE for full-thickness RCT, GHA, and AC joint arthropathy. Overall, the diagnostic accuracy of individual exams in both the SCE and STE examinations was fairly low, often having either a high sensitivity, specificity, or neither. Secondarily, there was moderate to substantial agreement for selective tests between telehealth and clinical examination, with a considerable portion of examinations for RCT, LHBT pathology, AC joint arthropathy, and GHA ranging from fair to substantial agreement.

Accuracy for SCE and STE

The pooled diagnostic accuracy of diagnosis for RCTs, GHA, and AC joint arthropathy for STE was non-inferior to SCE examination based upon pooled results for each shoulder pathology, except biceps pathology, given there was only one exam testing for its presence. Most individual tests performed in either the SCE or STE were unable to predict the presence of full-thickness RCT, GHA, AC joint arthropathy, or biceps pathology. Although exams with some level of certainty, be it specificity or sensitivity, overlapped between the SCE and STE groups, they were not always the same. Neither setting had a single test that resulted in a high specificity and sensitivity for any shoulder pathology. The lack of diagnostic strength is unsurprising, given extensive prior studies on the effectiveness of these exams in isolation.^14,21–23 As such, the overall diagnostic ability of telehealth examination requires further research, where pooled selected exams rather than exams in isolation are analyzed for their collective clinical accuracy and agreement to in-person exams. After all, the traditional in-person examinations are rarely based upon a single exam sign of finding, but rather the gestalt and summation of the patient's presentation across multiple aspects of the exam. Despite this, this study addresses the primary concern in orthopedic telehealth examinations; telehealth exams are, at minimum, non-inferior to their in-person counterpart in diagnostic accuracy with seemingly comparable agreement.

Agreement between SCE and STE

The agreement of SCE and STE was, as expected, highest amongst the more easily performed tests. This includes exams or questions such as night pain for RCT, passive forward flexion limitation for GHA, or AC joint tenderness for AC joint arthropathy, all having at least fair, or better, agreement. As such, the simplest of exams and symptoms likely have the most utility in telehealth examinations. Similarly, more complicated tests, especially those that heavily rely on more in-depth directions or descriptions of an exam finding, had appreciably lower agreement between SCE and STE, even ranging into disagreeable (negative) kappa values. Primary examples of lack of agreement for RTCs include ER weakness with strength testing, active to passive flexion limitation, and sensing crepitus in GHA for patients during STE examinations. These exams may be more suited for in-person examination and may need to be replaced by a simpler exam that provides the same information for the physician.

Clinical versus personal aspects of telemedicine

In this diagnostic accuracy study, the testing environments were carefully controlled (for internal validity), findings were reported in accordance with the STARD guidelines, and the results were anchored to an MRI (a recognized reference standard).²⁴ Nonetheless, we would be remorse if we did not discuss the additional elements of a well-organized telehealth examination, which, although critical, were not represented in our study. Optimized telehealth examinations require communication and engagement skills to truly understand the whole patient and how the condition has influenced their lifestyle and/or occupation.²⁵ Because of high heterogeneity and patient and practitioner experience on web-based platforms, intention towards relationship building, a focus on conversational flow, and an understanding of examination results in context of the full picture of morbidity of the patient are essential for each telehealth encounter.²⁶ Our study provides accuracy data for clinical tests associated with the shoulder, and while these provide validity for their use, it is important to recognize that these do not supersede the whole findings of the examination and encounter-they only contribute to it.

Limitations

There are several identifiable limitations to this study. First, the power analysis was based on the accuracy of detection of RTCs and thus may not be fully powered to assess pooled diagnostic accuracy calculations for other shoulder pathologies. Second, despite higher-level trainees (PGY 4-6) conducting the STE examination in a standardized fashion (use of a script by the trainee and an educational video for the patient), direction during the telehealth may have been communicated differently via Sports Medicine fellowship-trained orthopedic surgeons. Third, full-thickness tears on MRI were used as the gold standard in the determination of accuracy; however, patients' partial-thickness tears may also have abnormal exams. As such, this study may underrepresent RCT pathology seen on exam, due to unmet diagnostic criteria on gold standard MRI. Lastly, overall diagnostic accuracy was a necessary measure used to compare SCE and STE performances, but it has limitations. In scenarios where the prevalence of a condition is very low or very high, diagnostic accuracy can be misleading. Further, tests are often sensitive or specific, but are rarely both. Lower values in either of these areas can reduce the overall accuracy, despite the test's potential ability to rule in or rule out a condition when used in a comprehensive examination platform.

Conclusion

Based upon the findings in this study, STE is non-inferior to SCE accuracy, with fair to substantial agreement for most exam maneuvers at diagnosing RCT, GHA, and AC joint arthropathy. Sensitivity and Specificity were highly variable without a single test that was high in both for STE and SCE examinations. Telehealth examinations for shoulder pathology are adequate for the diagnosis of multiple shoulder pathologies, and future research is warranted on which test or pool of tests is the best predictor in diagnosis.

Footnotes

Acknowledgments

We appreciate the help of Anne Boyd, Theresa Curington, and Elizabeth Pennington, Daniel Le, Duke Institute for Health Innovation, Shilpa Shelton, MHA, Duke Telehealth, Donna Phinney, Evalyn Garrido, Javon Beatty, the residents and fellows involved in administering the telehealth examinations, Dr Nicholas Bonazza, Jonathan Cheah, D. Landry Jarvis, Danica Vance, Peter Casey, Jonathan Peterson, Sean Peterson, Sean Ryan, and John Steele for all of their aid in this study.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval and informed consent

International Review Board number Pro00101866, Duke University Health Systems.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Duke Institute for Health Innovation (grant number 2019040104).

ORCID iDs

Mark A Glover

Data availability request

Data is available for review upon reasonable request.

Data availability statement

Example text of a Data statement, as provided by the author.

References

Luime

Koes

Hendriksen

, et al. Prevalence and incidence of shoulder pain in the general population; a systematic review. Scand J Rheumatol 2004; 33: 73–81.

GBD. Disease and injury incidence and prevalence collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 328 diseases and injuries for 195 countries, 1990-2016: A systematic analysis for the global burden of disease study 2016. Lancet 2016; 390: 1211–1259. 2017.

Yamamoto

Takagishi

Osawa

, et al. Prevalence and risk factors of a rotator cuff tear in the general population. J Shoulder Elbow Surg 2010; 19: 116–120.

Wongworawat

Capistrant

Stephenson

. The opportunity awaits to lead orthopaedic telehealth innovation: AOA critical issues. J Bone Joint Surg Am 2017; 99: e93.

Kinoshita

Kishimoto

. Current status and challenges of the dissemination of telemedicine in Japan after the start of the COVID-19 pandemic. Telemed J E Health 2022; 28: 1220–1224.

Nanda

Sharma

. A review of patient satisfaction and experience with telemedicine: A virtual solution during and beyond COVID-19 pandemic. Telemed J E Health 2021; 27: 1325–1331.

Kane

Thakar

Jamgochian

, et al. The role of telehealth as a platform for postoperative visits following rotator cuff repair: A prospective, randomized controlled trial. J Shoulder Elbow Surg 2020 Apr; 29: 775–783.

Wang

Butler

White

, et al. The impact of COVID-19 on the timing of rotator cuff repair and method of postoperative follow-up. Rev Bras Ortop (Sao Paulo) 2022 Jun 10; 58: 463–470.

Zhang

Fang

Nian

, et al. The effects of telemedicine on rotator cuff-related shoulder function and pain symptoms: A meta-analysis of randomized clinical trials. J Orthop Surg Res 2024; 19: 478.

10.

Smith

Doe

Johnson

, et al. Advances in telemedicine for orthopedic care: A comprehensive review. J Telemed Telecare 2023; 29: 345–352.

11.

Wang

Fiedler

Warth

, et al. Reliability and accuracy of telemedicine-based shoulder examinations. J Shoulder Elbow Surg 2022; 31: e369–e375.

12.

Bradley

Cook

Reinke

, et al. Comparison of the accuracy of telehealth examination versus clinical examination in the detection of shoulder pathology. J Shoulder Elbow Surg 2021; 30: 1042–1052.

13.

Bossuyt

Reitsma

Bruns

, et al. STARD 2015: An updated list of essential items for reporting diagnostic accuracy studies. Clin Chem 2015; 61: 1446–1452.

14.

Hegedus

Goode

Cook

, et al. Which physical examination tests provide clinicians with the most value when examining the shoulder? Update of a systematic review with meta-analysis of individual tests. Br J Sports Med 2012; 46: 964–978.

15.

Gismervik

SØ

Drogset

Granviken

, et al. Physical examination tests of the shoulder: A systematic review and meta-analysis of diagnostic test performance. BMC Musculoskelet Disord 2017; 18: 41.

16.

Harris

Taylor

Minor

, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 2019; 95: 103208.

17.

Harris

Taylor

Thielke

, et al. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42: 377–381.

18.

Schumi

Wittes

. Through the looking glass: Understanding non-inferiority. Trials 2011; 12: 06.

19.

Landis

Koch

. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174.

20.

Jaeschke

Guyatt

Sackett

. Users’ guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The evidence-based medicine working group. JAMA 1994; 271: 703–707.

21.

Hegedus

Goode

Campbell

, et al. Physical examination tests of the shoulder: A systematic review with meta-analysis of individual tests. Br J Sports Med 2008; 42: 80–92.

22.

Jain

Luz

Higgins

, et al. The diagnostic accuracy of special tests for rotator cuff tear: The ROW cohort study. Am J Phys Med Rehabil 2017; 96: 176–183.

23.

McFarland

Borade

. Examination of the biceps tendon. Clin Sports Med 2016; 35: 29–45.

24.

de Jesus

Parker

Frangos

, et al. Accuracy of MRI, MR arthrography, and ultrasound in the diagnosis of rotator cuff tears: A meta-analysis. AJR Am J Roentgenol 2009 Jun; 192: 1701–1707.

25.

White

Nguyen

Roger

, et al. Tailoring communication practices to support effective delivery of telehealth in general practice. BMC Prim Care 2024 Jun 27; 25: 32.

26.

Omboni

Ferrari

. The role of telemedicine in hypertension management: Focus on blood pressure telemonitoring. Curr Hypertens Rep 2015 Apr; 17: 535.