Interrater and Intrarater Reliability of Subcutaneous Echogenicity Grade and Subcutaneous Echo-Free Space Grade in Breast Cancer-Related Lymphedema

Abstract

Background:

Subcutaneous echogenicity grade (SEG) and subcutaneous echo-free space (SEFS) grade allow semiquantitation of nonspecific subcutaneous tissue inflammation and fluid accumulation in breast cancer-related lymphedema. However, inter- and intrarater reliability of SEG and SEFS is yet to be reported. The objective of this study is to assess inter-rater and intrarater reliability of SEG and SEFS in patients with breast cancer-related lymphedema.

Methods and Results:

Two physiatrists performed SEG and SEFS grade to determine inter-rater and intrarater reliability. Inter-rater reliability for SEG and SEFS was excellent (Kappa [K] = 0.836, weighted Kappa [K_w] = 0.85; K = 0.884, K_w = 0.92). K and K_w values of SEG and SEFS grading systems indicated excellent intrarater reliability (K = 0.81, K_w = 0.83; K = 0.798, K_w = 0.82). This trial is registered with Clinicaltrials.gov, under number NCT03559296.

Conclusions:

Based on the findings of this study, SEG and SEFS demonstrated acceptable reliability. SEG and SEFS appear to be reliable and useful grading systems for the assessment of breast cancer-related lymphedema.

Introduction

Lymphedema can be defined as the abnormal accumulation of protein-rich interstitial fluid that occurs primarily as a consequence of malformation, dysplasia, or acquired disruption of lymphatic circulation.¹ Lymph stasis, which results in peripheral lymphedema, is characterized by edema and adipose tissue proliferation. As a vicious cycle of lymphedema progression, lymph stasis stimulates chronic inflammation because of uncontrolled responses of macrophages and CD4+ cells; fat accumulation also causes chronic inflammation by infiltration and activation of macrophages that produce inflammatory cytokines, which further promote lymph stasis directly or indirectly by decreasing lymphatic pumping and increasing capillary filtration.^2,3 Lymphedema secondary to breast cancer is caused by the disruption of the lymphatic system, which in the initial stages leads to the accumulation of fluid in the interstitial tissue space and eventually is clinically presented as swelling of the arm, shoulder, neck, or torso.⁴ Complex decongestive physical therapy is a widely used nonoperative treatment of breast cancer-related lymphedema. The extremity volume has been one of the major parameters representing the treatment results of complex decongestive physical therapy. However, the increase in extremity volume in lymphedema can be caused both by tissue fluid accumulation and by pathologic tissue proliferation, which cannot be assessed separately at present. Accordingly, the volume or circumference measurement alone may not clarify how these phenomena are modified by complex decongestive physical therapy.⁵

Musculoskeletal ultrasound has been widely used in the field of physical and rehabilitation medicine in the recent years. Ultrasonography using ordinary resolution is widely available, safe, inexpensive, and noninvasive, and therefore is highly recommended for the assessment of lymphedema. Skin and subcutaneous tissue ultrasonography can help diagnose, determine the severity of, and evaluate the treatment effects in patients with breast cancer-related lymphedema. Moreover, ultrasonographic assessments can be shared with patients and other medical staff, making them more comfortable and cooperative.^2,6

Suehiro et al. developed subcutaneous echogenicity grade (SEG) and subcutaneous echo-free space (SEFS) grade via B-mode ultrasonography, allowing semiquantitation of nonspecific subcutaneous tissue inflammation and fluid accumulation.⁷ Increase in SEG is attributed to increased cell density and increased collagen content in the tissue and it is considered to indicate the presence of ongoing or previous inflammation in the area. SEFS represents the fluid accumulated in the spaces between superficial fasciae, which is freely mobile in the spaces.⁵ Although these grading systems have been previously used to investigate the etiology of leg lymphedema or segmental distribution characteristics of arm and leg lymphedema, inter- and intrarater reliability of SEG and SEFS have yet to be reported.^2,8,9 These qualitative ultrasonographic assessment methods appear to be a promising objective way of assessing lymphedema-related changes, however, to be considered a useful tool to assess lymphedema, the reliability of the grading system needs to be determined.¹⁰ Therefore, the aim of this study was to investigate the inter-rater and intrarater reliability of SEG and SEFS grade in breast cancer-related lymphedema.

Materials and Methods

Study design

The study was designed as a prospective longitudinal study with two time points for data collection (Fig. 1).

FIG. 1.

Study flow.

Patients and setting

The study was approved by the Ethics Committee of Marmara University School of Medicine (approval number: 09.2018.466). It was registered on the Clinical Trials Registry and the registration number is NCT03559296. Oral and written informed consents were obtained from all patients after explanation of the procedure. This study was conducted in accordance with the principles of the Declaration of Helsinki. We enrolled 50 women with breast cancer-related arm lymphedema who were admitted to Lymphedema Management Outpatients Clinic of Department of Physical Medicine and Rehabilitation, Marmara University School of Medicine, between April 2018 and August 2018. Patient characteristics are presented in Table 1. Inclusion criteria were (1) unilateral lymphedema, (2) patients who underwent axillary lymph node dissection, and (3) stable lymphedema that has not required therapy in the past 3 months. Exclusion criteria were (1) bilateral lymphedema, (2) systemic edemagenic conditions (e.g., cardiac/hepatic/renal failure, terminal cancer, on chemotherapy, cancer recurrence), (3) current infection in upper extremity, and (4) neuropathy or myopathy (brachial plexopathy etc.), which may influence echogenicity.

Table 1.

Patient Characteristics

Age (years; mean ± SD)	52 ± 9.41
BMI (kg/m²; mean ± SD)	29.95 ± 5.64
ISL lymphedema clinical stage (median)	2
Duration of lymphedema (months; mean ± SD)	34.06 ± 27.27
Affected side
Right	26 (52%)
Left	24 (48%)
Dominant side
Right	48 (96%)
Left right	2 (4%)
Surgery type
Mastectomy	39 (78%)
Breast conserving surgery	11 (22%)
Chemotherapy	41 (82%)
Radiotherapy	42 (84%)
SEG
0	9 (18%)
1	33 (66%)
2	8 (16%)
SEFS
0	33 (66%)
1	12 (24%)
2	5 (10%)

BMI, body mass index; ISL, the International Society of Lymphology; SEFS, subcutaneous echo-free space grade; SEG, subcutaneous echogenicity grade.

Sample size

A power calculation was performed based on the pilot data from ultrasound images taken from patients diagnosed with breast cancer-related lymphedema.⁸ A sample size of 23 subjects was determined to be sufficient to detect differences in SEG and SEFS grade between involved and uninvolved arms with a 95% power and 5% type I error.

Outcome assessments

The International Society of Lymphology clinical stage

The clinical stage of lymphedema was determined according to the International Society of Lymphology (ISL). Patients without apparent symptoms were graded as stage 0 because all patients who underwent axillary lymph node dissection were considered to have impaired lymph transport.

Stage 0 (or Ia): A latent or subclinical condition in which limb swelling is not yet evident.

Stage I: An early accumulation of fluid that subsides with limb elevation.

Stage II: Tissue swelling that is not reduced by limb elevation alone. Pitting is manifested in earlier stage II, but the limb may or may not pit in later stage II as excess fat and fibrosis supervene.

Stage III: Lymphostatic elephantiasis in which pitting can be absent, and trophic skin changes, such as acanthosis, further deposition of fat and fibrosis, and watery overgrowths, have developed.

SEG grade

Grade 0: No or little increase in echogenicity in the subcutaneous layer. Horizontal or obliquely oriented echogenic lines caused by connective tissue bundles are clearly observed.

Grade 1: Diffused and monotonous increases in echogenicity in the subcutaneous layer. Echogenic lines are unclear but identifiable.

Grade 2: Diffused increases in echogenicity. Echogenic lines are not identifiable.

SEFS grade

Grade 0: No SEFS.

Grade 1: Horizontally oriented (<45 degrees to the skin) SEFS only.

Grade 2: Presence of vertically oriented (≥45 degrees to the skin) SEFS bridging the horizontally oriented SEFS (Fig. 2).⁸

FIG. 2.

Definition of SEG and SEFS grade. Reprinted from Suehiro et al. and Copyright (2018), with permission from the Editorial Committee of the Annals of Vascular Diseases.⁹ SEFS, subcutaneous echo-free space; SEG, subcutaneous echogenicity grade.

Because echogenicity is evaluated subjectively and is easily changed by controlling the encoder of the B-mode gain, the gain was first adjusted so that the subepidermal echogenic band was clearly seen as black or that otherwise normal subcutaneous fat in parts of the body other than the legs was seen as black.^2,8

Procedure

Assessments were completed by two physiatrists (the first physiatrist had a 5-year experience, while the second physiatrist had a 15-year experience in musculoskeletal ultrasonography-EULAR Level 3). They were both blinded to the results of each other's measurements. The first physiatrist recorded demographic data and clinical lymphedema stage of the patient, and obtained ultrasound images (Esaote MyLab ultrasound machine with a 6–18 MHz linear array probe). The scanning site was chosen as medial forearm because Suehiro et al. found that the differences in SEG and SEFS grade between the affected and unaffected arms were most evident in medial forearm, and SEFS severity in medial forearm represents extracellular fluid status in the entire arm.^8,11 All images were obtained between 11:00 AM and 15:00 AM. The probe was held in axial position in medial forearm over the flexor carpi radialis muscle. The depth of image captured was set at 2 cm. Because echogenicity evaluation is subjective and is easily influenced by variations in B-mode gain, this was first adjusted to black by reference to normal subcutaneous fat from another part of the body.¹⁴

First, physiatrist 1 performed grading by looking at ultrasound images without the knowledge of the clinical stage of patients. Three days later, after the initial examination, the two physiatrists independently assessed the images.

Statistical analysis

The sample size estimation was performed using the GPower V.3.1.7 (University of Kiel, Kiel, Germany). Statistical analysis was performed using IBM SPSS for Windows version 20.0 software (IBM Corp., Armonk, NY). Spearman's rank correlation was used to test the relationship between ISL clinical stage and SEG or SEFS. To test differences in SEG and SEFS grade among ISL clinical lymphedema stages, the chi-square test was used.

Data were analyzed using the simple Kappa (K) and weighted Kappa (K_w), to determine the reliability of SEG and SEFS grade. Cohen's K is a reliability index used for nominal and categorical data. K is a chance-corrected measure of agreement in which all disagreements are given equal weight. In contrast, the K_w takes into account the degree of disagreement among raters. K_w differentially weighs discrepancies between pairs of scores so that the further apart the two scores, the more effect that observation has on lowering the reliability.¹²

Overall interpretations of the simple K and K_w statistic were based on the criteria described by Fleiss et al.¹³ The level of reliability was defined as follows: K or K_w values of <0.40 reflect poor agreement; values of 0.40–0.75 reflect fair to good agreement, and values of >0.75 reflect excellent agreement.¹³ The ISL clinical lymphedema stage was grouped as reversible stage (stages 0,1) and irreversible stage (stage 2,3).¹⁴

Results

Tables 2 and 3 show distribution of SEG and SEFS grade among ISL clinical lymphedema stages. SEG and SEFS grade differed statistically significant among ISL clinical lymphedema stages (p = 0.0001, p = 0.026; consecutively). Hundred percent of patients with ISL stage 0 was graded as SEG 0, 76.9% of patients with ISL stage 2 was graded as SEG 1 and 66.7% of patients with ISL stage 3 was graded as SEG 2. Seven of the patients with ISL stage 3 were graded as SEG 3. SEFS was graded as 0 in 100% of patients at stage 0, in 92.5% of patients at stage 1, in 51.9% of patients at stage 2, and in 16.7 of patients at stage 3. A positive statistically significant correlation was detected between ISL clinical lymphedema and stages SEG (r = 0.626, p = 0.0001) and SEFS grade (r = 0.528, p = 0.0001).

Table 2.

Distribution of Subcutaneous Echogenicity Grade Among International Society Of Lymphology Clinical Lymphedema Stages

	ISL clinical lymphedema stage				Total
	Stage 0	Stage 1	Stage 2	Stage 3	Total
SEG
0
Count	4	3	2	0	9
% within stage	100.0%	23.1%	7.4%	0.0%	18.0%
1
Count	0	10	23	2	35
% within stage	0.0%	76.9%	85.2%	33.3%	70.0%
2
Count	0	0	2	4	6
% within stage	0.0%	0.0%	7.4%	66.7%	12.0%
Total
Count	4	13	27	6	50
% within stage	100.0%	100.0%	100.0%	100.0%	100.0%

Assessments of physiatrist 2 were used for the analyses.

Table 3.

Distribution of Subcutaneous Echo-Free Space Grade Among International Society of Lymphology Clinical Lymphedema Stages

	ISL clinical lymphedema stage				Total
	Stage 0	Stage 1	Stage 2	Stage 3	Total
SEFS
0
Count	4	12	14	1	31
% within stage	100.0%	92.3%	51.9%	16.7%	62.0%
1
Count	0	1	9	3	13
% within stage	0.0%	7.7%	33.3%	50.0%	26.0%
2
Count	0	0	4	2	6
% within stage	0.0%	0.0%	14.8%	33.3%	12.0%
Total
Count	4	13	27	6	50
% within stage	100.0%	100.0%	100.0%	100.0%	100.0%

Assessments of physiatrist 2 were used for the analyses.

Inter-rater reliability for SEG and SEFS was excellent (K_SEG = 0.836, p = 0.0001; K_wSEG = 0.85, p = 0.0001; K_SEFS = 0.884, p = 0.0001; K_wSEFS = 0.92, p = 0.0001). K values of SEG and SEFS grading systems indicated excellent intrarater reliability (K_SEG = 0.806, p = 0.0001; K_wSEG = 0.83, p = 0.0001; K_SEFS = 0.798, p = 0.0001; K_wSEFS = 0.82, p = 0.0001) (Table 4). Intrarater agreement on SEG at reversible stage lymphedema (ISL clinical stage 0–1) was higher than intrarater agreement on SEG at irreversible stage lymphedema (ISL clinical stage 2–3). Inter-rater agreement on SEG was higher at irreversible stage (ISL clinical stage 2–3) than SEG at reversible stage lymphedema (ISL clinical stage 0–1). No statistics evaluating the agreement on SEFS at reversible stage can be computed because SEFS data obtained by physiatrist 1 were a constant. Inter-rater agreement on SEFS grade at irreversible stage lymphedema was higher than inter-rater agreement on SEFS grade at whole stages (Table 4).

Table 4.

Inter- and Intrarater Agreement of Subcutaneous Echogenicity Grade and Subcutaneous Echo-Free Space Grade

	Intrarater K (95% CI)	Intrarater K_w, (95% CI)	Inter-rater K (95% CI)	Inter-rater K_w, (95% CI)
SEG of all images	0.81 (0.608–0.953)	0.83 (0.68–0.97)	0.84 (0.638–0.961)	0.85 (0.71–0.993)
SEFS grade of all images	0.8 (0.62–0.95)	0.82 (0.69–0.94)	0.88 (0.735–1)	0.92 (0.82–1)
ISL clinical stage 0–1
SEG	0.88 (0.600–1)	0.88 (0.66–1)	0.76 (0.336–1)	0.75 (0.44–1)
SEFS	^a	^b	^a	^b
ISL clinical stage 2–3
SEG	0.74 (0.460–0.999)	0.76 (0.54–0.97)	0.86 (0.605–1)	0.87 (0.68–1)
SEFS	0.8 (0.587–0.952)	0.81 (0.65–0.95)	0.9 (0.758–1)	0.93 (0.83–1)

Cannot be computed because at least one of the variables is constant.

All ratings same.

Kappa (physiatrist 1 first assessment/second assessment) was used for intrarater analyses.

Kappa (physiatrist 1 second assessment/physiatrist 2 assessment) was used for inter-rater analyses.

CI, confidence interval; K, Kappa; K_w, weighted Kappa.

Discussion

Upper extremity lymphedema is one of the most commonly reported complications after breast cancer surgery, which leads to impairment in quality of life of patients with breast cancer.^15,16 The most common treatment is complex decongestive therapy, including skin care, compression garments, exercise therapy, and manual lymph drainage.¹⁷ Baseline lymphedema severity was found to be the most important predictive factor for complex decongestive therapy. Therefore, it is crucial that the methods used to assess lymphedema severity are reliable.

Although volume and circumference measurements are widely used for the assessment of lymphedema, it is not possible to distinguish if increase in extremity volume is related to fluid accumulation or pathological tissue proliferation. Ultrasound is one of the best imaging modalities for assessing the severity of lymphedema since it is readily available in most of the physical medicine and rehabilitation clinics and it is noninvasive and inexpensive. It is preferable to computed tomography and magnetic resonance imaging because it is radiation free and less time-consuming.^6,7 The present study investigated the inter- and intrareliability of the two ultrasonographic grading systems, reflecting the severity of lymphedema.

Clinical lymphedema stages and SEG and SEFS grade were correlated. SEG and SEFS grade differed among ISL clinical lymphedema stages. Similar to the findings of the present study, it has been reported that SEG increased significantly between stages 0 and 1 and stages 1 and 2 in patients with lower extremity lymphedema.⁷

All patients with ISL stage 0 were graded as SEG 0, and most of the patients with ISL stage 2 were graded as SEG 1. SEFS was graded as 0 in all patients at stage 0, in 92.5% of patients at stage 1, in 51.9% of patients at stage 2, and in 16.7 of patients at stage 3. Suehiro et al. detected increased SEFS in only half of the patients with breast cancer-related stage 2 arm lymphedema in line with the results of the present study, which found increased SEFS in 49.1% of the patients at stage 2 lymphedema.⁸ Supportingly, another study showed SEFS grade was correlated with bioimpedance analysis of arm and leg lymphedema.¹¹ In agreement with our findings, previous researches on relationship between subcutaneous tissue echogenicity and ISL stage in secondary leg lymphedema have found positive correlation between ISL stage and SEG^7,9; as well as ISL stage and SEFS.⁹

Intrarater and inter-rater reliability of SEG and SEFS was found to be excellent. Johnson et al. used measures of entropy and average pixel intensity to assess the tissue composition in lymphedema.¹⁸ They found that raters were not consistent when rating edema and fibrosis, while they were consistent when measuring limb volume differences and entropy and average pixel intensity. They found that entropy and average pixel intensity measurements were reliable (Cronbach's α = 0.7 for entropy and 0.91 for average pixel intensity). Although these ultrasonographic measurements can be used for research purposes, they are not practical for clinical assessment purposes. Yang et al. investigated the diagnostic accuracy of SEG. Volume measurements using water displacement volumetry and truncated cone method, and SEG, were recorded and arm volume change and ISL clinical lymphedema stages were determined as standards for diagnosis. They found the accuracy of SEG was higher than volume difference in forearm, showing that SEG is an objective measure of severity of lymphedema.¹⁴

The results of the present study showed that intrarater agreement on SEG at ISL clinical stages of 0–1 was higher than intrarater agreement on SEG at ISL clinical stages of 2–3. Inter-rater agreement on SEG was higher at ISL clinical stages of 2–3 than SEG at ISL clinical stages of 0–1. Inter-rater agreement on SEFS grade at irreversible stage lymphedema was higher than inter-rater agreement on SEFS grade at whole stages. The SEG and SEFS grade are both reliable. However, it should be kept in mind that raters agree less when rating SEG of patients with ISL clinical stage 2–3 and raters agree more when rating SEFS grade of patients with ISL clinical stage 2–3.

Strengths and limitations

Limitations of this study should be noted. We could not calculate K statistics for the reliability of SEFS when assessing patients at ISL clinical stages of 0–1 because physiatrist 1 did not rate any patient as SEFS grade 1 due to the small number of patients at stage 0–1. The prospective design with power calculation and being the first study to show the robustness of SEG and SEFS can be viewed as strengths of the study.

Conclusion

Based on the findings of this study, SEG and SEFS demonstrated acceptable reliability. The SEG and SEFS grading system with ultrasound can be useful to follow progression, composition, and management of breast cancer-related lymphedema.¹⁹ In conclusion, SEG and SEFS appear to be reliable and useful for the assessment of breast cancer-related lymphedema.

Footnotes

Acknowledgment

The authors thank all subjects who participated in this study.

Author Disclosure Statement

No competing financial interests exist.

References

Liao

S-F

, Li

S-H

, Huang

H-Y

, et al. The efficacy of complex decongestive physiotherapy (CDP) and predictive factors of lymphedema severity and response to CDP in breast cancer-related lymphedema (BCRL). Breast, 2013; 22:703–706.

Suehiro

, Morikage

, Yamashita

, et al. Differentiation of functional venous insufficiency and leg lymphedema complicated by functional venous insufficiency using subcutaneous tissue ultrasonography. J Vasc Surg Venous Lymphat Disord, 2017; 5:96–104.

Tassenoy

, Vermeiren

, van der Veen

, et al. Demonstration of tissue alterations by ultrasonography, magnetic resonance imaging and spectroscopy, and histology in breast cancer patients without lymphedema after axillary node dissection. Lymphology, 2006; 39:118–126.

Bok

S-K

, Jeon

, Hwang

P-S

. Ultrasonographic evaluation of the effects of progressive resistive exercise in breast cancer-related lymphedema. Lymphat Res Biol, 2016; 14:18–24.

Suehiro

, Morikage

, Murakami

, et al. A study of increase in leg volume during complex physical therapy for leg lymphedema using subcutaneous tissue ultrasonography. J Vasc Surg Venous Lymphat Disord, 2015; 3:295–302.

Özçakar

, Kara

, Chang

K-V

, et al. Nineteen reasons why physiatrists should do musculoskeletal ultrasound: EURO-MUSCULUS/USPRM recommendations. Am J Phys Med Rehabil, 2015; 94:e45–e49.

Suehiro

, Morikage

, Murakami

, Yamashita

, Samura

, Hamano

. Significance of ultrasound examination of skin and subcutaneous tissue in secondary lower extremity lymphedema. Ann Vasc Dis, 2013; 6:180–188.

Suehiro

, Morikage

, Yamashita

, et al. Skin and subcutaneous tissue ultrasonography features in breast cancer-related lymphedema. Ann Vasc Dis, 2016; 9:312–316.

Suehiro

, Morikage

, Murakami

, et al. Subcutaneous tissue ultrasonography in legs with dependent edema and secondary lymphedema. Ann Vasc Dis, 2014; 7:21–27.

10.

Dylke

, Benincasa Nakagawa

, Lin

, Clarke

, Kilbreath

. Reliability and diagnostic thresholds for ultrasound measurements of dermal thickness in breast lymphedema. Lymphat Res Biol, 2018; 16:258–262.

11.

Suehiro

, Morikage

, Ueda

, et al. Local echo-free space in a limb with lymphedema represents extracellular fluid in the entire limb. Lymphat Res Biol, 2018; 16:187–192.

12.

Florence

, Pandya

, King

, et al. Intrarater reliability of manual muscle test (Medical Research Council scale) grades in Duchenne's muscular dystrophy. Phys Ther, 1992; 72:115–122.

13.

Fleiss

, Levin

, Paik

. Statistical methods for rates and proportions. Hoboken, New Jersey: John Wiley & Sons; 2013.

14.

Yang

, Kim

, Lee

, Lim

J-Y

, Lee

. Diagnostic accuracy of clinical measures considering segmental tissue composition and volume changes of breast cancer-related lymphedema. Lymphat Res Biol, 2018; 16:368–376.

15.

Soran

, D'Angelo

, Begovic

, et al. Breast cancer-related lymphedema–what are the significant predictors and how they affect the severity of lymphedema?. Breast J, 2006; 12:536–543.

16.

Chen

Y-W

, Tsai

H-J

, Hung

H-C

, Tsauo

J-Y

. Reliability study of measurements for lymphedema in breast cancer patients. Am J Phys Med Rehabil, 2008; 87:33–38.

17.

Coriddi

, Khansa

, Stephens

, Miller

, Boehmler

, Tiwari

. Analysis of factors contributing to severity of breast cancer related lymphedema. Ann Plast Surg, 2015; 74:22.

18.

Johnson

, DeSarno

, Ashikaga

, Dee

, Henry

. Ultrasound and clinical measures phedema. Lymphat Res Biol, 2016; 14:8–17.

19.

Balzarini

, Milella

, Civelli

, Sigari

, De

. Ultrasonography of arm edema after axillary dissection for breast cancer: A preliminary study. Lymphology, 2001; 34:152–155.