Inter-observer and Intra-observer Variability in Volume Measurements of the Lower Extremity Using Perometer

Abstract

Background:

The swelling of the extremities seen in lymphedema can be measured with many different volumetric devices; however, many methods lack important characteristics including reproducibility and independence from the subjectivity and skill of the operator. The aim of this study was to validate the use of the Perometer^® as a possible standard for volumetric measurement methods based on the inter-observer and intra-observer variability when using a standard method of Perometry^®.

Methods and Results:

Volumetric measurements were performed on 10 healthy test subjects by 5 individuals (the observers) who had been instructed in the measurement techniques to be used. The inter-observer variability was assessed by having the five observers measure all the test subjects both in the morning and in the early afternoon. The intra-observer variability was examined by having each observer measure all the 10 test subjects 4 times in a row in the aforementioned time frames. A data set was created using the measurements, allowing for the assessment of other parameters including variation of volume between the right and left leg and daily variation in swelling. Statistical measurements were performed using the Statistical Package for the Social Sciences (SPSS), from which it was determined that there was no statistically significant inter-observer (p-value 0.997) and intra-observer variation (p-value 0.995) based on a significance level of >5%. Furthermore, it was observed that a statistically significant difference in volume occurred in the leg volume during the day.

Conclusion:

It was concluded that the use of the Perometer provides consistent measurements of volume independent of the observer and therefore appears to provide a candidate standard for volumetric measurements.

Introduction

Volumetric measurement is a tremendously important tool in the treatment of lymphedema of the lower extremities. Despite this, international standards for measurement have not been established or accepted, and yet it remains the primary outcome measure for most clinical research. Precision in determining the volume of the extremities is not only essential for diagnosis but also for an assessment of the outcome of a course of treatment in patients with lymphedema.¹

The ideal method of measurement according to Chromy is that it should be precise, reproducible, independent of the operator, simple, fast, and economically advantageous.²

It is assumed that the characteristics that define the best volumetric measuring method are the same for both the upper and the lower limbs. Therefore, the characteristics listed above are considered when assessing the optimum method of volumetric measurement for the lower extremities. Different measurement methods are described for this purpose: water displacement, Peracutus Aqua Meth (PAM), tape measurement method, digital laser scanning technique, and dual-energy X-ray absorptiometry (DXA scanning).

Currently, volume measurements are made by using the “water displacement” method, which involves a direct and an indirect component.¹ The advantages and disadvantages of the method are judged in correlation with previously mentioned characteristics of ideal measuring. The advantages of the “water displacement” method are the precision of the method, its reproducibility, as well as its operator independence. Furthermore, the method is believed to be economically cost-effective, and these are the reasons why the “water displacement” method is ranked as the gold standard.¹

However, this view is debatable as the method appears practically unusable for volume measurement in the lower extremity. This is because volume measurement of the lower extremity requires a relatively extensive setup. Additionally, the patient's motor skills, flexibility, vulnerability to cross infection, test timing, and the complex setup of the equipment required are just a few of the factors, which serve as examples of the disadvantages of the method.² Furthermore, it should be taken into consideration that the measurements are taken on patients affected by edema who often have poor mobility posing as a clear additional disadvantage. In relation to lymphedema in the arms, this method is also complicated in its usage as it can be difficult to perform and to determine the level of measurement to the axilla.

PAM is a newly developed system designed as a volumetric measurement device. Water poses as a core element in this measurement.

Studies show the results of the PAM are with intraclass correlation coefficients of 0.99 comparable with other volumetric measurement devices such as the tape measurement method and inverse water volumetry. PAM appears as a precise and reliable volumetric device; however, it is not interchangeable with water displacement, Bravometer. This is possibly due to the fact that it is the first prototype of the PAM, and a next-generation device is needed for further elaboration. Hence, it is not possible to view this prototype of PAM as an ideal volumetric measurement device.³

It is also worth addressing possible issues of using these devices in different health care settings and the limitations this gives.

Another well-known and usual method is the “tape measure method.”⁴ With this volumetric measurement method, a tape measure is used to measure the perimeter of the extremity based on certain specific markings. Many variations in tape measure methods exist including the wide variation in the width of circumference measures (H-4-10) taken throughout the limb; these differences may affect the overall volume measurements recorded.

The tape measure method was considered in comparison with the Perometer^®. An experiment showed that the difference between the tape measure method and the Perometer was 157 mL, with a safety value of 95%. However, this difference was deemed too high compared to clinical measurements. It was therefore concluded that these methods cannot be used interchangeably.⁴ Factors such as the position of the limb when measuring the limb circumference are altered when the muscle is contracted. Differences may also occur because of the tension applied to the tape measure. This is a particular problem when measuring very large limbs. Evidence would also suggest that there is a great variation from simple use of circumference measures through to the calculation of the volume using different methods (e.g., truncated cone). The last level of circumference measurements on a limb will also affect the volume calculated, a problem that also occurs when using the Perometer.

More recent studies have been exploring circumferential tape measurements in comparison with the Perometer. Sharkey et al. found that the measurements made by the Perometer on average had lower standard errors and narrower confidence intervals compared with tape measurements; hence, the Perometer was found to be more accurate. Furthermore, it was concluded that there is excellent intra-observer reliability in the use of both the tape and Perometer measurements. However, the Perometer is more precise and less time consuming than the tape measurement method.⁵

Batista et al. found that 95% of the time the volumes measured with either the Perometer or the tape measurement differ by up to 200 mL, hence concluding that comparison between volume measurements performed by usage of Perometry^® or tape measurement method should be carried out with caution.⁶

However, Sun et al. found no significant difference in total arm volume measurements between the tape measurement method and the Perometer.⁷

Furthermore, it has been concluded that circumferential measurements are operator dependent. Thus, the volume determination depends on the specific metric, as the values, among other things, vary depending on how much the tape measure is tightened.²

Consequently, it can be concluded that the tape measure is neither precise, reproducible, nor operator independent, hence the method is not ideal.

Digital laser scanning technology is another volumetric measurement device. Laser scanning is used to develop three-dimensional imaging of the specific limb. First, the limb is scanned using a laser scanner, second, volume calculations are made using developed software.

Comparison of limb volume measurements made by digital laser scanning technique and water displacement has been explored in studies in which it appears that water displacement underestimates volume compared with digital laser scanning techniques. Furthermore, digital laser scanning technique appears to be a precise, reproducible, operator independent, and commercially available volumetric device.⁸ It is though questionable whether this is accurate with large distorted limbs with large lobes.

However, image processing on the computer and volume calculations are time consuming, which poses as a clear disadvantage.⁹

DXA scans appear superior to water displacement and the tape measurement method, and furthermore, this volumetric measurement method is reproducible, operator independent, and clinically available.¹⁰

However, disadvantages include the radiation dose, contraindications prohibiting scanning such as pregnancy, and the fact that this method is very time consuming.¹¹

Purpose

The current and applied methods are assessed based on the criteria for an ideal measurement method. As these measurement methods are not currently sufficient, the Perometer is considered as a possible improvement. Since there are already studies concluding the validity of the Perometer, the aim of this study was to investigate how the Perometer meets the requirements for inter-observer and intra-observer variability, variability between volume measurement of the right and left legs, as well as the daily variation that occurs in limb volume. Based on the above, and the characteristics of the ideal measurement method, the Perometer is considered as a possible representative of the ideal measurement method that meets the listed characteristics.

Methods and Materials

Perometer

The Perometer utilized in the following experiment is a Perl-System Meßgeräte GmbH, type 1000M.

The Perometer consists of a square plastic frame connected to a computer. The overall purpose of this installation is to make and register the measurements. The frame of the Perometer moves vertically from the ankle to the upper border of the lower limb. Infrared light signals are located opposite from each other on the inside of the plastic frame, allowing the measurements to be taken horizontally. The light signals are stopped by the limb located central to the frame, and thus, they measure the distance from the medial side of the frame to the extremity.

Diameter measurements are made at intervals of 4.7 mm as the frame is moved manually along the longitudinal axis of the extremity. The height of the frame of all measurements must be standardized otherwise readings are misleading in clinical practice. Furthermore, it is important to state that clothing can significantly change readings, which is why all subjects only wear underpants during the volume measurement, this should also be the case while measuring volume in clinical practice

The precision, reproducibility, operator independence, as well as the time-related aspect of the method are examined through analysis of the method's inter-observer and intra-observer variability. At first glance, the financial costs of the Perometer may appear expensive and therefore act as a disadvantage of the method. However, this article will also consider this aspect.^3,12

Test subjects

Ten healthy Caucasian test subjects volunteered to take part in this study. Six men and four women participated with an age range of 36–67 years. Inclusion criteria were age >18 years and good health. The exclusion criteria were edema, irregular or lobulated limbs, and being physically unable to stand still. We found no clinical signs of edema in the healthy test subjects, which was confirmed by a negative pitting test performed at the ankle area. The definition and reliability of the pitting test have been previously published in other studies.¹³

Table 1 shows the relevant data concerning the 10 test subjects.

Table 1.

The Distribution of Age, Body Mass Index, Volumes in the Left and Right Leg Represented in the Morning and in the Early Afternoon

Age (years)	BMI (kg/m²)	Volume, left, morning (mL)	Volume, right, morning (mL)	Volume, left, afternoon (mL)	Volume, right, afternoon (mL)
53.30 (10.53)	23.95 (4.19)	7144.6 (1008.9)	7151.8 (983.6)	7165.0 (970.1)	7308.0 (986.3)

The table shows mean values, and the spread is shown in parentheses.

BMI, body mass index.

Setup and procedure

The 10 test subjects were measured both in the morning and in the early afternoon. During the measurement, the test subject placed one foot corresponding to the marked area on the plate within the frame of the Perometer. One measurement was completed while the test person was standing with equal weight distribution on both legs, ensuring positional equilibrium.

The inter-observer variability was assessed based on five different observers, who measured all the test subjects both in the morning and in the afternoon.

The intra-observer variability was examined by having the same observer measure all 10 test subjects 4 times in a row in the morning, and 4 times in a row in the early afternoon.

Random selection of who took the measurements was undertaken. Thus, it was ensured that the measurements made occurred in a random order.

Furthermore, from this procedure, it was possible to calculate the volume of the right leg in comparison to the volume of the left leg. Likewise, it was possible to estimate the difference between the volume measurements taken in the morning and in the afternoon.

Taking measurements in the morning may have a number of advantages in ensuring standardization occurs. Bed rest and elevation of the limb may eliminate mild edema that was formed through the effects of gravity throughout the day. However, exact time of volume measurement in healthy test subjects is considered of minor importance as long as measurements were standardized during the day.¹⁴

Statistical analysis

Statistical measurements were performed using the Statistical Package for the Social Sciences (SPSS).

Based on 10 test subjects, it was assumed that this was not a representative sample with a normal distribution volume. Thus, it is not assumed that the values are normally distributed.

Lack of normally distributed data was the background of using nonparametric analytical approach to the measured volumes in this study.

The analysis of the differences between observers, and the inter-observer variance, was performed by using the Kruskal–Wallis test. Similarly, the difference between measurements made by the same observer was also analyzed using the Kruskal–Wallis test.

Furthermore, a standard significance level of 5% was used to determine the p-values in this study.¹⁵

The variation between the right and left leg as well as the variation between the morning and afternoon measurements were considered as paired data. With this clarified, these values were analyzed using the “Wilcoxon signed-rank test,” from which an “Exact Sig. (2-tailed)” p-value was also determined.^l,16

Results

Inter-observer variation

Table 2 shows the average volume measured by each observer. Thus, the inter-observer variance was analyzed. A p-value of 0.997 indicated that there was no statistically significant difference between the five different independent observers.

Table 2.

The Average Volume of 40 Measurements Performed Per Measurement-Taker

Measurement-taker 1	Measurement-taker 2	Measurement-taker 3	Measurement-taker 4	Measurement-taker 5
7194.78 mL	7192.35 mL	7173.48 mL	7148.7 mL	7199.53 mL

All results are shown in Supplementary Data S1. The average volumes are calculated based on these values.

Intra-observer variation

Table 3 expresses the mean volume measured by the same person. Each observer performed volume measurements 4 times on each of the 10 test subjects, both in the morning and in the afternoon. Using these results, the intra-observer variance was calculated. The p-value was found to be 0.995. This p-value is also considered in relation to a level of significance of 5%. These results showed no statistically significant difference between the four different independent measurements made by the same observer.

Table 3.

The Average Volume Measured by the Same Person (Measuring Person 2)

Measurement-taker 2	Measurement-taker 2	Measurement-taker 2	Measurement-taker 2
7192.35 mL	7183.275 mL	7192.375 mL	7207.0 mL

Variation between the right and left leg

The variation between the volume measurement of the right and left leg is, as previously mentioned, analyzed by using the Wilcoxon signed-rank test. The p-value was determined to be 0.000. Compared with a level of significance of 5%, it is clear that a statistically significant difference in volume of the right and left leg was determined.

Daily variation

The variation between the morning and afternoon measurements was also calculated using the Wilcoxon signed-rank test, with a p-value of 4.49 × 10⁻¹⁴; compared with the level of significance, it is estimated once more that a clear statistical significance existed between the volume of the legs measured in the morning and in the afternoon.

Discussion

Currently, identification of the ideal measurement method, which can be used for the measurement and monitoring of edema in the legs, is urgently needed. As previously discussed, the ideal measurement method was defined by Chromy, and in his article, Chromy lists the characteristics required of the ideal measurement method. In the assessment of the most optimum volumetric method, the Perometer appears as an excellent candidate. It is also important to consider the advantages and disadvantages of the Perometer in the evaluation of the best volumetric measurement method.

Among the advantages, dexterity and safety are taken into account while considering the Perometer as the best volume measurement device. The volumetric measurement is carried out within a few seconds, thus patients with motor limitations can be supported in the short amount of time it takes to complete a measurement. Compared with the water displacement method, the Perometer appears much safer and is able to accommodate varying degrees of immobility compared with the other methods. On the other side, the tape measurement method appears as the safest method with fewest demands to the patients' mobility.

While addressing the short time frame required for a single measurement, it is obvious how one patient can be measured multiple times a day, and thus, it is easy to determine the daily variance occurring as well as other parameters. This is very complicated to fulfill using the water displacement method due to its complex setting. Additionally, it is complicated to measure volume multiple times a day using the digital laser scanning technique due to its time-consuming image processing. Volume measurements are mostly needed in this way when considering clinical trials. However, it is also recorded daily in many centers while undertaking a period of complex decongestive therapy to determine treatment progress and when a plateau of limb reduction has occurred.

The localization of the edema appears as another important feature. The computer program connected to the Perometer produces graphs, which illustrate the lower extremities, and thereby it is possible to determine the exact location of the edema. This localization is also possible to deduce using the tape measurement method, the digital laser scanning technique, and the DXA scan; however, the water displacement method lacks this ability. This is quite important as the segmental change can be different even if the overall volume of the limb remains the same.

Presenting these advantages, it is also relevant to evaluate the disadvantages of the Perometer.

As presented throughout this article, the Perometer fulfills the most essential requirements when assessing the optimum volumetric method. However, it is important to state that the foot is not included in the volume measurement due to the frame of the Perometer. Thus, edema in the foot is not taking into consideration when assessing volume measurement using the Perometer. This limitation of the Perometer is also apparent in volume measurements of upper extremity, where there is experience of Perometer inconsistency when measuring the volume of the hand.⁷

This is notable as the volume measurement of the foot is included in the tape measurement method, the water displacement method, PAM, digital laser scanning technique, and DXA scanning.

Furthermore, it is important to note the fact that the Perometer has a size limitation, and this occurs with extreme edema or those who are morbidly obese.

The same situation applies to the use of the water displacement method. It is also possible to experience inconsistency in volume measurement using the Perometer in those with irregular or lobulated limbs. Lobulated parts of the leg may occur not only in lymphedema patients but also in those who are morbidly obese who may not have lymphedema. Measurement of these limbs can be difficult with all the methods

The Perometer measures volume changes, which include fat and water; therefore, it is important to record and adjust for the body mass index (BMI) over time in volume assessment. Furthermore, fibrotic tissue may also influence measurements.

Another important aspect taking into consideration while evaluating the optimum volumetric method is the clinical issues that may cause fluctuating volume. These issues are important in assessing the volume measurements both currently and in the long term. Such issues could be change or cessation of diuretic therapy, reduced mobility status, inability to elevate the limbs during the day, and a failure to go to bed at night. Development or worsening of comorbidities including cancer recurrence or other causes of edema, for example, cardiac or renal will influence volume change. Longitudinal volume may fluctuate due to obesity, change in BMI, unilateral limb volume injury, or reduction of muscle mass following neuropathic changes.

It is of course important to take into consideration whether the controls were standing, walking, or sitting during the day likewise in the clinical practice.

The reasoning behind a nonparametric analytical approach to the data was the lack of equal gender distribution of the test subjects. However, when looking at the mean values (Table 1), it is possible to assume that this distribution is in fact equal. As previously mentioned, the test subjects are healthy volunteers, and therefore, the range in variation is probably less than it would have been, had the test been carried out on patients affected with edema.

Currently, there is a great need for measurements of lower limb volume. Not only for the importance of estimating the difference between the legs (postoperatively) but also in terms of assessing the effects of the treatment. In addition to this, there is a need for a method with which measurement errors are minimized as much as possible. The Perometer has been validated in previous studies and found to have a high degree of precision.¹ This is consistent with the results of this study, which showed that there is no significant difference between intra-observer and inter-observer variability, thereby confirming that the measurements are independent. However, Batista et al. stated that volume measurements made by the Perometer are at risk of error due to imperfect calibration.⁶ Unfortunately, there is lack of clarification on how much effect this have.

Furthermore, DeSnyder et al. explained how the usage of the Perometer is limited in clinical centers due to the large footprint required as well as the high upfront cost of the Perometer.¹⁷ However, there are portable devices now that take up a smaller footprint and can be transported into primary care.

While addressing these disadvantages, it is important to state that there is a critical need to ensure volume measurements are taken in a standardized method, and despite the large footprint required and the upfront cost, the Perometer can provide volume measurements with a focus on minimizing measurement variability.

When considering the significant difference that existed between the right and left leg, it makes sense to draw a parallel to how the dominant arm is also larger than the nondominant arm, deeming the right–left leg difference to be a natural occurrence.

By comparing each test persons leg dominance and volume measurement, it appeared how volume was larger in the dominant leg compared with the nondominant leg. This is also supported in studies, concluding that the dominated leg is associated with greater volume.¹⁸

Thus, it can be concluded that the natural right–left leg variations are important to take into account, especially in the case of fitting for compression stockings, this is relevant in both unilateral and bilateral situations.

The significant difference between the daily variance is also recognized in other studies.⁴ Therefore, it is important that the volume is measured on the exact same time of day, especially when considering the significant difference found between measurements taken in the morning and in the afternoon, respectively. However, the clinical reality will not allow for this. Change in BMI should be noted, although this probably occurs in a bilateral manner.

The financial costs of the methods are also relevant, considering an ideal measurement method as it should be economical advantageous. The most representative way of addressing the health economic advantages and disadvantages of the Perometer, and especially in relation to currently used volumetric measurement method, is through cost–benefit studies. However, this demands a much bigger setup. Nevertheless, cost-effective outcomes can be expected and are urgently required.

Conclusion

In this study, the Perometer has shown that inter-observer and intra-observer variability is close to zero. It is concluded that the Perometer is up to the criteria for an ideal measuring method, with the acknowledgement of the limitations discussed within this study.

This gives rise to a new consideration of the Perometer as the new gold standard for measurement and monitoring of lymphedema in the lower extremity. It is hereby a useful tool in monitoring the effect of treatment of lymphedema.

The significant variation in volume shown between the right and left leg, as well as variations in volume found at different times of the day, appears self-explanatory. These conclusions are especially important to consider in the context of lymphedema treatment in clinics.

Ethical Considerations

This study was reviewed and approved by the Committees on Health Research Ethics in the Capital Region of Denmark.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

Funding Information

No funding was received for this study.

Supplementary Material

References

Tierney

, Aslam

, Rennie

, Grace

. Infrared optoelectronic volumetry, the ideal way to measure limb volume. Eur J Vasc Endovasc Surg, 1996; 12:412–417.

Chromy

, Zalud

, Dobsak

, Suskevic

, Mrkvicova

. Limb volume measurements: Comparison of accuracy and decisive paramters of the most used present methods. Springerplus, 2015; 4:707.

Wolfs

JAGN

, Bijkerk

, Schols

, Keuter

XHA

, van der Hulst

RRWJ

, Qiu

. Evaluation of a novel water-based volumetric device for measuring upper limb lymphedema: First experience with healthy volunteers. Lymphat Res Biol, 2019; 17:434–439.

Tan

, Coutts

, Bulley

. Measurement of lower limb volume: Agreement between the vertically oriented Perometer and a tape measure method. Physiotherapy, 2013; 99:247–251.

Sharkey

, King

, Kuo

, Bickerton

, Ramsden

, Furniss

. Measuring limb volume: Accuracy and reliability of tape measurement versus Perometer measurement. Lymphat Res Biol, 2018; 16:182–186.

Batista

, Baiocchi

JMT

, Campanholi

, Bergmann

, Duprat

. Agreement between perometry and sequential arm circumference measurements in objective determination of arm volume. J Reconstr Microsurg, 2018; 34:29–34.

Sun

, Hall

, Tighe

, Brunelle

, Sayegh

, Gillespie

, Daniell

, Taghian

. Perometry versus simulated circumferential tape measurement for the detection of breast cancer-related lymphedema. Breast Cancer Res Treat, 2018; 172:83–91.

McKinnon

, Wong

, Temple

, Galbraith

, Ferry

, Clynch

. Measurement of limb volume: Laser scanning versus volume displacement. J Surg Oncol, 2017; 96:381–388.

Preuss

, Killaars

, Piatkowski de Grzymala

, Binnebösel

, Neumann

Validity and reliability of three-dimensional imaging for measuring breast cancer-related lymphedema in the upper limb: A cross-sectional study. Lymphat Res Biol, 2018 [Epub ahead of print]; DOI: 10.1089/lrb.2017.0076.

10.

Gjorup

, Zerahn

, Juul

, Hendel

, Christensen

, Hölmich

. Repeatability of volume and regional body composition measurements of the lower limb using dual-energy X-ray absorptiometry. J Clin Densitom, 2017; 20:82–96.

11.

Gjorup

, Zerahn

, Hendel

. Assessment of volume measurement of breast cancer-related lymphedema by three methods: Circumference measurement, water displacement, and dual energy X-ray absorptiometry. Lymphat Res Biol, 2010; 8:111–119.

12.

Pannier

, Rabe

. Optoelectric volume measurements to demonstrate volume changes in the lower extremities during orthostasis. Int Angiol, 2010; 29:395–400.

13.

De Vrieze

, Gebruers

, Nevelsteen

, De Groef

, Tjalma

WAA

, Thomis

, Dams

, Van der Gucht

, Penen

, Devoogdt

Reliability of the MoistureMeterD compact device and the pitting test to evaluate local tissue water in subjects with breast cancer-related lymphedema. Lymphat Res Biol, 2019 [Epub ahead of print]; DOI: 10.1089/lrb.2019.0013.

14.

Rabe

, Stücker

, Ottillinger

. Water displacement leg volumetry in clinical studies—A discussion of error sources. BMC Med Res Methodol, 2010; 10:5.

15.

Kruskal–Wallis H Test using SPSS Statistics (Lærd Statistics). https://statistics.laerd.com/spss-tutorials/kruskal-wallis-h-test-using-spss-statistics.php (accessed April 5, 2019).

16.

Wilcoxon Signed-Rank Test using SPSS Statistics (Lærd Statistics). https://statistics.laerd.com/spss-tutorials/wilcoxon-signed-rank-test-using-spss-statistics.php (accessed April 5, 2019).

17.

DeSnyder

, Kheirkhah

, Travis

, Lilly

, Bedrosian

, Buchholz

, Schaverien

, Shaitelman

. Optimizing patient positioning to reduce variation in the measurement of breast cancer-related lymphedema. Lymphat Res Biol, 2019; 17:440–446.

18.

Teo

, Thompson

, Neo

, Lundie

, Munnoch

. Lower limb dominance and volume in healthy individuals. Lymphology, 2017; 50:197–202.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB