On the electronic measurement of pain intensity: Can we use different pain intensity scales interchangeably?

Abstract

The objective of this work was to study the agreement between four pain intensity scales when administered electronically: the Numerical Rating Scale-11, the Faces Pain Scale-Revised, the Visual Analogue Scale and the Coloured Analogue Scale. In all, 180 schoolchildren between 12 and 19 years old participated in the study. They had to report the maximum intensity of their most frequent pain using the electronic versions of the four scales. Agreement was calculated using the Bland–Altman method. Results show that the electronic versions of Numerical Rating Scale-11, Coloured Analogue Scale and Visual Analogue Scale can be used interchangeably.

Keywords

agreement assessment mHealth pain intensity young people

Introduction

Pain is a common problem in young people (Huguet and Miró, 2008b). For example, a recent systematic review reported a median prevalence of chronic pain between 11 and 38 per cent (King et al., 2011). Pain can have important detrimental effects on the life of young people and their families (Miró et al., 2007a). It has been shown to interfere with general quality of life (Huguet and Miró, 2008a), and psychological (Nieto et al., 2012) and physical functioning (Huguet et al., 2009). In order to best treat pain in young people, the instruments used must be psychometrically sound and validated. Self-report questionnaires are the most frequent way of collecting information about pain (Miró et al., 2015), and some of them have been found to provide valid and reliable reports when used with young people (McGrath et al., 2008).

Information and communication technologies (ICTs) are increasingly being used to help assess and treat people with pain (De la Vega and Miró, 2014; Fisher et al., 2015; Miró et al., 2007b; Nieto et al., 2008; Stinson and Jibb, 2013). Electronic devices, for example, are more accessible than paper-and-pencil questionnaires, and this facilitates collecting data in real time, wherever the subject is, thus increasing the accuracy of these data (Morren et al., 2009; Palermo et al., 2004; Stinson, 2009). Furthermore, the use of ICTs in the management of chronic conditions has been found to be appealing, which helps patients comply with treatment recommendations (Nieto et al., 2008). These are important issues for clinicians and researchers since the compliance with and accuracy of assessment are among the most important problems encountered when working with individuals with chronic health problems.

Four of the best-known and most widely used scales to measure pain intensity in young people are as follows: the 0–10 Numerical Rating Scale (NRS-11), the Faces Pain Scale-Revised (FPS-R), the Visual Analogue Scale (VAS) and the Coloured Analogue Scale (CAS); these are already available for use with electronic devices. Moreover, pain intensity scores provided with the electronic versions of these scales have proved to be valid. For example, in a recent study, Sánchez-Rodríguez et al. (2015) reported moderate-to-high convergent validity, and adequate discriminant and concurrent validity for these four scales. They also provided evidence on the agreement between the scores reported with the traditional (paper-and-pencil format) and the electronic versions of these pain intensity scales. These data suggest that clinicians and researchers could use both versions of the same scale (traditional and electronic) interchangeably, that is to say, if a traditional form of one of these scales is used during a face-to-face interview, for example, its scores can be compared with those obtained with the electronic version of the same scale that is used, for example, during a telephone follow-up. It is unclear, however, whether the scores on the electronic versions of these widely used scales are concordant. Thus, the objective of this study was to respond to a question that still needs to be solved: can clinicians and researchers use the electronic versions of the pain intensity scales interchangeably? That is, are reports obtained, for example, with the electronic version of the NRS-11 comparable to those obtained with the electronic version of the FPS-R? Although no studies are available about the agreement between the electronic versions of different pain intensity scales, on the basis of previous work with traditional versions (paper-and-pencil format) of pain intensity scales (Sánchez-Rodríguez et al., 2012), we hypothesized that there would not be complete agreement among electronic versions of pain intensity scales. That is to say, electronic pain intensity reports are not concordant, so the different pain intensity scales cannot be used interchangeably when administered electronically.

Method

Participants

We invited all the schoolchildren enrolled in grades 7–12 (12–19 years) from two public schools to take part (N = 280). Exclusion criteria were (1) having a cognitive impairment, (2) not being able to understand Catalan or Spanish, (3) not having pain in the previous 3 months and (4) not signing the informed consent.

Instruments

Pain intensity scores were reported with the electronic versions of the following scales: the 11-point NRS-11, the FPS-R, the VAS and the CAS.

Electronic version of the 11-point NRS

The traditional NRS-11 is widely used with children and adolescents. Users are asked to report the intensity of their pain by providing a number from 0 (‘no pain’) to 10 (‘very much pain’). In this study, participants had to select the number that best represented their pain intensity by sliding their finger across the screen so that all potential responses (ciphers from 0 (‘no pain’) to 10 (‘very much pain’)) appeared one by one in the centre of the screen, or by tapping on the selected number (response) at the top of the screen where the numbers are permanently on display.

The electronic version of the FPS-R

The traditional version of this scale shows six faces that demonstrate an increase in pain intensity from left to right (Hicks et al., 2001). The eFPS-R shows the six faces at the top of the screen, although the face that is selected is shown in the centre of the screen. In this study, participants had to select the face that best represented their maximum pain intensity by sliding their finger across the screen or by tapping on the chosen face at the top of the screen. The six faces in the scale show increasingly more pain from left to right. The leftmost face reflects ‘no pain’, whereas the rightmost reflects ‘very much pain’.

Electronic version of the VAS

The traditional form of the VAS consists of a 100-mm horizontal line with two limits representing ‘no pain’ (lower limit) and ‘very much pain’ (upper limit). Participants are asked to select the point on the line that best represents their pain intensity. The eVAS consists of a straight line with a marker that participants have to move along the line to the point representing the maximum intensity of their pain. Its lower limit represents ‘no pain’ and its upper limit represents ‘very much pain’. The pain intensity score was the distance between the lower limit and the point selected by the marker.

Electronic version of the CAS

This scale consists of a long triangular shape colour graduated from white (lower limit) to dark red (upper limit) (McGrath et al., 1996). The lower anchor means ‘no pain’ and the upper anchor means ‘most pain’. However, in this study, we changed the upper limit to ‘very much pain’ to avoid potential anchor effects and to make all the scales comparable (Castarlenas et al., 2013, 2015; Sánchez-Rodríguez et al., 2012, 2015). The procedure with the eCAS was the same as the one used and presented for the eVAS: participants had to move a marker along the triangular shape to the point that best represented the maximum pain they had experienced.

Procedure

This study was approved by Department of Education of the Catalan Government and by the principals of the participating schools. Participants were recruited from two public schools in Tarragona (Catalonia, Spain).

First of all, we sent a letter to the parents of all the children enrolled in grades 7–12 explaining the study and asking for their informed consent. When parents had signed and returned the informed consent, two experienced researchers conducted individual interviews with those children. Interviews took place during school hours. Participants were asked to remember the most frequent pain they have had in the last 3 months and to report the maximum intensity of that pain using the electronic version of the four pain intensity scales. The scales were administered using a mobile app through an iPod Touch 4^© which was the property of the research team. The iPod screen was 3.5 in diagonally (88.9 mm). The mobile application, named Painometer, includes four pain intensity scales that are widely used with adults, children or elderly patients. The application and its manual of use have been described in detail by De la Vega et al. (2014) and at the following webpage: http://algos-dpsico.urv.cat/wp-content/uploads/2013/02/Painometer-v2.-User-manual.pdf. Screenshots showing how the four pain intensity scales are presented in Painometer are available in Appendix 1.

Analyses

Analyses were conducted using MedCalc version 12.4.0 for Windows. In order to determine whether reports provided with the four electronic versions of the scales are in agreement, we used the Bland–Altman method (Bland and Altman, 1986). This method has been successfully used to determine agreement between scales in previous studies (Bailey et al., 2007; Castarlenas et al., 2015; Sánchez-Rodríguez et al., 2012, 2015). The Bland–Altman method uses a graph to represent the difference between the scores reported by each subject with two different scales or methods and compares this difference with the average of these two scores. We conducted analyses to determine the agreement between the electronic version of the 11-point NRS (eNRS-11)–electronic version of FPS-R (eFPS-R), eNRS-11–electronic version of VAS (eVAS), eNRS-11–electronic version of CAS (eCAS), eFPS-R–eVAS, eFPS-R–eCAS and eVAS–eCAS.

First, we converted all the data into a scale from 0 to 100. Then, we set the limit of agreement at ±20 (minimal clinically significant difference of CAS in children (Bulloch and Tenenbein, 2002). We decided to use this limit instead of the others because it is the most liberal one (for NRS-11 it was ±1 (Bailey et al., 2010; Voepel-Lewis et al., 2011); for FPS-R, it was ±2 (Tomlinson et al., 2010; Von Baeyer, 2006); and for VAS, it was −16 to +18 (Bailey et al., 2012)). We calculated agreement for two different confidence intervals (CIs): 80 per cent CI and 95 per cent CI (Von Baeyer, 2012).

Results

Sample characteristics

A total of 203 schoolchildren (72.5%) signed the informed consent in which their parents agreed to their participating in the study. Participants over 18 years old gave their own consent. Of the total sample, 23 were excluded because they did not report having pain in the previous 3 months. The final sample was made up of 180 schoolchildren with 76 boys (42%) and 104 girls (58%). The average age was 14.88 years (standard deviation (SD) = 1.63) and 63 participants (35%) reported having chronic pain, defined as pain in any area of their body lasting for at least 3 months and present at least once a month. A paper using data from this group of participants has already been accepted for publication and is now in press, but it addressed issues that are not discussed here (see Sánchez-Rodríguez et al., 2015). Table 1 summarizes descriptive information about this sample of participants.

Table 1.

Participants’ descriptive information.

Participants (N)	180
Mean age (SD)	14.88 (1.64)
Gender N (%)
Boys	76 (42%)
Girls	104 (58%)
Pain status N (%)
Acute pain	11 (65%)
Chronic pain^a	63 (35%)
With pain at the time of the interview	72 (40%)
Schooling grade N (%)
Seventh	34 (19%)
Eighth	22 (12%)
Ninth	29 (16%)
Tenth	61 (34%)
Eleventh	16 (9%)
Twelfth	18 (10%)
Localization of the most frequent pain N (%)^b
Head, face and mouth	43 (24%)
Cervical region	5 (3%)
Upper shoulder and upper limbs	17 (10%)
Thoracic region	2 (1%)
Abdominal region	22 (12%)
Lower back, lumbar spine, sacrum and coccyx	22 (12%)
Lower limbs	57 (31%)
Anal, perineal and genital region	2 (1%)

SD: standard deviation.

Defined as a pain in any area of their body (1) lasting for at least 3 months and (2) present at least once a month.

Information missing in 10 cases (5%).

Agreement

The data about agreement are shown in Table 2 and graphs can be found in Appendix 2. The agreement between the electronic versions of each pain intensity scale with each other can only be assumed for the eNRS-11 and eVAS (at 80% CI), the eNRS-11 and eCAS, and the eVAS and eCAS (for both 80% and 95% CI).

Table 2.

Limits of agreement according to the Bland–Altman method.

Scale	Mean difference	Limits of agreement (maximum limit of agreement = ±20)
		95% CI		80% CI
		Lower	Upper	Lower	Upper
eNRS-11–eFPS-R	11.7	−19.2	42.7	−8.51	32
eNRS-11–eVAS	5.4	−13.1	24	−6.7	17.6
eNRS-11–eCAS	0.6	−17.9	19.1	−11.5	12.7
eFPS-R–eVAS	−6.3	−36.3	23.7	−25.9	13.3
eFPS-R–eCAS	−11.1	−41.4	19.2	−31	8.7
eVAS–eCAS	−4.8	−18.2	8.5	−13.5	3.9

eNRS-11: electronic Numerical Rating Scale-11; eFPS-R: electronic Faces Pain Scale-Revised; eVAS: electronic Visual Analogue Scale; eCAS: electronic Coloured Analogue Scale; CI: confidence interval.

Discussion

The main objective of this study was to determine the concordance between the reports provided by four widely used pain intensity scales when administered electronically to young people. To the best of our knowledge, this is the first study to look at this specific issue. Previous studies of a similar nature have focused on the agreement between pain intensity scales when administered in their traditional versions (i.e. paper-and-pencil format) (Bailey et al., 2007; Sánchez-Rodríguez et al., 2012), or on the comparison and agreement between traditional versus electronic versions of the same scale (e.g. Castarlenas et al., 2015; Sánchez-Rodríguez et al., 2015; Wood et al., 2011).

For example, Bailey et al. (2007) studied the agreement of pain intensity reports from four pain intensity scales in their paper-and-pencil format: the VAS, the CAS, the Wong Baker FACES pain rating scale and the NRS-11. This study showed that these scales provide equivalent reports between the VAS and the CAS, whereas the reports obtained with the Wong Baker FACES pain rating scale and the NRS-11 did not agree with the scores reported by other scales. Similarly, in a previous study, we found there is no agreement between the reports provided by the NRS-11, the FPS-R, the CAS and the VAS when used in their traditional, paper-and-pencil format, and within a 95 per cent CI (Sánchez-Rodríguez et al., 2012).

On the basis of these previous studies with traditionally administered pain intensity scales (Bailey et al., 2007; Sánchez-Rodríguez et al., 2012), we anticipated non-agreement between the scores reported with the electronic versions of the scales. However, this hypothesis was only partially confirmed. That is to say, some comparisons between scales yielded comparable scores, namely, the eNRS-11 reports were comparable to those from eVAS (at 80% CI) and eCAS (at 80% and 95% CI). Similarly, eVAS reports were also comparable to those from eCAS (at 80% and 95% CI). One explanation for these results may be that by making all scales manipulable, they behaved in a similar way, so their reports were comparable. However, this is just a tentative hypothesis that future studies ought to examine.

Some limitations should be taken into account when interpreting the findings of this study. First, our sample is a convenience sample made up mainly by healthy participants. Thus, we do not know how representative it is. Although we can reasonably assume that the electronic versions of the pain intensity scales behave equally regardless of health condition, further studies with other samples are required to evaluate the reliability and generalizability of our findings. Second, we did not counterbalance the presentation order of the scales (i.e. in all cases the eNRS-11 scale was presented first, eCAS second, eFPS-R third and eVAS fourth). Nevertheless, previous studies have found that the order of presentation of the scales had no influence on the scores reported (Miró et al., 2009).

Despite these limitations, the findings of this study give important information about the agreement between the reports provided by different electronic versions of four pain intensity scales. More specifically, we found that reports provided by the eNRS-11 are in agreement with those provided by the eCAS and the eVAS, and that reports provided by the eVAS also agree with those provided by the eCAS.

These data have important implications. The eNRS-11 could be used interchangeably with the eCAS and the eVAS, and the eCAS could also be used interchangeably with the eVAS since the scores provided by one of them are concordant with those provided by the other. This means that researchers could replicate a study and compare data using a different scale from the original study because the results should be comparable.

As mentioned above, this study is the first to have provided data about the agreement between reports produced by different electronic versions of pain intensity scales. The results of this study contradict those of other studies conducted with the same scales but in their traditional paper-and-pencil versions (Bailey et al., 2007; Sánchez-Rodríguez et al., 2012). Therefore, additional research is needed to validate these results and, above all, to explain why this might be so.

Conclusion

Although additional research is needed to elucidate the characteristics of pain intensity scales in their electronic form, the results of this study support their use to report pain intensity in young people. Previous studies have shown that electronic pain intensity reports obtained with the NRS-11, the FPS-R, the CAS and the VAS using Painometer are valid (Sánchez-Rodríguez et al., 2015), and that there is agreement between reports provided with traditional and electronic versions of each scale (Castarlenas et al., 2015; Sánchez-Rodríguez et al., 2015). The present findings reveal that pain reports provided with some of the electronic versions of the scales are also concordant with others. As a whole, these data provide a good deal of evidence to support the use of electronic versions of these four pain intensity scales with young people.

Footnotes

Appendix

Acknowledgements

The authors would like to thank the participating schools (Escola Puigcerver and Col·legi Cardenal Vidal I Barraquer) for their interest and collaboration in the study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was partly funded by grants from Obra Social de Caixabank, Universitat Rovira i Virgili (PFR program), RecerCaixa, the Spanish Ministry of Innovation (MINECO; PSI2012-32471) and the Fundació La Marató de TV3. J.M.’s work is supported by the Institució Catalana de Recerca i Estudis Avançats (ICREA-Acadèmia) and Fundación Grünenthal.

References

Bailey

Bergeron

Gravel

. (2007) Comparison of four pain scales in children with acute abdominal pain in a pediatric emergency department. Annals of Emergency Medicine 50(4): 379–383, 383.e1–2.

Bailey

Daoust

Doyon-Trottier

. (2010) Validation and properties of the verbal numeric scale in children with acute pain. Pain 149(2): 216–221.

Bailey

Gravel

Daoust

(2012) Reliability of the visual analog scale in children with acute pain in the emergency department. Pain 153(4): 839–842.

Bland

Altman

(1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1: 307–310.

Bulloch

Tenenbein

(2002) Assessment of clinically significant changes in acute pain in children. Academic Emergency Medicine 9(3): 199–202.

Castarlenas

Miró

Sánchez-Rodríguez

(2013) Is the verbal numerical rating scale a valid tool for assessing pain intensity in children below 8 years of age? Journal of Pain 14(3): 297–304.

Castarlenas

Sánchez-Rodríguez

de la Vega

. (2015) Agreement between verbal and electronic versions of the numerical rating scale (NRS-11) when used to assess pain intensity in adolescents. Clinical Journal of Pain 31(3): 229–234.

De la Vega

Miró

(2014) mHealth: A strategic field without a solid scientific soul. A systematic review of pain-related apps. PLoS ONE 9(7): e101312.

De la Vega

Roset

Castarlenas

. (2014) Development and testing of painometer: A smartphone app to assess pain intensity. Journal of Pain 15(10): 1001–1007.

10.

Fisher

Law

Palermo

. (2015) Psychological therapies (remotely delivered) for the management of chronic and recurrent pain in children and adolescents. Cochrane Database of Systematic Reviews 3: CD011118.

11.

Hicks

von Baeyer

Spafford

. (2001) The Faces Pain Scale-Revised: Toward a common metric in pediatric pain measurement. Pain 93(2): 173–183.

12.

Huguet

Miró

(2008a) Development and psychometric evaluation of a Catalan self- and interviewer-administered version of the Pediatric Quality of Life Inventory^™ version 4.0. Journal of Pediatric Psychology 33(1): 63–79.

13.

Huguet

Miró

(2008b) The severity of chronic pediatric pain: An epidemiological study. Journal of Pain 9(3): 226–236.

14.

Huguet

Eccleston

Miró

. (2009) Young people making sense of pain: Cognitive appraisal, function, and pain in 8–16 year old children. European Journal of Pain 13(7): 751–759.

15.

King

Chambers

Huguet

. (2011) The epidemiology of chronic pain in children and adolescents revisited: A systematic review. Pain 152(12): 2729–2738.

16.

McGrath

Seifert

Speechley

. (1996) A new analogue scale for assessing children’s pain: An initial validation study. Pain 64: 435–443.

17.

McGrath

Walco

Turk

. (2008) Core outcome domains and measures for pediatric acute and chronic/recurrent pain clinical trials: PedIMMPACT recommendations. Journal of Pain 9(9): 771–783.

18.

Miró

Castarlenas

Huguet

(2009) Evidence for the use of a numerical rating scale to assess the intensity of pediatric pain. European Journal of Pain 13(10): 1089–1095.

19.

Miró

Castarlenas

de la Vega

. (2015) Validity of three rating scales for measuring pain intensity in youths with physical disabilities. European Journal of Pain 20: 130–137.

20.

Miró

Huguet

Nieto

(2007a) Predictive factors of chronic pediatric pain and disability: A Delphi poll. Journal of Pain 8(10): 774–792.

21.

Miró

Nieto

Huguet

(2007b) Realidad virtual y manejo del dolor Virtual reality and control of pain. Cuadernos de Medicina Psicosomática y Psiquiatría de Enlace 82: 52–64.

22.

Morren

van Dulmen

Ouwerkerk

. (2009) Compliance with momentary pain measurement using electronic diaries: A systematic review. European Journal of Pain 13: 354–365.

23.

Nieto

Miró

Huguet

(2008) Nuevas tecnologías de la información y la comunicación en el tratamiento del dolor crónico. Revista Española de Anestesiología y Reanimación 55: 426–433.

24.

Nieto

Raichle

Jensen

. (2012) Changes in pain-related beliefs, coping, and catastrophizing predict changes in pain intensity, pain interference, and psychological functioning in individuals with myotonic muscular dystrophy and facioscapulohumeral dystrophy. Clinical Journal of Pain 28(1): 47–54.

25.

Palermo

Valenzuela

Stork

(2004) A randomized trial of electronic versus paper pain diaries in children: Impact on compliance, accuracy, and acceptability. Pain 107(3): 213–219.

26.

Sánchez-Rodríguez

de la Vega

Castarlenas

. (2015) An app for the assessment of pain intensity: Validity properties and agreement of pain reports when used with young people. Pain Medicine 16(10): 1982–1992.

27.

Sánchez-Rodríguez

Miró

Castarlenas

(2012) A comparison of four self-report scales of pain intensity in 6- to 8-year-old children. Pain 153(8): 1715–1719.

28.

Stinson

(2009) Improving the assessment of pediatric chronic pain: Harnessing the potential of electronic diaries. Pain Research & Management 14(1): 59–64.

29.

Stinson

Jibb

(2013) New information and communication technologies for pain. In: McGrath

Stevens

Walker

. (eds) Oxford Textbook of Pediatric Pain. Oxford: University Press, pp. 569–577.

30.

Tomlinson

von Baeyer

Stinson

. (2010) A systematic review of faces scales for the self-report of pain intensity in children. Pediatrics 126(5): e1168–e1198.

31.

Voepel-Lewis

Burke

Jeffreys

. (2011) Do 0–10 numeric rating scores translate into clinically meaningful pain measures for children? Anesthesia and Analgesia 112(2): 415–421.

32.

Von Baeyer

(2006) Children’s self-reports of pain intensity: Scale selection, limitations and interpretation. Pain Research & Management 11(3): 157–162.

33.

Von Baeyer

(2012) Reported lack of agreement between self-report pain scores in children may be due to a too strict criterion for agreement. Pain 153: 2152–2154.

34.

Wood

von Baeyer

Falinower

. (2011) Electronic and paper versions of a faces pain intensity scale: Concordance and preference in hospitalized children. BMC Pediatrics 11: 87.