Abstract
Although chronic pain is relatively common in childhood, many teachers feel ill-prepared to work with students with chronic and recurrent pain in the classroom and would like to learn more about supporting these students. A web-based eHealth intervention designed to provide information about pain and pain management in the classroom was developed based on input from clinicians, and usability was tested using three groups of stakeholders (i.e., youth with chronic pain, parents of youth with chronic pain, and teachers). Preliminary testing indicated that the usability goals were met, with the majority of participants in all groups indicating that the website was easy to use and that they would either recommend it to teachers or use it themselves. Minor design and content changes were suggested and made, whereas major changes will be made in the future. Results provide encouraging preliminary support for the utility of eHealth interventions as knowledge translation and dissemination tools for educators.
Despite variability in specific definitions, the International Association for the Study of Pain usually defines chronic pain as pain that persists beyond 3 months (Merskey, Bogduk, & International Association for the Study of Pain, 1994). Chronic pain can be either persistent or recurrent and presents a significant health concern for many children (Huguet & Miro, 2008; King et al., 2011; Perquin et al., 2000; Stanford, Chambers, Biesanz, & Chen, 2008). Prevalence rates for pediatric chronic pain vary widely, but recent epidemiological findings estimate that approximately 20% of youth below the age of 18 experience chronic or recurrent pain at some point, with approximately 5% of these youth experiencing disabling pain (Huguet & Miro, 2008; King et al., 2011; Perquin et al., 2000). Youth with chronic pain report higher levels of internalizing symptoms (Rhee, 2000; Stanford et al., 2008), difficulties with peer and social functioning (Carter, Lambrenos, & Thursfield, 2002; Forgeron et al., 2010; Forgeron et al., 2011), and difficulties with family functioning (Chambers, 2003; Lewandowski, Palermo, Stinson, Handley, & Chambers, 2010).
Another area that is often affected by chronic pain is school and academic functioning (Dick & Pillai Riddell, 2010; Logan, Simons, & Kaczynski, 2009; Logan, Simons, Stein, & Chastain, 2008), with children and parents reporting a decline in grades (e.g., Logan et al., 2008) and difficulties across several areas of functioning at school, such as decreased attendance, social difficulties, and decreased ability to participate in extracurricular activities (e.g., Chalkiadis, 2001; Dick & Pillai Riddell, 2010; Forgeron et al., 2010; Forgeron et al., 2011; Sjolie, 2004). School is not just important with respect to meeting academic goals, but it is an important site for development of other skills, such as problem solving and managing peer and social relationships (Dick & Pillai Riddell, 2010; Forgeron et al., 2010; Greenberg et al., 2003). Given the importance of the school environment for development, it is important to appropriately address health concerns, such as pain, that may interfere with students meeting academic and developmental outcomes.
Teachers report a lack of training in and knowledge of pediatric chronic pain and how to support students with pain management in the classroom (Forsythe, 2010; Logan, Coakley, & Scharff, 2007; MacNevin, 2011) and can have preconceptions about pediatric chronic pain. For example, teachers have been found to base their opinions of students with pain on their own past experiences with pain, to believe that the environment influences whether a student reports pain (e.g., whether or not there is an “audience”), and may have difficulty understanding the waxing and waning of pain symptoms (Logan et al., 2007; Logan & Curran, 2005). As teacher expectations and perceptions can significantly affect student success (e.g., Good & Nichols, 2001; Jussim & Eccles, 1992; Saracho, 1991; Trouilloud, Sarrrazin, Martinek, & Guillet, 2002), it is important to provide teachers with accurate information about pediatric chronic pain to ensure that students with chronic pain can experience school success.
One way of providing teachers with evidence-based information about pediatric chronic pain is to have clinicians deliver information directly; however, it is difficult to do this in a convenient and consistent manner, and many children who experience chronic pain are not followed by specialized clinicians. In addition, it can be difficult to organize interprofessional case conferences that include educators and health care providers, meaning that communication across settings can be challenging. Using a broad educational dissemination strategy to reach teachers of students who either have a diagnosed pain condition or who complain of frequent pain may be an efficient way of translating clinical knowledge and helping teachers adapt their classroom environments to meet the needs of these students, thereby preventing pain-related negative outcomes.
eHealth interventions are emerging as a cost-effective and efficient way to provide clinical services to patients, families, and other professionals in many settings, and to bridge the gap between health care settings and the individual’s everyday environment (see Cushing & Steele, 2010). eHealth interventions are defined as “applications of technology that seek to either improve a client’s understanding of health information or use technology as a surrogate for the clinician in treatment delivery” (Cushing & Steele, 2010, p. 937). There is emerging evidence that eHealth interventions can be useful in increasing understanding of pain among youth and their parents (Cushing & Steele, 2010; Stinson, Wilson, Gill, Yamada, & Holt, 2009), as well as acting as a surrogate for the clinician among youth with chronic pain (e.g., Long & Palermo, 2009; Stinson et al., 2006). eHealth interventions are typically directed to the child or the family and are not often disseminated to educators. This is unfortunate because, if designed to meet the unique needs of educators, these interventions could be useful in providing information about pain management in the classroom.
It is important to engage stakeholders when developing eHealth interventions to ensure appropriate content and to encourage knowledge translation and dissemination of the final product (see Noordegraaf et al., 2012; van Gemert-Pijnen et al., 2011). When developing an intervention for educators, input should ideally be solicited from patients, parents, pain clinicians, and educators themselves, as each group offers a unique perspective on content of the intervention and can provide feedback about the most effective way to disseminate information. The purpose of this article is to describe the development and initial usability testing of an eHealth intervention, the Teaching Educators About Chronic Pain Project (TEACh-Pain), designed to increase educator understanding of pediatric chronic pain in the school setting.
Method
Phase I: Website Design and Content Development
The website was developed in collaboration with a local company who recommended a basic multimedia template for the website. Content for the TEACh-Pain website was developed using previous research in the area of pediatric chronic pain and school functioning (e.g., Forsythe, 2010; Logan et al., 2007; Logan & Curran, 2005; MacNevin, 2011), current literature describing best practice in cognitive-behavioural interventions for pediatric chronic pain (e.g., King, Boutilier, & Chambers, 2014; Noel, Petter, Parker, & Chambers, 2012), and materials currently being used by pain clinicians at a tertiary care health centre to communicate with schools (i.e., letters, pamphlets). The website consisted of six sections: (a) Home (i.e., an introduction to the purpose of the TEACh-Pain website), (b) Pain 101 (i.e., definition, prevalence rates, and descriptions of pediatric pain), (c) What Teachers Need to Know (i.e., signs, symptoms, and triggers of pain), (d) What Teachers Can Do (i.e., classroom adaptations to manage pain and strategies to facilitate transition back to school, relaxation videos), (e) Frequently Asked Questions (i.e., 10 commonly asked questions d accurate responses), and (f) Resources (i.e., links to community and academic resources). A statement on the home page indicated that the site is not intended to act as a substitute for medical treatment and/or evaluation. The website can be accessed at http://teachpain.wordpress.com. 1
Following content development, four pain clinicians from the fields of medicine (n = 1), nursing (n = 2), and psychology (n = 1) reviewed the website to provide feedback. These expert reviewers were all members of a complex pain team at a tertiary care health centre and provided several suggestions to improve the website content. Feedback primarily focused on improving the clarity of the message, providing additional information regarding the physical implications of pain in the classroom, potential classroom adaptations, and additional resources. Finally, a nurse in the position of Coordinator of Student Health Needs from a local school board was consulted to determine the feasibility of the potential classroom adaptations suggested by the expert reviewers. Minor changes to the original content were made based on reviewer suggestions; changes largely included clarifying various phrases and terminology, ensuring suggestions for adaptations were feasible, and providing additional resources (e.g., links to relevant websites).
Phase II: Usability Testing of Website
Usability is defined as the systematic testing of the utility of a product by collecting feedback from typical users (Kushniruk, 2002). With this in mind, the second phase of the study focused on assessing the usability of the website and collecting feedback about improvements from intended users. Usability testing is typically conducted in iterative cycles in which a product is created, tested with the target audience, adapted based on user feedback, and retested (Snodgrass & Coyne, 1992); this process continues until there is no further feedback and all usability goals have been met (Wichansky, 2000). Nielsen (2000) suggests that as few as five users can reliably identify approximately 85% of usability problems, and Kushniruk (2002) indicates that almost all usability problems can be detected with as few as eight participants. Based on recent usability research (e.g., Currie, McGrath, & Day, 2010; Stinson et al., 2006), the goal of the current study was to recruit between five and eight participants per group. Three usability goals were identified prior to testing. Goals would be met if the majority of participants (a) viewed the website as being easy to use, (b) viewed the website as being easy to understand, and (c) indicated that they would either use the website themselves (teachers) or would recommend it to teachers (youth and parents).
Participants
Three groups of participants were recruited for usability testing: (a) youth (i.e., patients) between the ages of 8 and 18 currently being treated at the pain clinic at a tertiary care children’s health centre, (b) parents of youth between the ages of 8 and 18 currently being treated at the same clinic, and (c) practicing teachers enrolled in graduate education programs. Although educators were the target audience and were considered to be the typical users of this website, youth and parents were recruited to provide feedback on the website, as they were able to provide useful suggestions for website improvement based on their experience working with schools.
Recruitment of patients and parents
Patient and parent participants were notified of the study by either receiving an information package outlining the study in the mail and/or being approached during their visit to a tertiary care pain clinic. In total, seven patients and six parents participated in the study; some parents and children participated as dyads, but this was not required and all interviews were conducted individually.
Recruitment of teachers
Practicing teachers with experience teaching at the elementary, junior high school, and high school levels were approached in various graduate education courses at a local university. Teachers were asked to provide their contact information to a researcher if they were interested in participating in the study. Potential participants were randomly contacted by telephone until recruitment goals were met. In total, eight teachers participated in this study.
Measures
Demographic questionnaire
A brief demographic questionnaire was administered to all participants. Patient participants provided information on age, grade, school, type of pain, duration of pain condition, and whether accommodations had been made in the school. Parent participants provided information on age, area of residence (i.e., rural, urban, suburban), duration of their child’s pain condition, and whether they or their child had been provided with access to resources (e.g., school psychologist, nurse) at school. Teacher participants provided information on number of years of teaching experience, experience working with children with chronic pain, and current teaching position.
Think-aloud
Think-aloud procedures are commonly used in usability research, as they allow for real-time expression of thoughts and opinions about the content being examined (see Kirwan, Duncan, Vandelanotte, & Mummery, 2013; Stinson et al., 2006). Participants were asked to independently examine the website while vocalizing their thoughts to a researcher. This procedure allowed participants to comment on the parts of the website they liked and disliked and also provided an opportunity to make suggestions for improvements. Participants were instructed to examine the website in whatever order they liked, to spend as much or as little time as they liked on each section, and to feel free to ask questions as they navigated through the content.
Website experience survey
Participants completed a brief semistructured interview and questionnaire to provide feedback about the website’s usability. Items on the interview and survey were adapted from previous usability studies (see Currie et al., 2010; Stinson et al., 2006, for examples) and focused on key elements of usability such as quality of content, ease of navigation, and general likability of the website. Participants were asked to provide both qualitative feedback (i.e., specific comments and suggestions for improvements) as well as quantitative ratings on a 5-point Likert-type scale ranging from strongly disagree to strongly agree.
Procedure
Participants completed the study in one of three settings: a health centre laboratory (n = 6 patients, n = 5 parents, n = 1 teacher), a university classroom (n = 7 teachers), or their home (n = 1 patient, n = 1 parent). For patients and parents who chose to participate from home, a researcher travelled to the home to administer the study protocol. Following consent, participants were instructed to independently explore the TEACh-Pain website while thinking aloud. When participants completed their review, they were asked to complete the website experience survey. The entire procedure was audiotaped and later transcribed. Each participant spent approximately 40 min completing the procedure. Responses on the think-aloud task and the open-ended questions on the website experience questionnaire were combined to allow for exploratory qualitative description of the data to determine perceived usefulness of the website. Qualitative description was used because, although there is a sizable preexisting literature on usability testing of websites (see Currie et al., 2010; George, 2005; Hinchliffe & Mummery, 2008; Stinson et al., 2006; Zimmerman, Akerelrea, Buller, Hau, & Leblanc, 2003), there is little preexisting knowledge of this topic to test specific hypotheses (see Sandelowski, 2000).
Qualitative content analysis was used to generate themes for suggestions for improvements to the website. Themes were divided into major (i.e., changes that could not be made independently by the researchers or changes requiring financial investment, such as producing videos or specialized animation) and minor (i.e., changes that could easily be made by the researchers, such as changing font size or clarifying phrasing). Minor suggestions were further examined to determine whether they were concordant or discordant from other suggestions. Major changes were not made at this time, but all changes deemed to be minor in nature were implemented.
Results
Demographics
A total of seven female patients participated in this study; however, due to recording equipment malfunction, the qualitative responses for two participants in this group were lost and are not included here. Patient participants ranged from 15 to 17 years old (M = 16 years) and had experienced chronic or recurrent pain for a mean of 3.3 years (SD = 1.49) at the time of the study. Patient participants reported experiencing full body, neuropathic, back and leg, and joint pain, and missing an average of 6.3 days of school each month due to pain (SD = 4.96). Four patients indicated that they had received classroom accommodations for their pain. A total of six parents (four mothers) participated in this study. Parents indicated that their children had been experiencing chronic or recurrent pain for an average of 3.2 years (SD = 1.33); two parents indicated that they had been able to access resources and supports at school. Finally, eight teachers (five women) participated in this study. Teacher participants had been teaching for a mean of 13.3 years (SD = 4.80); four teachers indicated that they had some experience working with students with chronic or recurrent pain.
Website Experience Survey
The majority of participants reported that the website was easy to use, easy to understand, and that they would either recommend it to teachers or use it themselves. Percentages of participants who either agreed or strongly agreed with each item on the website experience survey are presented in Table 1.
Percentages of Participants Selecting “Agree” or “Strongly Agree” on Survey Questions.
No patient participants watched the relaxation tutorial; one parent participant watched the relaxation tutorial; four teacher participants watched the relaxation tutorial.
Qualitative Findings
Qualitative results were collapsed across groups to facilitate coding of themes. Four themes emerged as a result of the analysis: interface design, website navigation, content, and animation. Each of these themes is discussed below.
Interface design suggestions
Interface design suggestions were defined as comments related to the website design (i.e., colours, icons, or graphics). The most consistent comment from participants was that the website design was “plain” and that more colour or graphics should be added to make it more visually stimulating. Participants made comments such as “I would like it to be more visually stimulating” and “[a]dding a little bit more visual or dressing it up a bit just to make it more eye appealing.” Participants also suggested adding the university and/or health centre logo to the website to give the impression of being a reputable source of information.
Navigation suggestions
Navigation suggestions were defined as comments related to moving from one section to another. Participants commented that drop-down menus or direct links to other sections of the website would facilitate navigation. Participants made suggestions such as “. . . add in a couple of links back to the pages that make it easier to access specific information.” Another participant indicated that navigation was not intuitive: “You had to go on the back button to click. I thought you would just click it and it would downsize again.”
Content suggestions
Content suggestions were defined as comments related to the material presented on the website. A common comment was that the TEACh-Pain acronym was confusing and that providing the meaning would have been helpful. One participant suggested, “. . . I would have at the top . . . ‘T’ and what it means, and then ‘A’ and what it means.” Participants also suggested adding information relating to symptoms of chronic pain, specific chronic pain conditions, and specific adaptations. For example, participants made suggestions such as “[p]rovide more information about symptoms and how to help students with specific chronic illnesses.” Teachers made several suggestions related to the amount of content on the website such as “. . . there was too much paragraph reading. It needs to be quicker.” Several participants suggested using bullet points to reduce the number of paragraphs and using variations in text features to highlight important information (e.g., “. . . using more text features like bold, italics, underline—would be really important”).
Animation suggestions
Animation suggestions were defined as comments related to the videos or video links on the website. Comments were largely related to the manner in which the videos were presented and the ease with which they could be accessed. Participants indicated that they would have liked all videos to be displayed prominently on the website, rather than linking to the video on an outside site. The majority of participants in all categories did not watch the relaxation videos; this might have been a result of the presentation format. Specifically, one participant indicated, “If I was going to make a change to this website, I would feature all the videos in the same way the two main videos are featured.” Other participants suggested adding more videos so participants could have the option to read the text or watch the video to learn the content. Participants indicated that they would have liked to see videos of local children attending the pain clinic or interviews with local teachers who had experience working with students with pain.
Discussion
The goal of the current exploratory study was to develop and conduct preliminary usability testing on an eHealth education dissemination intervention designed to provide educators with information about pediatric pain in the school setting. To our knowledge, this is the first usability study examining an eHealth intervention for pain specifically targeted to educators. This study was also unique, in that three groups of stakeholders (i.e., children with pain, parents of children with pain, and practicing teachers) participated in the usability testing. Results of this pilot usability study indicated that usability goals were met. Participants generally found the website easy to use, found it easy to understand, and indicated that they would either use it themselves or would recommend it to teachers. Based on these results, and given that research suggests 85% of usability problems can be identified by between five and eight participants (Kushniruk, 2002; Nielsen, 2000), it was determined that the usability and feasibility goals were met and that one iterative cycle was sufficient for this preliminary study.
Participants provided several useful suggestions for improvement to the website and those changes deemed to be minor were made. Most of the comments about the interface design were generally minor and related largely to the aesthetic properties of the website. Given the relative ease with which participant comments could be addressed, these changes were made. Similarly, the majority of comments relating to the content of the website were minor in nature and most comments were addressed. Participant comments regarding website navigation and animation were all deemed to be major and, due to limited resources, it was not feasible to make the changes. Future iterations of the website will be developed with these comments in mind to ensure that the website can appropriately meet the needs of educators.
Recent research has shown that eHealth interventions can be useful adjuncts to clinical services when managing pain in children and youth (e.g., Stinson et al., 2006). The findings of the current study add to this emerging literature and provide preliminary support for the potential utility of eHealth interventions in providing school personnel with accurate information about pediatric chronic pain and its management. Furthermore, these findings suggest that eHealth interventions may be an acceptable way of disseminating and translating current knowledge of research and clinical practice to stakeholders other than patients and their families.
Providing educators with evidence-based information about pediatric chronic pain is important, as school is an ideal setting in which to implement secondary prevention and intervention for children with chronic health and mental health conditions (e.g., Greenberg, Domitrovich, & Bumbarger, 2001; Larsson & Carlsson, 1996; Larsson & Sund, 2005; Wilson, Lipsey, & Derzon, 2003). Research indicates that behavioural management strategies (including those used to manage pain) are most effective when implemented consistently across settings (see Shapiro, Friedberg, & Bardenstein, 2006, for a discussion of behaviour management). Providing educators with accurate information about pediatric chronic pain may lead to fewer misperceptions of the condition, meaning that they may be better able to understand and implement behavioural pain management strategies developed by pain clinicians. In addition, an eHealth intervention such as this could be a useful way to provide educators with information about pediatric chronic pain that can be applied to students who have been identified as needing support for their pain in the classroom, but who may not be followed by specialist providers. Given the high prevalence of pediatric pain (see Huguet & Miro, 2008; King et al., 2011; Perquin et al., 2000), it is not unreasonable to assume that many teachers encounter children experiencing pain and that the information provided by the TEACh-Pain website could be useful when developing more formal adaptation/accommodation plans or as an adjunct to existing adaptation/accommodation plans.
It is interesting that only half of the teachers in this study reported experience with children with chronic pain in the classroom. This seems inconsistent with recent epidemiological data on chronic pain and may indicate teachers’ lack of awareness of the problem or students’ hesitancy to disclose. It is important to provide teachers with reliable information about pediatric chronic pain and to encourage students with pain and their parents to work with pain clinicians and school personnel such as school psychologists to develop school-based pain management strategies. Although interdisciplinary collaboration and communication is the ideal approach to pain management in the school setting (Papa, Rector, & Stone, 1998), it is often difficult to arrange opportunities for clinicians and educators to come together to meet about specific cases or more general health-related concerns; providing access to an eHealth intervention such as the TEACh-Pain website and using school psychologists as liaisons could be a way to bridge this gap, possibly resulting in decreased school disruption for children with pain.
Although preliminary findings of this pilot usability study were encouraging, some limitations should be addressed. First, only one iterative cycle was completed; it is possible that one more cycle of testing would have identified additional usability problems not previously identified. A second cycle of testing would have also been useful if the major changes suggested by participants had been implemented; unfortunately, due to limited resources, it was not feasible to make and test these changes. However, given that the majority of participants found the website easy to use and understand, making major changes would have likely resulted in a more aesthetically pleasing website and would have possibly made the material more engaging, but would not necessarily enhance ease of use for educators. Another limitation is that few participants viewed the relaxation video. Given that relaxation techniques are an important component of behavioural pain management programs, as well as being a simple strategy that does not require extensive clinical training to use, it is discouraging that this feature of the website was not appealing, especially to patients. With respect to the sample used in the current study, all teacher participants were graduate students and could therefore be more motivated to learn about health conditions affecting their students. Similarly, children and parents were all recruited from one pain clinic and, therefore, these results may not be applicable to other geographic locations or clinic settings. Participants across all three groups represented a very homogeneous sample, thereby limiting generalizability to diverse cultural and socioeconomic populations; the majority of participants were female and it is not clear whether males would provide similar feedback about the website. In addition, all participants in the pain patient group were adolescents, so it is unclear whether the website would be useful with younger children and their teachers. Finally, all parent participants were parents of adolescent females; future studies should investigate same- and mixed-sex parent–child dyads to determine whether sex plays a role in determining responses to the website content.
Despite these limitations, the results of this preliminary usability study are encouraging and suggest that further studies examining the usefulness of information-based eHealth interventions for educators are warranted. Future directions for the TEACh-Pain website include making major changes to the site and conducting further iterative cycles to test usability in a more diverse sample of children, parents, and educators (e.g., guidance counsellors, resource teachers). Furthermore, investigating the effectiveness of providing information about pediatric chronic pain on student outcomes and teacher knowledge, attitudes, and behaviour will be necessary to draw conclusions about the efficacy of this type of intervention, because a well-designed and usable website may not necessarily lead to better school outcomes for patients.
In summary, results of preliminary testing of this eHealth intervention for educators were encouraging, as they suggest that this may be a useful method of knowledge dissemination and translation in the area of pediatric chronic pain. The school setting has been identified as an important site in which to implement health interventions; the TEACh-Pain website is an important first step in developing a model of coordinated care to ensure positive school outcomes for children challenged by pain.
Footnotes
Acknowledgements
The authors wish to gratefully acknowledge the in-kind contribution from VIBE Creative Group that made this project possible. Special thanks to Sharon Amey and the research assistants in the Centre for Pediatric Pain Research for their assistance with data collection, transcription, and in the preparation of this article.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by a Nova Scotia Health Research Foundation (NSHRF) Scotia Scholars Award (Master’s) and a Pain in Child Health (PICH) student stipend awarded to J.A.B.