Abstract
The aim was to develop a new instrument for measuring length of sleep as well as television and computer habits in school-age children. A questionnaire was constructed for use when children visit the school health care unit. Three aspects of the validity of the questionnaire were examined: its face validity, content validity, and construct validity. Test–retest reliability was assessed by giving the questionnaire twice, 2 weeks apart, to the respondents. The questionnaire was assessed as being reasonably valid, the test–retest results (n = 138) showing 90.4% of the estimates regarding bedtime on weeknights on the two survey occasions to lie within ± 30 min of each other, the test–retest agreement also being rather close (κ > .600) regarding both sleep and media habits. The instrument can be a valuable tool in a clinical setting, both for measuring sleep habits in a class and for discussing sleep with individual school children and their families.
Background
School health programs aim at promoting healthy lifestyles, including healthy sleep habits. All school children in Sweden are provided the possibility of an individual visit to the school health care unit every 3 years, a school nurse discussing matters of health with them at the time, their height and weight also being measured then (Clausson, 2008). Sleep problems in children have been found to have increased in recent years, lack of sleep being one of the most common complaints of children and adolescents today (Bremberg, 2006). Sleep problems in childhood appear to be a possible indicator of later risk of neuropsychological difficulties (Gregory, Caspi, Moffitt, & Poulton, 2009), such as those involving depression (Gregory, Rijsdijk, Lau, Dahl, & Eley, 2009), anxiety, and aggression (Gregory, Van der Ende, Willis, & Verhulst, 2008). Nevertheless, the availability of clinical sleep-diagnostic and treatment services for children is generally inadequate in most countries, including Sweden (Owens, 2005). In order to have an adequate basis for providing children advice regarding their sleeping problems, reliable information on their sleeping habits and on factors that influence their length of sleep are called for. There was the need of a questionnaire suitable for use in a school health care setting for children with and without sleep problems that could provide a basis for discussions during health care visit.
One problem connected with many existing questionnaires is of their being designed primarily for use in diagnosing serious sleeping difficulties that are clinical in character, rather than for studying differences in children’s sleep habits generally. Another problem is the length of questionnaires. In the present study, the suitability of a new, short questionnaire was investigated. The questionnaire is designed so that school nurses can distribute it to school children at the time of their ordinary school health care visits. Development of the questionnaire was inspired by various other sleep-evaluation scales and questionnaires (Lomeli et al., 2008). Several of the existing questionnaires, specifically the School Sleep Habits Survey (Wolfson & Carskadon, 1998), the Sleep Evaluation Questionnaire (Mindell & Owens, 2003), the National Sleep Foundation Questionnaire (2006), and the Children’s Sleep Habit Questionnaire (Owens, Spirito, & McGuinn, 2000), were found to be rather time-consuming, each of them several pages long. For the purpose of guiding an interview leading to health counseling, what was needed was a short, easy-to-use questionnaire that children above a certain age could readily fill out and parents of younger children could complete on their child’s behalf. The instrument employed is a further development of an earlier 10-item questionnaire developed for use in a pilot study conducted in 2007–2008 of the length of sleep and of the television and computer habits of primary school children (Garmy, 2011).
Aim
The aim of the present study was to develop and evaluate the validity and reliability of a new instrument for measuring both the length of sleep and the television and computer habits of children of school age.
Method
The study has a psychometric, cross-sectional design. Approval of the study was obtained from the head principal of each school and from the school’s head physician. Respondents were told that participation was voluntary and those participating gave their informed consent. The study was also reviewed by the Advisory Committee on Research Ethics in Health Education (VEN 34-09).
The Questionnaire
A questionnaire (see Appendix A) consisting of 11 questions was constructed. The questions pertained to eight variables concerning sleep as well as possible effects of lack of sleep, and also to TV and computer habits (length of sleep on weeknights and on weekends, time for getting ready for bed, time of going to bed, amount of time spent watching television and at the computer). There were four questions regarding problems related to sleeping habits and their possible effects such as difficulties in falling asleep, difficulties in waking up, experiencing feeling tired at school, and enjoyment of school. Although asking the time the child got up on weekdays and weekends, and went to bed on weekends, would have provided a certain check on the child’s estimates of hours slept, these questions were not asked, due to the aim of keeping the questionnaire brief. The questionnaire was constructed for use at the regular, individual school health care visits children have while in the preschool class (age 6), as well as in grades 1 (age 7), 4 (age 10), and 8 (age 14) and in the first upper secondary school grade (age 16).
For children in the preschool class or in grade 1, administration of the questionnaire took place during the child’s first health care visit at school, when one of the parents or both came with their child to the school health care unit and filled out the questionnaire on the child’s behalf. For children in grades 4 and 8 and in upper secondary school, it involved the pupils filling out the questionnaire themselves during a school health care visit. The present questionnaire was developed initially on the basis of a review of pediatric literature on sleep and was inspired by earlier sleep questionnaires developed by Wolfson and Carskadon (1998), Mindell and Owens (2003), Sadeh (2004), Nordlund, Norberg, Lennerenäs, Gillberg, and Pernler (2004), and the National Sleep Foundation (2006). The instrument is also a further development of the 10-item questionnaire used in the pilot study referred to. Regarding the reliability of the 10-item questionnaire, results of the two separate testing conducted were found to correspond rather closely with one another (Garmy, 2011).
Data Collection
The questionnaires were distributed by six school nurses in eight schools during 2008–2009. The school nurses, each with more than 4 years of experience in school health care, were informed both in writing and orally that they should be sensitive in their approach and not try to persuade any of the potential respondents to participate. The respondents were told that there were no right or wrong answers, that it was a question of what they themselves thought.
Sample and Setting
The study, which was part of a larger study concerned with children attending the preschool class, grades 1, 4, and 8, and grade 1 in upper secondary school (n = 4692), was conducted in a municipality in southern Sweden of about 100,000 inhabitants. About 17% of the population there was born abroad, the unemployment rate was 2.4%, and 1.6% of the populace received welfare payments (Svärd & Nilsson, 2009). There are 44 schools in the municipality. Eight of these were selected randomly for inclusion in the study. The sample taken consisted of 160 respondents. The sample size was based on the recommendations of Fleiss (1986), that is, that 15–20 subjects would be required for estimating the reliability of a quantitative variable. There were 116 child respondents (32 being pupils in grade 4, 31 pupils in grade 8, and 31 pupils in first grade in upper secondary school) and 44 parent respondents whose children were in the preschool class or grade 1, 138 of them having filled out the questionnaire on both occasions on which it was given, 2 weeks apart. Fifty-three percent of those in the sample were girls (n = 73). Twenty-two of the respondents only answered the questionnaire once. For the pupils in grades 4 and 8 and the first upper secondary school grade, the reasons for failure to fill out the questionnaire the second time were either their being absent the second time the questionnaire was distributed (n = 7) or a lack of desire to participate then (n = 2), and for 13 of the parents of children in the preschool class or in grade 1 it was failure to fill out and return the second questionnaire. Cases in which the questionnaire was not filled out the second time (whether by the children themselves or their parents) were found to not differ from the remaining cases in terms of the age and gender of the children, or of geographic location.
Tests of Validity
Three aspects of the validity of the questionnaire were examined: its face validity, its content validity, and its construct validity. Validity refers to the degree to which an instrument measures what it is supposed to measure, face validity to the extent to which an instrument appears valid to those who use it, content validity to the extent to which a measure represents all facets of the content area being assessed, and construct validity to the degree to which a survey or test score correlates with some related measure (Polit & Beck, 2006; Streiner & Norman, 2003). The face validity of the instrument was evaluated, for the children who filled out the questionnaire themselves, by asking all of them, both orally and in writing, to comment on the questionnaire, and urging them to write down their comments on it. The time taken for completing the questionnaire was also noted. In addition, the content validity of the questionnaire was evaluated by letting an expert group consisting of four persons review and comment on the questionnaire. One of them had more than 10 years of experience as a registered nurse working with children and had been a school nurse for 7 years, another had more than 20 years of experience in the development and psychometric analysis of test instruments, and the remaining two persons had considerable experience in connection with use of sleep medicine.
The construct validity, the association between the length of sleep and feelings of being tired at school, as well as difficulties in sleeping, were assessed using the chi-square test. The length of sleep the children had was dichotomized so as to form two groups: those sleeping less than the median for their grade (short sleep) and those sleeping longer. The median length of sleep per night was 10 hr at the age of both 6 and 10, 8 hr at the age of 14, and 7½ hr at the age of 16.
Test of Reliability
The reliability of the questionnaire was tested by giving the respondents (n = 138) the same questionnaire twice within a 2-week period. This made it possible to evaluate the test–retest reliability or stability. An assessment of the test’s power indicated 30 individuals in each grade to suffice for acceptable conclusions to be drawn regarding the test’s reliability. Taking account of the fact that sleep and media habits can fluctuate, it was considered, in line with the expert group’s assessment, that ±30 min was a reasonable time span of differences between the two time estimates for there to be basic agreement between them.
Statistical Analysis
The chi-Square test and the Fisher Exact test were used to identify the association between length of sleep and both feelings of being tired at school and difficulties in sleeping (Altman, 1991). Associations with p values of <.05 were considered statistically significant. Kappa analysis and percentage of agreement were both used for assessing the agreement of the two measurements in test–retest terms. Unweighted Kappa was used for dichotomized variables, weighted Kappa for ordinal data (Altman, 1991), and percentage of agreement for continuous variables. For the test–retest assessment of sleep length, of bedtime, of time of getting ready for bed, and of time spent at a television set or at a computer, that is, of all the variables considered, an interval of ±30 min was considered by the expert group to represent an acceptable level of agreement. The kappa values were classified in the manner described by Altman (1991), such that a value of ≤0.20 was considered as “poor,” 0.21–0.40 as “fair,” 0.41–0.60 as “moderate,” 0.61–0.80 as “good,” and 0.81–1.00 as “very good.” The same criteria were used for percentage of agreement. The statistical analyses were performed using statistical package for the social sciences (SPSS) version 14.0 (Chicago, IL) and the VassarStats website for statistical computations (http://faculty.vassar.edu/lowry/VassarStats.html).
Results
Validity
Face validity
The parents of the children in the preschool class and of those in grade 1 tended to report no difficulties in answering the questions on behalf of their children. The pupils in grades 4 and 8 and in upper secondary school raised questions of what was meant by getting ready for bed, whereas none of the parents had such questions. The children in grade 4 often needed help in calculating their length of sleep. Some of the children wrote down the time they woke up rather than the length of time they slept. The time taken in filling out the questionnaire was generally about 3–7 min. The children in grade 4 took the longest time.
Content validity
The expert review of the instrument led to the development of a version of the questionnaire that also included two questions regarding stress and self-confidence since sleep problems are often associated with stress and low self-efficacy (Bremberg, 2006). The two questions were “I feel stressed” (never, seldom, often, or every day) and “I’m pleased with myself” (every day, often, seldom). A group of 30 pupils were asked to fill out this questionnaire. Since all of them answered these two questions in about the same way, both of the questions were excluded from further development of the questionnaire. Another result of the expert review of the test instrument was that a question specifically concerned with the length of time the child slept was included in the final questionnaire. Earlier, only questions about bedtime and the time the child got up were asked, which only enabled the time the child spent in bed to be calculated, rather than the length of time the child slept.
Construct validity
The median length of sleep on weeknights was 10 hr for the preschool class, as well as for grades 1 and 4, whereas for grade 8 it was 8 hr, and for the upper secondary school class considered it was 7 hr and 30 min. Those in each grade sleeping less than the median time formed a short-sleep group. Being short of sleep was found to be significantly related to being tired in school (χ2 = 6.265, df = 1, p = .012,) and to having difficulties in sleeping (χ2 = 7.672, df = 1, p = .006).
Reliability
The test–retest results showed a high level of agreement concerning bedtime on weeknights, 90.4% of the answers on the two testing occasions being within ±30 min of each other. The answers to length of sleep on weeknights also showed a high degree of concordance, 86.8% within a ±30-min interval on the two occasions, see Table 1 . All of the continuous variables (bedtime, time for getting ready for bed, length of sleep on weeknights and on weekends, and time spent watching television or at the computer) showed reasonable-to-excellent agreement (63.8–90.4% agreement). The Kappa and weighted Kappa values for having bedroom TV, having difficulties in falling asleep, feeling tired in school, having difficulties in waking up, and enjoying school showed reasonable-to-excellent agreement according to methods described by Altman (1991) (see Table 2).
Test–Retest Reliability Assessments a
Note. aThe percentage of the times for which the two assessments agree with each other to within ±30 min, n = 138.
Test–Retest Reliability Assessments a
Note. aKappa and weighted Kappa (quadratic) values for having bedroom TV, having difficulties in falling asleep, feeling tired in school, having difficulties in waking up, and enjoying school, n = 138.
Fact Box
Source. National Sleep Foundation: http://www.sleepfoundation.org/article/sleep-topics/children-and-sleep.
Discussion
Sleep is a fundamental need in terms of human health and is thus an obvious area for school health care. The availability of a reliable and valid instrument to screen students’ sleeping habits and the time spent on TV and computer is valuable. For one thing, it provides a unique entry point for a conversation regarding sleep and media habits of pupils at individual visits. In addition, it gives the school nurse a tool to use in identifying sleep habits in a class or in a school district.
The questionnaire was validated in terms of face validity, content validity, and construct validity. Regarding face validity, the questionnaire was found to be easy both to understand and to fill out with two exceptions. Children in grade 4 often needed help in calculating the length of time they slept, and many of the children, regardless of their age, had difficulties in understanding what getting ready for bed stands for. School nurses found the questionnaire valuable in providing an opportunity to discuss sleep hygiene with the pupils. Construct validity was assessed in terms of the degree of association between stated sleep length and answers to questions about being tired in school and having difficulties in sleeping. In both cases, the association was found to be highly significant, clearly supporting the construct validity of the test.
Close agreement was obtained in the test–retest comparison of the questionnaire results for length of sleep and for bedtime on weekday nights. The agreement was somewhat less but was still rather close for length of sleep and bedtime on weekend nights, which is rather understandable in view of weekends usually being less rigidly scheduled than weekdays are. The questionnaire showed a high degree of reliability regarding difficulties in waking up and bedroom TV. The other questions, concerning sleeping difficulties, feelings of tiredness at school and enjoying school, showed a reasonably close agreement (Altman, 1991).
Limitations
One potential drawback to the results is that the schools selected cannot be assumed to adequately represent the population of Sweden generally. However, the random selection of eight schools in different socioeconomic areas of the city in question does serve to counteract this weakness somewhat. Although some of the percentages of agreement and kappa values are lower than others, we can see no linear trend reflective of age differences in the test–retest responses. The answers of the pupils to the questionnaire showed a slightly lower degree of stability than the answers to the questionnaires by parents, yet the instrument seems to function relatively well for all the grades that were involved.
There is a clear advantage in employing a short and easy-to-use instrument in a clinical setting, although this does involve limitations, since the short length of the questionnaire makes it impossible to examine all aspects of the sleep habits and of the television and computer habits, as well as habits of other sorts that could be of interest related to sleep. Stress and lack of self-confidence, for example, represented here by the questions “I feel stressed” (never, seldom, often or every day) and “I’m pleased with myself” (every day, often, seldom), respectively, have been shown to be associated with sleeping difficulties (Bremberg, 2006). The focus of the present study and the breadth of the test instrument were necessarily limited. Investigating sleep habits in greater depth and studying other variables that contribute to sleep would require use of a longer instrument. The use of cellular phones by adolescents for text messaging and for calling after lights are out is prevalent and readily increases the degree of tiredness of many adolescents (Van den Bulck, 2007), yet is not examined here. Despite the limitations of the present instrument, the evaluation of its validity and reliability points to its being a potentially valuable tool in a clinical setting.
School Nursing Implications
School nurses should obviously continue their important work of promoting healthy lifestyles in children, including healthy sleep habits and healthy TV and computer habits. Regarding sleep, a first step that a school nurse should take is to teach children with sleep problems, and their families as well, sound sleep habits through emphasizing the need of maintaining a regular and consistent sleep schedule and bedtime routine, having the child’s bedroom conducive to sleep—dark, cool, and quiet—keeping TV and computers out of the child’s bedroom, and avoiding the child consuming caffeine (Mindell & Owens, 2003; National Sleep foundation 2006; Zarcone, 2002). Our screening questionnaire provides a point of entry for conversations with the children or with the parents regarding the sleep and media habits. If sleep problems are noted, it is important that the family be involved. Use of both a sleep diary and a questionnaire having more detailed questions than those involved here can be desirable. Judith Owens’ Children’s Sleep Habits Questionnaire (Owens et al., 2000) is an example of an instrument containing further questions regarding sleep habits. If problems persist, the child should see a doctor to rule out possible medical causes of insomnia or daytime sleepiness. Note that sleep apnea associated with snoring can also occur in school children.
Conclusions
The questionnaire shows rather close test–retest agreement regarding having sleeping difficulties, having bedroom TV, feelings of tiredness at school, enjoying school, and having difficulties in waking up. It also showed a reasonable level of reliability for length of sleep, bedtime, time for getting ready for bed, and time spent either watching television or at the computer. School nurses found that the questionnaire was a help in opening up a discussion with pupils regarding their sleep habits. Parents and school children alike found the questionnaire to be easy to fill out. It appears that the instrument could be a valuable tool in a clinical sense, both for measuring sleep habits and for discussing matters of sleep with individual school children and their families.
Footnotes
Appendix Questions Regarding Sleep and Lifestyle
I have a TV set in my room: No □ Yes □ About how long each day I typically spend watching TV: …… hrs. and ……. minutes About how long each day I typically spend at the computer: ….hrs. and …….
minutes How often I have difficulties in falling asleep: Never □ Seldom □ Often □ Every
night □ How often I feel tired in school. Never □ Seldom □ Often □ Every day □ To what extent I enjoy being in school: A lot □ Fairly much □ Not much at all □ When I have school the next day, I begin getting ready for bed at about:……………… When I have school the next day, I go to bed at about:…………… When I have school the next day, I usually sleep about: ……. hrs. ……… min. When I’m free the next day, I usually sleep about: ……. hrs. ……… min. I find it difficult to wake up in the morning: Never □ Seldom □ Often □ Every day
□
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Södra Sveriges Sjuksköterskehem; and the Fanny Ekdahl Foundation for Pediatric Research at Lund University.