Abstract
Introduction
Young children use computers frequently and for long durations [1]. Prolonged computer use has been associated with development of musculoskeletal complaints; and may be linked to secondary health conditions associated with sedentary behaviours [2, 3].
Therefore it is likely that children are at risk of physical health problems associated with high exposure to computer use. Physical health impairments associated with computers have been found to limit children’s performance in functional activities, such as taking notes in class, carrying books, playing video games and handwriting assignments [4, 5].
Preliminary evidence indicates that intervention may be necessary to prevent physical health problems associated with computer use in children. There are a limited number of studies that have developed interventions to educate children about ergonomic practices when using computers [4–10].
A common limitation of past intervention studies has been the lack of a valid and reliable questionnaire to determine effectiveness of the intervention. In addition, the focus of previous interventions has been on increasing knowledge with little to no acknowledgement of the attitude and behavioural components of health promotion strategies. Technology is evolving at an increasingly rapid pace. Interventions to support people to engage in healthy technology use will need to be updated to keep pace with these developments. For this to occur, exemplar methodology will need to be readily available to promptly develop and update outcome measures to assess the effectiveness of such interventions.
The aim of this study was to develop a valid and reliable self-report measure to assess primary school children’s knowledge, attitudes and behaviours related to healthy computer use.
Methods and materials
Participants
Sixteen non-government Catholic Education primary schools in Perth, Western Australia, agreed to take part in this cross-sectional survey. There were five phases in the study, three of which involved Year 5 students. Table 1 in the results provides an overview of participant demographics for each phase. Ethics approval for the study was granted by the Human Research Ethics Committee at Curtin University, and from the Catholic Education Office of Western Australia. Prior to any data collection, informed consent was obtained from the experts and from parents of the school children and the school children provided their assent to participate.
Methodology
The methodology for the study was based on the recommendations reported in the literature for best-practice in measure development [11–13], and on procedures used to develop existing measures [14–16]. The methodology ensured the questionnaire met criteria for the utility of a measure as defined in the Instrument Evaluation Process developed by Law [17]. According to this process, the purpose of the instrument determines the type of validity and reliability required. The current measure was intended for both descriptive and evaluative use as a pre-post measure, and so it required internal consistency and test-retest reliability, as well as content and construct validity. Since the measure was a self-report instrument neither observer nor inter-rater reliability was required.
A health promotion model was adopted as a framework for the questionnaire. Health promotion theory advocates behavioural change as a method of preventing chronic health problems and enhancing quality of life [18]. Health promotion literature asserts that to be effective, an intervention program must address an individual’s knowledge, attitudes and behaviours [19]. Therefore the questionnaire consisted of three sections addressing each of these components in relation to healthy computer use.
The methodology for the development and validation of the current questionnaire consisted of five phases, as shown in Fig. 1.
The guidelines for healthy computer use developed from this search were used to develop Version 1 of the questionnaire, which consisted of 77 knowledge, attitude and behaviour items.
In
In
During
The test-retest reliability of the questionnaire was determined in
Data analysis
Data were analysed using SPSS for Windows (Version 17.0) and SAS for Windows (Version 9.1). Simple descriptive statistics were used to summarise the feedback from the experts in Phase 2. Qualitative feedback about clarity and readability collected in Phases 2 and 3 was reviewed. Data collected in Phase 4 were analysed using simple descriptive statistics for the knowledge items; factor analysis to test construct validity of the attitude items; and Cronbach’s alpha was used on the variables within each main factor to determine internal consistency. The test-retest reliability of the data collected in Phase 5 was assessed using the Cohens Kappa statistic.
Results
Participant demographics
The demographic characteristics of the participating school children in each phase of the study are summarised in Table 1.
Questionnaire content
In the final version of the HCQC, Section One included 22 multiple choice questions that measured the children’s knowledge of basic computer ergonomics, including appropriate seat height, back support, forearm support and positioning of the keyboard, mouse and display, lighting, as well as frequency of breaks and response to discomfort. Questions were multiple-choice, and respondents were asked to select and circle the most appropriate answer from a choice of four responses.
Section Two measured children’s attitudes towards healthy computer use. The attitude items in the questionnaire were based on two health promotion theories:
1) Protection Motivation Theory [22] is based on the concept that people are motivated to protect themselves from physical, social and psychological threats; and
2) Theory of Reasoned Actions [23] is an approach to behavioural prediction. This theory asserts that there is an important distinction between a person’s own attitudes and their perception of other people’s attitudes. The latter is termed the subjective norm.
These theories contained constructs that were used to develop the attitude questions, including; ‘Evaluation and Perceived Severity of Consequences’, ‘Perceived Likelihood of Consequences’, ‘Response Efficacy’, ‘Self Efficacy’, ‘Normative Beliefs’, ‘Motivation to Comply with Others’. The scoring guide in the HCQC details the items that correspond with the constructs.
The responses to the attitude items were gathered using a five point Likert scale to indicate level of agreement with the statements. A pictorial scale using facial emotions was used to supplement the words related to level of agreement, as imagery of facial emotions is more concrete and easier for younger children to comprehend [24].
Section Three measured children’s self-reported behaviour related to their computer use. As there is currently no valid and reliable questionnaire assessing children’s behaviours relating to computer use, behaviour items were developed based on the recommendations by Cale [25] relating to measuring self-reported physical activity. Additional guidance was gained by reviewing existing physical activity measures [26, 27]. A ‘segmented day’ format was used, that required children to recall activities before, during, and after school; rather than during the day as a whole. This method provides time-related cues, which has a memory enhancing effect that can reduce errors in self-reported measurement. Respondents were asked to report on habitual or typical daily events, with wording such as: ‘think of a normal day last week’, as this also increases accuracy of recall [11]. One-day recall is more accurate for children, but three to four days of information about an activity are necessary for a representative profile of activity levels [11]. However, it was beyond the scope of the current questionnaire to assess more than one day of typical activity.
The presence and location of musculoskeletal discomfort was assessed using a Body Discomfort Chart [28], which has been used to assess discomfort in children in previous studies [29, 30].
Psychometric properties
Content validity
The experts reviewed Version 1 of the questionnaire, and their quantitative rating scores and qualitative feedback together with the pilot study on the 13 students (Phase Three) led to an improved Version 2 of the questionnaire. Items were removed if they received ratings of 1 or 2 out of 4 for relevance and clarity from the experts, or qualitative feedback indicating fundamental faults in a specific item and/or lack of supporting evidence for the corresponding guideline. Six items were removed in this phase, and the formatting of the measure was also altered to reduce the risk of missed items and to improve clarity. Version 2 of the questionnaire was distributed to 364 students.
The responses to the items in the Knowledge section are summarised in Table 2. Knowledge pertaining to the different constructs measured varied considerably among the children, with the percentage of correct answers ranging from 17.3% to 93.7%. A review of these responses was useful in developing the final version of this section of the questionnaire; with eight of the items being subsequently dropped; three items were slightly altered in their possible responses (one option was dropped for each question); and one new item was added (concerning action taken in the event that the computer monitor could not be seen clearly). The final Version 3 of the HCQC included a total of 18 knowledge items.
Factor analysis was used to identify any grouping of the Attitude items that may exist, and to identify any of these items that appear not to contribute to the various aspects of attitude. This analysis identified three main factors that accounted for 32% of the variance in the data for the attitude items. The scree plot of the attitude items is shown in Fig. 2, and shows that the factors beyond the third contribute small and relatively similar amounts to the total variance.
The factors were interpreted according to the underlying attitude theories and any overt themes. Table 3 describes the constructs corresponding to the three main factors. The first factor consisted of eight items (Table 4) and may be interpreted as measuring ‘external influences on healthy computer use’.
Internal consistency
Cronbach’s alpha was applied to the groups of items identified in the first three factors for theattitude items. The purpose of these analyses was to show whether these subsets of items were essentially aligned with the construct that they were meant to assess. Cronbach’s alpha for the attitude items were in the range 0.6–0.8 (Table 5), indicating that these items were generally well aligned in their groups to measure the construct as defined.
The internal consistency of the behaviour questions was not assessed, as these questions did not fall into natural groupings. The behaviour questions were asked in order to obtain essential demographic data about the respondent and to assess their behaviour profile with respect to computer use, experience of any discomfort that may be related to computer use, and their physical activity habits.
Test-retest reliability
Test-retest analysis was performed on responses from 63 students, using the final version of the HCQC. Table 6 shows the percentage of each question in the Knowledge section that was correctly answered at the initial administration and at re-test two weeks later. There was no net change in correct responses for nine questions; a move towards the correct answer in seven questions; and for two questions, a small move away from the correct answers.
The knowledge questions were re-coded to classify responses as either ‘incorrect’ or ‘correct’, and then Cohen’s Kappa statistic was calculated to compare agreement between responses given at the pre- and post-tests (see Table 6). These indicate that agreement was not particularly strong between pre- and post-test. This is partly due to the distribution of responses for the questions. Where Kappa was very low (near zero), there was often very high agreement. For example, for question 11 in the Knowledge section, 58/63 of the responses were in agreement (either both answers correct or both incorrect), but Kappa was -0.04. This occurred because the Kappa statistic measures agreement in excess of that expected by chance alone, and when the majority of respondents obtained the correct answer initially, it was expected that they would also do so at post-test.
Test-retest analysis was also performed on the responses to items in the Attitude section of the HCQC as shown in Table 7. The weighted Kappa for most questions indicated good agreement in responses when given two weeks apart.
Finally, Table 8 shows the measures of agreement for selected demographic and behaviour questions (where calculation was possible). Agreements varied from 76% to 100%. Values of Kappa were generally high, but were low on occasions where agreement on any one particular response was very high. When reading data in Tables 7 and 8, it should be noted that when the response to an item is not binary, weighted Kappa is used and the pre- and post- responses which differ by 1 point are considered to be in ‘close agreement’.. For binary responses, agreement must be exact and simple Kappa is quoted.
Discussion
Clinical utility
The HCQC has excellent clinical utility. It is a paper-based, self-report questionnaire, presented in a standardised format. It can be administered to large samples, by multiple administrators, and in a variety of contexts. It is cost-effective, and only takes an average of 20 minutes to complete, which is important so that children do not fatigue or lose interest. There is the potential for development of an online version of the measure. This would reduce the time required to collate scores, and allow administration to geographically diverse populations, including children in rural and remote communities. Scoring the HCQC is relatively simple, as each section provides information about a number of constructs, i.e. knowledge can be assessed in the areas of computer monitor, computer chair, etc.
Psychometric properties
Content validity
On average the experts rated the items between 3 (relevant) and 4 (extremely relevant). This indicates that the items were relevant to the outcomes being measured, and that the questionnaire was determined to have good content validity.
Construct validity
A questionnaire is considered to have good construct validity if factors align with the underlying constructs used to develop the items. In the case of the HCQC the factors did not align with the constructs from the attitude models. This is most likely because items were based on attitudes models that were developed for other health constructs, such as smoking or physical activity. Ideally, the attitude items should have been based on research about what influences children’s attitudes towards healthy computer use, but, as there is currently no evidence available, the next best option was to develop the attitude items in the HCQC based on existing models, and adapt them for the purpose of this measure. The results identified three factors which explained a modest total of 32% of the total variance. This means that the remaining variance in the dataset was explained by combinations of items which each made small contributions, and were not easily identifiable with particular constructs. The constructs which were identified, however, could be associated with interpretable themes (the items appeared to belong together).
Internal consistency
Cronbach’s alpha should be between 0.70 and 0.90 to indicate good internal consistency of items in ameasure [13]. The values for the 3 factors identified from Factor analysis of the attitude items were at the low end of this range (0.63–0.76), and were comparable to other health measures [14, 31]. Therefore the internal consistency for the attitude items was considered to be adequate. Healthy desktop computer use encompasses a wide variety of constructs, ranging from the placement of the computer monitor and keyboard, to sitting posture, to the total duration and number of breaks taken during computer use. Therefore, the knowledge items did not lend themselves to factor analysis, but that section of the questionnaire did collect information about a wide range of ergonomics risk factors.
Test-retest reliability
When interpreting test-retest results, it is generally recommended that Kappa values above 0.75 indicate excellent agreement, values between 0.40 and 0.75 show moderate agreement, and values below 0.40 indicate poor agreement [13]. Some authors have taken a value of 0.70 or higher to indicate good test-rest reliability [14].
Test-retest reliability for the current measure was poor to moderate for the Knowledge and Attitude sections overall, with coefficients varying considerably. Due to the way that Kappa is calculated, there were some occasions where its value was very low, but agreement was actually very good. This occurred when agreement on one particular response (either correct or incorrect for the knowledge questions, for example) was very high, leaving very few responses in disagreement. This is an artefact of the calculation, and in many cases, the percentages of agreement at test and re-test were very high for these cases. For example, cross-tabulation of participant responses for the fifth item in the Knowledge section identified that over 92% of responses were correct on both occasions. However, Kappa for this item was 0.31, indicating poor agreement. Kappa measures the agreement that is observed above that which would be expected by chance alone, and in this case, a high degree of agreement by chance alone is expected because of the large number of respondents who obtained the correct answer on each occasion. Thus, some items had excellent test-retest reliability, but this was not necessarily reflected in Kappa. This is a good example of how researchers should not rely on statistics alone to evaluate a questionnaire.
Limitations
The use of a convenience sample meant the questionnaire was validated on children from higher socio-economic areas in the metropolitan area of Perth, Western Australia. Therefore results may not be applicable to children from households with lower socioeconomic status (SES) or rural areas.
Since its development, the HCQC has been used with 537 school children to determine their knowledge of, and attitudes towards, healthy computer behaviour, including those from high, medium and low SES, and rural communities [31]. In that study, there were no significant differences in the Knowledge scores among the survey respondents based on their SES (p = 0.526). There were also no differences in responses to Attitudes items based on SES for the constructs: ‘Evaluation and Perceived Severity of Consequences’; ‘Response Efficacy’; ‘Self Efficacy’; ‘Motivation to Comply with Others’, and ‘Motivation to Learn Healthy Computing behaviour and Willingness to Disclose Discomfort’. With regard to the attitude construct ‘Perceived Likelihood of Consequences’, significantly more children from high SES schools believed using a computer could result in sore muscles, than children from low SES schools (p = 0.018). Also significantly more children from high SES homes than children from low SES homes believed their parents worry about their healthy use of computers in one of three items from the construct ‘Normative Beliefs’ (p = 0.008).
The knowledge items in the HCQC focus on healthy use of a desktop computer. Since the development of the measure, laptops and tablets are increasingly being used around the world by children enrolled in schools that offer 1 : 1 programs; wherein the students have and use their own portable electronic learning devices in everyday learning activities, in and out of the classroom [33]. The methodology described in this paper can be used as a model to create items to assess knowledge of healthy laptop or tablet use. In addition, the knowledge and behaviour sections of the questionnaire can easily be adapted to any type of computer device.
Conclusion
The Healthy Computing Questionnaire for Children (HCQC) was developed using a rigorous methodology and is based on a health promotion framework. It has adequate psychometric properties, pertaining to content validity and test-retest reliability. This methodology could be used to as an exemplar to develop further outcome measures related to healthy technology use among children. This will allow accurate appraisal of the effectiveness of intervention programs promoting healthy technology use. This is a key step in preventing physical health problems associated with computer use in children, and ensuring future generations do not acquire health problems that limit their ability to participate in current and future occupational roles.
Conflict of interest
The authors declare there are no conflicts of interest with the funding body for this project, and all authors have no financial ties with any private company to disclose.
Footnotes
Acknowledgments
The authors wish to thank the Catholic Education Commission of Western Australia for facilitating access to the students who participated in this study. A research grant from the School of Occupational Therapy and Social Work at Curtin University supported this study.
