Healthy Outcomes for Teens Project: Diabetes Prevention Through Distributed Interactive Learning

Abstract

Aim and Methods:

This study assessed whether distributed interactive learning via web-based modules and grounded in schema and social cognitive theory (treatment group, n = 101) would increase knowledge about diabetes prevention in adolescents from three middle schools to a greater extent than the control group (n = 80) and examined whether the school environment used to convey the education had an effect.

Results:

The treatment group showed substantially greater increases in overall and individual modular content knowledge, with 72 voluntarily choosing to retake evaluations that significantly improved their scores. The treatment (t[3.8], β ≥ 0.30, P < 0.001) and format of delivery (after school, pull out from physical education, or health education curriculum) (t[3.41], β ≥ 0.24, P < 0.001) influenced knowledge.

Conclusions:

Thus, distributed interactive learning was more effective than its passive counterpart, and a more structured delivery enhanced knowledge, as did opportunities to self-regulate learning. Attention to these process components will facilitate effective interventions by educators in schools.

Introduction

M ore than 206,000 people younger than 20 years of age have diabetes.¹ In parallel to diabetes, the prevalence of obesity among children 6–11 years of age has more than doubled in the past 20 years² and alarmingly tracks into adulthood.³ Some evidence exists that adolescents are aware of diabetes, but often misinterpret it as only an adult disease.⁴ A survey of high school students found a high rate of self-reported risk factors for diabetes, but fewer than half equated obesity, family history, or physical inactivity with risk for this disease.⁵ Given the preventable nature and lack of knowledge associated with diabetes, there is a need for effective physical activity and eating behavior interventions. Health-related interventions attempting to enhance awareness and knowledge can take place in varied forums: (a) school;^6

–12 (b) home;¹³ (c) community;^14,15 and (d) via the Internet.^15

–18 Our work in this research focuses on the latter, but in school settings. With limited school resources, Internet-based curriculum provides flexibility and a connection with the home environment lacking in many studies. Furthermore, almost all middle school-aged individuals have access to the Internet, as it is a valuable learning tool currently shaping teaching and learning in the 21^st century.^19,20

In order to investigate the effectiveness of using Internet-based learning environments to increase content knowledge regarding the prevention of diabetes, we grounded our work in schema theory. Schema theory—how information is interpreted, stored, and utilized by an individual—is considered the central memory construct of an information processing system.^21,22 Each person has the ability to form schemas on numerous topics, including nutrition and diabetes prevention. Connecting preexisting schemas of eating and nutrition with new information may help to change behavior, therefore increasing the likelihood of disease prevention.^23,24

Within schools, teachers help learners build schemata connecting new ideas with previous experiences. Discussion, role playing, illustrations, and explanations of how knowledge applies to a given situation are some of the strategies used to make and strengthen connections. Research suggests that the schemata formation improves retention through unique assimilations and storage of information. Schematas contain not only the knowledge itself, but also information about how this knowledge should be used.

Learning in web-based environments is divided into two categories: distributed passive learning and distributed interactive learning (DIL).²⁵ Distributed passive learning environments are traditional, linear learning environments containing sequential elements like a slide show. In contrast, DIL environments create an interactive atmosphere that permits universal accessibility, but also self-regulation in which a learner can explore material freely at his or her own pace while simultaneously interacting with the instructor and other learners. DIL environments have been associated with enhanced learning effectiveness and efficiency.

Given the need for diabetes prevention, the purpose of this study was to determine if web-based modules would increase knowledge about diabetes prevention in adolescents through an interactive environment. A secondary purpose of this work was to examine the contributory affects of different forms of interactivity on learning.

Subjects and Methods

The project was completed in two phases: (a) learning module development and (b) implementation of treatment. The study was enacted in a county in the Midwest containing three school districts with an average class size of 17.5 students, 62% of whom received free/reduced lunch. The districts contained similar racial demographics. The study protocols were approved by the Institutional Review Board, and parental consent and child assent were obtained.

Phase 1

In order to reflect participatory program design,²⁶ the researchers formed a teen council of high school students who had diabetes or were at risk for developing diabetes to facilitate web development for use by middle school-aged students. Risk for diabetes was defined as having two or more of the following: (a) self-reported body mass index at or above the 85^th percentile;²⁷ (b) having a sibling with diabetes; (c) having a parent or grandparent with diabetes; or (d) getting little or no exercise.²⁸

The council and researchers used a review and revise strategy to develop the materials for the web-based learning environment within the overall framework of the social cognitive theory.^29,30 The content and outcome measures were identified in a previous study,³⁰ modified to the reading level of the middle school participants, aligned with the formative evaluation, and modified to reflect the triadic reciprocality of the social cognitive theory variables. Specifically, the modules provided an overview of diabetes, energy expenditure, physical activity guidelines and suggestions, food label reading, and food portion control. The treatment website included interactive elements such as videos (observational learning), narrated text (social persuasion), and knowledge/skill-based games (outcome expectancies, self-efficacy), whereas the control website contained only text and images and was therefore void of interactivity. The only exception to this was the final module in which both groups were permitted to play a review game. Repeated internal pilot testing confirmed the functionality of the web materials. Once consensus was achieved among the group, the password-protected website was launched to the server for implementation.

Phase 2

Information was disseminated to students in the 6^th, 7^th, and 8^th grade in three middle schools. Inclusion criteria were being a student in one of these schools and grades, securing parental consent and participant assent, and having diabetes or being at risk for diabetes.

Eighty-five percent of those meeting our inclusion criteria provided consent/assent and were randomly assigned to treatment (interactive web-based modules) or control groups (noninteractive web-based modules). Sample size calculations were based on a power analysis for a 2 (treatment or control) × 2 (measures: pre/post) fixed effect of variance. The criterion for significance (α) was set at 0.05. The analysis of variance is nondirectional (i.e., two-tailed). For an effect size (f) of 0.25, 66 cases per level, with a total of 132 participants, yields a power of 81% (Sample Power version 2.0, 2000; SPSS, Chicago, IL). A 25% dropout rate was included for a recruitment goal of 165 participants. In total, 214 participants were recruited; however, full data sets were secured for 181 participants, 80 in the control group and 101 in the treatment group (Fig. 1). In order to minimize the risk of negative impact on the middle school students, the randomization was conducted slightly differently across the three schools. At School 1, those agreeing to participate in the afterschool program were individually randomized into treatment and control groups. However, in the school setting, the participants were randomized into treatment and control groups by intact class cohorts. Each participant received an MP3 audio player valued at $50 for participation and used to access the podcasts containing diabetes-related content.

FIG. 1.

Consort diagram for the Healthy Outcomes for Teens Project.

Four decisive knowledge tests and one review game were developed for use during the Healthy Outcomes for Teens (HOT) project intervention. The knowledge tests were developed in a similar multiple-choice style, with one correct answer and three incorrect answers based upon a previous intervention for adults.³⁰ The readability of each question, across all of the tests, ranked between 3.6 and 9.5 on the Fletch–Kincaid readability scale.

Session 1 focused on the completion of the pretests and using passwords that permitted simultaneous data collection. In Session 2 participants completed modules 1 and 2; in Session 3 they completed modules 3 and 4. Participants were permitted to return to previous modules. The fourth session focused on the completion of module 5 (a review game) and the post-testing. Treatment group participants could retake the post-test as many times as they wanted. An additional fifth session was offered as a make-up day.

Data were collected via the Internet and secured in an Excel (Microsoft, Redmond, WA) database, which were then exported into SPSS for analysis (version 17.0, 2008; SPSS). Among the variables included in the database were (a) demographics, (b) knowledge test scores, (c) frequency of the HOT Project website use, and (d) diabetes risk questionnaire. Analysis of variance was used to confirm that there were no differences in pre-test scores across the control and treatment groups and that random sampling techniques had resulted in the two groups having similar knowledge of content (mean_control = 4.75, SD = 1.85; mean_treatment = 5.07; SD = 1.94; P = 0.61).

An analysis of variance was used for post-test data comparison. A hierarchical regression was used to identify the contributory effect of each module, specifically the different forms of interactivity, to overall knowledge. Pearson product-moment correlation coefficients were calculated to determine which demographic variables were associated with the overall knowledge. Variables with significant correlations were then included in the hierarchical regression, which was previously described. A repeated-measures analysis was used to examine the potential influence of this retaking of tests on outcomes. An α level of 0.05 was used for all quantitative analyses.

Results

Knowledge pre-test scores showed no significant differences between genders (males = 4.78, females = 5.00; P = 0.43) or across the three schools (control, P = 0.40; treatment, P = 0.17). Paired sample t tests of post-test data revealed that both the control and treatment groups significantly increased their knowledge of content (mean_control = 5.96, SD = 2.23, P < 0.05; mean_treatment = 8.54; SD = 3.92; P ≤ 0.001). However, the treatment group showed substantially greater increases in overall and individual modular content knowledge over their control counterparts (Table 1). Additionally, there was a significant difference between School 2 and School 3 in post-test scores (mean_{School 2} = 6.58, SD = 2.98; mean_{School 3} = 8.68; SD = 3.29; P ≤ 0.001).

Table 1.

Treatment Versus Control Content Knowledge

	Control		Treatment
	Pretest (n = 181)	Post-test (n = 181)	Pretest (n = 181)	Post-test (n = 181)	Final-pre (n = 79)	Post-post (n = 74)
Module 1	0.60 ± 0.72	0.96 ± 0.72^*	0.68 ± 0.72	1.39 ± 0.79^*	1.85 ± 0.45	1.86 ± 0.45
Module 2	0.78 ± 0.65	1.00 ± 0.64^**	0.85 ± 0.74	1.28 ± 0.81^**	1.87 ± 0.40	1.82 ± 0.47
Module 3	2.36 ± 1.36	2.71 ± 1.47^*	2.39 ± 1.22	4.43 ± 2.08^*	6.51 ± 1.07	6.26 ± 1.16
Module 4	2.09 ± 1.21	2.69 ± 1.43^*	2.18 ± 1.09	3.85 ± 1.86^*	5.64 ± 0.87	5.42 ± 0.95
Module 5	1.39 ± 0.79	1.65 ± 0.76^**	1.44 ± 0.96	2.00 ± 1.05^**	2.89 ± 0.32	2.76 ± 0.60
Overall knowledge	4.75 ± 1.85	5.96 ± 2.23^*	5.07 ± 1.94	8.55 ± 3.52^*	11.29 ± 2.90	11.69 ± 2.54

Data are mean ± SD values.

P < 0.01, **P < 0.05.

Only participants within the treatment group were allowed to retake knowledge tests, with 72 of the 101 participants electing to retake these tests. To determine the impact of feedback on the learning experience, the number of attempts to complete the knowledge test was tracked. The final pre-test score (mean = 11.29, SD = 2.90) was significantly higher (t [1,79] = − 18.10, P ≤ 0.001) than the first pre-test score (mean = 5.09, SD = 1.96) among the treatment group, quantifying the effects of retaking the knowledge test. The participants significantly increased (F[1,63], P ≤ 0.001) his or her post-post test score (mean = 11.92, SD = 2.32, best retake score) over the post-test (mean = 9.39, SD = 3.41), but the value was not significantly different from the final pre-test score (mean = 11.36, SD = 2.49, best retake score; P = 0.12).

Correlations (Table 2) indicated that school (r = 0.30), assignment (r = 0.39), number of test retakes (r = 0.24), and post-post-test score (r = 0.51) reached significance (P < 0.01) with post-test scores. The variables of gender (r = − 0.72, P = 0.34), diabetes risk (r = −0.66, P = 0.38), and exercise (r = −0.11, P = 0.14) were not significantly correlated with post-test scores. The first step of the regression analysis regressed the content knowledge measures (post-test performance) on to the demographic and learning experience variables and exhibited a significant relationship: adjusted R ² ≥ 0.19, F(8,169) ≥ 6.31, P < 0.001. This result indicated that the treatment (HOT Project curriculum) (t[3.8], β ≥ 0.30, P ≤ 0.001) and format of delivery (after school, pull out from physical education, or health education curriculum) (t[3.41], β ≥ 0.24, P ≤ 0.001) influenced the acquisition of content knowledge. The variable measuring test retakes was not a significant contributor to the post-test performance (t[1.24], β ≥ 0.09, P = 0.22). In the second step, content knowledge was regressed on school, assignment, module 1, module 2, module 3, module 4, and module 5. Specifically, the second step exhibited a significant relationship: adjusted R ² ≥ 0.20, F(1,178) ≥ 23.60, P ≤ 0.001, which included the variables of assignment and school, therefore suggesting that the treatment was effective and influenced by the format of delivery.

Table 2.

Pearson Product-Moment Correlations

Variable	1	2	3	4	5	6	7	8
1. Post-test score	1.00	0.51^**	0.39^**	0.31^**	−0.07	−0.07	0.23^**	−0.11
2. Post-post-test score	0.51^**	1.00	—	0.22	0.00	−0.13	0.07	−0.04
3. Assignment	0.39^**	—	1.00	0.22^**	−0.16^*	−0.07	0.37^**	−0.19^**
4. School	0.31^**	0.22	0.22^**	1.00	0.110	−0.07	0.18^*	−0.05
5. Gender	−0.072	0.00	−0.16^*	0.11	1.00	0.12	−0.13	0.15^*
6. Diabetes risk	−0.07	−0.13	−0.07	−0.07	0.12	1.00	0.09	0.63^**
7. Number of test retakes	0.24^**	0.07	0.37^**	0.18^*	−0.13	0.09	1.00	0.04
8. Amount of physical activity	−0.11	−0.04	−0.19^**	−0.05	0.15^*	0.64^**	0.04	1.00

Post-test scores were collected at the last session; post-post-test scores were collected at the last or make-up session; assignment represents either control or treatment group; school represents three schools.

P < .05, **P < .01.

Discussion

To prevent overweight or diabetes, risk individuals must have some working knowledge about healthy eating and physical activity.³¹ Telecommunications are one way that adolescents access and become aware of information.²⁰ As predicted, the adolescents who experienced highly interactive modules learned more than those who did not. Their scores might be considered failing in a class (46% or 66% correct), but it is important to consider that the treatment group did significantly improve their knowledge test scores over their control counterparts. These scores demonstrate the importance of diabetes prevention knowledge being disseminated to this population because they lack declarative knowledge. This knowledge deficit has also been reported for a small group (n = 21) of adolescents and their parents, leading to the conclusion that improving diabetes risk knowledge was a critical public health need.³² In another study, high schoolers also had little self-awareness of their own high risk for diabetes status.⁵ When high schoolers were taught using either a traditional curriculum or an online curriculum, both groups improved knowledge, although the computer-based knowledge scores were significantly better,³³ similar to our results.

An unexpected outcome was the element of test retaking. Participants who retook post-tests received an average score of 89.9%. Indeed, mastery is one of the four primary strategies for self-efficacy improvement²⁸ and as such has specific pedagogical ramifications, which can be supported easily online.

In accordance with previous literature^34,35 the HOT project website utilized DIL learning environments, thus creating independent, self-regulated learners. DIL, or edutainment, is increasingly prevalent in both the learning community and personal or home environment.³⁶ The presence of interactive elements such as a competitively social atmosphere, high engagement, and reinforcement resulted in the creation of unique schemata about the knowledge and its application. As predicted, the more interactive the module, the greater the “kid appeal.” Adolescents enjoyed replacing traditional school with creative interactive elements. This is similar to a group of adolescents with type 1 diabetes who favorably reviewed three computer games targeting self-management of their diabetes.³⁷ Interestingly, in the present study, no single module had a greater impact than another, as it was more the collective interactivity (presences of video, audio, gaming, and voiced-over text) that led to enhanced learning. Given these findings, it may be worthy to explore if there is an upper limit to the amount of interactivity that may be included in a learning experience, or a dose–response to the depth of this exposure similar to that found with multicomponent school interventions not using online venues.¹¹ Disease prevention knowledge can be improved, but it remains unclear if there is a point when learning is compromised because the interactivity is a distraction. From another perspective, researchers have found no correlation between ratings of software fun and knowledge gained.³⁸

Self-regulated learning includes strategies such as planning to study, rehearsal of content, elaboration of the topic, monitoring oneself for correct responses, and regulating behavior relative to the task.³⁹ The use of this web-based intervention, in the treatment website, promoted self-regulated learning.⁴⁰ Participants who required more time to comprehend the materials were allowed to go back through modules. The treatment group could retake test questions, which implied they could use the tests as a monitor of correct responses rather than as a “grade.” This type of situation decreases explicit evaluation of the student and social comparison, which may be especially important to middle school children.³⁹

The self-regulation of learning and the creation of a social atmosphere are both desirable attributes for this age group³⁹ and add value to the learning process. When young adults perceive the information as meaningful, they have a tendency to increase the regularity and intensity of their engagement in the learning process.⁴⁰ As evidenced by the frequency of hit counts outside of class, these learning modules created a value-added learning experience that may have influenced knowledge. Structure within the instructional delivery and support for autonomous learning were both present in the intervention and have been shown to enhance self-regulated learning.⁴¹

With class size growth, it is becoming increasingly difficult to provide quality instruction, particularly in the area of health education. School expectations include not only academic achievement, but social development and health promotion. The diversity of needs may be overwhelming for teachers and school systems.⁴² Web-based learning can be used as an effective method of instruction in a teacher's toolbox.

Findings from this study suggest that the delivery format (after-school programming, pull out from physical education, or during health education) also affects learning. Those participants who completed the modules as part of the health education, disease prevention unit displayed the most knowledge gains. This might represent the venue with the most structure, which has been shown to enhance self-regulatory learning.⁴¹ We know from previous research that those students who spend out of class time on the content are more likely to have better comprehension and application at the end of the course.⁴³ These findings suggest that the use of an interactive website should be utilized as a supplemental learning for teachers. Given that health education curricula are intended to improve the dietary patterns that contribute to disease among adolescents, it is not surprising that this was the venue where the most learning took place.

No research project is conducted without its limitations. Changes in behavior were not measured and should be included in future studies. However, knowledge is important in effecting behavior change, although not the only determinant. Studies that have not assessed knowledge in nutrition and middle school environments cannot evaluate the curriculum effectiveness.¹⁰

Conclusions

This study provides a unique look into the usefulness of applying online interactive technologies to an adolescent population. Clearly, DIL was more effective than its passive counterpart. A more structured delivery time, within a health class, led to enhanced knowledge, as did opportunities to self-regulate learning for both genders. Being able to retake tests in itself is a learning opportunity and can increase knowledge. However, continued research needs to quantify the effects of increased knowledge on behavior change, such as improvement in dietary choices, greater engagement in physical activity, or enhanced self-efficacy toward making healthy choices. Furthermore, it remains unclear how outside of class access and the download of audio files directly influence learning, as we only recorded hit counts.

With limited time and resources, educators need to choose wisely from an array of educational programs, including content and venue. Research has demonstrated the value of using multiple, varied approaches to enhance outcomes with repetitive, reinforcing messages. This study examined the process aspects of an educational program, showing that DIL learning with a self-paced component is effective, especially within the framework of a health class.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

References

Centers for Disease Control and Prevention: National Diabetes Fact Sheet. General Information and National Estimates on Diabetes in the United States. 2003 rev.. Atlanta: Centers for Disease Control and Prevention, 2004.

Ogden

, Flegal

, Carroll

, Johnson

. Prevalence and trends in overweight among US children and adolescents. JAMA, 2002; 288:1728–1732.

Ferraro

, Thorpe

, Wilkinson

. The life course of severe obesity: does childhood overweight matter? J Gerontol, 2003; 58:110–119.

Hains

, Berlin

, Davies

, Smothers

, Sato

, Alemzadeh

. Attributions of adolescents with type 1 diabetes related to performing diabetes care around friends and peers: the moderating role of friend support. J Pediatr Psychol, 2007; 32:561–570.

Mahajerin

, Fras

, Vanhecke

, Ledesma

. Assessment of knowledge, awareness, and self-reported risk factors for type II diabetes among adolescents. J Adolesc Health, 2008; 43:188–190.

Zenzen

, Kridli

. Integrative review of school-based childhood obesity prevention programs. J Pediatr Health Care, 2009; 23:242–258.

Robinson-O'Brien

, Story

, Heim

. Impact of garden-based youth nutrition intervention programs: a review. J Am Diet Assoc, 2009; 109:273–280.

Dobbins

, De Corby

, Robeson

, Husson

, Tirilis

. School-based physical activity programs for promoting physical activity and fitness in children and adolescents aged 6–18. Cochrane Database Syst Rev, 2009; 1:CD007651.

Reynolds

, Franklin

, Leviton

, Maloy

, Harrington

, Yaroch

, Person

, Jester

. Methods, results, and lessons learned from process evaluation of the high 5 school-based nutrition intervention. Health Educ Behav, 2000; 27:177–186.

10.

Lytle

, Murray

, Perry

. School-based approaches to affect adolescents' diets: results from the TEENS study. Health Educ Behav, 2004; 31:270–287.

11.

Birnbaum

, Lytle

, Story

, Perry

, Murray

. Are differences in exposure to a multicomponent school-based intervention associated with varying dietary outcomes in adolescents? Health Educ Behav, 2002; 29:427–443.

12.

Schwartz

, Novak

, Fiore

. The impact of removing snacks of low nutritional value from middle schools. Health Educ Behav, 2009; 36:999–1011.

13.

McCall

, Raj

. Exercise for prevention of obesity and diabetes in children and adolescents. Clin Sports Med, 2009; 28:393–421.

14.

Wilson

. New perspectives on health disparities and obesity interventions in youth. J Pediatr Psychol, 2009; 34:231–244.

15.

Story

, Kaphinst

, Robinson-O'Brien

, Glanz

. Creating healthy food and eating environments: policy and environmental approaches. Annu Rev Public Health, 2008; 29:253–272.

16.

Doyle

, Goldschmidt

, Huang

, Winzelberg

, Taylor

, Wilfley

. Reduction of overweight and eating disorder symptoms via the Internet in adolescents: a randomized controlled trial. J Adolesc Health, 2008; 43:172–179.

17.

Thompson

, Baranowski

, Cullen

, Watson

, Canada

, Bhatt

, Liu

, Zakeri

. Food, Fun and Fitness Internet program for girls: influencing log-on rate. Health Educ Res, 2008; 23:228–237.

18.

Reuters: American Youth Trail Peers in Internet Usage. Magazine

. [serial on the Internet] November 25, 2008. www.pcmag.com/article2/0,2817,2335412,00.asp. 2009 December 3.

19.

Guan

, Subrahmanyam

. Youth Internet use: risks and opportunities. Curr Opin Psychiatry, 2009; 22:351–356.

20.

Schmidt

. A schema theory of discrete motor skill learning. Psychol Rev, 1975; 82:225–260.

21.

Shea

, Wulf

. Schema theory: a critical appraisal and reevaluation. J Mot Behav, 2005; 37:85–101.

22.

Mayer

. Thinking, Problem Solving, Cognition. 2 ^nd. New York: W.H. Freeman and Co., 1992.

23.

Israeli

, Stewart

. Memory bias for forbidden food cues in restrained eaters. Cogn Ther Res, 2001; 25:37–47.

24.

Anderson

, Spiro

, Anderson

. Schemata as scaffolding for the representation of information in connected discourse. Am Ed Res J, 1978; 15:433–440.

25.

Ruland

, Starren

, Vatne

. Participatory design with children in the development of a support system for patient-centered care in pediatric oncology. J Biomed Inform, 2008; 41:624–635.

26.

Daniels

. The use of BMI in the clinical setting. Pediatrics, 2009; 124,Suppl 1:S35–S41.

27.

Magge

, Stettler

, Jawad

, Levitt Katz

. Increased prevalence of abnormal glucose tolerance among obese siblings of children with type 2 diabetes. J Pediatr, 2009; 154:562–566.

28.

Bandura

. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs, NJ: Prentice Hall, 1986.

29.

Northrup

. Concurrent formative evaluation: guidelines and implications for multimedia designers. Ed Tech, 1995; 35:24–31.

30.

Herrejon

, Hartke

, Scherer

, Chapman-Novakofski

. The creation and impact evaluation of “Your Guide to Diet and Diabetes,” an interactive web-based diabetes tutorial. Diabetes Technol Ther, 2009; 11:171–179.

31.

Katz

. School-based interventions for health promotion and weight control: not just waiting on the world to change. Annu Rev Public Health, 2009; 30:253–272.

32.

Cullen

, Buzek

. Knowledge about type 2 diabetes risk and prevention of African American and Hispanic adults and adolescents with family history of type 2 diabetes. Diabetes Educ, 2009; 35:836–842.

33.

Casazza

, Ciccazzo

. The method of delivery of nutrition and physical activity information may play a role in eliciting behavior changes in adolescents. Eat Behav, 2007; 8:73–82.

34.

Kahalifa

, Lam

. Web-based learning: effects on learning process and outcomes. IEEE Trans Ed, 2002; 45:350–356.

35.

Wouters

, Tabbers

, Pass

. Interactivity in video-based models. Ed Psychol Rev, 2007; 19:327–342.

36.

Moreno-Ger

, Burgos

, Martínez-Ortiz

, Sierra

, Fernández-Manjón

. Educational game design for online education. Comput Hum Behav, 2008; 24:2530–2540.

37.

Aoki

, Ohta

, Masuda

, Naito

, Sawai

, Nishida

, Okada

, Oishi

, Iwasawa

, Toyomasu

, Hira

, Fukui

. Edutainment tools for initial education of type 1. Stud Health Technol Inform, 2004; 107:855–859.

38.

Sim

, MacFarlane

, Read

. All work and no play: measuring fun, usability, and learning in software for children. Comput Educ, 2006; 46:235–248.

39.

Wang

, Pomerantz

. The motivational landscape of early adolescence in the United States and China: a longitudinal investigation. Child Dev, 2009; 80:1272–1287.

40.

Zhao

, Kuh

. Adding value: learning communities and student engagement. Res Higher Educ, 2004; 45:115–138.

41.

Sierens

, Vansteenkiste

, Goossens

, Soenens

, Dochy

. The synergistic relationship of perceived autonomy support and structure in the prediction of self-regulated learning. Br J Educ Psychol, 2009; 79:57–68.

42.

Greenberg

, Weissberg

, Utne

O'Brien M

, Zins

, Fredericks

, Resnik

, Elias

. Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning. Am Psychol, 2003; 58:466–474.

43.

Kuh

. The other curriculum: outside of class experiences associated with student learning and personal development. J Higher Ed, 1995; 66:123–155.