Abstract
Background:
Flexible (or functional) insulin therapy method is a self-management education approach for intensive insulin therapy in patients with type 1 diabetes. The serious game (or applied game) “L'Affaire Birman” (“Mr. Birman's File”) (available at
Materials and Methods:
This prospective multicenter pilot study evaluated the effect of this videogame on the therapeutic knowledge and behavior of children with type 1 diabetes. PedCarbQuiz (PCQ) and Diabetes Self-Management Profile (DSMP) questionnaires were used before (T0), immediately after (T1), and 6 months after (T2) the unstructured use of the videogame.
Results:
The 38 children enrolled in the study were 42% boys and 58% girls; they had a mean age of 13.7 ± 2.1 years old, a diabetes duration of 6.0 ± 3.8 years, and hemoglobin A1c (HbA1c) levels of 8.5 ± 1.4% (69.4 ± 9.4 mmol/mol). The children connected to the game 3.3 ± 2.8 times during this 6-month study. Their PCQ score increased from 31.6 ± 4.9 at T0 to 36.0 ± 4.0 at T2 (P < 0.05). Two PCQ subscores also increased significantly: the insulin titration score at T1 and T2 and the carbohydrate quantification score at T2. Conversely, the DSMP score was not different at T0, T1, and T2 (59.1 ± 9.9, 60.2 ± 9.8, and 60.0 ± 10.0, respectively), and HbA1c levels also remained stable throughout the study (8.4 ± 1.3%, 8.4 ± 1.2%, and 8.5 ± 1.5% at T0, T1, and T2, respectively). Subgroup analysis found a greater impact of the game in children with poor glycemic control and low knowledge at baseline. Adherence to the game was rather low (half of the children played less than 2.5 bouts), but no criterion was found to be predictive of this low attractiveness.
Conclusions:
Nonsupervised usage of the serious game “L'Affaire Birman” was able to improve insulin titration and carbohydrate quantification in children with type 1 diabetes.
Introduction
T
Flexible (or functional) insulin therapy (FIT) is considered a reference education method for managing intensive insulin therapy. 3 Two studies assessing the FIT method have demonstrated an improvement in quality of life and a greater freedom in food consumption, together with a stable glycemic control and no increase in the occurrence of hypoglycemia. 3,4 However, education programs focusing on FIT were poorly developed due to several limitations, including the use of nonintensified insulin regimens with premixed insulin, the lack of educational involvement of healthcare professionals, the scarcity of education facilities or educational tools, and the fear of hypoglycemia. 5 A serious (or applied) game is a game designed for a primary purpose different from pure entertainment. Such software products were previously developed in the general setting of T1D but were not yet devoted to FIT learning. 6
To address this question, we developed the serious videogame “L'Affaire Birman” (“Mr. Birman's File”), specially designed for education of children and adolescents on FIT. We report here the LUDIDIAB study designed to evaluate the impact of the videogame “L'Affaire Birman” in a population of young T1D patients.
Materials and Methods
LUDIDIAB was a prospective, multicenter pilot study conducted in the pediatric departments of four French university hospitals. The primary objective was to investigate if the usage of the serious game “L'Affaire Birman” increases the amount of knowledge and skills of young T1D patients. Secondary objectives were to determine the game's adherence and satisfaction, hemoglobin A1c (HbA1c) change, and the number of episodes of severe hypoglycemia (defined as hypoglycemia requiring assistance) and emergency hospitalizations related to diabetes.
The serious game “L'Affaire Birman”
This serious game is a Web-based free videogame, the medical content of which was designed by an academic diabetes care team, including physicians, dieticians, nurses, and expert patients with diabetes. Game play, graphic design, soundscape, and story were created by a multimedia development team. The theoretical background for the “situated learning” design of this game was largely inspired by the conceptual model for the design of a serious game from Thompson et al. 7
“L'Affaire Birman” is indeed based on problem solving about diabetes: the player has to manage the T1D of Alex, the main character of the game, during a playful investigation. Interaction is made possible by a semiquantitative glycemic simulator specially developed by our team for this game; the player adapts insulin dose injection for each meal according to Alex's parameters of FIT. Glucose level, physical activity, and carbohydrate intake have to be processed for insulin dose calculation. In case of inadequate insulin dose, hypoglycemia or hyperglycemia may occur and require the use of carbohydrates or rapid-acting insulin to promptly manage the glycemic drift (screenshots of the game are presented in Fig. 1).
Screenshots of the serious game “L'Affaire Birman” (freely available at
Regarding the absence of a consensus statement on the FIT method, the recommendations included in the game mainly relied on the expertise of the healthcare professionals involved in its development. Additionally, experts from the French Diabetes Society (physicians and dieticians) validated the educational content of this game before it was launched. This game is not customizable regarding the type of treatment (insulin pump or multiple daily injections) or the age of the player. However, the game was developed to create interest in boys and girls 11–18 years of age, treated with intensive insulin therapy.
The serious game “L'Affaire Birman” is freely available at
Subjects and protocol
This pilot study involved a population of children with T1D. All subjects and their family provided informed written consent to participate in the study, which was approved by the local ethics committee. All participating centers recruited children among their outpatients' records, according to the following inclusion criteria: age 11–18 years; diabetes duration of ≥12 months; intensive insulin therapy with multiple daily injections or continuous subcutaneous insulin infusion for at least 6 months; and prior therapeutic education about diabetes self-management according to the usual care of each center. Patients with celiac disease, long-term corticosteroid treatment, and/or inflammatory disease were excluded, as were illiterate subjects and those without access to a computer. No specific education was required prior to the study. However, all children had a diabetes duration of ≥12 months, allowing time for each participating center to provide conventional education (fixed insulin doses) or FIT education (carbohydrate quantification to adapt insulin doses) for self-care management of intensified insulin therapy, according to their usual practice.
The PedCarbQuiz (PCQ) and the Diabetes Self-Management Profile (DSMP), two validated questionnaires, were administered to the patients 1–3 months before serious game use (baseline [T0]), 1–2 weeks after first game use (immediate effect [T1]), and after 6 months of ad libitum use of the game (long-term effect [T2]). The Internet link to the game was given to participating children with the only recommendation being to complete the entire game at least one time all the way through until end credits. Children and their families were not taught how to play the game because it was assumed that the tutorial at the beginning of the software would be enough to facilitate use. The use of the game was unstructured; children played independently without supervision by the healthcare study team. Of note is that quarterly routine visits were scheduled (prescription refill, routine monitoring, etc.), but, to avoid bias, no additional education was provided throughout the participation of each subject. A game bout was recorded each time children connected to the game (regardless of the duration of the game). Duration of each game bout and cumulative duration of all game bouts were not collected. PCQ score change from T0 represented the primary evaluation criteria. Secondary evaluation criteria at T0, T1, and T2 were the DSMP score, plasma HbA1c levels, and retrospective assessment of severe hypoglycemia (defined as hypoglycemia episode requiring assistance) and emergency hospitalization related to diabetes.
PCQ
This validated questionnaire was specially designed for children with diabetes. 8 It explores carbohydrate recognition, carbohydrate quantification, and ability to adjust insulin doses according to carbohydrate intake and glycemic level (FIT concept). Three corresponding subscores can be analyzed at the end of the test: carbohydrate recognition, carbohydrate quantification, and insulin titration. The PCQ is a 78-item, self-administered, multiple-choice, paper-based questionnaire requiring 20–30 min to complete. Higher scores indicate a greater degree of knowledge. The maximum PCQ global score is 47 points.
DSMP
We used exactly the DSMP-F version of this questionnaire that was especially created to evaluate patients using the FIT method. 9 This questionnaire is also validated. The DSMP-F interview quantifies adherence to the prescribed regimen and self-management behaviors, such as remediation or prevention of unwanted glucose excursions through adjustment of insulin, diet, or exercise. The maximum DSMP score is 86 points.
Satisfaction questionnaire
A satisfaction questionnaire specifically designed for this serious game was administered to the children at the end of their participation to the study. We asked them to rate, on a 0–10 scale (from 0 = low to 10 = high) their overall satisfaction with this game (acceptance), their wish to play this game (usability/playability), their satisfaction regarding the educational content, and their wish to modify self-care management after playing this game. Additionally, patients could qualitatively evaluate the software in a free response area.
Statistical analysis
The absence of previous results and the exploratory feature of this pilot study did not allow any sample size calculation. Parameters were expressed as mean ± SD values and n (percentage) for quantitative and qualitative variables, respectively. PCQ scores and subscores and DSMP scores were expressed in numerical values and as the percentages of correct answers at each time point. The change of the PCQ score from T0 to T1 was expressed as a relative percentage; the median of this relative percentage was used to define the subgroups “low effect” (LE)/“high effect” (HE) that were below/above this median, respectively. The subgroups “nonusers”/“users” were defined according to the median number of game bouts performed: nonusers/users were below/above this median, respectively. The t test and Fisher's exact test were used for intergroup comparison of qualitative and quantitative data, respectively. An analysis of variance for repeated measures was used for intragroup comparisons of quantitative values at different time points. Prism version 6.0f software (GraphPad Software Inc., La Jolla, CA) was used for statistical analysis. A P value of <0.05 was chosen to denote significance.
Results
Forty-seven children were included in this trial. Nine subjects were excluded from analysis for noncompliance to the protocol (lack of PCQ and/or DSMP questionnaire completion). Finally, we present the results from 38 patients who completed the trial. Their principal baseline characteristics were as follows: age, 13.7 ± 2.1 years old; 42% boys; diabetes duration, 6.0 ± 3.8 years; and HbA1c level, 8.5 ± 1.4% (69.4 ± 9.4 mmol/mol). Other baseline characteristics are presented in Table 1.
Low/High Effects subgroups were defined according to the median value of the PedCarbQuiz (PCQ) score relative to the percentage change after game use; Low/High Effects subgroups were below/above this median, respectively. Nonusers/users were defined according to the median number of game bouts completed; nonusers/users were below/above this median, respectively. Data are expressed as mean ± SD values for quantitative parameters and as the number (percentage) for qualitative parameters.
Intergroup comparison concerns Low versus High effects subgroups and nonusers versus users.
CSII, continuous subcutaneous insulin infusion; DSMP, Diabetes Self-Management Profile; HbA1c, hemoglobin A1c; MDI, multiple daily injection.
During this 6-month study, children used the game 3.3 ± 2.8 times, with a wide range from one time to 15 times. Among the 38 patients, 23 (60%) were able to tell the end of the game scenario, proving they completed the entire game at least once. Mean baseline PCQ scores were not different among the four centers (30.6 ± 5.5, 32.0 ± 5.7, 32.0 ± 4.2, and 34.3 ± 3.6, respectively). For the entire population, the PCQ score increased from 31.6 ± 4.9 at T0 to 33.8 ± 5.0 at T1 (difference not significant) and 36.0 ± 4.0 at T2 (P < 0.05). PCQ subscore analysis showed a significant increase for insulin titration at T1 and T2 and for carbohydrate quantification only at T2 (Fig. 2). The DSMP score was not different at T0, T1, and T2 (59.1 ± 9.9, 60.2 ± 9.8, and 60.0 ± 10.0, respectively; difference not significant) (Fig. 2). HbA1c levels also remained stable throughout the study (8.4 ± 1.3%, 8.4 ± 1.2%, and 8.5 ± 1.5% at T0, T1, and T2, respectively; difference not significant). Severe hypoglycemia occurred in the past 6 months in 10.5% and 13.1% of subjects at T0 and T2, respectively (difference not significant). Hospitalizations in the past 6 months were also stable, involving 15.8% and 13.1% of the subjects at T0 and T2 (difference not significant).
The change of the PCQ score between T0 to T1 ranged from −16.2 to +31.8% with a median of +8.33%. This median value was used to define the LE/HE subgroups: LE (n = 18)/HE (n = 20) had a change of their PCQ score below/above this median value, respectively. The comparison of these two subgroups showed that baseline HbA1c levels were higher in the HE compared with the LE subgroup (9.0 ± 1.4% [74.9 ± 9.4 mmol/mol] vs. 7.8 ± 0.9% [61.7 ± 6.0 mmol/mol], respectively; P < 0.01). Baseline PCQ score was also lower in the HE compared with the LE subgroup (29.9 ± 4.7 vs. 33.4 ± 4.5, respectively; P < 0.05). Children from the HE subgroup also had their parents divorce more frequently, compared with the LE subgroup (35% vs. 6%, respectively; P < 0.05). Other comparisons did not show significant differences between these subgroups (Table 1).
The median number of game bouts performed in the entire population was 2.5. This threshold was used to define the subgroups nonusers/users of the game that were below/above this value, respectively. Comparison of baseline characteristics from nonusers/users subgroups did not show any significant differences (Table 1).
Satisfaction questionnaire
Overall satisfaction with this game (acceptance), wish to play this game (usability/playability), satisfaction regarding the educational content, and wish to modify self-care management after playing this game were rated 6.8 ± 1.3, 5.2 ± 1.7, 5.4 ± 2.4, and 1.4 ± 1.9 out of 10, respectively. Forty percent of patients classified the game as interesting, but 30% complained about a lack of action in the game.
Discussion
This is the first time, to our knowledge, that a serious game has been specially designed to promote FIT education. Furthermore, we have shown that the use of this education support has improved the knowledge of children with T1D, in particular, concerning carbohydrate quantification and insulin dose adaptation. The satisfaction questionnaire also reflected a good acceptance and a mild usability/playability of this game.
Development of serious games for diabetes began two decades ago with “Captain Novolin” and “Packy & Marlon.” These two games relied on the same concept of situational problem solving addressing diet, self-monitoring of blood glucose, exercise, or psychosocial aspects. These games were designed for game console and personal computer to reach a target audience of children with T1D. Small studies that evaluated these educational tools have shown improvement of intermediate qualitative criteria, such as self-efficacy, communication with parents, and self-care behaviors. 10,11 A decrease of unscheduled doctor visits was also found with “Packy & Marlon” in one of the sole randomized controlled trials in the field. 11 Other games targeting T1D education were created after these two first productions: “The DAILY,” “Dbaza,” “Egg Breeder,” “Insulot,” and “Starbright Life Adventures.” All of these software products were mainly assessed by convenience sample pre- and/or post-test showing mild improvement of qualitative criteria (knowledge, engagement, self-efficacy, communication, and self-care behaviors). 6
The results of our study are consistent with previous reports as we found a moderate improvement of dietetic knowledge without changes in the therapeutic behaviors and the metabolic parameters. This modest result may be explained by a lack of statistical power due to the small sample size of this pilot study, lack of effectiveness of the game, or low adherence of children to the game, which was played only three times per patient. This latter hypothesis represents, in our opinion, the main reason explaining the mild results of this intervention. This low adherence may be linked to the unstructured use of the game, which was not implemented in an educational program. We hoped that the motivation to play would be sufficient to promote an adequate use, but integration within an educational path would have been more effective. The difficult integration of such a tool in an existing education program is probably one of the main barriers for the wider use of serious games. This issue stems not only from technical difficulties in using these multimedia materials but also from the reluctance of caregivers with new technologies.
We have shown that the benefit of our game was mainly observed on carbohydrate counting and insulin dose adaptation, in accordance with the type of situational problem presented to the player during the adventure; for each meal (three times per day), the player had to practice determining the amount of carbohydrate visually presented and the appropriate insulin dose. This result is in line with our aim when designing the game for FIT. However, the transfer of knowledge toward diabetes self-management does not appear to operate because the score on the DSMP, a self-management behavior questionnaire, was not improved by this game. We showed a better PCQ score improvement in children with lower baseline knowledge about diabetes, more precarious family structure, and poorer metabolic control. In addition, most of our population had received no previous education regarding FIT. These findings suggest that our game is well suited for beginners inexperienced with FIT.
The frequency of utilization of the game was not a determinant of learning efficacy in this study, as we have shown no difference in game use between LE and HE subgroups. However, in the absence of data on utilization duration, a “dose-effect” of the game may not be ruled out. We also did not find a different impact of the game among boys or girls or according to age or grade level, suggesting that the game may be widely used in children or adolescent populations. This finding is related to the quality of the game design and the situational problems in which players can identify. However, we found that adherence to our game was very variable among subjects, reflected by the number of game bouts performed, ranging from only one to more than 10. Analysis of baseline characteristics of nonuser and user subgroups did not give any clue to understanding the variable attractiveness of our game among children.
A previous study conducted by Lu et al. 12 has shown, for example, that identification may be of paramount importance for game impact; in the game “Escape from Diabetes” (an obesity prevention game), benefits were lower in white children compared with Hispanic and African American players whose ethnic similarity with videogame characters improved story immersion and positive health outcomes. A conceptual model for the creation of serious game targeting education of T1D was published in 2010 and details the main parameters to be taken into account in the development of such a support to maximize its subsequent impact. 7
We also highlight the need for qualitative evaluations of existing interactive tools, as quantitative assessment does not always help improve the attractiveness of these games. Indeed, the qualitative assessment would help to finely define the cognitive process involved, such as the trace-based approach (history of users' action collected in real time from their interaction with the support) that was recently proposed to identify users' engagement and qualify their engaged behaviors in interactive systems. 13
Production of new serious games about T1D appears to have decreased in recent years, which is probably related to their weak medical impact. We have previously mentioned the difficulty of integrating these tools into a treatment course. Other barriers hamper the development of new games. First, there is a major conceptual limitation in the difficult balance between the game itself and the educational content; a game with rich educational content should not be playful and should obtain low player adherence. Conversely, a playful serious game should hinder the educational content and finally be associated with low knowledge transfer. Second, the structural limitations are mainly represented by the technological lability of media for which the videogames are intended, requiring the development of costly multiplatform interoperable software. Finally, medical limitations also hinder the development and use of serious games for T1D; healthcare professionals may be reluctant to use such recreational educative tools without previous control or regulation. The Swiss nongovernmental organization Health On the Net actually tries to address this question and offers a certification of sites and health applications according to a quality label. 14 Other approaches for using serious games look promising, especially for healthcare professional training. The game “InsuOnline,” developed to train general practitioners to the initiation of basal insulin therapy in type 2 diabetes subjects, is currently being evaluated in a randomized study. 15
We acknowledge that our study has several weaknesses. The before–after design without any control group is probably the main pitfall. The study population is also small in this pilot trial, and therefore subgroup analyses on such a small sample size should be cautiously interpreted, regarding potential statistical bias. Subjects were all recruited in university centers. The unstructured use of the game corresponds to real-life conditions but resulted in a heterogeneous use of the tool, which could lead to a bias in the analysis of its effectiveness. Finally, the assessment process may have been skewed by the heaviness of the two questionnaires, PCQ and DSMP, which include dozens of items. A direct evaluation of the answers in the game would certainly have been most suitable, especially to assess the progression of learning in different areas, but such “online monitoring” was not developed because of technical and financial issues.
However, the strengths of our study are the originality of the concept, the focus of the issue on FIT for the first time, the use of the validated PedCarbQuiz and DSMP questionnaires as evaluation criteria, and the assessment over a 6-month period.
The results of this pilot study will allow us to calculate the number of subjects necessary for the assessment of our newest serious game, “Time Out” (
In conclusion, we have shown that the unstructured use of our game “L'Affaire Birman” can slightly improve FIT knowledge in children with T1D. Improvement is higher in children with poor baseline metabolic control and knowledge. The impact of the game integrated in the usual care course with supervision by health professionals remains to be determined in future studies.
Footnotes
Acknowledgments
We acknowledge the following persons for their work and support of this project: The LUDIDIAB study group (V. Ribault, D. Kauffman, C. le Tallec, D. Martin, and C. Stuckens), H. Leray, N. Topor, L. Troncy, I. Duvelleroy, D. Chatelin, C. Thouroude, and K. Rose. The serious videogame “L'Affaire Birman” was funded by the French Health Ministry, Sanofi-Aventis, and Accu-Chek.
Author Disclosure Statement
A.G. is an employee and shareholder of Zippyware Company, which administers the website that hosts the game. M.J., C.A., J.M., L.T., and Y.R. declare no competing financial interests exist.