Evaluation of a web-based lifestyle coach designed to maintain a healthy bodyweight

Healthy Weight Assistant

The Healthy Weight Assistant (HWA) is a web-based intervention developed by the Netherlands Nutrition Centre. The goal is to support people with a healthy weight and those who are slightly overweight (i.e. a body mass index of 18–28 kg/m²) to achieve and maintain a healthy weight. The aim is not to achieve a given weight loss, but to support the achievement of healthy dietary and physical activity behaviour. The development of the HWA was based on a pilot study that assessed the possibilities for an interactive Internet application to support healthy behaviour. ⁶ The theoretical basis of the HWA is the Transtheoretical model. ⁷

The log-on screen of the HWA is shown in Figure 1. The HWA consists of four stages which are marked in the application by a ‘to-do list’ and tabs in the ‘diary’ (Figure 2). When users enter the program for the first time, they start by assessing their baseline status. At this stage the users answer questions about their bodyweight, dietary behaviour, physical activity behaviour and emotions. This results in a tailored advice which can be applied in the next stages of the application. The second stage is motivation and goal setting. Users are asked about their motivation to change behaviour and the application assists them in making these motivations clear to themselves, thereby also focusing on clarifying the emotions related to behaviour. In addition, users are coached to set useful and realistic goals. The third stage is ‘difficult moments’. Users are encouraged to reflect on their difficult moments (i.e. moments at which it is tempting to perform non-healthy behaviour) and to provide solutions for these moments. The HWA coaches the user throughout this stage by giving automated feedback. The final stage is monitoring behaviour and achievement of goals by means of a diary. Users can add dietary and physical activity information in the diary and provide feedback on the achievement of their own goals.

OPEN IN VIEWER

Figure 1

The log-on page of the HWA

OPEN IN VIEWER

Figure 2

My diary

Web-based applications

Web-based interactive applications combine the tailored and interactive nature of successful interventions with the wide reach of traditional mass media campaigns. Recent studies have shown the potential of such applications for achieving weight loss ^8,9 and to some degree weight management. ¹⁰ However, most studies have focused on applications aimed at treatment or secondary prevention. There is currently a lack of understanding of the requirements for successful interactive web-based applications for preventive interventions. There is a growing need for a holistic framework for the design of sustainable web-based applications in health care. The results of this study will be used as input for this framework.

The objective of the present study was to evaluate the HWA. The research questions were: how do users value the HWA? What are the requirements to successfully change behaviour via an interactive web-based application?

Online questionnaires

Respondents completed two online questionnaires: questionnaire 1 at the start of the study, before they had access to the HWA and questionnaire 2 after they had access to the application for six weeks. Invitations including a link to the online questionnaire were emailed to the respondents at the start of the study and six weeks after completing the first questionnaire. Both questionnaires were available for three weeks to complete. When no response was received after two weeks a reminder was sent including the final date for completing the questionnaire. The questionnaires were theory-based and, where possible, validated (Table 1). The constructs ‘motivation for participation’, ‘expectations of the HWA’, subjective behaviours and ‘usage of the HWA’ were assessed using one item measures like ‘Why do you want to use the HWA’ (motivation) and ‘How often did you use the HWA’ (usage). Knowledge was assessed by a 10-item true/false questionnaire based on the Netherlands classification model ¹¹ (for diet) and a 10-item true/false questionnaire based on the Dutch Standard for Healthy Physical Activity ¹² (for physical activity). The stage of change for healthy diet behaviour and stage of change for healthy physical activity behaviour were both assessed with an item with distinct answering categories (one for diet and one for physical activity). ¹³ Satisfaction and usability were measured by 19 items (five-point Likert scales, from 1 = I don't agree at all to 5 = I totally agree) based on work of Steele et al. ¹⁴ and of Vandelanotte and De Bourdeaudhuij. ¹⁵

Real-time usability tests

Real-time usability tests were conducted in order to assess the usability of the application, to identify problems experienced by users, to gain insight in the way the application was used and in the way the users would like to use the application. Respondents for these tests were invited via online questionnaire 2 and could indicate their interest in participating. In total, nine respondents were included: four respondents did not actually use the application; three respondents used the application occasionally (i.e. 1 to 4 times within the six week period); and two respondents used the system more regularly (i.e. more than 4 times within the six week period).

The real-time usability test started with a short interview to assess motivation and expectations (first time users) or impressions (occasional and regular users). First time users then registered to use the HWA and started following the steps for first use. Occasional and regular users used the application according to their normal habits. After that the scenario-based test was conducted, in which they performed certain tasks (i.e. scenarios) guiding them through the application. These scenarios consisted of tasks that a typical user would want to complete when using the HWA to achieve their goals. Finally, the usability test ended with a short interview in which the respondents stated their experiences and commented on their overall impression of the application.

The real-time usability tests were recorded using MORAE software (TechSmith Corporation, Michigan, USA). The screens were captured, all entries were saved and the respondent's facial expressions, as well as all audio, were recorded. The respondents were instructed to think aloud during the whole test.

Log-file analysis

Google Analytics was used to gain insight into the usage of the application. This tool allowed us to monitor the number of users of the HWA, the most frequently visited pages of the HWA and the average length of visits. The application itself stored all user-entered data. This information about demographics, behaviour, goals which might be achieved and diary entries was analyzed to gain insight into the way the HWA was used and what the effects of the application were. This information was combined with data from both questionnaires.

Qualitative analysis

A qualitative analysis of forum posts, email messages and free-text responses in the questionnaires was carried out. The HWA was linked to a forum where users could share experiences with the application, could ask questions and could make recommendations about the HWA. The posts of this forum were collected and qualitatively analyzed to gain insight into the user experience. Furthermore, all email contacts between respondents and researchers were stored and analyzed qualitatively. Finally, the free-text responses of questionnaire 2 were part of the qualitative study. All gathered data were coded according to the codebook in Table 2. This codebook was based on usability literature ^16,17 and adapted to the specific needs of our study.