Abstract
The Groningen Overweight and Lifestyle (GOAL) intervention effectively prevents weight gain. The present study describes a process evaluation in which 214 participants in the intervention group received a structured questionnaire within 7 months (a median of 5 months) after the end of the intervention. The authors investigated the content of the intervention (on basis of the participants’ recall), the participants’ satisfaction of the intervention, the participants’ satisfaction with the nurse practitioners (NPs), and the determinants of the participants’ satisfaction. In general, the results show that the content corresponded well with the protocol for the intervention, except for the number of telephone calls and the percentage of participants with individualized goals for a healthy lifestyle. The overall satisfaction of the participants was high, and success and perceived success and a low educational level were important determinants for a higher overall satisfaction grade. Furthermore, the NP was considered to be an expert and motivational to learning and keeping up a healthy lifestyle. The authors therefore conclude that the GOAL study is feasible and indicates that the NP is well equipped to treat these patients. However, it is recommended to reinforce the advice given and the lifestyle goals after the first contact sessions.
Obesity is a global problem with serious consequences in terms of public health. The World Health Organization has indicated that worldwide approximately 1.6 billion adults were overweight or obese in 2005 and has predicted that this would rise to 2.3 billion in 2015 (World Health Organization, n.d.). Overweight and obesity are associated with health issues such as diabetes, high blood pressure, high cholesterol concentrations, asthma, different types of cancers, coronary heart disease, and a diminished quality of life (Bogers et al., 2007; Carroll, 1998; Field et al., 2001; Fontaine & Barofsky, 2001; Mokdad et al., 2003). Because of these serious health consequences, counteracting the ongoing obesity epidemic remains a high priority.
On an individual level, lifestyle changes are needed to improve the health of overweight and obese patients, and the Groningen Overweight and Lifestyle (GOAL) intervention has assisted its participants in this aim. Lifestyle interventions, such as the GOAL intervention, are interventions guided by health professionals aiming at changes in the dietary and physical activity behavior of participants. Lifestyle interventions have already shown that lifestyle changes are effective in reducing weight (Curioni & Lourenco, 2005; Douketis, Macie, Thabane, & Williamson, 2005), but lifestyle changes without weight loss can also result in improved health (e.g., increased insulin sensitivity, improved blood glucose control, and a decrease in total and visceral fat; Duncan et al., 2003; Gannon & Nuttall, 2006; Lee et al., 2005). Moreover, lifestyle interventions can be considered as relatively inexpensive when compared with pharmacological interventions and surgery (Bogers et al., 2010).
In the Netherlands, participants with hypertension and/or dyslipidemia are usually treated by general practitioners (GPs), but these doctors report a lack of time and knowledge to achieve behavioral changes (Hiddink, Hautvast, van Woerkum, Fieren, & Van’t Hof, 1997). The GOAL intervention is designed to investigate long-term effects of lifestyle counseling by specially trained nurse practitioners (NPs) compared with usual care by GPs. It has already been shown that the GOAL-intervention positively changes physical activity and nutrition intake (ter Bogt, Milder, et al., 2011). Furthermore, weight gain was prevented in the intervention group. After 1 year of follow-up, a mean weight loss of 1.9% was found (ter Bogt et al., 2009), and after 3 years follow-up, the mean weight loss was 1.2% (ter Bogt, Bemelmans, et al., 2011).
Besides an evaluation of the effects of an intervention program, its process also needs to be evaluated (Oakley, Strange, Bonell, Allen, & Stephenson, 2006; Parry-Langdon, Bloor, Audrey, & Holliday, 2003; Rosecrans et al., 2008). A process evaluation describes specific program components, can assess the quality of an intervention program, and is used to investigate whether a program has been carried out as planned (Linnan & Steckler, 2002). Evaluating the implementation of the intervention can avoid incorrect conclusions about its effectiveness due to inadequate implementation (Linnan & Steckler, 2002). Furthermore, a process evaluation examines the participants’ views of the intervention (Oakley et al., 2006). The satisfaction of participants is an important health outcome, predicts attrition, and is used to evaluate quality of care and to determine aspects to improve (Jackson, Chamberlin, & Kroenke, 2001; VanWormer, Martinez, Cosentino, & Pronk, 2010).
However, the determinants of participants’ satisfaction remain largely unknown (Jackson et al., 2001; VanWormer et al., 2010). Previous research has shown that age and baseline health status may influence satisfaction, whereas gender and race do not seem to (Jackson et al., 2001). Also, health-related outcomes are important to patient satisfaction (Jatulis, Bundek, & Legorreta, 1997). A recent evaluation of a behavioral weight loss program showed that success (body mass index [BMI] loss, physical activity increase, and improved diet quality) and perceived success (improved body image) were predictors of overall program satisfaction (VanWormer et al., 2010). It also showed that educational level was related to whether or not participants would refer the program to others.
Thus, a process evaluation can evaluate and improve the quality of an intervention, which is useful in case of further implementation of the investigated and other lifestyle interventions. The present study aims to evaluate the content and the participants’ satisfaction of the GOAL intervention. We focus on the participants’ recall of the intervention content as compared with the protocol, the participants’ satisfaction with the intervention components, the participants’ satisfaction with the NP, and the overall satisfaction of the GOAL intervention. We also investigate what factors of success and perceived success and which characteristics of the participants determine the overall satisfaction of the participants.
Method
GOAL Study Description
For the GOAL study, overweight and obese individuals (with a BMI of 25-30 and ≥30 kg m−2, respectively) from the northern part of the Netherlands were screened for eligibility. From 11 general practice locations, 457 participants (aged 40-70 years) with a BMI between 25 and 40 kg m−2 with hypertension and/or dyslipidemia were randomized into an intervention group (N = 225) and a control group (N = 232). Table 1 shows the baseline characteristics for the intervention group.
Baseline Characteristics for the Intervention Group
Note. BMI = body mass index.
The participants in the intervention group had lifestyle counseling by NPs. The intervention is shown in Table 2 and contained four individual visits and one telephone call in the first year. In the second and third years of the intervention, participants visited the NP once a year and telephone calls were made twice a year. During the contact sessions, the NPs were guided by a standardized software program, for which they were specially trained.
Visits (Including Measurements) and Contents of the Lifestyle Intervention for the GOAL Study
Note. GOAL = Groningen Overweight and Lifestyle intervention.
Measurements on, for example, body weight and height, blood pressure, serum lipids, fasting glucose, physical activity, and nutrition intake.
V1-V6 = visit nurse practitioner.
F1-F5 = feedback moment by telephone by nurse practitioner.
In the first two visits, participants received information and individual advice on healthy lifestyle (healthy eating and physical activity); were given feedback on their lifestyle by discussing their food diary, their pedometer results, and their baseline questionnaires; and developed a treatment plan with individual goals. Standard advice for healthy living was based on national and international guidelines (NHLBI, 1998, 2000; Zelissen & Mathus-Vliegen, 2004). According to the protocol, every participant had to set a goal to lose weight, to prevent weight gain, or to achieve a healthy lifestyle without a focus on weight. In addition, for every participant, individual goals on healthy eating and physical activity were set. Participants with a weight loss goal were advised to reduce caloric intake by 300 to 500 kcal/day.
In the contact sessions thereafter, the NP evaluated lifestyle and lifestyle changes, discussed potential barriers and experiences, and adjusted the goals if necessary. Participants in the control group visited the GP once to discuss the screening results, and thereafter they received usual care according to the guidelines (Dutch Institute for Healthcare Improvement CBO, 2006). Weight measurements for both groups were at baseline, after 1 year follow-up and after 3 years follow-up. More details about the GOAL study can be found elsewhere (ter Bogt et al., 2009; ter Bogt, Bemelmans, et al., 2011; ter Bogt, Milder, et al., 2011).
Process Evaluation
All participants who completed the study were invited for this process evaluation. The dropouts were also included in the evaluation, except for 12 dropouts who moved to another area, were unreachable, had a serious disease, or had died. A request to complete a questionnaire with multiple-choice questions was sent to 213 participants of the intervention group. This questionnaire was sent within 7 months after the end of the 3 years of intervention, with a median time of 3 years and 5 months after baseline. The main topics were the content of the intervention and the participants’ satisfaction with it. The items from the questionnaire are shown in Tables 3 to 5. Furthermore, the questionnaire contained questions about the perceived success of participants (whether they felt healthier, felt fitter, and had fewer health complaints than before the GOAL study and whether they were satisfied with the results on weight control). If participants did not respond to this questionnaire, we sent the questionnaire again together with a shortened version, which could be returned if the participant was not willing (or not able to) complete the original questionnaire. This shortened questionnaire contained questions about the reason for not responding to the original questionnaire and about the overall score of the intervention.
The Participants’ Recall of Intervention Intensity and Components
Note. NP = nurse practitioner.
Satisfaction With the Intervention and the Intervention Components
Note. BMI = body mass index; NP = nurse practitioner.
The number of participants for the overall appreciation of the intervention was 159. For the subgroups low educational level, successful, BMI < 30 kg m−2, and males the number of participants for the overall appreciation was 48, 87, 103, and 76, respectively.
p < .05 for the difference between participants with a low educational level and participants with an intermediate or high educational level.
p < .05 for the difference between successful and unsuccessful participants.
p < .05 for the difference between participants with a baseline BMI of <30 kg m−2 and ≥30 kg m−2.
The Participants’ Satisfaction With the Advice Given on Healthy Eating and Physical Activity
All participants of the intervention group who returned the original questionnaire were included in the analyses, except for participants who had not had any contact sessions with the NP and participants who did not answer the large majority of the questions. The participants who returned the shortened questionnaire were only included in the analyses of the overall score of the intervention.
Statistical Analyses
Differences in baseline characteristics (gender, educational level, baseline BMI, age, and history of dieting), percentage of weight loss after 3 years, and number of dropouts between the participants who responded and did not respond on the original questionnaire were investigated using an independent sample t test and a chi-square test.
The content and the participants’ satisfaction with the GOAL intervention were described by frequencies of the answer categories. Also, the mean grade of the intervention was calculated. All aspects of the participants’ satisfaction of the intervention were also compared for the following subgroups: successful participants (<1% weight gain) versus unsuccessful participants (≥1% weight gain), men versus women, a BMI between 25 and 30 kg m−2 versus a BMI of at least 30 kg m−2, and a high or intermediate educational level versus low educational level. Differences between these subgroups were tested using independent t tests and chi-square tests.
To examine which factors determine the participants’ satisfaction, the association of the overall grade of the participants’ satisfaction with success and perceived success and the characteristics of participants were investigated using independent sample t tests, Pearson correlations, and a linear trend analyses. Success or perceived success contained the following items: feeling healthier than before the GOAL study, feeling fitter than before the GOAL study, having fewer health complaints than before the GOAL study, satisfaction with the results on weight control, and weight loss after 3 years. The characteristics of participants that were investigated as determinants of the participants’ satisfaction were gender, a low educational level, baseline BMI, age, and history of dieting.
SPSS 18.0 was used to perform the analyses, and a significance level of .05 was considered significant.
Results
Response to Questionnaire
Of the original questionnaires sent, 150 (70%) were returned. Of the 64 nonresponders, 13 (20%) returned the shortened version, and the reported reasons for not responding were “it takes too much time” (N = 4), “it was too long ago, I cannot remember the intervention very well anymore” (N = 4), and other reasons (N = 5). After excluding two participants because they had had no contact sessions with the NP and two others who had not answered most of the questions, we included 146 participants in our analyses and 159 participants in the analyses for the overall appreciation of the study. Baseline characteristics (gender, educational level, baseline BMI, age, and history of dieting) and weight loss after 3 years did not significantly differ between responders and nonresponders. However, intervention dropouts (N = 38) responded less to the questionnaire (p < .001).
Content of the Intervention
Table 3 describes the intervention intensity and components on basis of the participants’ recall. The total number of visits corresponded well to the six visits prescribed in the protocol, because the participants indicated that the number of visits was less than 5 for 19%, 5 to 10 for 63%, and more than 10 for 19% of the participants. However, the majority of the participants did not have contact with their NP by phone (63%), whereas the protocol prescribed five phone calls. Furthermore, the percentage of the participants who set individual goals for eating, physical activity, and weight control was 67%, 60%, and 85%, respectively. Furthermore, 81% of the participants recalled receiving advice on healthy eating, whereas 59% recalled receiving advice on physical activity.
Participants’ Satisfaction of the GOAL Intervention
Table 4 describes the participants’ satisfaction regarding the different components of the intervention. Most participants perceived the total number of contact sessions as good (63%) and most of the participants who received phone calls were satisfied with these calls (76%). More than 80% of the participants were positive or neutral about the usefulness of the food diary and the pedometer, and 76% of the participants agreed with the usefulness of the individual lifestyle goals.
The participants’ satisfaction on the advice given by the NP is shown in Table 5. More than 80% of the participants who received advice found the advice on healthy eating and physical activity useful. The advice on healthy eating fitted in the daily practice of 77% of the participants, whereas this was only 61% for the advice on physical activity. About 60% of the participants agreed that the advice on healthy eating and physical activity added knowledge to their existing knowledge on these topics.
The participants were satisfied with the NP (as shown in Table 4). Most participants were satisfied with the visits to the NP (91%), agreed that the NP was an expert (87 %), and agreed that the NP was motivational regarding learning and keeping up a healthy lifestyle (82%). Moreover, some participants agreed (43%) and some disagreed (25%) that they had discussed other personal matters (besides lifestyle) with the NP. However, most participants agreed that it was possible to discuss everything they wanted with the NP.
Furthermore, Table 4 shows the overall satisfaction with the GOAL intervention. Most participants agreed that the lifestyle advice (as in the GOAL study) is a useful addition to the existing health care service, and prolongation of the intervention was preferred by 24% of the participants, whereas 43% did not want prolongation and 30% was unsure. Overall, the participants were happy with the intervention and 61% would certainly participate again. The average score of the intervention was 7.5 (SD = 1.2) out of a possible 10. It appeared that the 146 respondents of the original questionnaire scored the intervention higher than the respondents of the shortened questionnaire (7.5 vs. 6.4, respectively; p < .01).
Subgroup Analyses for Aspects of Participants’ Satisfaction
Table 4 shows the results of the subgroups for low educational level, successful participants, overweight participants, and males. The subgroup analyses of the aspects of the participants’ satisfaction showed that successful participants agreed more often than unsuccessful participants that the NP was motivational (89% vs. 76%, p = .02) and that advice on healthy lifestyle added knowledge (71% vs. 44%, p < .01, for healthy eating; 69% vs. 38%, p = .02, for physical activity). Of the males, 71% agreed that the advice on healthy eating added knowledge to their existing knowledge compared with 51% of the females (p = .02). Participants with a BMI of at least 30 kg m−2 agreed more often that it was possible to discuss everything with the NP (89%) than participants with a BMI below 30 kg m−2 (84%, p = .04), and they also discussed personal matters more often (62% vs. 35%, p < .001). Participants with a low educational level agreed more that they were satisfied with the telephone calls (91% vs. 68%, p = .04), that it was possible to discuss everything with the NP (100% vs. 79%, p = .02), that they discussed personal matters with the NP (52% vs. 40%, p = .02), that they were satisfied with the visits to the NP (100% vs. 87%, p = .04), and that advice on physical activity added knowledge to their existing knowledge on that topic (79% vs. 44%, p < .001). Also, it seemed that participants with a low educational level agreed more that advice on healthy eating added knowledge (69% vs. 56%, p = .08) and that advice on physical activity fitted into their daily practice (69% vs. 57%, p = .05).
Associations With Participants’ Overall Satisfaction
We also investigated whether success and perceived success and the characteristics of participants could explain the participants’ overall satisfaction grade of the intervention. Feeling healthier after the GOAL study (p < .001), feeling fitter after the GOAL study (p < .001), and perceiving fewer health complaints after the GOAL study (p < .001) were significantly associated with a higher overall score of the intervention. Also, the percentage of weight loss after 3 years (p < .001) and being satisfied with results on weight control (p < .001) correlated positively with the overall score. Concerning the characteristics of the participants, only educational level showed a significant association, that is, a low educational level was associated with a higher overall score of the intervention (p = .04).
Discussion
The present study evaluates the process of the GOAL intervention and focuses on the content of the intervention and the satisfaction of participants. In general, the participants’ recall of the content (i.e., the number of visits to the NP, individual goals for weight control, and advice on healthy eating) was similar to the protocol of the intervention, and the participants were satisfied with the intervention (components). More than 90% were happy with their participation, and the overall satisfaction grade was 7.5 out of a possible 10, which is comparable both to satisfaction grades of health care and to a recent Dutch lifestyle intervention (Hall & Dornan, 1988; Harting, van Assema, & de Vries, 2006).
Nevertheless, some elements of the intervention were not well implemented for all participants and should be improved in further implementation. First, 63% of the participants did not receive any telephone calls during the intervention, although the protocol prescribes five telephone calls. Previous studies indicated that transportation is a perceived barrier to self-management (Jerant, von Friederichs-Fitzwater, & Moore, 2005) and a reason for nonadherence (Jorgensen, Polivka, & Lennie, 2002) and that telephone calls may be useful in an intervention as an alternative to expensive face-to-face visits (Sherwood et al., 2006). However, a few NPs indicated to the investigators that they substituted telephone contacts for face-to-face contacts, because they preferred face-to-face contact due to practical issues involved with the telephone calls, for example, not easily being able to contact the participants and a higher adherence to face-to-face visits. This means that the implementation of the intervention did not fully comply with the protocol, which could lead to incorrect conclusions about the effectiveness of the intervention. However, these alterations in the program were only conducted by a few NPs and therefore the protocol was followed to a large extent. Second, the percentages for participants with individual goals on healthy eating (67%) and physical activity (60%) were surprisingly low. This may partly be explained by participants who already met the guidelines on baseline and therefore did not receive an individual goal on this topic. However, for healthy eating it is less likely that participants already met all criteria on baseline (ter Bogt, Milder, et al., 2011). Therefore, the number of individual goals on a healthy lifestyle should be improved in further implementation, since goals on behavior change have a positive effect on dietary and physical activity behaviors (Shilts, Horowitz, & Townsend, 2004). Finally, some of the participants reported that they did not receive any advice on healthy eating and physical activity (19% and 41%, respectively). However, the instructions for the NP in the protocol and the standardized software program were strict regarding advice on a healthy lifestyle. Therefore, it is more likely that the participants did not remember the advice anymore or did not recognize it as advice for a healthy lifestyle. This may be due to the time (more than 3 years) between the beginning of the intervention and the moment the questionnaire was sent.
Participants were on average satisfied with all intervention components, and the advice on a healthy lifestyle was considered useful and fitted in reasonably well with the daily lives of the participants. Participants who felt healthier, who felt fitter, and who had fewer health complaints than before the GOAL study were more satisfied with the GOAL intervention. Weight loss after 3 years was also associated with the overall grade, which implies that success and perceived success of the intervention were important determinants for the participants’ satisfaction of the intervention.
Participants with a low educational level also regarded the advice as useful, and they indicated that the advice on healthy lifestyle added more knowledge compared with the participants with a higher educational level. Advice on physical activity also fitted better into the daily lives of participants with a low educational level. This can be explained by the study of Ball, Salmon, Giles-Corti, and Crawford (2006), who found that women with a low socioeconomic status participate more in transport-related activity and are more likely to be active at work. However, they also found that people from a low socioeconomic status tended to lack time due to work commitments and a lack of flexibility of working hours. Furthermore, advice on healthy eating added more often knowledge to men than to women. This may be due to the higher number of weight loss attempts for women compared with men (Kruger, Galuska, Serdula, & Jones, 2004), which can lead to more knowledge on healthy eating for women at the start of this intervention.
A great strength of the present study is that we systematically investigated the intervention intensity and components (on basis of participants’ recall) and the participants’ satisfaction of participants in the intervention group using standardized questionnaires, while often data are not systematically collected for a process evaluation (Oakley et al., 2006). However, a few limitations of this evaluation should also be mentioned. First, the analyses on the content and the participants’ satisfaction of the intervention are based on self-reported data, and therefore, data on the content of the intervention may be less accurate than measured data would have been. Second, the intervention was not totally implemented as designed, because not all participants were contacted by telephone. The reasons for missing the phone calls were not measured in this evaluation; possible reasons could be due to the NP as well as the participant. However, a few NPs indicated to the investigators that it was not always easy to contact the participants and participants had a higher adherence to face-to-face visits than to the phone-call appointments. Third, only univariate analyses were used to investigate the determinants of the participants’ satisfaction, because the multivariate outcomes were susceptible to changes in the equation. However, results were similar for the multivariate analyses. Finally, participants who dropped out of the intervention responded less to the questionnaire, and participants who responded to the shortened questionnaire scored lower in the intervention. Despite the high response in our study, this could bias our results resulting in a small overestimation of the quality of the intervention and the participants’ satisfaction.
To summarize, the obesity epidemic is a problem with serious health consequences. The GOAL study investigates the long-term effects of lifestyle intervention by NPs, and it has been shown that it effectively prevents weight gain in contrast to the Dutch population in which BMI increases on average 0.05 kg m−2 per year (Gast, Frenken, van Leest, Wendel-Vos, & Bemelmans, 2007). This study evaluated the process of the intervention and showed that the participants’ recall of the content corresponded with the protocol to a large extent and that the participants’ satisfaction was high, which makes this intervention feasible. Furthermore, the GOAL intervention has a relative low intensity and the majority of the participants thought no more contact sessions were needed, which makes the intervention relatively inexpensive.
Implications for Practitioners
This evaluation shows some elements (e.g., telephone calls, healthy lifestyle goals, and advice on a healthy lifestyle) that should be improved in further implementation of the GOAL study and which should be kept in mind for the implementation of other interventions.
First, the protocol with regard to telephone calls should be changed. An appointment for this contact session should be made in advance (preferably during the face-to-face session). Also, there should be an opportunity for NPs and participants to replace the phone calls when participants are difficult to contact by phone or are unwilling to be contacted for sessions by phone. For example, the phone calls could be replaced by email or face-to-face contacts.
Second, the number of individual goals on a healthy lifestyle, and especially healthy eating, were lower than expected and should be improved in further implementation. Therefore, the NPs should remind the participants of these goals on every contact session. Furthermore, the participants should commit themselves to the behavioral goals by a contract.
Third, some participants could not remember the advice given anymore. Therefore, the manner in which the advice is given should be improved and additional reinforcement after the first contact sessions is needed for these recommendations. This may be realized by supporting the advice with additional written information that is given during the face-to-face visits. This information should also be reinforced in the second and third year of the intervention by pamphlets with the mail. This written information has to be comprehendible for participants with a lower educational level, because this group agreed more that the advice added knowledge to their existing knowledge. Besides this written information, the NP should also reinforce the advice in person during the later contact sessions.
Fourth, this study shows that higher educated participants were less satisfied with the GOAL study, which can be explained by the fact that the advice on a healthy lifestyle did not add to their existing knowledge and that advice on physical activity did not fit well into their daily lives. The degree to which advice on healthy eating added knowledge also differed by gender. These differences should be borne in mind by NPs (and other health professionals) when adjusting advice on a healthy lifestyle to the characteristics of the participant. For some participants advice on a healthy lifestyle may not even be the priority, and the main task of the NP will be to motivate the participants involved.
Finally, within our evaluation, participants were asked what should be changed in the GOAL intervention to raise their satisfaction, but most participants (69%) did not have any suggestions and the suggestions given were very diverse. However, some recommendations for further implementation were mentioned by a small group of participants such as “I prefer more contact sessions” (5%) and “I would like (more direct) feedback on my blood samples” (4%).
In conclusion, the GOAL study evaluates a lifestyle intervention conducted by NPs. Previous research (ter Bogt et al., 2009; ter Bogt, Bemelmans, et al., 2011) showed that this intervention was effective in preventing weight gain, and the present study shows that the participants were satisfied with the NP. In general, the NP was considered both expert and motivational regarding the participants’ learning and keeping up a healthy lifestyle. Therefore, the GOAL study indicates that the NP is well equipped to treat these patients.
Footnotes
Acknowledgements
We are grateful to the participating general practices and their patients for their enthusiasm and cooperation. Thanks also to all the students of the research team for performing measurements and data collection. We also thank Ivon Milder for composing the questionnaires. Finally, we thank the members of the Hypertension Service Groningen for generating the GOAL study.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article:
Funding was provided by the Netherlands Organization for Health Research and Development (Zon-Mw, Project No. 6200.0016), the Dutch Ministry of Health, Welfare and Sport, and Foundation Fund “De Gavere.” The funding agencies had no role in the study design, data collection, data analysis, data interpretation, or the writing of the report.