What Design Features Are Used in Effective e-Health Interventions? A Review Using Techniques from Critical Interpretive Synthesis

Abstract

The effectiveness of e-health interventions varies greatly. Despite this, there has been relatively little formal consideration of how differences in the design of an intervention (i.e., how the content is delivered) may explain why some interventions are more effective than others. This review primarily examines the use of the Internet to provide educational and self-management interventions to promote health. The article develops hypotheses about how the design of these interventions may be associated with outcomes. In total, 52 published reports from both a diversity sample and a representative sample were reviewed using techniques from Critical Interpretive Synthesis. Four core interactive design features were identified that may mediate the effects of intervention design on outcomes: Social context and support, contacts with intervention, tailoring, and self-management. A conceptual framework to summarize the design of e-health interventions delivered using the Internet is proposed. The framework provides a guide for systematic research to identify the effects of specific design features on intervention outcomes and to identify the mechanisms underlying any effects. To optimize the design of e-health interventions more work is needed to understand how and why these design features may affect intervention outcomes and to investigate the optimal implementation and dosage of each design feature.

Introduction

The term “e-health” describes a range of information and communication technologies that are used to provide healthcare,¹ such as Internet or computer-based technologies, telemedicine, remote patient monitoring, electronic health records, and videoconferencing. This review will focus primarily on the use of the Internet to provide educational and self-management interventions that support users and patients to change health-related behaviors. Using the Internet to deliver health interventions currently has a small but significant overall effect on behavior.^2
–4 However, the reported efficacy and effectiveness of individual interventions vary hugely.⁵ Effectiveness may be improved by optimizing the design of interventions, that is, the ways in which the content of an intervention is delivered.

There are already well-established frameworks to guide development of the content of health interventions.^6
–8 These need to be supplemented by frameworks to guide how that content should be delivered using digital communication technologies like the Internet. Existing frameworks have identified a range of design features that may influence users' satisfaction and behavior change.^8

–12 However, these frameworks do not provide a comprehensive description of design features that can be used to deliver interventions and have not systematically examined how specific features contribute to outcomes.

The design features of published Internet-based health interventions are not always explicitly described or systematically varied, which precludes definitive hypothesis testing of the effects of design features on outcomes. This review aims to develop a conceptual framework to define the range of design features that could be used to deliver the content of health interventions delivered using the Internet. The purpose of developing this conceptual framework is to permit analysis of how specific feature(s) of intervention design may influence health-related outcomes. We will develop hypotheses and questions about possible associations between intervention design and outcomes that should be addressed by future research. We used techniques from Critical Interpretive Synthesis (CIS), an approach to review that is well suited to the task of hypothesis generation.

Methods

CIS¹³ provides an appropriate methodological alternative to Cochrane-style systematic review¹⁴ when the literature is not suitable for meta-analysis. CIS uses techniques from qualitative research (e.g., diversity sampling, constant comparison, deviant case analysis, theoretical saturation¹⁵) to guide a dynamic and iterative approach to the review process. CIS does not use the conventional search processes of the systematic review (e.g., exhaustive searching for all available articles, rigid inclusion and exclusion criteria, quality assessments). Instead, the aim is to identify and select a diverse sample of articles to represent the variation found within the literature. To identify and compare the range of design features used to deliver Internet-based health interventions we collected a diversity sample, a theoretical sample, and a representative sample of articles (see Fig. 1).

Fig. 1.

Flow chart for the identification and selection of studies.

PHASE 1. DIVERSITY SAMPLING

Aim

To select a sample of articles, which used a diverse range of design features to deliver Internet-based health interventions.

Inclusion criteria

Included were articles published between 2000 and 2009 reporting quantitative analyses of fully automated Internet-based health interventions.

Exclusion criteria

Excluded were computer-based interventions, interventions delivered solely by synchronous computer-mediated communication, interventions delivered by face-to-face contact, or interventions designed specifically to treat mental health disorders. These criteria ensured that diversity in the design of interventions was examined in a homogeneous sample.

Search strategy

Studies were identified between June and September 2009 using electronic bibliographic databases: Ovid, ISI Web of Knowledge, PubMed, Science Direct, and Google scholar. Reference, related article, and cited article lists were checked for additional relevant studies. Search terms were as follows: Internet; health; intervention; quantitative; behavior; review; efficacy; evaluation; use. Sampling ceased when no substantially different design features were identified from including further articles (i.e., when saturation was achieved).¹⁶ Twenty-seven articles were included in the final diversity sample, reporting interventions for physical activity,^{17

–29} smoking cessation,^30

–34 weight,^35
–37 physical activity and nutrition,³⁸ nutrition,³⁹ chronic pain,⁴⁰ dyspnea,⁴¹ arthritis,⁴² and breast cancer risk.⁴³ Four of these programs contained telemedicine components.^30,35,40,41

Review strategy

The design features used to deliver interventions were first identified. The associations between the presence of design features and intervention effectiveness were then examined, paying attention to possible reasons for variability in the effect of design features. The output for Phase 1 was as follows: The development of a conceptual framework to summarize the design features used in Internet-based interventions; the generation of hypotheses about the potential association between design features and intervention outcomes; and identification of research questions to address unresolved issues. In order to generate hypotheses, interventions were coded as more effective, less effective, or ineffective (see Table 1). The diversity sample contained 15 more effective interventions,^{18,20,21,25

–31,34

–37,41} 11 less effective interventions,^{17,19,22
–24,32,33,38
–40,42} one intervention reporting only process evaluations,⁴³ and no ineffective interventions.

Table 1.

Criteria for Defining Intervention Effectiveness

INTERVENTION CODE	CRITERIA
More effective	• The intervention led to improvement on the majority of outcome measures.
	• The intervention was at least as effective as comparison groups.
	• The intervention was more effective than waiting list or no intervention control groups.
Less effective	• The intervention led to improvement on a minority of outcome measures.
	• The intervention was not necessarily as effective as comparison groups.
	• The intervention was more effective than waiting list or no intervention control groups.
Ineffective	• The intervention did not lead to improvement on any of the outcome measures.
	• The intervention was no more effective than waiting list or no intervention control groups.

PHASE 2: THEORETICAL SAMPLING

Aim

To clarify the classification of design features identified during Phase 1 and provide further insight as to the possible reasons for variability in the effect of design features.

Search strategy

Search for articles was conducted in parallel but coordinated with reviewing the diversity sample. Twenty-three articles were identified comprising four conceptual articles,^44

–47 four qualitative articles,^48

–51 five review articles,^4,52

–55 six empirical studies not related to the evaluation of Internet-based health interventions,^56

–61 and four published guidelines for website design.^62

–65

PHASE 3: REPRESENTATIVE SAMPLING

Aim

To “test” the emerging conceptual framework by examining whether the hypotheses generated from Phase 1 were consistent and generalizable.

Inclusion criteria and exclusion criteria

As for the diversity sample except that computer-based interventions or interventions delivered using synchronous computer-mediated communication were not excluded. These interventions were included to enable a more comprehensive test of the framework against a more representative sample of the literature.

Search strategy

Based on consultation with two experts in behavior change interventions (S.M.) and e-health (J.P.), two predefined sets of articles were sampled from two published systematic reviews.^66,67 Twenty-five articles published between 2001 and 2005 were included, reporting interventions for physical activity,^68

–72 dietary behavior,^{73

–82} and physical activity and dietary behavior.^{83

–92} Fourteen articles reported Internet and e-mail-based interventions,^{68

–71,75,79,81

–88,90,92} three of which contained telemedicine components,^84
–86 seven articles reported computer-based interventions,^{73,74,76,78,80,89,91} and two articles reported telemedicine interventions.^72,77

Review strategy

To test the conceptual framework, interventions were reviewed to identify any additional design features not identified from the review of the diversity sample. The associations between the presence of design features and intervention effects were then examined. Interventions included in the representative sample were also coded as more effective, less effective, and ineffective (see Table 1). The representative sample contained 15 more effective interventions,^{68,70
–72,74
–76,78,81,83,84,86
–88,91} eight less effective interventions,^{73,77,79,80,85,89,90,92} and two ineffective interventions.^69,82

Results

Eleven design features were identified from the review of the diversity sample (Fig. 2 and Table 2). No additional features were identified from the review of the representative sample. In Figure 2 the inner circle contains four interactive design features hypothesized to mediate intervention outcome; the outer circle contains eight features hypothesized to moderate the effects of the four interactive features.

Table 2.

Conceptual Definitions for the Design Features Identified

DESIGN FEATURE	CONCEPTUAL DEFINITION
Social context and support	Facilitate perceptions of social context, human or human-like interaction, and social support
Contacts with intervention	Provide direct or mediated contact with the intervention, or individuals responsible for the intervention
Tailoring and targeting	Provide optimally relevant information matched to individual users (tailoring⁶⁰) or groups of user (targeting⁴⁶)
Self-management	Use of personal information for reflective self-management and monitoring of health behavior
Entertainment	Provide content-based entertainment activities
Aesthetics	Provide an aesthetically pleasing or engaging intervention
Updated information	Provide regularly updated information throughout the intervention period
Usability	Facilitate users' ability to successfully navigate the intervention
Credibility	Provide information or services to facilitate perceptions of credibility
Information architecture	Access to information and navigation through the intervention⁴⁵
Program exposure	The length of time a user is entitled to engage with the intervention

Fig. 2.

Conceptual framework for the relationship between the design features described in studies of Internet-based health behavior interventions.

The review presented here focuses on the four interactive features: Social context and support, contacts with intervention, tailoring, and self-management. The aim of this review was to generate hypotheses about the associations between the presence of specific design features and intervention outcomes, not to engage in definitive hypothesis testing. Table 3 summarizes the hypotheses generated from the review of the diversity sample. Each of the four interactive design features is considered below, presenting (a) a description of how has each design feature is implemented, (b) an examination of the associations of that feature with intervention outcomes in the diversity, theoretical, and representative samples, and finally a summary of (c) the hypotheses and (d) research questions generated by the review.

Table 3.

Findings from Review of the Diversity Sample Regarding the Association Between the Presence of Design Features and Intervention Effectiveness

DESIGN FEATURES ASSOCIATED WITH MORE EFFECTIVE INTERVENTIONS
• Automated dialogue components^20,21,30,34
• Synchronous- and asynchronous-mediated peer-to-peer communication^{18,32,34,41,42}
• Additional contacts containing behavior change techniques^{18,20,21,25,28,31,34}
• Tailoring based on more than one variable (e.g., theoretical, behavioral, or demographic)^{20,21,28,29,33,35,41}
• Tailoring based on a large and varied number of constructs (e.g., several theoretical constructs)^{18,20,29,30,34,36}

DESIGN FEATURES ASSOCIATED WITH LESS EFFECTIVE INTERVENTIONS
• Use of avatars¹⁷
• Providing information about other users^24,39
• Additional contacts containing usage promotion techniques²⁹

DESIGN FEATURES NOT ASSOCIATED WITH INTERVENTION INEFFECTIVENESS
• “Ask the expert” components
• Activity planning
• Self-monitoring

SOCIAL CONTEXT AND SUPPORT

Implementation

Features providing social context and support could be grouped into three subtypes: Simulation of person-to-person interaction (e.g., automated dialogue, avatars),^{17,20,21,30,34} provision of synchronous- (e.g., chat rooms)^36,41 or asynchronous- (e.g., discussion forums)^18,32,34,42 mediated contact with other users, and provision of information about other users (e.g., social norms, vignettes).^{24,28,32,34,42}

Associations with outcomes

From the review of the diversity sample it appears that simulation of person-to-person interaction using automated dialogue is more effective than using avatars. Two interventions in the representative sample—one more effective and one less effective—reported using simulation of person-to-person interaction using digitized speech.^72,77 This suggests that the use of digitized speech is not always associated with intervention effectiveness. Automated dialogue may be more effective than avatars for simulating person-to-person interaction because users expect more from a system that looks and behaves like a human than from an overtly automated system. Qualitative feedback from users does suggest that avatar-based systems are an unrealistic substitute for human interaction.¹⁷ Current technology may not yet permit the development of systems sufficiently sophisticated to simulate real interaction. Users may also have differing preferences for interacting with human-like systems versus machine-like systems.⁴⁹

From the review of the diversity sample there appeared to be a positive association between peer-to-peer–mediated communication and intervention outcome. However, although some studies reported that discussion forums and chat rooms were frequently used and helpful,^33,34,42 others reported infrequent use and few changes in perceptions of social support.^18,37,41 From the review of the representative sample, mediated communication with peers appeared to be used in both more and less effective interventions, but dissatisfaction with mediated communication did not appear to have adverse effects.⁸⁴ No effects were found for the use of synchronous and asynchronous peer-to-peer communication in a systematic review,⁵³ although this may be because the review was published in 2004, before the onset of Web 2.0, when peer-to-peer communication was less widely used. The effectiveness of mediated communication with peers appears to be variable and may depend on several factors including perceptions about the credibility of Internet-based peer advice,^44,48,50 perceived quality of interaction,^69,86 and active user involvement (i.e., users involved in posting and responding to messages may benefit more than “lurkers”).^37,51

From the review of the representative sample, providing information about other users appeared to be associated with more effective interventions,⁷⁸ whereas from the review of the diversity sample it was not. This may be because interventions from the diversity sample tended to include testimonials from hypothetical users,²⁴ whereas interventions from the representative sample included testimonials from real users.⁷⁸

Hypotheses

Social context and support features have varied outcomes, but providing automated dialogue components, synchronous- or asynchronous-mediated communication with peers, or information about other real users may have a positive effect on intervention outcomes.

Research questions

What makes features that provide social context and support effective? Is it because they increase perceptions of social support? Or is it because of enhanced personal relevance of the information and advice provided?

CONTACTS WITH INTERVENTION

Implementation

Two types of contacts with intervention were identified: Expert-initiated contact^{18

–21,24,25,28
–30,36,40,42} and user-initiated contact.^{32,34,36,38,41} Expert-initiated contact could be grouped into two further subtypes: Contact delivering behavior change techniques (e.g., motivational emails) and contact promoting intervention usage (e.g., e-mail reminders).

Associations with outcomes

Contacts delivering behavior change techniques were more common in more effective interventions than in less effective interventions.^76,81 Simple reminders were found to have a small effect on health behavior in a recent systematic review⁴ but were more common in less effective interventions.⁷⁷ “Ask the expert” services were used in both more and less effective interventions reviewed in the diversity sample. However, “ask the expert” services were not a common feature of more effective interventions reviewed in the representative sample, suggesting that this feature may not be an essential component for an effective intervention.

The effectiveness of contacts delivering behavior change techniques may be influenced by several factors. There is some evidence that these contacts may only be effective for already engaged users of the intervention and may not engage “new” or infrequent users.²² They may also be subject to ceiling effects or context effects. Motivational e-mails provided to users who are already performing the desired behavior at baseline are likely to be redundant.²⁸ Motivational e-mails provided within a workplace context run the risk of being ignored in an already overflowing inbox.^28,42 Allowing users to choose to receive mobile phone or e-mail reminders increased their perceived personal control over implementing their physical activity intentions.²¹

Hypotheses

Contacts delivering behavior change techniques may be more effective than simple reminders to use the intervention.

Research questions

Are contacts delivering behavior change techniques (e.g., tailoring, social support, or goal reminders) more effective than those that do not? Why do user-initiated contacts such as “ask the expert” services appear to have little influence on intervention outcome? Do users prefer to seek advice from peers for some health behaviors (e.g., smoking cessation)?³³ How important are the characteristics of the “expert,” the user, and the health topic?

TAILORING

Implementation

Tailoring is the provision of information relevant to one individual person⁶⁰ and can be based on theoretical constructs, behavior, or demographic characteristics.⁵⁵

Associations with outcomes

Nearly all the studies used a tailored design, so no comparison could be made with non-tailored designs. The number of variables of individual constructs used to deliver tailoring did not appear to differ between more and less effective interventions reviewed in the representative sample. From the review of the diversity sample and in a systematic review of tailored print materials⁵⁵ information and advice appeared to be more effective if they were tailored to more than one variable. The effect of tailoring variables has been proposed to be hierarchical (i.e., tailoring based on theoretical constructs is more effective than tailoring based on behavior), which is more effective than tailoring based on demographic characteristics.⁵⁵

Hypotheses

Tailoring based on greater numbers of variables may be more effective than tailoring based on just one variable.

Research questions

How should tailoring be implemented? What is the optimal balance between tailored and non-tailored information? Is tailoring more effective if users choose to receive it? Is tailoring effective because it enhances perceptions of personal relevance or because it resembles face-to-face interaction?

SELF-MANAGEMENT

Implementation

Two types of self-management features were identified: Activity planning and self-monitoring.

Associations with outcomes

The review of both the diversity and representative samples found that activity planning and self-monitoring were used in both more and less effective interventions. Self-management features appeared to be well liked by users,^21,27,33 but recent evidence suggests they are not always frequently used.¹⁸ This finding is unexpected as there is good evidence for the effectiveness of self-management strategies in contexts other than Internet-based interventions.^52,54,56 Recent meta-analyses have found that interventions that included self-monitoring components were significantly more effective, particularly when provided in conjunction with other components (e.g., goal setting and feedback on performance).^52,54

A recent meta-analysis of interventions delivered using the Internet reported that goal setting and action planning had a significant impact on behavior but self-monitoring did not.⁴ Published intervention protocols often do not specify how activity was planned, what behaviors were monitored, or how behavior was monitored, which makes it difficult to explain when and why these techniques may be effective or ineffective. These mixed findings may reflect qualitative differences between the processes of activity planning versus self-monitoring. The latter may require more intensive and sustained effort, which may benefit from human interaction and support.

Hypotheses

Self-management strategies that include behavior change techniques theoretically predicted to function synergistically will be more effective than those omitting theoretically relevant techniques.

Research questions

What makes self-management features effective? Are these features more effective when they are structured (e.g., users choose from predefined options) or unstructured? How important is providing feedback on activity planning and self-monitoring? Is activity planning more effective than self-monitoring?

Discussion

This review used techniques from CIS to develop a conceptual framework summarizing the design of Internet-based health interventions. The framework proposes that four interactive design features mediate intervention outcomes: Social context and support, contacts with intervention, tailoring, and self-management. It is hypothesized that interventions may be associated with more positive outcomes if they provide social support using automated dialogue, peer-to-peer–mediated communication, or information about other real users, additional contacts that incorporate behavior change techniques, and tailoring based on combinations of variables. Further research is needed to understand the effects of user-initiated “ask the expert” services and self-management features. These design features are not specific to Internet-based health interventions. Therefore the hypotheses and questions generated by this review may also have implications for the delivery of other e-health and telemedicine applications such as remote patient monitoring, telepsychiatry, home healthcare, and decision support tools.

The conceptual framework includes some of the design features proposed in existing frameworks^9

–12 (e.g., tailoring, self-monitoring, aesthetics, credibility, usability, and information architecture). However, the framework also includes new features not defined within existing frameworks (e.g., social context and support and contacts with intervention). The conceptual framework proposed in this review goes beyond existing frameworks by specifying which design features contribute to “interactivity” and by developing hypotheses about how these interactive features may influence health-related outcomes.

The aim of this review was to provide a detailed critical analysis of a diversity sample and a representative sample of the literature, rather than an exhaustive systematic review of all Internet-based health interventions. The hypotheses and framework proposed do not represent all possible interpretations but present one potentially useful perspective. Analysis of intervention design was limited by the level of detail provided by authors. The literature could benefit from more explicit reporting of the development and implementation of interventions⁹³ and/or the sharing of intervention details.⁹⁴ It is likely that factors other than intervention design (e.g., quality of theoretical content) will also have important effects on intervention outcome. Because few ineffective interventions could be identified an important comparison group was missing. To fully understand effective intervention design there is a real need to publish reports on both effective and ineffective interventions.

Improving our understanding about how the design of e-health interventions promotes optimal outcomes will help these interventions to realize their full potential. Further research is needed to understand how and why specific design features may affect intervention outcome by addressing the questions generated from this review. Systematic quantitative approaches⁹⁵ can be used to identify what dosage and combination of features will produce optimal intervention outcomes. Furthering our understanding of which design features add to or take away from intervention effectiveness will also help researchers and practitioners to decide if, when, and how the Internet or other e-health technologies will be the most effective mode of delivery for a given intervention.⁹⁶

Footnotes

Disclosure Statement

No competing financial interests exist.

References

Eng

. The ehealth landscape: A terrain map of emerging information and communication technologies in health and health care. Princeton, NJ: The Robert Wood Johnson Foundation, 2001.

Portnoy

, Scott-Sheldon

LAJ

, Johnson

, Carey

. Computer-delivered interventions for health promotion and behavioral risk reduction: A meta-analysis of 75 randomized controlled trials, 1988–2007. Prev Med, 2008; 47:3–16.

Wantland

, Portillo

, Holzemer

, Slaughter

, McGhee

. The effectiveness of web-based vs. non-web-based interventions: A meta-analysis of behavioral change outcomes. J Med Internet Res, 2004; 6:e40.

Webb

, Joseph

, Yardley

, Michie

. Using the Internet to promote health behavior change: A systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and modes of delivery on efficacy. J Med Internet Res, 2010; 12:e4.

Nguyen

, Carrieri-Kohlman

, Rankin

, Slaughter

, Stulbarg

. Internet-based patient education and support interventions: A review of evaluation studies and directions for future research. Comput Biol Med, 2004; 34:95–112.

Abraham

, Michie

. A taxonomy of behavior change techniques used in interventions. Health Psychol, 2008; 27:379–387.

Green

, Kreuter

. Health promotion and planning: An educational and ecological approach. Mountain View, CA: Mayfield, 1999.

Kok

, Schaalma

, Ruiter

RAC

, Van Empelen

, Brug

. Intervention mapping: Protocol for applying health psychology theory to prevention programmes. J Health Psychol, 2004; 9:85–98.

Crutzen

, De Nooijer

, Brouwer

, Oenema

, Brug

, De Vries

. A conceptual framework for understanding and improving adolescents' exposure to Internet-delivered interventions. Health Promot Int, 2009; 24:277–284.

10.

Fogg

. Persuasive technology: Using computers to change what we think and do. San Francisco: Morgan Kaufmann Publishers, 2003.

11.

Ritterband

, Thorndike

, Cox

, Kovatchev

, Gonder-Frederick

. A behavior change model for Internet interventions. Ann Behav Med, 2009; 38:18–27.

12.

Walther

, Pingree

, Hawkins

, Buller

. Attributes of interactive online health information systems. J Med Internet Res, 2005; 7:e33.

13.

Dixon-Woods

, Cavers

, Agarwal

, Annandale

, Arthur

, Harvey

, Hsu

, Katbamna

, Olsen

, Smith

, Riley

, Sutton

. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol, 2006; 6:35.

14.

Higgins

JPT

, Green

. Cochrane handbook for systematic reviews of interventionsVersion 5.1.0 [updated March 2011] [Online].www.cochrane-handbook.org/. 2010 October 1.

15.

Strauss

, Corbin

. Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage Publications, 1983.

16.

Marshall

. Sampling for qualitative research. Fam Pract, 1996; 13:522–526.

17.

Bickmore

, Gruber

, Picard

. Establishing the computer-patient working alliance in automated health behavior change interventions. Patient Educ Couns, 2005; 59:21–30.

18.

Ferney

, Marshall

, Eakin

, Owen

. Randomized trial of a neighborhood environment-focused physical activity website intervention. Prev Med, 2009; 48:144–150.

19.

Hageman

, Walker

, Pullen

. Tailored versus standard Internet-delivered interventions to promote physical activity in older women. J Geriatr Phys Ther, 2005; 28:28–32.

20.

Hurling

, Catt

, De Boni

, Fairley

, Hurst

, Murray

, Richardson

, Sodhi

. Using Internet and mobile phone technology to deliver an automated physical activity program: Randomized controlled trial. J Med Internet Res, 2007; 9:e7.

21.

Hurling

, Fairley

, Dias

. Internet-based exercise intervention systems: Are more interactive designs better? Psychol Health, 2006; 21:757–772.

22.

Leslie

, Marshall

, Owen

, Bauman

. Engagement and retention of participants in a physical activity website. Prev Med, 2005; 40:54–59.

23.

Marks

, Campbell

, Ward

, Ribisl

, Wildemuth

, Symons

. A comparison of web and print media for physical activity promotion among adolescent girls. J Adolesc Health, 2006; 39:96–104.

24.

Marshall

, Leslie

, Bauman

, Marcus

, Owen

. Print versus website physical activity programs. A randomized trial. Am J Prev Med, 2003; 25:88–94.

25.

Napolitano

, Fotheringham

, Tate

, Sciamanna

, Leslie

, Owen

, Bauman

, Marcus

. Evaluation of an Internet-based physical activity intervention: A preliminary investigation. Ann Behav Med, 2003; 25:92–99.

26.

Parrott

, Tennant

, Olejnik

, Poudevigne

. Theory of planned behavior: Implications for an email-based physical activity intervention. Psychol Sport Exerc, 2008; 9:511–526.

27.

Sciamanna

, Lewis

, Tate

, Napolitano

, Fotheringham

, Marcus

. User attitudes toward a physical activity promotion website. Prev Med, 2002; 35:612–615.

28.

Spittaels

, De Bourdeaudhuij

, Brug

, Vandelanotte

. Effectiveness of an online computer-tailored physical activity intervention in a real-life setting. Health Educ Res, 2007; 22:385–396.

29.

Spittaels

, De Bourdeaudhuij

, Vandelanotte

. Evaluation of a website-delivered computer-tailored intervention for increasing physical activity in the general population. Prev Med, 2007; 44:209–217.

30.

Brendryen

, Drozd

, Kraft

. A digital smoking cessation program delivered through the Internet and cell phone without nicotine replacement (Happy Ending): Randomized controlled trial. J Med Internet Res, 2008; 10:e51.

31.

Danahar

, Smolkowski

, Seeley

, Severson

. Mediators of a successful web-based smokeless tobacco cessation program. Addict, 2008; 103:1706–1712.

32.

Patten

, Croghan

, Meis

, Decker

, Pingree

, Colligan

, Dornelas

, Offord

, Boberg

, Baumberger

, Hurt

, Gustafson

. Randomized clinical trial of an Internet-based versus brief office intervention for adolescent smoking cessation. Patient Educ Couns, 2006; 64:249–258.

33.

Patten

, Rock

, Meis

, Decker

, Colligan

, Pingree

, Dornelas

, Offord

, Boberg

, Gustafson

. Frequency and type of use of a home-based, Internet intervention for adolescent smoking cessation. J Adolesc Health, 2007; 41:437–443.

34.

Severson

, Gordon

, Danahar

, Akers

. ChewFree.com: Evaluation of a web-based cessation program for smokeless tobacco users. Nicotine Tob Res, 2008; 10:381–391.

35.

Booth

, Nowson

, Matters

. Evaluation of an interactive, Internet-based weight loss program: A pilot study. Health Educ Res, 2008; 23:371–381.

36.

McConnon

, Kirk

SFL

, Cockroft

, Harvey

, Greenwood

, Thomas

, Ransley

, Bojke

. The Internet for weight control in an obese sample: Results of a randomised controlled trial. BMC Health Serv Res, 2007; 7:206.

37.

McConnon

, Kirk

SFL

, Ransley

. Process evaluation of an Internet-based resource for weight control: Use and views of an obese sample. J Nutr Educ Behav, 2009; 41:261–267.

38.

Franko

, Cousineau

, Trant

, Green

, Rancourt

, Thompson

, Ainscough

, Mintz

, Ciccazzo

. Motivation and self-efficacy, physical activity, and nutrition in college students: Randomized controlled trial of an Internet-based education program. Prev Med, 2008; 47:369–377.

39.

Park

, Nitze

, Kritsch

, Kattelmann

, White

, Boeckner

, Lohse

, Hoerr

, Greene

, Zhang

. Internet-based interventions have potential to affect short-term mediators and indicators of dietary behavior of young adults. J Nutr Educ Behav, 2008; 40:288–297.

40.

Berman

RLH

, Iris

, Bode

, Dregenberg

. The effectiveness of an online mind-body intervention for older adults with chronic pain. J Pain, 2009; 10:68–79.

41.

Nguyen

, Donesky-Cuenco

, Wolpin

, Rineke

, Benditt

, Paul

, Carrieri-Kohlman

. Randomized controlled trial of an Internet-based versus face-to-face dyspnea self-management program for patients with chronic obstructive pulmonary disease: Pilot study. J Med Internet Res, 2008; 10:e9.

42.

Lorig

, Pitter

, Laurent

, Plant

. The Internet-based arthritis self-management program: A one-year randomized trial for patients with arthritis or fibromyalgia. Arthritis Rheum, 2008; 59:1009–1017.

43.

Bowen

, Ludwig

, Bush

, Unruh

, Meischke

, Wooldridge

, Robbins

. Early experience with a web-based intervention to inform risk of breast cancer. J Health Psychol, 2003; 8:175–186.

44.

Cline

RJW

, Haynes

. Consumer health information seeking on the Internet: The state of the art. Health Educ Res, 2001; 16:671–692.

45.

Danahar

, McKay

, Seeley

. The information architecture of behavior change websites. J Med Internet Res, 2005; 7:e12.

46.

Kreuter

, Skinner

. Tailoring: What's in a name? Health Educ Res, 2000; 15:1–4.

47.

Kreuter

, Strecher

, Glassman

. One size does not fit all: The case for tailoring print materials. Ann Behav Med, 1999; 21:276–283.

48.

Eysenbach

, Köhler

. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in depth interviews. Br Med J, 2002; 324:573–577.

49.

Farzanfar

. When computers should remain computers: A qualitative look at the humanization of health care technology. Health Inform J, 2006; 12:239–254.

50.

Kelly

, Connolly

, Bilous

, Stewart

, Nag

, Bowes

, Loader

, Muncer

. Reader beware: Diabetes advice on the web. Pract Diabetes Int, 2002; 19:108–110.

51.

Smaglik

, Hawkins

, Pingree

, Gustafson

, Boberg

, Bricker

. The quality of interactive computer use among HIV-infected individuals. J Health Commun, 1998; 3:53–68.

52.

Dombrowski

, Sniehotta

, Avenell

, Johnston

, MacLennan

, Araújo-Soares

. Identifying active ingredients in complex behavioural interventions for obese adults with obesity-related co-morbidities or additional risk factors for co-morbidities: A systematic review. Health Psychol Rev 10.1080/17437199.2010.513298.

53.

Eysenbach

, Powell

, Englesakis

, Rizo

, Stern

. Health related virtual communities and electronic support groups: Systematic review of the effects of online peer to peer interactions. BMJ, 2004; 328:1166–1170.

54.

Michie

, Abraham

, Whittington

, McAteer

, Gupta

. Effective techniques in healthy eating and physical activity interventions: A meta-regression. Health Psychol, 2009; 28:690–701.

55.

Noar

, Benac

, Harris

. Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychol Bull, 2007; 133:673–693.

56.

Anderson

, Winett

, Wojcik

. Self-regulation, self-efficacy, outcome expectations, and social support: Social cognitive theory and nutrition behavior. Ann Behav Med, 2007; 34:304–312.

57.

Fogg

, Marshall

, Kameda

, Soloman

, Rangnekar

, Boyd

, Brown

. Web credibility research: A method for online experiments and early study results. Proceedings of the ACM CHI Conference on Human Factors in Computing Systems. Seattle, WA. New York: ACM SGICHI, 2001; 295–296

58.

Fogg

, Marshall

, Laraki

, Osipovich

, Varma

, Fang

, Paul

, Rangnekar

, Shon

, Swani

, Treinen

. What makes websites credible? A report on a large quantitative survey. Proceedings of the ACM CHI Conference on Human Factors in Computing Systems. Seattle, WA. New York: ACM SGICHI, 2001; 61–68.

59.

Fogg

, Marshall

, Osipovich

, Varma

, Laraki

, Fang

, Paul

, Rangnekar

, Shon

, Swani

, Treinen

. Elements that affect web credibility: Early results from a self-report study. Proceedings of the ACM CHI Conference on Human Factors in Computing. Systems, The Hague, The Netherlands. New York: ACM SGICHI, 2000; 287–288.

60.

Kreuter

, Bull

, Clark

, Oswald

. Understanding how people process health information: A comparison of tailored and nontailored weight-loss materials. Health Psychol, 1999; 18:487–494.

61.

van der Heijden

. Factors influencing the usage of websites: The case of a generic portal in the Netherlands. Inform Manage, 2003; 40:541–549.

62.

Fogg

. Standford guidelines for web credibility. A research summary from the Stanford Persuasive Technology Lab [Online] 2002. http://credibility.stanford.edu/guidelines/index.html. 2010 October 12.

63.

Krug

. Don't make me think. A common sense approach to web usability. Berkeley, CA: New Riders, 2006.

64.

Sillence

, Briggs

, Fishwick

, Harris

. Guidelines for developing trust in health websites. Proceedings of the 14th International World Wide Web Conference, Chiba, Japan. New York: ACM, 2005; 1026–1027

65.

U.S. Department of Health and Human Services. Usability.gov. Your guide for developing usable and useful websites. www.usability.gov/. 2010 October 12.

66.

Norman

, Zabinski

, Adams

, Rosenberg

, Yaroch

, Atienza

. A review of ehealth interventions for physical activity and dietary behavior change. Am J Prev Med, 2007; 33:336–345.

67.

van den Berg

, Schoones

, Vliet Vlieland

TPM

. Internet-based physical activity interventions: A systematic review of the literature. J Med Internet Res, 2007; 9:e26.

68.

Hager

, Hardy

, Aldana

, George

. Evaluation of an Internet, stage-based physical activity intervention. Am J Health Educ, 2002; 33:329–335.

69.

Kosma

, Cardinal

, McCubbin

. A pilot study of a web-based physical activity motivational program for adults with physical disabilities. Disabil Rehabil, 2005; 27:1435–1442.

70.

McKay

, King

, Eakin

, Seeley

, Glasgow

. The Diabetes Network Internet-based physical activity intervention. A randomized pilot study. Diabetes Care, 2001; 24:1328–1334.

71.

Palmer

, Graham

, Elliot

. Effects of a web-based health program of fifth grade children's physical activity knowledge, attitudes and behavior. Am J Health Educ, 2005; 36:86–93.

72.

Pinto

, Friedman

, Marcus

, Kelley

, Tennstedt

, Gillman

. Effects of a computer-based, telephone-counseling system on physical activity. Am J Prev Med, 2002; 23:113–120.

73.

Anderson

, Winett

, Wojcik

, Winett

, Bowden

. A computerized social cognitive intervention for nutrition behavior: Direct and mediated effects on fat, fiber, and vegetables, self-efficacy, and outcome expectancies among food shoppers. Ann Behav Med, 2001; 23:88–100.

74.

Baranowski

, Baranowski

, Cullen

, Marsh

, Islam

, Zakeri

, Honess-Morreale

, deMoor

. Squire's Quest! Dietary outcome evaluation of a multimedia game. Am J Prev Med, 2003; 24:52–61.

75.

Block

, Block

, Wakimoto

, Block

. Demonstration of an e-mailed worksite nutrition intervention program [abstract] Prev Chronic Dis, 2004; 1:A06.

76.

Block

, Wakimoto

, Metz

, Fujii

, Feldman

, Mandel

, Sutherland

. A randomized trial of the Little by Little CD-ROM: Demonstrated effectiveness in increasing fruit and vegetable intake in a low-income population [abstract] Prev Chronic Dis, 2004; 1:A08.

77.

Delichatsios

, Friedman

, Glanz

, Tennstedt

, Smigelski

, Pinto

, Kelley

, Gillman

. Randomized trial of a “talking computer” to improve adults' eating habits. Am J Health Promot, 2001; 15:215–224.

78.

Irvine

, Ary

, Grove

, Gilfillan-Mortan

. The effectiveness of an interactive multimedia program to influence eating habits. Health Educ Res, 2004; 19:290–305.

79.

McKay

, Glasgow

, Feil

, Boles

, Barrera

Jr . Internet-based diabetes self-management and support: Initial outcomes from the Diabetes Network Project. Rehabil Psychol, 2002; 47:31–48.

80.

Oenema

, Tan

, Brug

. Short-term efficacy of a web-based computer-tailored nutrition intervention: Main effects and mediators. Ann Behav Med, 2005; 29:54–63.

81.

Papadaki

, Scott

. The Mediterranean Eating in Scotland Experience Project: Evaluation of an Internet-based intervention promoting the Mediterranean diet. Br J Nutr, 2005; 94:290–298.

82.

Verheijden

, Bakx

, Akkermans

, van den Hoogen

, Godwin

, Rosser

, van Staveren

, van Weel

. Web-based targeted nutrition counselling and social support for patients at increased cardiovascular risk in general practice: Randomized controlled trial. J Med Internet Res, 2004; 6:e44.

83.

Frenn

, Malin

, Brown

, Greer

, Fox

, Greer

, Smyczek

. Changing the tide: An Internet/video exercise and low-fat diet intervention with middle school students. Appl Nurs Res, 2005; 18:13–21.

84.

Harvey-Berino

, Pintauro

, Buzzell

, Casey Gold

. Effect of Internet support on the long-term maintenance of weight loss. Obes Res, 2004; 12:320–329.

85.

Harvey-Berino

, Pintauro

, Buzzell

, DiGiulio

, Casey Gold

, Moldovan

, Ramirez

. Does using the Internet facilitate the maintenance of weight loss? Int J Obes, 2002; 26:1254–1260.

86.

Harvey-Berino

, Pintauro

, Casey Gold

. The feasibility of using Internet support for the maintenance of weight loss. Behav Modif, 2002; 26:103–116.

87.

Kypri

, McAnally

. Randomized controlled trial of a web-based primary care intervention for multiple health risk behaviors. Prev Med, 2005; 41:761–766.

88.

Plotnikoff

, McCargar

, Wilson

, Loucaides

. Efficacy of an email intervention for the promotion of physical activity and nutrition behavior in the workplace context. Am J Health Promot, 2005; 19:422–429.

89.

Prochaska

, Sallis

. A randomized controlled trial of single versus multiple health behavior change: Promoting physical activity and nutrition among adolescents. Health Psychol, 2004; 23:314–318.

90.

Southard

, Southard

, Nuckolls

. Clinical trial of an Internet-based case management system for secondary prevention of heart disease. J Cardiopulm Rehabil, 2003; 23:341–348.

91.

Vandelanotte

, De Bourdeaudhuij

, Sallis

, Spittaels

, Brug

. Efficacy of sequential or simultaneous interactive computer-tailored interventions for increasing physical activity and decreasing fat intake. Ann Behav Med, 2005; 29:138–146.

92.

Veverka

, Anderson

, Auld

, Coulter

, Kennedy

, Chapman

. Use of the stages of change model in improving nutrition and exercise habits in enlisted Air Force men. Mil Med, 2003; 168:373–379.

93.

Michie

, Fixsen

, Grimshaw

, Eccles

. Specifying and reporting complex behaviour change interventions: The need for a scientific method. Implement Sci, 2009; 4:40.

94.

Kraft

, Yardley

. Current issues and new directions in psychology and health: What is the future of digital interventions for health behaviour change? Psychol Health, 2009; 24:615–618.

95.

Collins

, Murphy

, Strecher

. The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART), New methods for more potent eHealth interventions. Am J Prev Med, 2007; 32,5 Suppl:S112–S118.

96.

Griffiths

, Lindenmeyer

, Powell

, Lowe

, Thorogood

. Why are healthcare interventions delivered over the Internet? A systematic review of the published literature. J Med Internet Res, 2006; 8:e10.