A study of a Dutch online decision aid for parents of children with ADHD

Abstract

During a three-month study period, visitors to an online decision aid (DA) for attention deficit hyperactivity disorder (ADHD) were invited to complete an online questionnaire before and after working through the DA. Some 75,000 unique visitors found their way to the page on DAs, although fewer than 1 in 10 visited the DA for ADHD, staying there for about six minutes on average and using 8–9 clicks to navigate. A total of 195 people completed the first questionnaire (a response rate of about 3%). Only 12 of the respondents to the first questionnaire (6%) completed the second questionnaire. There was no significant effect of the DA as measured by three decisional outcome measures. Respondents moderately appreciated the information received. Even though the DA was constructed according to evidence-based guidelines and International Patient Decision Aids Standards wherein expert and patient involvement are assured, these preliminary results suggest that the online DA for ADHD needs further work to support the decision-making process of parents with regard to the most appropriate treatment for their child.

Introduction

Since the 1990s, reform of the Dutch health-care system has been a prominent matter in Dutch health policy.¹ Competition and choice became guiding principles for change. The main rationale had been the continuing rise in health expenditures in combination with demographic trends and technological developments, i.e. increasing diagnostic and curative possibilities. A more consumerist, choice-oriented approach to the delivery of public services took place in other Western countries as well.² The enactment of the National Health Insurance Act in 2006 brought about a shift towards privatized health insurances and there were major changes in the rules and regulations concerning health care in the Netherlands. For consumers and patients this implied the possibility of increased choice in arrangements for cure, care and prevention.

The role of information with regard to choice and competition in health care is essential. All parties in the health ‘market’ need the same (comparative) information on cost and quality if proper choices are to be made and competition is going to work.³ The Dutch Ministry of Health therefore initiated a health-care portal on the Internet.

The National Institute for Public Health and the Environment (RIVM) was requested to develop a new portal on the Internet and to open telephone lines and help desks in local information centres. The portal was dubbed kiesBeter.nl (‘make better choices’) and aimed at a general Dutch audience (over 16 years old) of average educational background. The portal has been available since 2004. Consumers have access to integrated, independent and reliable information on health, care and health-care insurance services.

In 2004 the Netherlands Organisation for Health Research and Development initiated the Dutch Decision Aids implementation Programme (D-DAP).⁴ Decision support technology assists patients in making informed choices about treatment options, including the in dubiis abstine decision (in case of doubt, abstain from intervening), from a patients' perspective. The information is based on evidence-based medical guidelines. Decision aids therefore support the decision-making process, empower patients and supplement patient-clinician interactions. DAs thus fit into the model of shared decision-making where the care provider and care consumer aim for mutual agreement and effort with regard to medical decisions⁵ thus contributing to patient empowerment. From a meta-analysis by O'Connor et al. ⁶ it appears that DAs are effective with regard to improved knowledge, realistic expectations as to the benefits and risks of treatment options, patient participation in the decision-making process and decreasing decisional conflict.

D-DAP was part of a wider research and development schedule supported by the Dutch government and designed to provide health-care consumers with tools to strengthen their position in a demand-driven health-care system. All DAs within the D-DAP were developed according to the International Patient Decision Aid Standards,⁷ based on a theoretical framework⁸ and subject to a production and validation process involving experts in medical decision-making, researchers, health professionals and patients.⁴ They are made available through the national health portal. As such they are constructed using ASP.NET according to government guidelines and W3C standards with regard to accessibility and sustainability. Currently there are 17 DAs available online, four of which concern mental health problems such as depression, anxiety disorders and attention deficit hyperactivity disorder (ADHD) among children.

Recent approaches in decision support technology research have evaluated either the outcome decision or the decision-making process itself.⁹ In the present study we took the latter approach with regard to the decision-making process of parents or caretakers and their care providers on the treatment of ADHD among children. Extrapolated from foreign studies it is estimated that 3–5% of Dutch children (under 16 years old) suffer from ADHD.^10,11,12 Some 60,000–100,000 children (5–14 years old) meet the criteria for this diagnosis. It is further estimated that 40,000 children need treatment for ADHD.¹⁰

Three aspects of decision-making were investigated in the present study: the stage of decision-making, decisional conflict and the knowledge of parents who are looking for a treatment for their ADHD-diagnosed child. The acceptability of the DA for parents was also measured.

Methods

The study entailed a pre-/post-test design using an online survey of visitors of the DA for ADHD. The log-on screen of the general DA starting page is shown in Figure 1. Visitors who clicked on the link about DAs for ADHD were directed to a page where they were asked to complete a questionnaire before and after completing the DA. Visitors could decide to carry on using the DA or – by default – to participate in the study anonymously (Figure 2).

Figure 1

The log-on screen of the general DA (‘Keuzehulpen’) starting page

Figure 2

Intermediate page for the DA on ADHD among children (‘keuzehulp ADHD’)

If they decided on the latter, then a new window opened and they were referred to the first online questionnaire (Q₁). When respondents had completed it they could navigate through the DA on ADHD at their own pace. Once finished they were referred to the second questionnaire (Q₂).

The convenience sample was taken from parents or caretakers of children (6–18 years old) recently diagnosed with ADHD. They were either referred by schools or by care providers, or found the DA by referrals from websites on ADHD or via search engines.

The data in the present study were collected during a three-month period.

Questionnaires

The first questionnaire started with basic background variables and two exclusion questions about ADHD actually and recently being assessed and about the respondents' experience with the present DA. The semi-structured questionnaires were constructed using O'Connor's work and manuals,^13,14,15 as well as expert opinion (P Stalmeier, personal communication).

Five valid and relevant constructs were measured in the two questionnaires in order to establish an effect of using the DA. The first is the stage of the decisional process, to determine how far respondents are in deciding about the treatment options for the child in question (e.g. ‘Several treatments are available for children with ADHD. Have you thought about a treatment for your child?’ with a response on a five-point Likert-type scale). The second is decisional conflict meaning the extent of uncertainty the respondent experiences about making a decision (e.g. ‘I know the advantages of every treatment’ with a response on a five-point Likert-type scale). The third is knowledge (‘Would you please rate your knowledge on ADHD and its treatment possibilities’ with a response on a 1–10 numerical scale) as assessed by the respondents themselves. Apart from these decisional constructs, two other features were measured: acceptability, the degree of satisfaction with the (amount of) information and preparation, the extent to which respondents feel the DA contributed to a better preparation for the doctor's visit and for deciding.

The instruments were pre-tested by representative testers in an online environment.

Results

The portal (http://www.kiesBeter.nl) attracted approximately 900,000 unique visitors during the three-month study period. (A unique visitor is usually defined as a visitor logging on from a single IP address and not returning within 24 hours). About one-third of them visited the subsite on medical information. About one-quarter of these, some 75,000 unique visitors, found their way to the DAs (Figure 3). Fewer than 1 in 10 DA visitors visited the DA on ADHD, staying there for about six minutes on average and using 8–9 clicks to navigate (Figure 4). Thus in the 3 months of the study, about 7500 people visited the ADHD page.

Figure 3

Numbers of unique visitors to the health portal, to the medical information subsite and to all DAs during a four-month period

Figure 4

Number of unique visitors to the DA, duration of visit and number of clicks

A total of 195 people completed the first questionnaire (a response rate of about 3%). Only 12 of the respondents to the first questionnaire (6%) completed the second questionnaire (Table 1). Most of them were female, about 38 years of age and of more than average educational background. The children concerned were 6–10 years of age.

Table 1

Background and dependent variables for respondents completing each questionnaire

	Mean values Q₁ (n = 195)	Mean values Q₂ (n = 12)
Background variables
Sex
No. of males (%)	50 (26)	2 (17)
No. of females (%)	145 (74)	10 (83)
Age
Male (years)	41 (6)	39 (4)
Female (years)	39 (5)	39 (3)
Child concerned (years)	9 (3)	8 (2)
Educational level* (SD)	4.7 (1.5)	4.8 (1.8)
Dependent variables
Stage of decisional process^† (SD)	2.4 (1.1)	2.6 (1.1)
Decisional conflict^‡ (SD)	68 (26)	55 (30)
Knowledge^§ (SD)	6.2 (1.9)	6.5 (1.9)

*Educational level was the highest education completed by the respondent, ranging from 1 (primary school) to 7 (academic education)

^†Stage of decisional process measured how far respondents were in deciding about the treatment options for the child, on a five-point Likert type scale ranging from 1 = at the very beginning of the decision-making process (‘I haven't given it a thought’) to 5 = the end of the process (‘I have chosen a treatment for my child and I feel confident about it’)

^‡Decisional conflict measured the extent of uncertainty that the respondent experienced about making a decision. The scale was composed of three subscales (Uninformed [3 items, e.g. ‘I know from which treatment options I can choose’], Values [2 items, e.g. ‘It is clear to me which advantages are most important for me’] and Uncertainty [2 items, e.g. ‘I know the disadvantages of every treatment option’]) all with five-point Likert type answers. The scores were aggregated and recoded on a 0–100 scale where 0 = no decisional conflict whatsoever and 100 = full force decisional conflict

^§Knowledge was measured by self-rating on a 10-point scale (‘Would you please rate your knowledge on ADHD and its treatment possibilities’)

For respondents (n = 12) completing both questionnaires, there were no significant differences between them with regard to the stage of decisional process, decisional conflict and knowledge (Table 2). There appeared to be a small, non-significant increase in both decisional conflict and the self-report on knowledge, as well as a small, non-significant decrease of the stage of decision making of the respondents.

Table 2

Mean values of the dependent variables from those respondents who completed both questionnaires

	Mean Q₁ (n = 12)	Mean Q₂ (n = 12)	z value	P value
Stage of decisional process (SD)	2.6 (1.1)	2.2 (0.7)	−1.4	0.16
Decisional conflict (SD)	55 (30)	57 (27)	−0.1	0.94
Knowledge (SD)	6.5 (1.9)	6.7 (1.5)	−0.1	0.89

The respondents' answers indicated that the DA contributed to better preparation for decisions and facilitated decision-making (63%). Satisfaction with the information received through the DA was moderate, and 60% felt that the information was too limited. The overall acceptability of the DA on ADHD was average (i.e. approximately midway between unacceptable at the one end of a Likert-type scale and complete acceptance at the other).

Coherence, as determined by the correlation coefficient between the three dependent variables appeared to be average to strong (Table 3).

Table 3

Correlation coefficients between the three dependent variables (n = 195)

	Decisional conflict	Knowledge
Stage of decisional process	−0.62*	1.00**
Knowledge	−0.62*	−

*P < 0.05; **P < 0.01

There was moderate correlation between decisional conflict and stage of decisional process, and between decisional conflict and knowledge of ADHD. There was a strong correlation between the stage of decisional process and knowledge. Higher decisional conflict implies that respondents are in the first stages of their decision-making process, and that they possess relatively little knowledge about ADHD and treatment options. When respondents progress in their decision-making process their knowledge of ADHD increases.

Consistency, reliability and internal validity of the constructs built into the questionnaires as measured with Cronbach's alpha appeared to be high (data not reported).

Discussion

In the present study, the DA on ADHD did not seem to influence transitions of respondents to another stage in their decision-making process. The respondents indicated more decisional conflict and increase of knowledge, although at a non-significant level. They rated the acceptability of the DA, in terms of satisfaction and information needs, at an average level. The observation that most respondents (94%) did not complete both questionnaires suggests that the navigational flow and usability of the online DA may have been too complicated.

These outcomes concern the group of respondents and cannot be generalized. The present study is still ongoing and a substantial increase in respondents may change the strength and direction of the effect. These preliminary results are nonetheless important with regard to the DA on ADHD. It appears that the basic objectives are not sufficiently met. Development of decision support technology is in its initial stage in the Netherlands, although serious steps have been taken that are widely supported by scientific, professional and patients' organizations. Little research has been done with regard to its effectiveness. The emphasis has been on the scientific underpinning of both content and process, while underestimating the importance of the online user-experience. Improvement may be reached by increasing the usability of the DA and its acceptability by users.

Footnotes

Acknowledgments

We thank ICJ van Miert, V Siemers Reiter and FLM Taken (Utrecht University) for their work on data collection, JJ Colijn (RIVM) for her valuable input and P Stalmeijer (Radboud University Nijmegen) for his advice.

References

Helderman

, Schut

, van der Grinten

, van de Ven

WP.

Market-oriented health care reforms and policy learning in the Netherlands. J Health Polit Policy Law 2005;30:189–209

Clarke

, Newman

, Smith

, Vidler

, Westmarland

Creating Citizen-Consumers. London: SAGE Publications, 2007

VWS (Ministry of Health Welfare and Sport). Choosing with Care. The Equipping of Patients and Consumers in a Demand Driven Care Sector. International Publication Series Health Welfare and Sport. The Hague: VWS, 2001

Raats

, van Veenendaal

, Versluijs

, Burgers

JS.

A generic tool for development of decision aids based on clinical practice guidelines. Patient Educ Couns 2008;73:413–7

Charles

, Gafni

, Whelan

Shared decision-making in the medical encounter: what does it mean? (or it takes at least two to tango). Soc Sci Med 1997;44:681–92

O'Connor

, Bennett

, Stacey

, Decision aids for people facing health treatment or screening decisions. Cochrane Database of Systematic Reviews 2006;3:CD001431

International Patient Decision Aid Standards (IPDAS) Collaboration. See http://www.ipdas.ohri.ca (last checked 19 October 2009)

Durand

, Stiel

, Boivin

, Elwyn

Where is the theory? Evaluating the theoretical frameworks described in decision support technologies. Patient Educ Couns 2008;71:125–35

Stalmeier

, Roosmalen

, Verhoef

, The decision evaluation scales. Patient Educ Couns 2005;57:286–93

10.

Gezondheidsraad (Health Council of the Netherlands). Diagnostiek en behandeling van ADHD [Diagnostics and treatment of ADHD] . Den Haag: Gezondheidsraad, 2000 [Dutch]

11.

Buitelaar

, Kooij

Aandachtstekort-hyperactiviteitstoornis (ADHD); achtergronden, diagnostiek en behandeling [ADHD; backgrounds, diagnostics and treatment]. Nederlands Tijdschrift voor Geneeskunde 2000;144:1716–23 [Dutch]

12.

Swanson

, Sergeant

, Taylor

, Sonuga-Barke

, Jensen

, Cantwell

DP.

Attention-deficit hyperactivity disorder and hyperkinetic disorder. Lancet 1998;351:429–33

13.

O'Connor

AM.

User Manual – Decisional Conflict Scale. See http://decisionaid.ohri.ca/eval.html#DecisionalConflictScale (last checked 27 October 2009)

14.

O'Connor

AM.

User Manual – Stage of Decision Making. See http://decisionaid.ohri.ca/eval.html#StageDecisionMaking (last checked 27 October 2009)

15.

O'Connor

AM.

Validation of a decisional conflict scale. Med Decis Making 1995;15:25–30