User testing digital,multimedia information to inform children,adolescents and their parents about healthcare trials

Abstract

Digital, multimedia information resources (MMIs) containing text, video, animation and pictures are a promising alternative to written participant information materials designed to inform children, adolescents and parents about healthcare trials, but little research has tested whether they are fit for purpose. This study employed a consecutive groups design and user testing questionnaire to assess whether participants were able to find and understand key information in multimedia resources. Two rounds of testing were completed. In each round, seven children aged 7–11 tested the MMI with a parent; six adolescents aged 12–17 and seven parents tested the MMI independently. After round 1, the resources were revised based on participant scores, behaviour and feedback. Round 1 identified problems with 2/10 information items (length of trial and use of insulin pump); only 3/20 participants could locate all information items without difficulty. After revisions, 14/20 participants scored a clear round. Information comprehension was high: 96% understood in round 1 and 99% in round 2. Participant feedback on the multimedia resources was positive, although presentation preferences varied. User testing was employed successfully with children, adolescents and parents to identify issues with, and improve, multimedia resources developed to inform potential healthcare trial participants.

Keywords

Adolescents children education informed consent multimedia information parents patient information trials user testing website

Introduction

Randomized control trials (RCTs) potentially provide unbiased evidence of effectiveness (Ross et al., 1999; Treweek et al., 2013). Potential participants must be given sufficient information about the trial, traditionally via printed participant information sheets (PIS), to inform their decision about participation (World Medical Association, 2013). This requirement is one of a number which must be met in order for participant consent to be deemed valid; consent must also be voluntarily given by a person with capacity (Health Research Authority, 2018a). In paediatric trials, children and adolescents deemed competent provide consent; when not, parents or legal guardians consent for the child and, where appropriate, the child assents (Health Research Authority, 2018b). Indeed, research indicates that children prefer to be informed and involved in healthcare decision-making (Baker et al., 2013; Grootens-Wiegers et al., 2015). Therefore, it is crucial that accessible information is provided (Health Research Authority, 2018b; Hunter and Pierscionek, 2007).

However, participants do not always understand key trial concepts, such as their right to refuse or withdraw from research (Cox et al., 2006; Tam et al., 2015). Even when prospective participants feel informed (as indicated by participant ratings of their own understanding), they may not recall important details such as the trial aim and duration (Jefford and Moore, 2008; McCarty et al., 2007). In paediatric trials, children, adolescents and their parents may find it more difficult than adult participants to comprehend trial information (Barfield and Church, 2005; Simon et al., 2004); for example, those making decisions about a paediatric clinical trial were less able to correctly answer questions regarding the meaning of randomization and voluntariness (Simon et al., 2004). This lack of understanding is one of many factors which could impact negatively on informed consent, recruitment, protocol fidelity and retention (Hallinan et al., 2016; Jefford et al., 2011; Ross et al., 1999; Stryker et al., 2006). For example, a participant who does not fully understand the requirements of the trial may be more likely to withdraw from it due to unanticipated demands.

Typical printed PIS are consistently criticized as too long and complex (Caldwell et al., 2012). For adults, the US National Institutes of Health recommends materials should be written at an 11–12-year-old reading level (National Institutes of Health, 2013), a recommendation not always met (Ennis and Wykes, 2016; Eltorai et al., 2015; Sheppard et al., 2014; Simonds et al., 2017; Terblanche and Burgess, 2010). PIS for children and adolescents are often simplified but may still be too complex for the intended age group (Ford et al., 2007; Grootens-Wiegers et al., 2015; Grossman et al., 1994). A recent study involving researchers and participants highlighted a need for research into the best approach for delivering trial information (Healy et al., 2018). One promising alternative is digital, multimedia information resources (MMIs): websites combining text, videos, animations and pictures. Child and adult participants may prefer MMIs over print (Baker et al., 2013; Nishimura et al., 2013; O’Lonergan and Forster-Harwood, 2011) and they can improve trial understanding, particularly by children (O’Lonergan and Forster-Harwood, 2011; Tait et al., 2015). Current evidence suggests that one in three Internet users around the world is a child or adolescent (UNICEF, 2017), although literacy rates and Internet access vary considerably. Nevertheless, current everyday use of digital devices by a large number of children and adolescents means that MMIs may offer greater familiarity and accessibility than written content.

The TRials Engagement in Children and Adolescents (TRECA) study has two phases. The first phase involved the development and user testing of prototype MMIs designed to provide information to children, adolescents and parents when invited to participate in healthcare trials, the second involves embedding trial-specific versions of the MMIs in six healthcare trials, to evaluate whether they can improve recruitment and retention to trials, and users’ quality of decision-making (Martin-Kerry et al., 2017). In the first part of TRECA (phase 1), two age-appropriate prototype MMIs were developed (one for children aged approximately 5–11 years and their parents – MMI-1 and another for adolescents aged approximately 12–17 years and parents – MMI-2). The aim of this aspect of the TRECA study was to user test the prototype MMIs to ensure they are easy to use and understand, prior to their use in TRECA phase two.

User testing is an iterative process in which changes are made to an information resource in response to data obtained (Raynor et al., 2011). Potential problems can be identified by asking participants to locate and demonstrate understanding of key information items; revisions are based on how easy or difficult participants find these tasks (Knapp et al., 2009a, 2009b, 2010, 2011; Raynor et al., 2011; Raynor, 2013). While the ability of user testing to increase recruitment to trials has been questioned (Treweek et al., 2018), it has successfully improved printed medicine leaflets (Raynor, 2013; Raynor et al., 2011; Yamamoto et al., 2017), clinical trial information (Knapp et al., 2009a, 2009b, 2010, 2011; Man et al., 2015), booklets to facilitate patient–clinician communication (Ahmed et al., 2014) and a website to inform adolescents’ self-management of haemophilia (Breakey et al., 2013).

Method

Participants

Participants were recruited through secondary schools in northern England, investigator networks and publicity flyers. To approximately match the MMIs’ target age-groups, participants’ user testing prototype MMI-1 were 7–11 years old (accompanied by a parent); for prototype MMI-2, participants were adolescents aged 12–17 years and parents of adolescents. Adolescents and parents tested prototype MMI-2 separately, acknowledging adolescents’ greater independence in decision-making.

User testing produces its most valid, insightful data when participants are potential information users without significant relevant prior knowledge or experience (Knapp et al., 2009a), thus participants did not have type 1 diabetes (the focus of the prototype MMIs). Similarly, new participants were recruited for each user testing round. Participants were purposively sampled (i.e. targeted for recruitment in response to the characteristics of participants already recruited) to attempt to ensure maximum variation in age, sex and educational attainment (reading age was not assessed). Participants aged 7–15 years old gave assent; a parent provided consent. Other participants provided consent. Each participant received a £10 voucher to compensate their time.

Tested materials

The tested prototype MMIs were intended for younger children and their parents (MMI-1) and adolescents and parents (MMI-2). Both prototype MMIs contained trial-generic content (e.g. explanations of randomization and research design) and trial-specific content (e.g. procedures and risks) intended to be informative yet engaging. For example, trial-specific content was developed for the prototypes based on subcutaneous insulin: pumps or injections (SCIPI), a recently completed insulin delivery trial recruiting children and young people with type 1 diabetes (Blair et al., 2015). All information from the SCIPI trial PIS was incorporated into the MMIs. Video content was developed from interviews with four people from the SCIPI trial (child and adolescent participants, a participant’s parent and a clinician).

The MMIs, animations and videos were developed in collaboration with a UK commercial digital company, Morph (www.morph.co.uk). Both prototype MMIs comprise a home page and five additional tabs: ‘About the trial’, ‘Taking part’, ‘After the trial’, ‘Questions’ and ‘Contact’. The first three tabs have subtabs for additional information. Both MMIs contained the same information; MMI-1 used simpler text and brighter colours to appeal to younger children. The prototype MMIs were informed by the needs and preferences of children and adolescents with long-term health conditions, their parents and clinicians involved in paediatric research (Martin-Kerry et al., 2018). During initial development of the MMIs (prior to user testing), it was identified that the reading level of the draft text was too demanding, and it was simplified using readability software and feedback from an education professional (SH) with expertise in text comprehension. The development of the MMIs is reported elsewhere (Martin-Kerry et al., 2018).

During user testing, the MMIs were displayed on a laptop or desktop computer, although they can also be viewed on a tablet computer or mobile phone.

Procedure

The researcher (RS) first communicated with participants to build rapport, before explaining that after interacting with the prototype MMI, they would be asked about its content. They were reassured that user testing assesses the MMIs, not participants. Sessions took place at a time and location convenient to the participant (either at home, at school or at the university).

Testing of the original MMIs

Participants were allowed up to 15 minutes (timed by the researcher using a stopwatch) to interact with the prototype MMI; participants informed the researcher when they had finished (if sooner). The researcher then asked participants 10 questions in a fixed order (Table 1) based on key information items in the MMIs; the question order deliberately did not reflect the order of information presentation in the MMIs.

Table 1.

User testing scores for round one.

	Round 1 – Original prototype MMIs, N = 20
	Finding			Understanding
	NF	D	F	NU	UD	U
Q1: Imagine you decided to take part in the SCIPI trial. How long would you take part in the trial for?	6	4	10	0	2	18
Q2: If you took part in the SCIPI trial, what extra things would you need to do at your diabetes clinic visit?	0	2	18	0	1	19
Q3 What is the SCIPI trial testing?	1	1	18	1	0	19
Q4 Imagine you are in the group that uses the pump, when should you wear it?	6	4	10	1	1	18
Q5: Imagine you are in the SCIPI trial. What will happen when the SCIPI trial ends?	0	0	20	0	0	20
Q6: How often would you visit the diabetes clinic as part of the SCIPI trial?	0	0	20	0	0	20
Q7: Are there any risks to taking part in the SCIPI trial – Is it safe?	0	1	19	0	0	20
Q8: Who can take part in the SCIPI trial?	3	1	16	0	1	19
Q9: If you decided you didn’t want to take part in the SCIPI trial anymore, what would you need to do?	0	0	20	0	0	20
Q10: Why do people with type 1 diabetes need to take/inject insulin?	0	2	18	0	1	19
Total (%)	16/200 (8.0)	15/200 (7.5)	169/200 (84.5)	2/200 (1.0)	6/200 (3.0)	192/200 (96.0)

Note: NF: not found; D: found with difficulty; F: found; NU: not understood; UD: understood with difficulty; U: understood; SCIPI: subcutaneous insulin: pumps or injections.

For each question, participants located the required information in the prototype MMI, then explained what it meant using their own words. If participants struggled to find or understand information, the researcher repeated the question and prompted gently for an answer. A two-minute cut-off per question ensured that participants did not become unnecessarily distressed if they could not find or understand information. If participants could not locate the information, the researcher directed them to the relevant content and asked them to explain the information.

Finally, participants were asked for their general opinions of the MMI to determine preferences for style, layout and overall design. Interviews were audio recorded and transcribed verbatim.

Revision and testing of the revised MMIs

Revisions made between rounds were based on user testing scores and participants’ observed behaviour and feedback. After revisions were made, a second user testing round (as above) was completed.

Data analysis

User testing data are indicative, not analysed statistically. Each question was scored by the researcher (RS) for finding (found; found with difficulty e.g. found with prompting; not found) and understanding (understood; understood with difficulty e.g. understood with prompting; not understood). The number of participants able to find and understand each target information item was calculated.

Viewing time, response time and any further behaviour indicating the ease or difficulty of finding or understanding information were recorded by the researcher (RS) to inform revisions. Quotations from participants are used to illustrate points noted in the user testing and as a rationale for MMI revisions. They have not been analysed or developed thematically.

Research ethics

Approval was granted by the Yorkshire and the Humber – Bradford Leeds Research Ethics Committee (16/YH/0387), Health Research Authority (HRA; IRAS ID 213557) and University of York Department of Health Sciences Research Governance Committee (HSRGC/2016/169/A).

Results

Pilot testing

Four participants (one parent, one adolescent and one parent and child pair) took part in pilot user testing to assess questionnaire suitability and timings. No significant issues were identified.

Round 1

Participants

The original prototype MMIs were tested by 26 participants: seven children aged 7–11 years (mean 8.4 years) tested prototype MMI-1 alongside a parent (n = 6; one parent took part with two of their children); six adolescents aged 12–17 years (mean 14.2 years) and seven parents of adolescents tested prototype MMI-2. As such 20 user testing sessions were completed. Fourteen participants were male (53%). Of the parent participants, 10 (76%) had higher education qualifications. All participants spoke English as their first language. The majority of sessions took place during the morning or afternoon (n = 14); six sessions took place in the evening (after 5 pm).

Findings

Participants spent an average of 11.5 minutes (range 5.0–15.0) looking at the MMI before testing and 17.9 minutes (range 6.9–26.5) completing user testing and providing feedback. Child and parent pairs spent longer looking at the MMI (mean 14.1 minutes, range 12.0–14.1) than adolescents (mean 8.6 minutes, range 5.0–13.5) or parents of adolescents (mean 11.5 minutes, range 7.1–14.5).

Ninety-two percent of requested information was found (Table 1). However, 7.5% of this was found with difficulty and only 3 of 20 participants scored a ‘clear round’ in locating all answers without difficulty. In particular, participants had difficulty locating information about trial length (Q1) and the pump (Q4). Comprehension was high: 17 of 20 participants scored a ‘clear round’ for understanding.

MMI revisions

In response to user testing scores, observed behaviour and feedback, some changes were made. Data suggested the MMI layout was suboptimal; participants felt they had to scroll too much, causing frustration and resulting in participants missing information at the bottom of pages:

I scrolled down too fast I think, there’s lots of this, scroll, scroll, scroll sort of thing. (P11, parent of adolescent, 14 years)

You’re having to scroll up and down to see everything and a lot of kids, people won’t bother they’ll just look at the front page. (P23, parent of adolescent, 12 years)

Some participants did not notice the two subtabs under ‘About the trial’ (‘Why is it happening?’, ‘What is being tested?’), possibly due to the first subtab opening automatically (Online Supplementary Appendix 1). The tabs and subtabs also disappeared from the top of the page when scrolling:

There is a problem with the tabs that first one you read and then, even the first time, [child’s name] and I didn’t spot there was a second tab. (P27, parent of two children, 7 and 9 years)

I didn’t see those {tabs}, I think maybe because they’re not quite as bright as the tabs at the bottom…You go down don’t you in order to read the bottom, so you lose the top. (P25, parent of adolescent, 17 years)

This produced specific difficulties in locating information about the insulin pen and pump, exacerbated by the video layout on the page; a line of three videos taking up the width of the screen was located underneath information about the pen (Online Supplementary Appendix 2). Participants assumed this was the end of the web page and did not scroll down for information about the pump:

Because you said about the difference between the pump and the pen, I’d have done a separate page…because it’s confusing because you had videos so you think it stops there. (P34, parent of two adolescents, 14 and 17 years)

Following this feedback, edits included changing subtab colour and size, fixing the tabs and subtabs panel to the top of the web page, separating information about the pen and pump onto different subtabs and re-arranging the video layout.

Participants highlighted that important information should not solely be within video as connectivity issues may mean they do not load, or participants may prefer text. This issue caused difficulties in finding out the trial length (Q1) as it was only located in video. Furthermore, this video was located at the bottom of a web page, but participants considered this information important and expected it to be more prominent. Finally, participants reported being confused and discouraged by some video content, due to language complexity and regional accents:

There was one thing which I found quite hard, it was one she {the clinician} just said stuff a bit fast. (P14, child, 8 years)

Some of the videos were quite hard to understand, certainly the professional, the doctor talking. (P17, parent of two children, 7 and 8 years)

Consequently, additional text was incorporated to ensure that important information was always available as text; the trial length was written at the top of the ‘Taking part’ tab. While it was not possible to re-film video content, comments on complexity were noted for developing the phase 2 MMIs.

Round 2

Participants

The revised prototype MMIs were tested by 26 new participants: seven children aged 7–11 years (mean 8.9 years) tested prototype MMI-1 alongside a parent (n = 6; one parent took part with two children); six adolescents aged 12–17 years (mean 15.0 years) and seven parents of adolescents tested MMI-2. As such, a total of 20 user testing sessions were completed. Twelve (46%) participants were male. Of the parent participants, 12 (92%) had higher education qualifications. All participants spoke English as their first language. The majority of sessions took place during the morning or afternoon (n = 16) and four sessions took place in the evening (after 5 pm).

Findings

Participants spent an average of 9.2 minutes (range 4.4–14.5) looking at the MMI and 15.4 minutes (range 6.8–31.2) completing user testing and providing feedback. Once again younger children and their parents spent longer looking at the MMI (mean 12.4 minutes, range 8.4–14.5) compared to adolescents (mean 7.3 minutes, range 5.5–11.1) and parents of adolescents (mean 9.1 minutes, range 4.4–12.6). Second round participants spent less time looking at the MMI and answering questions than first round participants.

The MMI revisions appeared successful; 93.0% of information was found without difficulty compared to 84.5% in round 1 (Table 2). Fourteen of 20 participants scored a ‘clear round’ for finding information. Persistent difficulties with locating information were largely due to participants reaching the two-minute limit allocated per question. Finally, 99% of the information was understood, and 19 of 20 participants scored a ‘clear round’ for understanding.

Table 2.

User testing scores for round 2.

	Round 2 – revised prototype MMIs, N = 20
	Finding			Understanding
	NF	D	F	NU	UD	U
Q1: Imagine you decided to take part in the SCIPI trial. How long would you take part in the trial for?	0	2	18	0	0	20
Q2: If you took part in the SCIPI trial, what extra things would you need to do at your diabetes clinic visit?	1	2	17	1	0	19
Q3: What is the SCIPI trial testing?	0	1	19	0	0	20
Q4: Imagine you are in the group that uses the pump, when should you wear it?	0	2	18	0	0	20
Q5: Imagine you are in the SCIPI trial. What will happen when the SCIPI trial ends?	0	0	20	0	0	20
Q6: How often would you visit the diabetes clinic as part of the SCIPI trial?	0	1	19	0	0	20
Q7: Are there any risks to take part in the SCIPI trial – Is it safe?	0	0	20	0	0	20
Q8: Who can take part in the SCIPI trial?	1	1	18	0	0	20
Q9: If you decided you didn’t want to take part in the SCIPI trial anymore, what would you need to do?	0	1	19	0	0	20
Q10: Why do people with type 1 diabetes need to take/inject insulin?	2	0	18	1	0	19
Total (%)	4/200 (2.0)	10/200 (5.0)	186/200 (93.0)	2/200 (1.0)	0/200 (.0)	198/200 (99.0)

Note: NF: not found; D: found with difficulty; F: found; NU: not understood; UD: understood with difficulty; U: understood; SCIPI: subcutaneous insulin: pumps or injections.

MMI revisions

The revised MMIs in round 2 performed well and required no further revisions. Previously identified problems with the subtab layout were resolved with round 2 participants noting the ease of navigation:

I think the website is very good, you know, it flows really well and you can clearly see where everything is. (P47, adolescent, 15 years)

The fact that [child’s name] can navigate his way round it, it’s pitched at the right level. (P56, parent of child, 9 years)

Further recommendations were identified for consideration when developing later trial MMIs, including adding information to the ‘Questions’ page about (i) potential risks and (ii) contacting trial staff with unanswered questions. It was suggested that hyperlinks could be used to link relevant MMI pages.

Participant feedback

Feedback from both testing rounds was combined to determine the overall acceptability of the approach. Feedback was largely positive; most participants liked the MMIs’ appearance and found them reassuring and child-friendly:

It’s like child-friendly as well, like the pictures aren’t like, scary or anything. (P54, adolescent, 17 years)

The colours and the presentation and the style of it is quite nice, it does seem good. (P15, parent of two children, 7 and 8 years)

I like how there’s all the little pictures and, there’s on the videos, they look very easy to look at, there’s nothing complicated. (P57, child, 9 years)

I think it’s really good, like the website looks a lot better than even usual websites. It’s very clean, it’s clear navigation as well. (P20, adolescent, 16 years)

The videos were generally received as informative, engaging and reassuring, although not all were found relevant, such as the SCIPI trial participant’s brief video description of the pen and pump, which was also written in text:

There are real people speaking, like, which is a good thing ‘cause it kind of shows evidence that people are actually safe. (P12, child, 10 years)

The sound bites are short enough in length so you don’t lose concentration. (P59, parent of adolescent, 15 years)

There’s a lot of information, and I’m not sure to be honest whether some of these videos were relevant. (P11, parent of adolescent, 14 years)

Participants generally liked the different media, although individual preferences varied:

Some of the cartoony videos were kind of fun to watch and they were also very simple and easy to understand and I think, well I do know a lot more now. (P57, child, 10 years)

It’s quite well balanced in terms of pictures and I suppose it’s those kind of learning styles stuff, some people are quite visual so pictures work and then you’ve got your detail of the wording. (P25, parent of adolescent, 17 years)

Videos is not for me, so I’d just prefer to see it as text so more easily found, so what will happen and it’s there rather than having to click. (P32, parent of two adolescents, 14 and 17 years)

For adults as well I think it’s better than reading, it’s an inclusive thing because you’ve got the, you know, you can hear it, you can see it, and you can read it. (P41, parent of adolescent, 13 years)

Participants liked the diagrams, noting their role as visual cues to locate information. For example, participants remembered the calendar alongside information about clinic visit frequency on the SCIPI trial. However, it is important that the appropriate format is selected; it was noted that the insulin pen and pump were abstract and that photos would be preferred, although participants did not have the prior knowledge of people with diabetes.

Participants found most of the information understandable (with the exception of some video content), although they appreciated that the topic almost necessitated some technical language. Views on the amount of included information differed: some found it succinct, while others noted the large quantity:

It was the right balance of not patronising, but understandable. You feel you’ve come away, well I felt I came away understanding what the thing was. (P10, parent of child, 9 years)

There were certainly bits that I thought were really quite complicated but that she remembered, you know like whether it’s safe to take part. (P17, parent of two children, 7 and 8 years)

Some of the words are quite hard…but I don’t think there’s a way of avoiding some of this terminology. (P11, parent of adolescent, 14 years)

There’s quite a lot of information but it doesn’t feel unmanageable. (P43, parent of adolescent, 13 years)

They’re all simple {words} or if they are big words they’ve been explained to what they mean, which is good. (P54, adolescent, 17 years)

Discussion

User testing of the prototype MMIs highlighted initial difficulties with locating specific information. Nevertheless, most participants could understand the information without difficulty, even if they needed directing to it. Second round participants took less time to review the prototype MMIs and answer user testing questions. Overall, the revisions to the prototype MMIs after round 1 improved their ability to inform, with most participants able to find and understand all requested information. No further revisions were required.

The original prototype MMIs performed reasonably well, especially for understanding. This is likely due to the significant input from relevant stakeholders in the qualitative work during development, as well as attention to text readability during drafting (Martin-Kerry et al., 2018). Similar to other studies (Dickinson et al., 2016; Knapp et al., 2009b, 2011; Raynor et al., 2011), user testing identified problems with locating information. These difficulties may influence the willingness and ability to provide valid consent to trials. That these issues were not identified by stakeholders, including children, adolescents and experts during MMI development, is consistent with previous research showing that experts cannot always anticipate the way that information materials will perform for a range of users (Lentz and De Jong, 1997). This highlights the added value of a performance-based approach to the analysis of participant information.

A time limit (two minutes) was imposed for locating information. This was arbitrary but lower than in other studies (some of which used no time limit) (Knapp et al., 2009a, 2009b, 2010) due to the inclusion of children, who may become bored or distressed by not finding information. While no timing issues were identified during pilot testing, this time limit may be too short given the nature of the MMI content, for example, watching a video may use half the allotted time. As the remaining finding difficulties were largely due to participants not locating the answer in time, it is possible that this time limit has falsely deflated testing results. Further, the external validity of findings is reduced as patients actually approached about trial participation would probably consider recruitment materials for longer. It does suggest that unresolved difficulties with the MMIs may be less problematic in a real scenario.

Understanding of information in the prototype MMIs was high in both rounds, 96% and 99%, respectively. Despite the MMIs containing technical and unfamiliar information, participants generally felt this was communicated well. However, the trial scenario (SCIPI) was relatively simple with two-arm randomization and no identified intervention risks; many trials are more complex. Furthermore, sample sizes were small, and parents in this study were relatively well educated which may have elevated user testing scores: previous research suggests that participants with higher education qualifications are better able to understand informed consent information (Flory and Emanuel, 2004). However, the prototype MMIs were designed to be used primarily by children and adolescents, and their inclusion in user testing should ensure that they are usable by people from a range of backgrounds. Future studies should aim to include a more varied sample to ensure that trial information is accessible to all within the relevant study population. We acknowledge that while the consecutive groups study design employed is appropriate and useful for the iterative development of materials, it is a relatively weak indicator of effectiveness; however, conventional random allocation to groups is not straightforward in user testing.

To the authors’ knowledge, there is little existing user testing research on paediatric medical information materials. However, usability testing has been successfully employed with adolescents and young people to improve digital self-management programmes for haemophilia (Breakey et al., 2013) and asthma (Davis et al., 2017); both studies highlighted the importance of including potential users in their development. Furthermore, young people involved in the usability testing of other media (e.g. video games), highlight different issues to adults (Andersen et al., 2016). In the present study, child and adolescent participants could use the prototype MMIs and provide useful feedback, further indicating the importance of including these age groups in the development and evaluation of relevant materials. However, younger children’s feedback was often less descriptive, meaning the researchers placed more reliance on parental reflections and observing the child’s behaviour, for example, the child not noticing a tab or video. In addition, younger children sometimes relied on parental assistance during user testing, but the level of parental assistance given varied greatly. This may be a result of the experimental setting; during actual trial recruitment, parents would presumably provide as much assistance as required. Finally, the youngest children (aged seven years) struggled to hold their attention throughout the session. While this may continue to be a factor during actual trial recruitment, the MMIs would have more personal relevance and thus be more likely to hold attention.

The feedback from participants on the prototype MMIs was mostly very positive. It highlighted varied preferences about information presentation, with some participants relying solely on text and others, especially younger children, preferring videos and animations. This endorses the value of using different media to meet information needs and learning preferences. Our findings are consistent with previous research which found that adolescents appreciated a variety of media (Breakey et al., 2013), and research demonstrating participant preferences for MMIs (Baker et al., 2013; Nishimura et al., 2013; O’Lonergan and Forster-Harwood, 2011) and improved understanding when using MMIs (O’Lonergan and Forster-Harwood, 2011; Tait et al., 2015). We acknowledge that user testing was limited to the United Kingdom, for a study of MMIs being used within UK-based trials. If the MMIs were used in countries with very different health or social settings, the evaluation would need to be adapted accordingly.

Conclusion and implications

Informed consent for research can only be obtained if participants are able to understand the information provided (Tam et al., 2015). Developers of patient information materials should consider user testing to ensure material is fit for purpose. This method has previously improved a range of printed medical information materials (Ahmed et al., 2014; Knapp et al., 2009a, 2009b, 2010, 2011; Raynor, 2013; Raynor et al., 2011); this study demonstrated its positive effects on multimedia information. User testing highlighted issues not previously identified during the MMIs initial development and guided further development through performance-led revisions. Positive participant feedback suggests that MMIs are a promising alternative to written trial information; trial researchers may consider using MMIs as this study shows that they can be used and understood sufficiently by children, adolescents and parents. The impact of MMIs on behaviour and cognitions (recruitment, retention and decision-making quality) will now be evaluated in six embedded recruitment trials hosted by paediatric healthcare trials during phase 2 of TRECA.

Supplemental material

Supplemental Material, Online_Supplementary_Appendix_1 - User testing digital, multimedia information to inform children, adolescents and their parents about healthcare trials

Supplemental Material, Online_Supplementary_Appendix_1 for User testing digital, multimedia information to inform children, adolescents and their parents about healthcare trials by Rebecca Sheridan, Jacqueline Martin-Kerry, Ian Watt, Steven Higgins, Simon R Stones, Danielle Horton Taylor and Peter Knapp in Journal of Child Health Care

Supplemental material

Supplemental Material, Online_Supplementary_Appendix_2 - User testing digital, multimedia information to inform children, adolescents and their parents about healthcare trials

Supplemental Material, Online_Supplementary_Appendix_2 for User testing digital, multimedia information to inform children, adolescents and their parents about healthcare trials by Rebecca Sheridan, Jacqueline Martin-Kerry, Ian Watt, Steven Higgins, Simon R Stones, Danielle Horton Taylor and Peter Knapp in Journal of Child Health Care

Footnotes

Authors’ Note

The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. The funders were not involved in the study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.

Acknowledgements

We acknowledge the help of children, adolescents and parents who participated in user testing of the prototype MMIs. We thank the Bradford District Care NHS Foundation Trust and the Young Dynamos for their assistance in recruiting participants, and the schools involved in helping us identify suitable adolescents and providing us with a space in which to undertake user testing. We also acknowledge the support of the National Institute for Health Research Clinical Research Network (NIHR CRN). We would like to thank the members of the TRECA Study Advisory Group who are not authors of this article (Prof. Paul Baines, Prof. Michael Beresford, Prof. Peter Bower, Prof. Carrol Gamble, Ms Jennifer Preston, Dr Catherine Stones, Prof. Bridget Young), the Study Steering Committee (Prof. Faith Gibson, Prof. Louise Locock, Dr Bob Philips, Dr Louca-Mai Brady and Dr Matt Sydes) and members of the Patient and Parent Advisory Group who are not authors of this article (Mrs Sammy Ainsworth, Ms Sophie Ainsworth and Ms Robyn Challinor) for their continued input into the study. Finally, we thank Morph, the digital agency responsible for developing the MMIs ().

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The TRECA study is funded by the NIHR Health Services & Delivery Research Programme (project number 14/21/21).

ORCID iD

Rebecca Sheridan

Supplemental material

Supplemental material for this article is available online.

References

Ahmed

Raynor

McCaffery

, et al. (2014) The design and user-testing of a question prompt list for attention-deficit/hyperactivity disorder. BMJ Open 4(12). DOI: 10.1136/bmjopen-2014-006585.

Andersen

Khalid

Brooks

(2016) Considerations and methods for usability testing with children. In: Interactivity, Game Creation, Design, Learning, and Innovation: 5th International Conference, ArtsIT 2016 and First International Conference, DLI 2016 (eds Brooks

Brooks

), Esbjerg, Denmark, 2–3 May 2016, pp. 228–238. Berlin: Springer.

Baker

Leek

Salas

, et al. (2013) Suggestions from adolescents, young adults, and parents for improving informed consent in phase 1 pediatric oncology trials. Cancer 119: 4154–4161.

Barfield

Church

(2005) Informed consent in pediatric clinical trials. Current Opinion in Pediatrics 17: 20–24.

Blair

Gregory

Hughes

, et al. (2015) Study protocol for a randomised controlled trial of insulin delivery by continuous subcutaneous infusion compared to multiple daily injections. Trials 16: 163.

Breakey

Warias

Ignas

, et al. (2013) The value of usability testing for Internet-based adolescent self-management interventions: “Managing Hemophilia Online”. BMC Medical Informatics and Decision Making 13: 113.

Caldwell

Dans

de Vries

, et al. (2012) Standard 1: consent and recruitment. Pediatrics 129(Suppl 3): S118–S123.

Cox

Fallowfield

Jenkins

(2006) Communication and informed consent in phase 1 trials: a review of the literature. Supportive Care in Cancer 14: 303–309.

Davis

Peters

Calvo

, et al. (2017) “Kiss myAsthma”: Using a participatory design approach to develop a self-management app with young people with asthma. Journal of Asthma 24: 1–10.

10.

Dickinson

Raynor

Knapp

, et al. (2016) Do patients use a headline section in a leaflet to find key information about their medicines? Findings from a user-test study. Therapeutic Innovation & Regulatory Science 50: 581–591.

11.

Eltorai

Naqvi

Ghanian

, et al. (2015) Readability of invasive procedure consent forms. Clinical and Translational Science 8: 830–833.

12.

Ennis

Wykes

(2016) Sense and readability: participant information sheets for research studies. The British Journal of Psychiatry 208: 189–194.

13.

Flory

Emanuel

(2004) Interventions to improve research participants’ understanding in informed consent for research: a systematic review. Journal of the American Medical Association 292: 1593–1601.

14.

Ford

Sankey

Crisp

(2007) Development of children’s assent documents using a child-centred approach. Journal of Child Health Care 11: 19–28.

15.

Grootens-Wiegers

De Vries

Vossen

, et al. (2015) Readability and visuals in medical research information forms for children and adolescents. Science Communication 37: 89–117.

16.

Grossman

Piantadosi

Covahey

(1994) Are informed consent forms that describe clinical oncology research protocols readable by most patients and their families? Journal of Clinical Oncology 12: 2211–2215.

17.

Hallinan

Forrest

Uhlenbrauck

, et al. (2016) Barriers to change in the informed consent process: a systematic literature review. IRB 38: 1–10.

18.

Health Research Authority (2018a) Principles of consent: general principles and role of Participant Information Sheets. Available at: http://www.hra-decisiontools.org.uk/consent/principles-general.html (accessed 13 June 2018).

19.

Health Research Authority (2018b) Principals of consent: children and young people. Available at: http://www.hra-decisiontools.org.uk/consent/principles-children.html (accessed 16 July 2018).

20.

Healy

Galvin

Williamson

, et al. (2018) Identifying trial recruitment uncertainties using a James Lind alliance priority setting partnership—the PRioRiTy (Prioritising Recruitment in Randomised Trials) study. Trials 19: 147.

21.

Hunter

Pierscionek

(2007) Children, Gillick competency and consent for involvement in research. Journal of Medical Ethics 33: 659–662.

22.

Jefford

Mileshkin

Matthews

, et al. (2011) Satisfaction with the decision to participate in cancer clinical trials is high, but understanding is a problem. Supportive Care in Cancer 19: 371–379.

23.

Jefford

Moore

(2008) Improvement of informed consent and the quality of consent documents. The Lancet Oncology 9: 485–493.

24.

Knapp

Raynor

Silcock

, et al. (2009a) Performance-based readability testing of participant information for a Phase 3 IVF trial. Trials 10: 79.

25.

Knapp

Raynor

Silcock

, et al. (2009b) Performance-based readability testing of participant materials for a phase I trial: TGN1412. Journal of Medical Ethics 35: 573–578.

26.

Knapp

Wanklyn

Raynor

, et al. (2010) Developing and testing a patient information booklet for thrombolysis used in acute stroke. International Journal of Pharmacy Practice 18: 362–369.

27.

Knapp

Raynor

Silcock

, et al. (2011) Can user testing of a clinical trial patient information sheet make it fit-for-purpose?—A randomized controlled trial. BMC Medicine 9: 89.

28.

Lentz

De Jong

(1997) The evaluation of text quality: expert-focused and reader-focused methods compared. IEEE Transactions on Professional Communication 40: 224–234.

29.

Man

M-S

Rick

Bower

(2015) Improving recruitment to a study of telehealth management for long-term conditions in primary care: two embedded, randomised controlled trials of optimised patient information materials. Trials 16: 1–11.

30.

Martin-Kerry

Bower

Young

, et al. (2017) Developing and evaluating multimedia information resources to improve engagement of children, adolescents, and their parents with trials (TRECA study): study protocol for a series of linked randomised controlled trials. Trials 18: 265.

31.

Martin-Kerry

Knapp

Atkin

, et al. (2018) Supporting children and young people when making decisions about joining clinical trials: qualitative study to inform multimedia website development (under review).

32.

McCarty

Nair

Austin

, et al. (2007) Informed consent and subject motivation to participate in a large, population-based genomics study: the Marshfield Clinic Personalized Medicine Research Project. Public Health Genomics 10: 2–9.

33.

National Institutes of Health (2013) How to write easy-to-read health materials. Available at: https://medlineplus.gov/etr.html (accessed 13 June 2018).

34.

Nishimura

Carey

Erwin

, et al. (2013) Improving understanding in the research informed consent process: a systematic review of 54 interventions tested in randomized control trials. BMC Medical Ethics 14: 28.

35.

O’Lonergan

Forster-Harwood

(2011) Novel approach to parental permission and child assent for research: improving comprehension. Pediatrics 127: 917–924.

36.

Raynor

Knapp

Silcock

, et al. (2011) ‘User-testing’ as a method for testing the fitness-for-purpose of written medicine information. Patient Education and Counseling 83: 404–410.

37.

Raynor

(2013) User testing in developing patient medication information in Europe. Research in Social and Administrative Pharmacy 9: 640–645.

38.

Ross

Grant

Counsell

, et al. (1999) Barriers to participation in randomised controlled trials: a systematic review. Journal of Clinical Epidemiology 52: 1143–1156.

39.

Sheppard

Hyde

Florence

, et al. (2014) Improving the readability of online foot and ankle patient education materials. Foot and Ankle International 35: 1282–1286.

40.

Simon

Siminoff

Kodish

, et al. (2004) Comparison of the informed consent process for randomized clinical trials in pediatric and adult oncology. Journal of Clinical Oncology 22: 2708–2717.

41.

Simonds

Garroutte

Buchwald

(2017) Health literacy and informed consent materials: designed for documentation, not comprehension of health research. Journal of Health Communication 22: 1–10.

42.

Stryker

Wray

Emmons

, et al. (2006) Understanding the decisions of cancer clinical trial participants to enter research studies: factors associated with informed consent, patient satisfaction, and decisional regret. Patient Education and Counseling 63: 104–109.

43.

Tait

Voepel-Lewis

Levine

(2015) Using digital multimedia to improve parents’ and children’s understanding of clinical trials. Archives of Disease in Childhood 100: 589–593.

44.

Tam

Huy

Thoa

LTB

, et al. (2015) Participants’ understanding of informed consent in clinical trials over three decades: systematic review and meta-analysis. Bulletin of the World Health Organization 93: 186–198.

45.

Terblanche

Burgess

(2010) Examining the readability of patient-informed consent forms. Journal of Clinical Trials 2: 157–162.

46.

Treweek

Lockhart

Pitkethly

, et al. (2013) Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis. BMJ Open 3. DOI: 10.1136/bmjopen-2012-002360.

47.

Treweek

Pitkethly

Cook

, et al. (2018) Strategies to improve recruitment to randomised trials. Cochrane Database of Systematic Reviews 22. DOI: 10.1002/14651858.MR000013.pub6.

48.

UNICEF (2017) The state of the world’s children 2017: children in a digital world. Available at: https://www.unicef.org/publications/files/SOWC_2017_ENG_WEB.pdf (accessed 16 July 2018).

49.

World Medical Association (WMA) (2013) World Medical Association declaration of Helsinki: ethical principles for medical research involving human subjects. Journal of the American Medical Association 310: 2191–2194.

50.

Yamamoto

Doi

Yamamoto

, et al. (2017) Adaptation of the European Commission-recommended user testing method to patient medication information leaflets in Japan. Drug Healthcare and Patient Safety 9: 39.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.70 MB

0.62 MB