Comprehension of multimedia health information among older adults with chronic illness

Health knowledge and comprehension processes in self-care

Older adults with chronic illness manage multiple self-care tasks such as taking medications and monitoring symptoms. Effective self-care in turn depends on understanding health information (e.g. about illness, treatments) obtained by reading articles, discussions with health-care providers, and from other sources. Older adults who have limited health literacy, the ability to obtain and understand information needed to make health decisions (Nielson-Bohlman et al., 2004), may have specific trouble with self-care. While older adults vary widely in health literacy, inadequate health literacy is pervasive among those with chronic illness and is associated with poor health outcomes, in part because of difficulty understanding the information needed for self-care.

Health literacy depends on broader cognitive abilities necessary for comprehension. These abilities include processing capacity (e.g. working memory) and health-related and domain-general knowledge (e.g. Chin et al., 2011). For example, comprehension processes such as integrating the concepts conveyed by words into idea units (the textbase representation of explicit information in the text) depend on working memory (Kintsch, 1998). Knowledge, on the other hand, can make these processes more efficient, for example by guiding conceptual integration. Knowledge is also crucial for elaborating the textbase into a situation model, which represents what the text is about and reflects deeper understanding of the text in terms of a broader context.

Knowledge accumulates with age-related experience, even as processing capacity declines with age (Baltes, 1997). For example, older adults with hypertension know more about their illness the longer they have had the illness (Chin et al., 2011). Older adults can use their knowledge to maintain comprehension, especially at the situation model level, although they may need to invest more cognitive resources than younger adults in order to do so (Miller et al., 2004). An implication of this finding for designing health materials is the need to help older adults leverage their knowledge to support comprehension. We explored whether older adults with more knowledge about hypertension better understand information about hypertension that was conveyed by multimedia passages, focusing on possible benefits of visual–graphic formats (e.g. pictures) for comprehension (although in this exploratory study we did not experimentally manipulate the presence of pictures).

Multimedia and comprehension processes among older adults

Multimedia formats are pervasive in patient education, in part reflecting the belief that pictures especially benefit low-literacy patients by reducing the need for reading processes such as recognizing words and integrating concepts (Houts et al., 2006). Pictures may improve text comprehension for several reasons, such as providing an alternative route to encoding and retrieving information, or providing a more efficient format than text for conveying some kinds of information (e.g. Mayer, 2005; Schnotz, 2005). However, pictures are only likely to improve comprehension if they are relevant to the text and help organize concepts mentioned in the text by explicitly signaling this organization (Schnotz, 2005). Such pictures help readers create situation models from text (Gyselinck et al., 2008).

Evidence that pictures improve older adults’ text comprehension is mixed, with some studies showing benefit (e.g. Morrow et al., 1998) and others not (e.g. Liu et al., 2009; for a review, see Houts et al., 2006). This pattern suggests that benefits may depend on characteristics of the participants and pictures. For example, pictures that are not relevant to the text may impair comprehension among older adults, who may have trouble inhibiting distracting information.

For several reasons, older adults who know more about the passage topic may benefit more from the presence of pictures than those with less knowledge. First, older adults often must invest more cognitive resources than younger adults do in order to benefit from knowledge during reading, such as by drawing knowledge-based inferences to elaborate the textbase into a situation model (e.g. Miller et al., 2004). Relevant pictures may provide environmental support for knowledge use by signaling relationships between concepts in the text and how these concepts relate to knowledge. Second, experts are more likely than novices to attend to relevant information in multimedia passages (e.g. Canham and Hegarty, 2010), suggesting that more knowledgeable older adults better discern relevant information in graphics and its relationship to the text, perhaps facilitating integration of text and graphics into a situation model. Finally, research on allocation of study time to multiple texts suggests that more knowledgeable learners sample more broadly from multiple texts (the less knowledgeable spend most of their time studying one or a few [easy] texts; Reader and Payne, 2007). Similarly, older adults who know more about hypertension may allocate time to graphic as well as text-based sources of information.

Dynamics of attention during multimedia learning: when do pictures matter?

To explore the impact of visual–graphic formats on older adults’ comprehension of multimedia passages about hypertension, we used eye-tracking to measure attentional processes during different phases of processing. We were interested in when older adults with different levels of knowledge were likely to use pictures to elaborate text comprehension. Eye-tracking techniques are increasingly used to measure attentional processes in multimedia learning (for review, see Hyönä, 2010), although few studies have investigated attention allocation strategies among older adults viewing multimedia materials (but see Liu et al., 2009). While many of these studies have used global measures of attention allocation (e.g. percent dwell time to text and to pictures over the total trial), it is also important to investigate the dynamics of attention allocation to better understand how knowledge may influence the integration of text- and picture-based information during comprehension (Hyönä, 2010).

Our study of attention allocation while viewing multimedia health materials was guided by eye-tracking studies of multimedia learning and the literature on re-reading. First, people tend to focus on text when first viewing multimedia passages, reading the text before looking at graphics (Hegarty and Just, 1993; Rayner et al., 2001; Schmidt-Weigand et al., 2010). Second, the re-reading literature suggests that people create a textbase representation during their first pass through a text (Millis et al., 2000), although older adults may also focus on the situation model during this phase (Stine-Morrow et al., 2004). Readers who know more about the topic may be especially text-focused when first viewing multimedia passages because they can use their knowledge rather than accompanying graphics to create the textbase and an initial situation model (Hegarty and Just, 1993). People tend to develop the situation model when re-reading the text by elaborating the textbase with knowledge (Millis et al., 2000; Stine-Morrow et al., 2004). Therefore, relevant graphics may be most helpful during later passes through the text. At this point, the graphics may scaffold readers’ knowledge use when creating the situation model to the extent that the graphics signal relationships among text concepts, as well as how these concepts relate to knowledge structures (Gyselinck et al., 2008; Hegarty and Just, 1993). Thus, following the multimedia and re-reading literatures, we analyzed attention to text and graphics during two phases of processing multimedia passages: during and after the first read of the text.

We also explored the impact of these eye-tracking measures on passage comprehension by relating them to offline measures of comprehension. For example, an increase in saccades between text and picture across passage conditions could either indicate processing difficulty (more saccades without corresponding increase in comprehension accuracy) or an effective strategy (more saccades that occur with more accurate comprehension; Holsanova et al., 2008). Therefore, we related allocation strategies to measures of passage comprehension (question accuracy).

The present study

We explored these issues in a study in which older adults viewed (at their own pace) multimedia passages about hypertension, with text accompanied by both relevant and irrelevant pictures. Their fixations on text and pictures were measured by eye-tracking. After completing each passage, they answered questions about information from the text and pictures (for more information, see D’Andrea, 2010). We explored the following issues:

Materials and apparatus

Six passages about hypertension topics were adapted from material on MedlinePlus, a National Institutes of Health-supported online resource for hypertension information (two passages served as practice trials in order to familiarize the participants with the types of passages, questions, and eye-tracking apparatus). Each passage contained one paragraph and two pictures. One picture was relevant to the text content, explicitly depicting and elaborating concepts in the text (see Figure 1). The other picture was less relevant to the text content, for example, showing photographs of people. Both types of pictures are often found in online patient education materials. In a pilot study in which 12 older adults with hypertension viewed the four experimental passages, the relevant versus irrelevant pictures were rated as more useful for understanding the text (4.5 versus 1.3; 7-point scale with 7 = most useful; 1 = least useful) and as containing more information from the text (4.6 versus 1.3; 7-point scale with 7 = all picture information is from the text; 1 = no picture information is from the text). Multimedia passages about exercise and medication were also presented to participants after they finished responding to the six hypertension passages described above, but the data from these passages are not reported in this article (for more information, see D’Andrea, 2010).

OPEN IN VIEWER

Figure 1

Example display from the study. The relevant picture is on the top right and the irrelevant picture on the bottom right (position of text and pictures was counterbalanced across participants).

Texts, modified to ensure their length was consistent with eye-tracking requirements (mean length = 102 words; mean Flesh-Kincaid readability score = 9.96), were presented in 20-point font on either the left or right half of the display (pictures were presented on the other half, with the relevant picture either above or below the other picture), with position of text and picture counterbalanced across participants. Passage order was also counterbalanced across participants. The text and pictures took up a roughly equal amount of display space and were displayed simultaneously to participants.

Stimuli were presented on a 19-inch cathode ray tube monitor located 24 inches from the subject. A desk-mounted SR Limited EyeLink 2000 system remotely tracked and recorded eye movements as participants viewed the displays. Viewing was binocular, but only eye movements of the right eye were recorded. Fixations were sampled at 1000 Hz. The system was calibrated until the average eye position error was less than 1°. A Microsoft Sidewinder Game Pad controller was used to allow participants to move through stimulus displays at their own pace and respond to comprehension questions.

Measures

Comprehension of information from the text/picture displays was measured by 12 True/False questions (3 questions for each of the four passages from which data were analyzed). The questions were designed to measure comprehension at both the textbase (information explicitly provided by text) and situation model level (six of each type of question occurred across the four passages). The latter questions required integration of information across text and pictures, or between passage information and prior knowledge. For the passage in Figure 1, an example textbase question was ‘Is the size and flexibility of the arteries the only thing that determines blood pressure?’, while a situation model question was ‘Does blood pressure apply force from outside of the artery wall?’.

Participants’ hypertension knowledge was measured by a questionnaire modified from Gazmararian et al. (2003). It contained 37 questions about different facets of the illness (risk factors, self-care practices, etc.; Cronbach alpha = 0.90; for more information, see Chin et al., 2011). Vocabulary (a component of general knowledge) was measured by the Advanced Vocabulary Test. Health literacy was measured with the Short Test of Functional Health Literacy (STOFHLA; Baker et al., 1999), a 36-item test that predicts the ability to understand and remember a wide range of health-related concepts.

Procedure

The session lasted about 90 minutes. After providing consent, participants completed a demographic questionnaire and the vocabulary test. They were then seated in front of the eye-tracking system’s computer monitor. The eye-tracking equipment was calibrated, and participants began two practice trials (hypertension passages followed by comprehension questions). They were told to examine the text and picture of each display (at their own pace) so they could later answer questions about the displayed information. Next, calibration was adjusted if needed, followed by the four experimental trials. After viewing each display, participants answered comprehension questions before the next trial. Finally, they completed the STOHFLA and hypertension knowledge tasks.

Passage comprehension and knowledge

Passage comprehension was measured by the proportion of correctly answered questions (collapsed over question type because there were so few questions of each type per trial). For the total sample (N = 41), comprehension (mean = 0.68; SD = 0.14) was related to previous health (hypertension) knowledge (r = .44), vocabulary (r = .49), and STOFHLA (r = .39, all ps < .01). Health knowledge (r = .41) and vocabulary (r = .61) also predicted comprehension for the 29 participants in the comprehension process analysis. We next investigated how the comprehension differences associated with knowledge related to the processing (eye-fixation) measures in order to explore possible reasons for the association of knowledge and comprehension.

Comprehension processes

Eye-fixations with durations of less than 100 ms were excluded from the analysis because passage information is unlikely to be extracted for such fixations (Hegarty and Just, 1993). We first examined looking behavior over the entire trial. As in Rayner et al. (2001), participants spent more time looking at the text (29.9 sec/trial) than at the two pictures (6.1 sec/trial) in the displays, t(28)=9.2, p < .001. They also made more fixations on the text (mean = 119 fixations/trial) than on pictures (21 fixations/trial), t(28) = 11.7, p < .001. Mean fixation duration did not differ for text (251 ms) and pictures (270 ms/trial), t(28)=1.40, p > .10. Passage comprehension and health knowledge were not associated with measures of overall time allocation on text and pictures (total trial time, total reading time, total picture viewing time, ps > .10, for all measures).

We then investigated whether comprehension and health knowledge related to allocation of processing time during different processing phases: (i) first pass: during the first read-through of the text (defined as the interval from the time of display onset to when the participant fixated the final word of the text, regardless of whether this first read-through was interrupted in order to look at the picture); (ii) after first pass.

First-pass phase. Participants spent on average 0.80 (SD = 0.14) of their total trial time during this first pass. To understand how individuals spent their time during this phase, the proportion of first-pass time spent on the text (time on text during first-pass/total first-pass time; the complement of the proportion of first-pass picture viewing time) was analyzed. The proportion of this first-pass reading time (mean = 0.94, SD = 0.06) was correlated with hypertension knowledge (r = .42, p < .05), but not with health literacy (r = –.06, p > .10) or passage comprehension (r = .22, p > .10). In other words, more knowledgeable participants spent more time reading text and less time viewing pictures during their first pass compared to less knowledgeable participants, although this measure was not significantly associated with comprehension.

After first-pass phase. More knowledgeable participants also appeared to allocate their time to text and pictures differently after their first-pass through the passage. The proportion of after first-pass time spent reading text (mean = 0.39, SD = 0.28) was negatively correlated with hypertension knowledge (r = –.44, p < .05). That is, more knowledgeable participants now spent less time reading and more time viewing pictures compared to less knowledgeable participants. There was some evidence that this strategy was successful because the correlation with comprehension accuracy was marginally significant (r = –.32, p = .09).

Because picture viewing time seemed more consequential after, rather than during the first pass through the text, we analyzed in more detail how participants allocated time to text and pictures during this phase. Participants with more knowledge and a better understanding of the passage tended to look at the more relevant pictures: the proportion of time spent on the relevant picture (time on relevant picture after first pass/total time after first pass) was significantly correlated with knowledge (r = .52, p < .01) and marginally with comprehension (r = .32, p = .09), while the proportion of time on the irrelevant picture was not correlated with either measure (p > .10 for both). We also examined mean picture run time (mean duration from initial fixation on, to saccade out, of pictures), which indicates for how long participants interrupted their reading to examine the pictorial information. More knowledgeable participants tended to look longer at relevant pictures compared to the less knowledgeable (r = .36, p = .06).

Finally, we calculated the proportion of time spent on text during versus after first-pass reading in order to examine whether shifts in looking behavior across phases were related to knowledge and comprehension. This overall allocation measure was marginally related to comprehension (r = .38, p = .07), suggesting an optimal overall pattern of looking across phases. It also correlated with knowledge (r = .48, p < .05), suggesting this strategy characterized the more knowledgeable participants. To sum up, higher knowledge participants did most of their looking at pictures after they had already read through the text once and developed an initial understanding of the information, while lower knowledge participants were more likely to distribute their looking at the picture throughout the trial. The fact that more knowledgeable participants spent more time looking at the more relevant picture after reading the text suggests they used the picture to elaborate the text-based representation constructed during their first pass into a situation model.

Comprehension of multimedia health information: implications for patient education

We investigated older adults’ comprehension of multimedia passages about hypertension, using eye-tracking to measure attentional processes during different phases of viewing the passages, and relating these processes to offline measures of passage comprehension. As in other studies (e.g. Miller et al., 2004), older adults with more health knowledge understood the passages better. Of most interest was whether this advantage was rooted in how they processed the health-related information.

While knowledge-related differences in comprehension were unrelated to participants’ overall allocation of time when viewing the passages (over the total trial), relationships between allocation, knowledge, and comprehension emerged when the fixation data were partitioned into phases (during and after first reading the text). Older adults with more health knowledge spent more time than those with less knowledge fixating on the text rather than viewing the pictures during the first pass. After this reading-based first pass, they spent more time viewing the relevant picture rather than re-reading, and there was some evidence that this strategy (reading and then viewing the relevant picture) was associated with how well they understood the passages.

Older adults who have more domain knowledge may spend time integrating and organizing concepts to construct accurate representations of the explicit text information (the textbase) and perhaps drawing inferences to elaborate the textbase into a situation model (Miller et al., 2004). They tend to do so when first reading the text (Stine-Morrow et al., 2004). They may have spent less time looking at the pictures during this first pass because they could rely on their knowledge rather than the picture in order to integrate and elaborate on text concepts (Hegarty and Just, 1993; Mayer, 2005). After their first pass, participants with more knowledge appeared to wrap up their understanding of the passage by spending more time viewing the relevant picture and less time re-reading the text compared to lower knowledge participants. They may have done so because readers generally elaborate the textbase into a situation model during later passes through a text (Millis et al., 2000), and pictures that depict and interrelate text concepts support development of situation models (Gyselinck et al., 2008), and thus may be most helpful at this point in processing. Because knowledge use during text comprehension is especially resource-demanding for older adults (Miller et al., 2004), the pictures may have helped reduce these cognitive demands, scaffolding use of knowledge to elaborate the textbase into a situation model.

Similar findings occurred in another recent study from our lab (Gao et al., 2011). In this study, older adults viewed passages about hypertension that contained text with or without a concept-map that illustrated relationships among key concepts mentioned in the text. Concept maps may be especially likely to improve comprehension because they semantically overlap with the text and explicitly convey relationships among text concepts, supporting development of the textbase and situation model. As we found in the study described above, older adults with more hypertension knowledge in this second study better remembered information from the passages. Moreover, this advantage was greater when the text was accompanied by the concept map, and a mediation analysis suggested that the advantage was partially due to more time spent viewing the concept map by the more knowledgeable participants. Phase-based analyses are planned to explore whether concept map-related benefits for the more knowledgeable participants can be traced to viewing the graphic after reading the text.

Our finding that older adults’ picture viewing was associated with better comprehension differs from Liu et al. (2009), who found that older adults did not benefit from pictures, despite looking at them more than younger adults did. Although the studies cannot be directly compared because of different materials, participants, and procedures, it is possible that at least some of the older adults in our studies had sufficient domain knowledge to capitalize on the pictures in the context of the passage.

Our findings suggest that general recommendations for using pictures to support older adults’ comprehension of health information should be qualified. Pictures as well as text may require knowledge to be accurately interpreted, so that picture-based benefits for comprehension are more likely for older adults with more knowledge about the topic. An important challenge is to design pictures that benefit older adults with lower as well as high knowledge.

Comprehension of multimedia health information: methodological implications

Our findings have two implications for measuring multimedia passage comprehension processes. First, the finding that relationships between processing and knowledge only emerged for the phase-based analyses suggests the value of measuring and analyzing the dynamics of attention allocation during comprehension, in addition to more global processing measures (Hyönä, 2010). Second, relating these process measures to offline comprehension measures provided a richer context for exploring their role in comprehension.