The Impact of Numeracy Ability and Technology Skills on Older Adults’ Performance of Health Management Tasks Using a Patient Portal

Abstract

Patient portals, which allow patients to access their health record via the Internet, are becoming increasingly widespread and are expected to be used by diverse consumer populations. In addition to technology skills, numeracy skills are also likely to be critical to performing health management tasks, as much of the data contained in the portal are numeric. This study examined how factors such as Internet experience, numeracy, and education impacted the performance of common tasks using a simulated patient portal among a sample of older adults. In addition, information was gathered on the ability of older adults to estimate their numeracy skills. Results indicated that numeracy and Internet experience had a significant impact on their ability to perform the tasks and that older adults tended to overestimate their numeracy skills. Results from this study can help to identify interventions that may enhance the usability of patient portals for older adults.

Keywords

objective numeracy ability patient portals self-management of health simulation methods subjective numeracy ability

Introduction

Electronic personal health records (PHRs) are transforming healthcare by providing patients with increased access to personal health information. The types of PHRs available include “standalone” models, in which information is entered by the patient, “integrated” models that extract information from insurance claims and pharmacy data, and “tethered” PHRs that are linked to the patient’s electronic medical record (EMR) and offer the patients access to parts of their medical record via web portals (Detmer, Bloomrosen, Raymond, & Tang, 2008). Patients using portals have the ability to view their medical history, review laboratory results and medication lists, communicate with their provider, and follow links to credible health information online (Yamin et al., 2011).

Diverse patient populations are increasingly using patient portals. To date, many of the studies on portals have focused on access and use, and in this regard, disparities have been found. Specifically, it has been reported that older adults, less-educated adults, and African Americans, Latinos, and Filipinos were less likely than younger, more educated, and non-Hispanic Whites to request a password to use a patient portal, but among those who did request a password, older adults were more likely than younger adults to actually log on (Sarkar et al., 2011). Similar results pertaining to older adults have been found in other studies as well. For example, in a study exploring the differences in adoption and use by 74,368 patients, older patients were less likely to receive an activation code from their provider. However, those who did receive access were more likely to activate their account (Ancker et al., 2011). In another study, patients older than 65 years were found to use a PHR to a greater extent than patients aged 18 to 35 years (Yamin et al., 2011). Given the interest older patients exhibit in using portals and the fact that the proportion of the population that is older is continuing to grow, a question of particular importance concerns the capability of older adults to appropriately use the information contained in their PHRs. Also, these patients are more likely to have medical problems and consequently face an increasing number of doctor visits as they age; thus, effective use of a patient portal for health management is especially critical for this population.

Despite the increasingly widespread use of patient portals, few studies have investigated the ability of patients to effectively use portals to manage their health. The limited data that are available indicate that patients encounter problems using portals. In a recent usability study of an electronic PHR, participants (aged 27 to 84) had difficulty performing common tasks such as finding lab results, making appointments, and interpreting medication instructions (Segall et al., 2011). This result is especially disconcerting as the analysis of several years of usage data from a widely used PHR indicated that viewing test results was the most-used feature (Silvestre, Sue, & Allen, 2009).

While it has been shown that usability problems occur, little is known about the factors that influence the ability of patients, and especially older adults, to perform common health management tasks using a portal. Some of the portal usability problems may be attributed to computer anxiety or limited computer skills in older patients (Lober et al., 2006). Problems may also arise due to lower health literacy in this group, as it is well documented that older adults are disproportionately affected by lower health literacy than younger individuals (Baker, Gazmararian, Sudano, & Patterson, 2000; Kutner, Greenberg, Jin, & Paulsen, 2006; Williams et al., 1995). In particular, the numeracy component of health literacy may be critical in governing effective use of a PHR as many of the tasks performed with a portal depend on numeracy skill. Health numeracy has been defined as “the degree to which individuals have the capacity to access, process, interpret, communicate, and act on numerical, quantitative, graphical, biostatistical, and probabilistic health information needed to make effective health decisions” (Golbeck, Ahlers-Schmidt, Paschal, & Dismuke, 2005, p. 375). Portal tasks relying on numeracy include managing appointment dates and times, understanding medication dosage instructions, reviewing lab results, and interpreting health information from charts, tables, and graphs. However, there are no data available regarding the numeracy ability of older patients and how this ability affects their use of a patient portal.

This article reports on a study that examined the ability of older adults to use a patient portal to perform common health-management tasks. A sample of older participants interacted with a simulated patient portal that was based on MyChart, a PHR available from Epic. The participants were required to answer 15 questions reflecting common health-related tasks that ranged in complexity with respect to demands on numeracy ability. The unique focus in this study was in determining the degree to which numeracy skills impact the ability to perform health management tasks using a portal, as many of the tasks rely on effective use of numeric information. In addition, we examined the importance of technology skills for portal use. This is also an important issue as currently there is still an age-related digital divide. Additionally, we examined older adults’ perceptions of their numeracy ability. As much of the information contained in the portal is numeric, it may be of importance to find out how older adults view their abilities to deal with numbers. Furthermore, we gathered data on older adults’ feelings about portals in general and the problems they had using the numeric information contained in the portal simulation. The outcomes from this research will help identify design changes and interventions that can enable older patients to overcome barriers to use and to enhance their ability to use these portals to manage their health. Changes made to help older adults use portals may make adoption of these systems easier for all age groups as, generally, the human factors literature indicates that design interventions that benefit older adults also benefit most user groups (e.g., Fisk, Rogers, Charness, Czaja, & Sharit, 2009).

Method

Sample

Participants for the study were recruited from the Miami area through placement of flyers in community organizations and senior centers and by word of mouth. Interested individuals contacted the study investigator by telephone. The study investigator provided an overview of the study and administered a telephone prescreening, which included screening questions (e.g., age, primary language) and the Wechsler Memory Scale III (WMS-III; Wechsler, 1997). Participants who were eligible and interested were scheduled for participation. All participants were required to be English-speaking and noncognitively impaired as measured by a score greater than 26 on the Mini Mental Status Examination (MMSE; Folstein, Folstein, & McHugh, 1975), adjusted for age and education using the correction established by Mungas, Marshall, Weldon, Haan, and Reed (1996). Participants were not required to have any prior experience with computers, the Internet, or patient portals. Study participants included 51 adults ranging in age from 60 to 85 years (M = 69.31, SD = 7.44). Table 1 displays the demographic profile of the participants in the study. The sample was ethnically and educationally diverse, had fairly low income, and the majority reported to be in good to excellent health.

Table 1.

Sample Characteristics (N = 51).

Age in Years, M (SD)	69.31 (7.44)
Gender, n (%)
Male	20 (39.2)
Female	31 (60.8)
Ethnicity, n (%)
Hispanic	16 (31.4)
Non-Hispanic White	20 (39.2)
Non-Hispanic Black	15 (29.4)
Education, n (%)
High school or less	11 (21.6)
Some college	24 (47.1)
College graduate/postgraduate	16 (31.4)
Yearly household income, n (%)
Less than $20,000	28 (54.9)
$20,000 to $49,999	11 (21.6)
$50,000 or more	12 (23.5)
Ratings of general health, n (%)
Poor	0 (0.0)
Fair	6 (11.8)
Good	25 (49.0)
Very good	15 (29.4)
Excellent	5 (9.8)

Measures

Background Questionnaire

This questionnaire gathered demographic data such as gender, age, ethnicity, education, and income (Czaja et al., 2006a). It also gathered information on participants’ perceptions of their health, their medical conditions, and medications taken. This questionnaire also assessed participants’ attitudes toward computers (Czaja et al., 2006a; Jay & Willis, 1992).

Technology Experience Questionnaire

This questionnaire assessed use of common technologies such as ATMs, cell phones, and computers (Czaja et al., 2006b). Those who reported having experience with computers responded to questions concerning their frequency and duration of computer use. Those who reported having Internet experience responded to questions concerning their frequency and duration of Internet use, as well as where they use the Internet and what types of activities they perform on the Internet.

Health literacy and numeracy measures

The Test of Functional Health Literacy in Adults (TOFHLA; Parker, Baker, Williams, & Nurss, 1995) consists of a 50-item reading comprehension test and a 17-item numeracy component that consists of hospital forms and prescription bottles. The reading portion of the test includes passages on preparation for a medical procedure, the patient rights and responsibilities section of a Medicaid application form, and a standard hospital informed consent form, while the numeracy portion tests one’s ability to understand directions for taking medications, monitoring glucose, keeping clinic appointments, and obtaining financial aid. TOFHLA scores range from 0 to 100 and are categorized as follows: inadequate (0 to 59), marginal (60 to 74), and adequate (75 to 100). Individuals who have “adequate” functional health literacy should be able to read, understand, and interpret most health texts. However, those who have “marginal” or “inadequate” functional health literacy will likely have difficulty reading, understanding, and interpreting most health materials.

The objective numeracy measure developed by Lipkus, Samsa, and Rimer (2001) is a frequently used measure that consists of 11 questions: 3 general numeracy questions developed by Schwartz, Woloshin, Black, and Welch (1997) and 8 additional questions that focus on numeracy in a health context. The general questions assess one’s ability to convert a percentage to a proportion, convert a proportion to a percentage, and determine how many times out of 1,000 rolls a fair die would come up an even number. The eight additional questions use similar mathematical operations as the general questions but are phrased in the context of health risks. Correct answers are given 1 point, resulting in scores that range from 0 to 11.

The Subjective Numeracy Scale (SNS) developed by Fagerlin et al. (2007) is a self-report measure of perceived ability to perform various mathematical tasks and preference for the use of numerical versus prose information. It is significantly correlated (r = 0.68) with the Lipkus et al. scale (Fagerlin et al., 2007). The SNS consists of eight items: four questions that assess respondents’ beliefs about their skill in performing various mathematical operations and four questions that assess respondents’ preferences for presentation of numerical information. There are no right or wrong answers; participants answer each question on a 6-point Likert-type scale. Possible scores on the SNS range from 8 (for those participants rating themselves lowest on ability to perform mathematical tasks and preference for the use of numerical information) to 48 (for those participants rating themselves the highest on numeric abilities and preference for numerical information).

Usability Questionnaire

A usability questionnaire was developed for the study to assess how participants felt about using the patient portal simulation. There were two sections to the questionnaire. The first section contained seven questions concerning how they felt in general about using a patient portal like the simulation they had just used (e.g., would it help them to perform health management tasks more quickly, would it be useful). The second section contained 10 questions that concerned the experience they had just had using the simulated patient portal (e.g., was it difficult to locate information, were the numerical tables confusing). Each question was answered on a 5-point Likert-type scale (1 = agree; 5 = disagree). In addition, there was a yes/no question to assess whether they would use a patient portal like the simulation if it was available from their doctor.

Patient Portal Simulation

The simulated patient portal was based on Epic’s MyChart, which allows patients to schedule appointments, view test results and x-rays, renew prescriptions, send and receive emails from their healthcare providers, and link to health information from trustworthy sources. MyChart was chosen because of its widespread use; an estimated 50 million patients see healthcare providers who use the Epic software system (Kaelber, Jha, Johnston, Middleton, & Bates, 2008). Figure 1 shows the homepage of our simulated patient portal, referred to as the CREATE (Center for Research and Education on Aging and Technology Enhancement) Patient Portal Simulation, which captured all of the relevant features of the existing MyChart system. The portal was populated with data for a fictitious patient referred to as “Pat.” Pat had conditions such as diabetes, high blood pressure, and high cholesterol. This enabled the simulated portal to be populated with a variety of information on which to base the tasks.

Figure 1.

Homepage of the Patient Portal Simulation.

Tasks

Fifteen tasks were developed to test participants’ performance on health-management tasks commonly carried out using a portal (i.e., locating the date and time of an upcoming appointment, reviewing test results). These tasks were designed to span the spectrum of numeracy ability proposed by Golbeck et al. (2005): (1) basic numeracy, which involves identifying numbers and making sense of quantitative data that do not involve manipulation of numbers; (2) computational numeracy, which involves counting, quantifying, computing, and performing simple manipulation of numbers, quantities, items, or visual elements in a health context; (3) analytical numeracy, which involves inference, estimation, and understanding proportions, percentages, and frequencies, and often requires information to be integrated from multiple sources and formats; and (4) statistical numeracy, which involves understanding probability statements, having the skills to compare information presented on different scales (probability, proportion, percent), having the ability to critically analyze quantitative health information such as life expectancy and risk, and understanding statistical concepts such as randomization.

To determine the task’s difficulty, four independent raters were asked to evaluate each of the 15 tasks. The raters were asked to review all of the tasks and assign the value of “5” to the task/tasks that they determined to be the most complex and assign the value of “1” to the easiest task/tasks. The rest of the tasks were ranked in relation to these endpoints. The computation of Cronbach’s alpha revealed a high inter-rater reliability (α = .842). The four ratings given to each task were averaged to get an overall rating of the difficulty of that task. After averaging the four ratings for each task, the resulting weight given to the tasks ranged from 1.25 to 4.50. Based upon these weights, tasks were divided into two categories: 7 “simple” tasks (weights from 1.25 to 2.25) and 8 “complex” tasks (weights from 2.50 to 4.50).

The total possible scores for simple and complex tasks were determined by summing the number of possible points in each category. Answers by participants that were left blank or incorrect were given a score of 0, partially correct answers (on tasks that had multiple parts) were given 1 point, and completely correct answers were given 2 points. Thus, the maximum scores for the simple and complex task sets were 14 and 16, respectively.

Table 2 displays examples of tasks, the steps necessary to perform each task, the corresponding type of numeracy skill involved in performing the task, and the assigned difficulty rating. The patient portal pages that were used to perform the example tasks are displayed in Figures 2 through 5.

Table 2.

Examples of Tasks and the Corresponding Steps Necessary to Perform Tasks, Type of Numeracy Skill Required, and Difficulty Rating.

Scenario/Task	Elemental Tasks	Type of Numeracy	Difficulty Rating/Category
Pat has an appointment with a new doctor scheduled soon, but cannot remember the date of the appointment. Fortunately, this information is available for you in the patient portal. What is the date and time of Pat’s next doctor’s appointment?	In the menu, click on “Appointments” button, locate and click on “Future Appointments” button in the sub-menu, click on appropriate appointment, and read the date and time.	Basic	1.25/Simple
Pat’s doctor has included in the health record a table of Pat’s target glucose levels for before and after meals. Use the information in this table and the information in the glucose monitoring weekly summary to determine if Pat’s average glucose levels are on target. Is Pat’s average glucose level after lunch in the range it should be?	In the menu, click on “My Medical Record” button, locate and click on “Test Results” button in the sub-menu, locate and click on “Glucose Monitoring Weekly Summary” link, locate Pat’s average after lunch, click on link to “View Table of Target Glucose Levels,” and determine if glucose is in the range given in table of target glucose levels.	Computational	1.50/Simple
Pat checks his or her blood sugar just before eating. Pat takes 1 unit of insulin (Apidra injection) for every 10 grams of carbohydrates eaten. Along with basic dosage instructions, Pat’s doctor has also included a schedule in the patient portal that indicates the amount of insulin that should be added to the usual dose based upon blood sugar levels. Use the link to the insulin dose schedule to answer the following question: Pat’s blood sugar is 284 and Pat ate 40 grams of carbohydrate at breakfast. How much total insulin does Pat need to take?	In the menu, click on “My Medical Record” button, locate and click on “Medications” button in the sub-menu, locate “Apidra Injection” in the medications list, click on “Insulin dose schedule” link, use information in the table to determine how much insulin to add to usual dose, and calculate dose.	Analytical	4.50/Complex
During Pat’s last visit the doctor explained that high blood pressure can lead to health problems such as heart attack, stroke, heart failure, and kidney disease. Based upon Pat’s personal profile (age, gender, height, weight, and current blood pressure), the doctor created a graph of Pat’s estimated risks for these conditions and put the graph in the patient portal of Pat’s health record. Use the link to “High Blood Pressure Health Risk Calculator” to view the graph and answer the following question: What does the first graph show about Pat’s risk of heart failure compared with the normal risk?	Click on “High Blood Pressure Information” link, locate and click on the link to “High Blood Pressure Risk Calculator,” view graph, locate the bar that represents risk of heart failure, and comprehend that “2.1×” means “2.1 times greater than normal risk.”	Statistical	3.75/Complex

Figure 2.

Detailed Appointment Information.

Figure 3.

Table Used to Answer Glucose Task.

Figure 4.

Insulin Dose Schedule.

Figure 5.

Graph of Health Risks.

Procedure

Participants first completed the Background Questionnaire, Technology Experience Questionnaire, numeracy measures, and TOFHLA. Irrespective of participants’ Internet experience, all participants worked through a tutorial on basic computer skills (such as using a mouse and scrolling) to ensure that they had adequate knowledge of basic operations required for interacting with the simulated patient portal. They were then given a brief training session on how to use the portal. Participants were told to pretend they were a relative of Pat and were to use the portal to help Pat manage his or her health. Participants were given a packet that contained the 15 tasks, with space provided below each question for them to record their answers. They were allowed up to 2 hours to complete all of the tasks. Upon finishing the tasks, the participants were given the usability questionnaire to gather more information about their feelings concerning use of the portal.

Results

Statistical Analysis

All analyses were conducted with IBM SPSS Statistics Version 19. Participants’ self-reported Internet experience; participants’ scores on measures of health literacy, subjective numeracy, and objective numeracy; and participants’ responses to the usability questionnaire were summarized using descriptive statistics. The correlation between subjective and objective numeracy scores was determined by using Pearson’s r correlation. Two hierarchical regression models were constructed for predicting the effects of education, Internet experience, and objective numeracy scores on task performance. In one model, the dependent measure was performance on the simple tasks; in the second model, the dependent measure was performance on the complex tasks. The predictor variables were entered in the following order: education, Internet experience, and objective numeracy. Education was entered first as a control variable due to the variability in the level of education among participants.

Internet Experience

Eleven participants (21.6%) reported having no experience with the Internet. The remaining participants had varying levels of experience. Table 3 indicates how long the participants had been using the Internet, as well as how often, on average, they used the Internet per week. To create a variable that captured the participants’ overall Internet experience, the responses to the duration question (coded 1 to 4) were multiplied by the responses to the intensity question (coded 1 to 4), resulting in scores ranging from 1 to 16 for those participants who had Internet experience (participants with no prior Internet experience received a score of 0). This variable was used in the hierarchical regression models.

Table 3.

Participants’ Internet Experience (N = 40).

	N	%
Length of time using the Internet
Less than 6 months	6	11.8
Between 6 months and a year	3	5.9
More than 1 year, but less than 5 years	5	9.8
5 years or more	26	51.0
Hours/week using the Internet
Less than 1 hour	9	17.6
Between 1 hour and 5 hours	12	23.5
More than 5 hours, but less than 10 hours	7	13.7
10 hours or more	12	23.5

Note: Table does not included data for the 11 participants with no Internet experience.

Functional Health Literacy

TOFHLA scores in this sample ranged from 59 to 99 (M = 85.37, SD = 10.73). There was not much variation in the scores, and most participants performed very well. Forty-three participants (84.3%) had scores in the “adequate” range (75 to 100). Of the remaining participants, one participant had a score in the “inadequate” range (0 to 59), and seven participants had scores in the “marginal” range (60 to 74). Due to the lack of variability in TOFHLA scores among participants, this variable was not included in the hierarchical regression models.

Subjective and Objective Numeracy

The scores on the SNS ranged from 14 to 48 (M = 32.76, SD = 9.20), while the participants’ objective numeracy scores ranged from 0 to 11 (M = 5.29, SD = 2.69). There was a small but significant positive correlation between the two scores (r = .395, p < 0.01), indicating that for this sample only 15.6% of the variance in objective numeracy score could be explained by the variance in subjective numeracy score. The majority of participants (52.9%) correctly answered 5 or fewer objective numeracy questions, while subjectively rating their skills as quite high. Thus, most participants tended to overestimate their numeracy ability.

Patient Portal Task Performance

Scores for the simple tasks ranged from 0 to 14 (M = 8.67, SD = 4.41). Six participants (11.8%) were able to correctly complete all of the simple tasks. On the complex tasks, scores ranged from 0 to 15 (M = 5.90, SD = 4.01), indicating that none of the participants were able to correctly perform all of the complex tasks.

As noted, two hierarchical regression models were constructed: One model was used to examine the effects of education, Internet experience, and objective numeracy scores, in that order, on performance of the simple tasks, and the second model was used to assess the impact of these predictor variables on performance of the complex tasks. Education was not found to be a significant predictor of performance on either the simple or complex tasks. The addition of Internet experience resulted in both models being significant, F_simple(3, 47) = 6.735, p < .001; F_complex(3, 47) = 7.038, p < .001. These models remained significant after the addition of the objective numeracy score; thus, objective numeracy was a significant predictor of both simple and complex task performance even after education and Internet experience were accounted for, F_simple(4, 46) = 12.081, p < .0001; F_complex(4, 46) = 16.289, p < .0001. As indicated in Table 4, after inclusion of education and Internet experience, numeracy accounted for more of the variance in performance on the complex tasks (27.6%) than on the simple tasks (21.2%).

Table 4.

Summary of Hierarchical Regression Models for Simple and Complex Task Performance Scores.

	Simple Tasks			Complex Tasks
	R ²	ΔR²	p	R ²	ΔR²	p
Education	0.108	0.108	ns*	0.076	0.076	ns*
Internet experience	0.301	0.193	0.001	0.310	0.234	0.000
Numeracy	0.512	0.212	0.000	0.586	0.276	0.000

ns, not significant.

Usability Ratings

The vast majority (86.3%) of participants indicated that they would use a patient portal like the simulation if it was available from their doctor. Participants tended to have a positive opinion of patient portals in general. Ninety-four percent of participants either agreed or somewhat agreed that a patient portal would improve their ability to perform health management tasks (i.e., review test and lab results, schedule a doctor’s appointment, or look for information about a medical condition), and 92% either agreed or somewhat agreed that a patient portal would allow them to get information that would help them understand issues related to their health. Table 5 summarizes participants’ responses to questions regarding their difficulty in comprehending information contained in the simulation.

Table 5.

Participants Feelings About Information Contained in the Portal (N = 51).

	N	% Agree
I thought that the numerical tables (for example, the glucose tables) used in the portal were confusing.	20	39.2
I thought the graphs about health risks used in the patient portal were confusing.	15	29.4
I thought the graphs about blood test results used in the patient portal were confusing.	14	27.5
In general, I thought that the information I needed to answer the questions regarding health management tasks was difficult to understand.	16	31.4

Discussion

PHRs have the ability to deliver useful and trustworthy health information and data to patients to help them better manage their health and chronic conditions. This benefit of patient portals, however, is contingent on the ability of patients to be able to use the information provided in these systems in a meaningful way. Results from this study strongly suggest that older adults may encounter problems performing basic tasks using patient portals because of the Internet and numeracy skills required for use of these portals.

An interesting result from this study is the discrepancy that was found between health literacy and health numeracy skills in this sample. Approximately 84% of the participants in this study were determined to have “adequate” health literacy based upon their TOFHLA scores, implying that they should be able to read, understand, and interpret most health texts. However, the sample had health numeracy scores that were quite low; 52.9% of participants could not correctly answer the majority of objective numeracy questions. This result indicates that if the health texts used by older adults in patient portals involve numeric concepts, they may encounter problems even if they are considered to have “adequate” health literacy. It has been noted that “[t]he reporting of health literacy without disaggregating prose from numeracy obscures health numeracy skill” (Donelle, Hoffman-Goetz, & Arocha, 2007, p. 652) and the results from this study clearly underscore the importance of separately evaluating the health literacy and health numeracy of an individual.

Compared to a multisite study of 3,260 Medicare enrollees aged 65 years or older (Gazmararian et al., 1999), this sample scored quite highly on the TOFHLA. In the study of Medicare enrollees, which used the short version of the TOFHLA, the prevalence of inadequate or marginal health literacy among English speakers ranged from 26.8% to 44.0%. The majority of participants included were White females with at least a high school education and a yearly income of more than $15,000/year. Characteristics associated with higher rates of inadequate health literacy included African American race, older age, and fewer years of school completed. However, despite the fact that the sample of participants in our study contained older individuals, African Americans, and those with a high school or less education, the overwhelming majority were still considered to have “adequate” health literacy based on the results of the TOFHLA, a finding that strongly suggests the need for more sensitive measures of health literacy.

Another interesting result of this study was the weak correlation (r = .395) found between subjective and objective numeracy in this sample of older adults, with most of the participants tending to overestimate their skills. This result implies that older adults may believe that they can comprehend and use the numeric information provided in their PHR correctly when, in fact, they cannot. This could result in false assumptions that could easily lead to serious problems such as taking medications incorrectly or believing that abnormal test results are in the proper range.

The hierarchical regression models provide some important insights into factors that impact older adults’ performance of common health management tasks using a patient portal. While the sample was not highly educated—21.6% of the sample had only an education level of high school or less, and an additional 47.1% had only some college education—education was not found to significantly impact task performance. However, the Internet skills and numeracy skills of this sample were determined to have a significant impact on the performance of the tasks, both simple and complex. After accounting for education, the addition of Internet experience to the model significantly improved the prediction of task performance, accounting for 19.3% of the variance in simple tasks and 23.4% of the variance in complex tasks. When numeracy was added to the model, it resulted in an additional 21.2% of the variance in simple task performance being explained and 27.6% of the variance in complex task performance. The results of the regression analysis clearly indicate the importance of both Internet experience and numeracy skill in performance of typical portal tasks, whether simple or complex.

Results from this study can help identify interventions that may enhance the usability of patient portals for older adults. For example, results indicated that Internet experience had a significant impact on task performance. This could be expected, as many of the functions of the portal require skills that are consistent with those necessary for Internet use (i.e., scrolling, clicking on links, and closing windows). One implication is that healthcare providers and designers of these portals should be able to identify patients, especially older adults, with little or no prior Internet experience and provide instructional resources that could facilitate their proper use of the functions in their patient portals.

The results from this study also strongly suggest that careful consideration needs to be given to the presentation of numerical information in patient portals. For instance, 39.2% of participants found the numerical tables (such as the glucose table shown in Figure 3) to be confusing, indicating that tables displaying numeric information in the portal need to be formatted to provide information in a way that is more readily understood by those with low numeracy. Numbers given in a table or in a list of lab results that are out of the proper range for the patient could be highlighted to call attention to the fact that they are too high or too low, and audio and/or video explanations could be added to help patients understand and interpret this and other types of numeric information.

The great potential of patient portals to deliver important health information to patients lies in the ability for information to be tailored to meet the needs of the individual using the PHR. Krist et al. (2011) note that preventive care recommendations given to patients through their PHRs are already personalized according to the established guidelines, but point out that content and presentation of the PHR could be further personalized based upon other factors, including race/ethnicity, socioeconomic status, literacy, and numeracy. Results from this study strongly suggest that the numeracy factor may be one of the most critical factors to consider when tailoring PHRs to meet the needs of older adults.

One limitation of this study is the relatively small sample size. However, the sample was a diverse group of participants aged 60 to 85 years old, which makes the results more generalizable to older adults. There was variability in gender, ethnicity, education, income, and Internet experience, making the results more powerful. Future studies should test portal usability in a larger sample of older adults.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This project was supported by grant number R36HS018239 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. This project was also supported by CREATE grant number 3 P01 AG017211.

Author Biographies

Jessica Taha, MS, is a doctoral candidate in Ergonomics at the University of Miami. Her research interests include health literacy, health numeracy, and aging and technology.

Joseph Sharit, PhD, is a research professor in the Department of Industrial Engineering, and has appointments in the Department of Psychiatry and Behavioral Science and the Department of Anesthesiology at the University of Miami’s Miller School of Medicine. He is one of the principal investigators in the Center on Research and Education for Aging and Technology Enhancement (CREATE) at the University of Miami and a research scientist with the GRECC’s (Geriatric Research Education and Clinical Center) Laboratory for E-Learning and Multimedia Research at the Miami VA. His research interests in aging include assessment of older adult interaction with technological systems, decision making, and instructional and work systems design.

Sara J. Czaja, PhD, is a Leonard M. Miller Professor in the Department of Psychiatry and Behavioral Sciences, and a Professor of Industrial Engineering at the University of Miami. She is a fellow of the American Psychological Association and the Human Factors and Ergonomics Society and the Gerontological Society of America. She is also the Scientific Director of the Center on Aging at the University of Miami and the Director of the Center on Research and Education for Aging and Technology Enhancement (CREATE). Her research interests include: aging and cognition, aging and healthcare access, family caregiving, aging and technology, and functional assessment.

References

Ancker

J. S.

Barron

Rockoff

M. L.

Hauser

Pichardo

Szerencsy

Calman

(2011). Use of an electronic patient portal among disadvantaged populations. Journal of General Internal Medicine, 26 (10), 1117-1123.

Baker

D. W.

Gazmararian

J. A.

Sudano

Patterson

(2000). The association between age and health literacy among elderly persons. Journal of Gerontology, 55B (6), S368-S374.

Czaja

S. J.

Charness

Dijkstra

Fisk

A. D.

Rogers

W. A.

Sharit

(2006a). Background questionnaire (Tech. Rep. No. CREATE-2006-02). Miami: University of Miami.

Czaja

S. J.

Charness

Dijkstra

Fisk

A. D.

Rogers

W. A.

Sharit

(2006b). Computer and technology experience questionnaire (Tech. Rep. No. CREATE-2006-03). Miami: University of Miami.

Detmer

Bloomrosen

Raymond

Tang

(2008). Integrated personal health records: Transformative tools for consumer-centric care. BMC Medical Informatics and Decision Making, 8, 45.

Donelle

Hoffman-Goetz

Arocha

J. F.

(2007). Assessing health numeracy among community-dwelling older adults. Journal of Health Communication: International Perspectives, 12(7), 651-665. doi:10.1080/10810730701619919

Fagerlin

Zikmund-Fisher

B. J.

Ubel

P. A.

Jankovic

Derry

H. A.

Smith

D. M.

(2007). Measuring numeracy without a math test: Development of the subjective numeracy scale. Medical Decision Making, 27(5), 672-680.

Fisk

A. D.

Rogers

W. A.

Charness

Czaja

S. J.

Sharit

(2009). Designing for older adults: Principles and creative human factors approaches. Boca Raton, FL: CRC Press.

Folstein

M. F.

Folstein

S. E.

McHugh

P. R.

(1975). Mini-Mental State: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12(3), 189-198.

10.

Gazmararian

J. A.

Baker

D. W.

Williams

M. V.

Parker

R. M.

Scott

T. L.

Green

D. C.

…Koplan

J. P.

(1999). Health literacy among Medicare enrollees in a managed care organization. JAMA, 281(6), 545-551. doi:10.1001/jama.281.6.545

11.

Golbeck

A. L.

Ahlers-Schmidt

C. R.

Paschal

A. M.

Dismuke

S. E.

(2005). A definition and operational framework for health numeracy. American Journal of Preventive Medicine, 29(4), 375-376.

12.

Jay

G. M.

Willis

S. L.

(1992). Influence of direct computer experience on older adults’ attitudes toward computers. Journal of Gerontology: Psychological Sciences, 47(4), 250-257.

13.

Kaelber

D. C.

Jha

A. K.

Johnston

Middleton

Bates

D. W.

(2008). A research agenda for personal health records (PHRs). Journal of the American Medical Informatics Association, 15(6), 729-736.

14.

Krist

A. H.

Peele

Woolf

S. H.

Rothemich

S. F.

Loomis

J. F.

Longo

D. R.

Kuzel

A. J.

(2011). Designing a patient-centered personal health record to promote preventive care. BMC Medical Informatics and Decision Making, 11, 73.

15.

Kutner

Greenberg

Jin

Paulsen

(2006). The health literacy of America’s adults: Results from the 2003 National Assessment of Adult Literacy (NCES 2006-483, U.S. Department of Education). Washington, DC: National Center for Education Statistics.

16.

Lipkus

I. M.

Samsa

Rimer

B. K.

(2001). General performance on a numeracy scale among highly educated samples. Medical Decision Making, 21(1), 37-44.

17.

Lober

W. B.

Zierler

Herbaugh

Shinstrom

S. E.

Stolyar

Kim

E. H.

Kim

(2006). Barriers to the use of a personal health record by an elderly population. AMIA Annual Symposium Proceedings, 514-518. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1839577&tool=pmcentrez&rendertype=abstract

18.

Mungas

Marshall

S. C.

Weldon

Haan

Reed

B. R.

(1996). Age and education correction of mini-mental state examination for English- and Spanish-speaking elderly. Neurology, 46(3), 700-706.

19.

Parker

R. M.

Baker

D. W.

Williams

M. V.

Nurss

J. R.

(1995). The test of functional health literacy in adults: A new instrument for measuring patients’ literacy skills. Journal of General Internal Medicine, 10(10), 537-541.

20.

Sarkar

Karter

A. J.

Liu

J. Y.

Adler

N. E.

Nguyen

Lopez

Schillinger

(2011). Social disparities in internet patient portal use in diabetes: Evidence that the digital divide extends beyond access. Journal of the American Medical Informatics Association, 18, 318-321.

21.

Schwartz

L. M.

Woloshin

Black

W. C.

Welch

H. G.

(1997). The role of numeracy in understanding the benefit of screening mammography. Annals of Internal Medicine, 127(11), 966-972.

22.

Segall

Saville

J. G.

L’Engle

Carlson

Wright

M. C.

Schulman

Tcheng

J. E.

(2011). Usability evaluation of a personal health record. Proceedings of the AMIA 35th Annual Symposium on Biomedical and Health Informatics 2011, 1233-1242.

23.

Silvestre

Sue

V. M.

Allen

J. Y.

(2009). If you build it, will they come? The Kaiser Permanente model of online health care. Health Affairs, 28(2), 334-344.

24.

Wechsler

(1997). Wechsler Adult Intelligence Scale III (3rd ed.). San Antonio, TX: The Psychological Corporation.

25.

Williams

M. V.

Parker

R. M.

Baker

D. W.

Parikh

N. S.

Pitkin

Coates

W. C.

Nurss

J. R.

(1995). Inadequate functional health literacy among patients at two public hospitals. Journal of the American Medical Association, 274(21), 1677-1682.

26.

Yamin

C. K.

Emani

Williams

D. H.

Lipsitz

S. R.

Karson

A. S.

Wald

J. S.

Bates

D. W.

(2011). The digital divide in adoption and use of a personal health record. Archives of Internal Medicine, 171(6), 568-574.