Abstract
Background:
Chronic heart failure (HF) is a complex and costly disease. Daily weight and symptom monitoring is the cornerstone of HF management. Little information exists about feasibility of a mobile monitoring intervention among minority patients with HF.
Methods:
We developed and tested usability of a mobile-monitoring system in minority patients with HF in a 2:1 randomized controlled trial. We tracked usage and obtained feedback on usability and the system overall at 1, 2, and 3 months.
Results:
Forty-two participants aged 53.0 ± 9.4 years (mean ± standard deviation) were randomized to the mobile-monitoring intervention group. They included the following: 67% males, 76% White Hispanics, 21% African Americans, and 52% with high school education or less. Over the 3-month intervention period, 26 (62%) participants used the system over 50% of the time. Overall, on a 1.0–7.0 scale for both, program satisfaction scores were excellent (mean 6.84 ± 0.46), and the usability ratings were all above 6.0. Comparing 1- to 3-month responses, there was a substantial increase in the percentage of participants who felt the system was easy to use after they had gotten used to it (84% vs. 94%) and that navigating the system was not complicated (78% vs. 84%). Almost all participants said that the program made them feel more secure about their health and that they would stay enrolled in a program like this. None of them had used a similar system before.
Conclusions:
A mobile phone-based disease management program is feasible in a minority county hospital population and offers a modality to help reduce ethnic disparity.
Introduction
Chronic heart failure (HF) is a complex and costly disease. 1,2 The American College of Cardiology/American Heart Association Guidelines for the Management of HF recommend daily weight monitoring as the cornerstone for management of patients with HF. 3 Traditional telemedicine programs for HF have used devices installed in the home as information and transmission hubs, but this has resulted in increased program costs. 4,5 One innovative method to follow changes in weight and symptoms is using mobile phones. 6 However, the usability results from previous HF trials using mobile phone interventions have been inconsistent. A previous study by Scherr et al. required participants to enter readings using a mobile phone's Internet browser. 7 The authors reported that many participants found it difficult to use the intervention, and 12 out of the 54 participants never used the technology. Scherr et al. highlighted the importance of system design on the success of the telemonitoring system. 7 Seto et al. showed that participants had high adherence to a mobile phone-based telemonitoring system, that the program was feasible for patients, including the elderly and those with no experience with mobile phones, and that it improved quality of life through improved self-care. 8,9
Mobile phones have been proposed as the new telemonitoring modality because they are ubiquitous, already a part of peoples' everyday lives, and thus, low cost and scalable. 10 However, few HF disease-management programs have used a mobile phone system. 9,11 There is currently a lack of insight into the usability of a mobile telemonitoring system in minority populations with HF 12 and the desirable features required for successful telemonitoring solutions from the patients' perspective in such populations.
We received a “technology transfer–commercial partnership grant” to pilot a mobile phone-based disease management system for monitoring HF patients. We designed the telemonitoring system with a commercial partner, Generation One, a private firm offering mobile solutions. We evaluated the monitoring platform in a 2:1 randomized controlled prospective study in 61 HF participants, with 42 of them randomized to the intervention arm, in a 3-month study. The primary intent of the trial was to pilot the telemonitoring system and determine its impact on self-efficacy, knowledge, and quality of life. The quantitative findings from the trial suggested that the telemonitoring system improved self-efficacy for chronic disease and for HF, and quality of life, but it did not affect knowledge in a predominantly Hispanic group of participants. These quantitative results are presented in a separate manuscript. 13 This article discusses the mobile-telemonitoring system usage data, usability and qualitative findings from the trial based on patient surveys at three time points during the study, and in-depth interviews conducted at 3 months. The objective of the qualitative analysis was to attempt to obtain an understanding of the patient perspectives regarding using a mobile phone-based telemonitoring system among a minority county hospital population that comprised mostly Hispanics.
Methods
Participants
We recruited HF patients from the Heart Failure Clinic at Jackson Memorial Hospital, a county hospital in Miami. The intervention group had 42 participants. To be eligible for participation, patients had to be adult, community dwelling, and ambulatory patients diagnosed with HF, able to speak and read English or Spanish, with anticipated survival ≥6 months, able and willing to operate the mobile phone, and read, comprehend, and respond to the mobile phone-based queries. The University of Miami Institutional Review Board approved the study before its start.
Study Intervention
Participants in the intervention group received a mobile phone (model FG 630) for use during the 3-month study period for daily monitoring. The program covered all costs for data use on the mobile phone and 30 free minutes of calling each month. During the enrollment session, we instructed participants one-on-one on how to use the mobile phone Web browser messaging and gave them a two-page step-by-step instruction manual with pictures ( Fig. 1 ). We provided telephone support throughout the study and additional in-person training if needed. Participants chose their preferred time and language to receive the daily questions. The telemonitoring system required the participants to answer 10 daily questions regarding their weight and HF symptoms for 3 months. Other than the weight, all questions required simple yes–no answers. The daily responses took about 2 to 3 min. Participants received 3 messages 15 min apart if they did not respond to the first automated message. The server used the cell number for patient identification. If responses indicated possible HF worsening (based on preconfigured algorithms), the patient received a message asking him/her to contact the study coordinator. The study coordinator was able to view the data on a secure Web site and received an alert on his/her study mobile phone. The study coordinator then coordinated the patients' care with the providers in the Heart Failure Clinic and acted as the intermediary between the patients and the providers. We contacted participants in the intervention group at least once a month, or more often as the need arose, and administered the scheduled questionnaires.

Two-page pictorial instruction provided to the participant.
Measures
We conducted interviews in English or Spanish as per the participant's preference to collect the following data in addition to the outcomes data.
Clinical and Sociodemographic Characteristics
We collected these data at baseline through participant interviews and chart reviews. Included were age, gender, ethnicity, education, marital and employment status, current smoking and alcohol consumption status, mobile phone ownership and use, NYHA class, and comorbidity information (Table 1).
Clinical, and Sociodemographic Characteristics of Participants (n = 42)
Married or in a marriage-like relationship.
Divorced, Widowed, Never Married.
New York Heart Association Heart Failure Classification.
Additional comorbidities include history of stroke, myocardial infarction, arrhythmia, peripheral vascular disease, type II diabetes, and hypertension.
HF, heart failure.
Usability Survey
We developed a 31-item survey for this study, which we conducted at the end of 1, 2, and 3 months, over the phone or in person. The survey included questions regarding six aspects of the mobile monitoring system: ease of use, navigation, readability, messages, confidence, and motivation to use the system; it also asked about satisfaction with the program overall. The participants answered the questions on a 7-point Likert scale, with a score of 1 signifying that the statement was very inaccurate and 7 signifying that the statement was very accurate. The final total score for each aspect was the mean of the responses for that aspect, with 7 being the best possible score.
Program Evaluation
In face-to-face interviews at the end of the study, we asked the participants 29 open-ended questions to obtain an in-depth understanding of individual experiences, perceptions, motivating factors, and recommendations.
System Usage Data
System usage data were obtained from data files that were downloaded into the study's database from the server where patient responses were stored.
Statistical Methods
Descriptive statistics were used to assess and summarize the distribution of each of the baseline characteristics as well as system use, usability, and satisfaction measures. Differences between subgroups of interest with respect to the latter variables and their change over time were ascertained using, as appropriate, independent or paired t tests, Mann–Whitney tests, analysis of variance, or Kruskal–Wallis tests. Finally, simple and multiple linear regression analyses were used to assess different baseline characteristics as predictors of usability and satisfaction. All statistical analyses were performed using the Statistix 10 (Analytical Software, Tallahassee, FL). Reported in the results are 95% confidence intervals (95% CIs) and p-values corresponding to two-sided hypothesis tests. Results of all tests with a p-value <0.05 were considered as statistically significant at the 5% Alpha level. At the qualitative level, program evaluation interviews were analyzed for meaningful segments of data (a phrase and a sentence) to identify patterns.
Results
Baseline sociodemographic characteristics of the 42 participants randomized to the mobile phone intervention group are shown in Table 1. Their age ranged from 27 to 83 years with a mean ± standard deviation (SD) of 53.0 ± 9.4, a 95% CI of 50.1–56.0, and a median of 53.0. This group included 31 (73.8%) White Hispanics and 9 (21.4%) African Americans. Forty (95.2%) of the participants had at least one additional comorbidity, besides HF. Thirty-one participants used the system all 3 months, and 32 responded to the surveys at all three time points.
Mobile Monitoring System Use
Thirty-six of the 42 participants (85.7%) used the mobile phone system, and 6 participants (14.3%) never used it. Dividing the intervention period into three 1-month intervals, 36 participants (85.7%) used the cell phone system in month 1, range = (2–30) days, mean ± SD = 21.5 ± 7.6 days; 33 (78.5%) in month 2, range = (3–30) days, mean ± SD = 21.1 ± 7.2 days; and 31 (73.8%) in month 3, range = (2–30) days, mean ± SD = 18.2 ± 7.2 days. Twenty-six (62%) of 42 participants used the system >50% of the time and 31 (73.8%) used it in all 3 months. Figure 2 shows the percent of days each patient responded to the mobile phone system, as a percent of the 90 total days in the intervention. Among the 31 participants that used the system in all 3 months, the total number of times they used it ranged from 21 to 87 with a mean ± SD of 62.0 ± 15.9 and was not significantly related to any of the variables listed in Table 1. Figure 3 shows the change in mean system use by participants over the 3 months. There was no difference in mean system use rate in the first month among the participants that ever used the system and the participants that used the system all 3 months.

Patient response percentage over 3-month intervention period (n = 42).

Patient mean response rate in each month as a percent of messages received ± SE. Line 1 shows the mean response rate for all participants who ever responded (n = 36). Line 2 shows the mean response rate for those who responded in all 3 months (n = 31). The range used in this figure attempts to show clearly the change over the 3 months. Line 1 shows that among the participants that ever used the system (n = 36), mean system use rates (percent ± SE) were higher in month 1 (71.7 ± 4.2) compared to month 2 (64.4 ± 5.3) and month 3 (52.3 ± 5.5). Line 2 shows that among the participants that used the system all 3 months (n = 31), mean system use average rates were higher in the first 2-month periods (72.2 ± 4.8 and 73.7 ± 3.7, respectively) and then declined to 60.8 ± 4.3 during the third month. SE, standard error.
Technical Challenges
Figure 2 shows that the first 4 participants never used the system. This was because they never received the messages due to an error on part of the study coordinator, who did not complete the registration process. This error was noticed after the first few weeks and was resolved by providing additional training to the study coordinator. Three participants had problems with the mobile phones or chargers, which were resolved by changing the phones.
Usability
Ten participants expressed difficulty in understanding how to use the system: five of them understood the process by explaining one additional time, five needed two additional sessions. This prompted us to develop a two-page instruction manual for participants using a picture for each step ( Fig. 1 ), rather than the several page manual that we used initially.
We collected the usability survey from 35 participants at month 1, 32 participants at month 2, and 34 participants at month 3. Only 32 participants who answered surveys at all three time points are included in the data presented in Figure 4, which shows the final mean score for each aspect assessed. Below each graph in Figure 4 are the specific questions asked for each aspect. The final scores for all aspects ranged between six and seven, showing high satisfaction.

Participants answered questions on a 7-point Likert scale, with a score of 7 signifying that the statement was very accurate to 1 signifying that the statement was very inaccurate. A range of 6 to 7 is used on the Y-axis to highlight the change at different time points. a When scoring it, this question was back coded, as agreement to the statement would constitute a more negative response. The average scores improved progressively from 1 to 3 months for navigation, readability, and confidence in the system. The scores improved and then decreased for ease of use, messages, and motivation. The score for ease of use changed most from 6.3 at month 1 to 6.8 at month 2 and then deteriorated to 6.4 at month 3.
We used item responses of only 6 and 7 to reflect “complete agreement,” 1–2 “complete disagreement,” and 3–5 as neutral responses. When comparing 1- to 3-month responses, there was a substantial improvement in the percent of participants who felt the system was easy to use after they had gotten used to it (84% vs. 94%), navigating the system was not complicated (78% vs. 84%), it was easy to make corrections (53–65%), or easy to make mistakes (38–22%). Eighty-eight percent of the participants felt that they were able to read the queries on the mobile phone. All participants said that the program made them feel more secure about their health and that they would stay enrolled in a program like this.
Patient satisfaction with the program overall is shown in Table 2. The scores for overall satisfaction with the program were >6.8 at all three time points, showing a high level of satisfaction overall.
Frequency Distribution of Responses to Questions on Overall Satisfaction with Program (n = 32)
Participants answered questions on a 7-point Likert scale, with a score of 1 signifying that the statement was very inaccurate and 7 signifying that the statement was very accurate.
User Experiences, Perspectives, and Recommendations
We elicited user experience with and perspectives regarding the mobile monitoring system and the program using open-ended questions at study end, which are summarized in Table 3. We assessed participants' willingness to pay to be in a similar program every month (Table 4).
Users' Experiences, Perspectives, and Recommendations for a Heart Failure Mobile Monitoring System (n = 34)
Other positive themes that emerged from the participants' comments included the following: (1) mobile phones are a fast and easy way to respond to questions; (2) since mobile phones are part of everyday life, it is easy to remember to answer daily questions; and (3) since participants received daily reminders to weigh themselves, it had become a habit. Some of the notable positive comments were as follows: the “system” “is useful because it helped me to control weight,” “sent me good messages to improve my health,” “sent me useful messages,” “kept me on track and to remember to weigh myself daily,” “reminded me to take care of my health,” “motivated me to take care of my health,” “helped me learn more about my CHF,” and “good way to control my weight.” One participant said that it reminded him that “people care about my health and help me know what I need to do to take care of my health.”
Participant Willingness to Pay for a Heart Failure Mobile Monitoring System (n = 32)
Contacts with Study Coordinator
The number of contacts with the study coordinator over the 3-month period ranged between 7 and 17, with a mean of 11.66 ± 2.18.
Discussion
This randomized controlled trial was the first of its kind, to our knowledge, to use a mobile phone Web browser messaging application that monitored daily weights and HF symptoms in a minority, predominantly Hispanic, county hospital population for 3 months. Participants reported high acceptance of the mobile phone monitoring system and found the intervention highly feasible and helpful in their care.
Most participants in our study were able to enter and transmit their daily responses regarding weight and symptoms using the Web browser. Sixty-two percent of the participants used the system more than 50% of the time and 74% used it for all the 3 months. However, the use declined over time, similar to that reported in other studies. 9,11,12,14 It was noteworthy that system use increased with increasing age. As shown in Figure 3 , there was no difference in mean system use rate in the first month among the participants that ever used the system and the participants that used the system all 3 months.
There were high levels of satisfaction with all usability aspects, including efficiency and effectiveness, with little change in scores over the 3-month period (Fig. 4). The slight increase and then decrease in ease of use scores over the 3 months probably reflects a burnout on the participants' part to using the system daily, rather than the system becoming difficult to use.
Our participants voiced their desire to have the feedback communication from providers who had direct authority to make medical decisions. Desai and Stevenson agreed with our participants when they deliberated why the Telemonitoring to Improve Heart Failure Outcomes (Tele-HF) Trial 15 had failed to show positive health outcomes. They proposed that the team member receiving the data should be empowered to contact the patient directly with a treatment plan, without having to “triangulate” the discussion with a physician before recommending a plan to the patient. 16 Other studies also suggested that making the intervention more interactive with more automated feedback and connectivity to their actual providers may increase participant interest. 8
Most participants expressed they would be willing to pay a small monthly fee to be in a similar program. However, the willingness to pay a relatively higher monthly amount of >$20 decreased from 75% to 47% of the participants from 1 to 3 months (Table 4). This may reflect burnout, or a decrease in the novelty of the intervention. Alternatively, it may be a reflection of an increase in participants' self-efficacy, which would make them feel that it was less valuable. In a previous study, some patients did not want to be watched long-term, while others were concerned they might become dependent on the system. 8 However, our study participants did not express that sentiment, and almost all said they would like to stay enrolled in a similar program for longer periods.
Patients reported that they thought mobile intervention was easy, useful, and fast, and it helped them manage their weight. Participants commented that the daily reminders to weigh themselves had made that a habit. On further probing, participants reported that it helped them realize the importance of daily weights and that they would continue monitoring their daily weight even after the program ended. These comments were validated by their improved self-efficacy scores and reflect findings from other studies. 17
The best features of our intervention were that it was offered on a mobile phone, available in 2 languages, allowed participants to choose a time to receive the daily messages, had simple response options, and same familiar questions every day.
All participants except one had a personal mobile phone, corroborating data from national surveys, which show that low-income minority populations have high rates of mobile phone ownership. 10,18 Although 76% of the participants reported having texting experience before enrollment, only 21% had used Internet on their mobile phones. We observed low participant response rates, requests for additional training in use of the Web browser messages, and feedback that the browser system was difficult to use. When we broke down the steps to access and respond to each browser message, this seemingly simple task had 10 steps, as shown in Figure 1 . This prompted us to develop the pictorial instruction sheet, but some participants had already lost interest in the study by that time. Perhaps participants who had experience with using Internet on their mobile devices might have found it easier to use Web browser messages; however, we did not find that among our small sample.
Our study required use of a study-specific phone. Participants did not have the option to receive messages on their personal mobile phones. However, some participants felt that it was difficult to carry around two mobile phones and they often forgot to respond because the study mobile phone was not with them at all times. We provided study-specific mobile phones with data and voice coverage because of our low-income county hospital target population. However, that may have been unnecessary. Text messaging interventions may be more intuitive to use with fewer steps compared to browser messages. In addition, text messaging interventions may also pose fewer connectivity challenges because of lower data needs.
We learned several lessons. Staff training in intervention setup should be rigorous before patient enrollment begins. Developing a short, pictorial step-by-step instruction manual is helpful in participant training. Additional training of the participants in using the Web browser messages often resolves the usability challenges in motivated participants. It is essential to have the intervention in the languages spoken by the target audience. Steps to correct data entered by participants should be built into the system. Modifying questions and information frequently and having the daily responses monitored by providers who can modify the treatment plan may increase participant engagement. Text messaging mobile interventions, rather than browser-based interventions, may be more usable and conceivably lead to greater use and larger effects in similar populations. 19 These lessons and our participants' insightful suggestions for program improvement (Table 3) may be valuable to other similar programs.
The biggest limitation in our study was that we enrolled participants from the Heart Failure Clinic at a large county hospital, who may not be representative of the general population with HF. The low intensity of our remote monitoring system may not be suitable for patients with severe HF. We also did not use standardized tools for usability evaluation. 20 Future studies should use monitoring systems that are compatible across multiple devices and use participants' existing mobile phones, thus making such interventions more scalable.
To the best of our knowledge, this is the first study of its kind to evaluate the use and outcomes of a cellular Web browser messaging program to monitor daily symptoms and engage predominantly Hispanic participants with HF in self-managing their disease. We have shown that a mobile phone monitoring system is feasible in a predominantly low income, less educated multiethnic population, and most usability challenges may be averted with proper planning and training. Monitoring and reminder systems leveraging commonly used mobile devices that can increase self-efficacy and detect deterioration preemptively would be of immense benefit in management of HF and other chronic diseases. 21,22 The results from this study underscore the unique opportunity mobile phones present to address ethnic disparity in chronic disease management in minority populations. 23 There is a need for more research in how to design mobile interventions that are usable across a wide range of patients and devices and how to best target mobile phone-based interventions to patients most likely to benefit.
Footnotes
Acknowledgments
This research was supported by the State of Florida Department of Health's James and Esther King Biomedical Research Program (Grant No. 09KC-01). The authors thank Drs. Bernard A. Roos, Herman Cheung, Sara Czaja, and Anne E. Burdick for mentorship and guidance; Sandeep Dayanand and Silvana Cobian for helping with patient recruitment, data collection, and study implementation; and Diana Ruiz for help with article preparation. The authors also thank the staff in the Jackson Heart Failure Clinic for allowing them to work with their patients and collaborating with them on their care. The authors gratefully acknowledge the support of the Geriatric Research, Education, and Clinical Center, Bruce W. Carter Department of Veterans Affairs Medical Center, Miami, FL, and the Miami VA Research.
Disclosure Statement
No competing financial interests exist.
