User-Centered Evaluations with Older Adults: Testing the Usability of a Mobile Health System for Heart Failure Self-Management

Many older adults living with heart failure struggle to follow recommended self-management routines. To help older adults with heart failure more effectively and efficiently self-manage their disease, we developed Engage, a mobile health application promoting the performance, logging, and sharing of routine self-management behaviors. This paper reports on the usability evaluation of the Engage system with 15 older adults with heart failure and informal caregivers. In two phases, participants used Engage during a task-based usability test (n=5) and a scenario-based usability test (n=10). Usability and performance data were assessed through video-recorded observation and the administration of the system usability scale (SUS) and NASA Task Load Index (TLX). We found that task-based testing was useful in quickly identifying problems within our application, but scenario-based testing elicited more valuable feedback from older adults. A comparison of the different evaluation methods used and the discussion of the challenges encountered provide multiple implications for the practice of usability testing of mobile health products with older adults.