Incorporating Reporting Efforts to Manage and Improve Health and Wellness Programs

Program evaluation and reporting

Managers of wellness programs want to know if their programs are meeting expectations. Commonly used tools include program evaluations, such as a return on investment (ROI) evaluation, as well as some form of periodic reporting. ³ Both these methods often utilize the same data that are generated and stored during routine program operations. Examples include administrative records such as personal information for participants, information related to insurance coverage, medical and pharmaceutical claims data, and survey data. Even though this article is on reporting, an overview of evaluations will be given first to provide context, and then the remainder of the article will focus on periodic reporting for the AtYourBest wellness program.

Program evaluations, such as the previously mentioned ROI, can be used to help gauge a program's success. Often, the frequency of the program's evaluation (eg, annual, biennial) is driven by how long it takes to generate a sufficient volume of data to ensure that analyses are large enough to make statistically valid conclusions. When these evaluations are performed in a scientifically rigorous manner, they can provide an accurate measure of the program's impact. ⁴ The study team has previously published an evaluation of a high-risk case management program that generated savings in excess of what it cost to run the program, and participants had better quality of care compared with nonparticipants. ⁵ However, a limitation of the evaluation process is that the time lag of about 1.5 years between program start and having actionable data is inadequate to support day-to-day operations.

In contrast, periodic reporting can be used to monitor program performance on a more frequent basis. Systematic and data-driven reporting processes are increasingly being used to improve health care programs. Walter Shewhart is recognized as a pioneer in the field of statistical quality control with the publication of his book, The Economic Control of Quality of Manufactured Product, in 1931. ⁶ More recently, Nolan and Provost have published on the importance of understanding variation when managers are making programmatic decisions. ⁷ In 2007, Thor et al provided a comprehensive review of statistical process control in health care improvement. ⁸ Thor et al describe Shewhart as the father of statistical process control and note how such methods have been used to improve health care processes or enabled patients to better manage their own health. More recently, Dowding et al conducted a literature review of reporting tools being used for improving patient care and found “considerable heterogeneity” regarding who was using them, the program being managed, and the indicators used. ⁹

Although there has been considerable discussion on statistical process control and reporting tools to manage health care quality improvement in the scientific literature, there is little discussion on how these processes can be used to help manage a health and wellness program, such as AtYourBest. The exceptions include an article by Ozminkowski and Serxner ³ that provides an overview of how to use reporting to improve health and productivity in the workplace. The objective of the current article is to describe how periodic reporting is being used to provide performance metrics needed to manage and improve the AtYourBest wellness program. The article describes how weekly and monthly reporting processes inform program management as they work through the challenges of managing and improving a relatively new wellness program, with plans to expand to all states.

Conceptual overview

In their article, ³ Ozminkowski and Serxner described the 8 key issues required for a solid reporting process. These include metrics to report information about (1) who the program serves, (2) the health risks those individuals have, (3) the physical and mental health conditions they have, (4) the operating metrics for the programs they use to manage their health risks and conditions, (5) the quality of care that people receive to address these risks and conditions, (6) associated health care utilization and expenditures, (7) productivity-related utilization and expenditures, and (8) satisfaction with access to care, health status, and program services. Although productivity-related utilization and expenditures are most likely not applicable for the AtYourBest population, the majority of whom are 65 years of age or older, the remainder of these issues are addressed in reporting about the AtYourBest population. As will be explained in more detail, the AtYourBest reporting process describes individuals who are eligible to participate in the program and covers the operating metrics in detail. Health risk information is collected from health risk appraisal surveys that are conducted independent of the AtYourBest program. Similarly, information about health conditions, quality of care, access to care, health status, and program satisfaction also are assessed through surveys that are independent of the AtYourBest program, although these data are available to program management. This article goes beyond the article by Ozminkowski and Serxner by showing how the framework can be executed in real life and how AtYourBest reporting has evolved from this framework.

Data acquisition, cleaning, and storage

To support the AtYourBest reporting process, the reporting team receives weekly data inputs from 4 different program components, including the online portal, information on gym memberships, telephonic coaching, and participation in the in-market health events. These data are loaded, cleaned, and placed into a data warehouse. The data are then compiled into weekly and monthly AtYourBest reports, which are used by program staff to manage day-to-day operations and determine if goals are being met.

Data provided from all 4 sources (the online portal, gym membership, telephonic coaching, and in-market event attendance) include personal identifiers as well as membership information and utilization/attendance metrics. These data are merged with administrative data related to the individual's Medicare Supplement Insurance, such as plan-specific information and dates of coverage. Next, the data are cleaned, which includes looking for missing data and null values that are outside of expected ranges. Dates, such as date of birth and enrollment dates, are checked to make sure they fall within established parameters. Personal identifiers are checked to make sure they are unique and accurate; these include first and last name, date of birth, and residential address. In addition to checking the data, the format of each incoming data file is checked to make sure the column headings are correct, and in the correct order. Once cleaned, the data are stored in the AtYourBest data warehouse, which is designed to ensure the data are secure, yet accessible for use. Private health information is protected by ensuring that only the minimum necessary information is used and that access to these data is restricted to those with proper authorization. In addition, all staff receive annual training with regard to UnitedHealth Group's data privacy and data release governance policies and must pass an annual certification test to demonstrate their knowledge of how to handle confidential data and related issues.

AtYourBest reporting

Weekly and monthly reports are used, and there is a trade-off in efficiencies between them. The weekly reports provide the best opportunity for near real-time reporting but are based on a relatively small sample, as the report only captures weekly increments of Activated Members and Ongoing Engagement data for the snapshots. Newly engaged individuals are called “Activated Members,” while those who remain engaged are listed in the “Ongoing Engagement” section of the reports. In contrast, the monthly reports have some latency but are based on a larger sample, as they cumulatively include all individuals in the program. Having a larger sample helps to account for random patterns in the data that can occur on a daily or weekly basis.

The monthly report

The monthly report is compiled as a Microsoft Excel (Microsoft Corp., Redmond, WA) spreadsheet with separate worksheets for each state and a summary worksheet that includes all states combined. Trend lines are used to provide a quick visual summary of how the metric has performed over time. Each worksheet is organized into 5 sections. The first section lists the number of Medicare beneficiaries who are eligible to participate in that state, while the summary worksheet includes the total of all eligible individuals from the 4 states where AtYourBest is being piloted.

The second section of the monthly report contains information about program participation for Activated Members and those with Ongoing Engagement, as well as data for all who are registered for the program, where total registered is a cumulative measure of everyone who has participated in the program during the current year. It also includes membership reporting metrics for the 4 program components, including the online portal, gym membership, telephonic coaching, and in-market event attendance. In this section, targets are established by program management and then tracked to determine if they are met. Reported values are “On Track” if they exceed the target, or are not more than 10% below the target. The reported value is “At Risk” if it lies between 11% and 20% below the target. Finally, reported values are “Not On Track” if they are more than 20% below the target. Tracking results are highlighted using green, yellow, and red colors for On Track, At Risk, and Not On Track, respectively. This section also tracks how many program participants have utilized 2, 3, or all 4 of the AtYourBest program components. Figure 1 shows the program participation section from a monthly report.

OPEN IN VIEWER

FIG. 1.

Example metrics for program participation, monthly AtYourBest wellness program report, September 2015. TBD, to be determined; YTD, year to date.

The third section of the monthly report lists program activities and interventions (Fig. 2). This section tracks the number of visits to the online health portal or gym, as well as the frequency of telephonic wellness coaching or participation statistics for the in-market events. For example, gym usage is tracked by calculating the average participation days per month, as well as tabulating categorically the percent of individuals who visited the gym 0–3, 4–7, 8–11, or 12 or more days per month. The specific wellness coaching categories tracked include weight management, stress reduction, nutrition, tobacco cessation, exercise, heart health, diabetes lifestyle, and fall avoidance.

OPEN IN VIEWER

FIG. 2.

Example metrics for program activities and interventions, monthly AtYourBest wellness program report, September 2015.

The fourth section of the monthly report includes data about member demographics (Fig. 3). Age categories, sex, and hierarchical condition category (HCC) score are tracked, including the total numbers who are eligible to engage, the number of Activated Members, and the number with Ongoing Engagement. The age categories include ≤69, 70–74, 75–79, and ≥80, while the HCC categories are 0–1.5, 1.6–2.0, and >2.0. The HCC score is obtained from a standard risk adjustment tool utilized by the Centers for Medicare & Medicaid Services (CMS). The HCC model developed by CMS uses a sample of more than 1 million individuals with Medicare to predict health care costs based on demographic and health conditions, which are then converted into relative risk factors. ¹⁰ Thus, the average Medicare insured has an HCC score of 1.0, while scores greater or less than 1.0 reflect expectations, respectively, for higher or lower future costs based on the individual's age, sex, and medical conditions.

OPEN IN VIEWER

FIG. 3.

Example metrics for program demographics, monthly AtYourBest wellness program report, September 2015. CRM, Customer Reporting Mart (database); HCC, Hierarchical Condition Category; N/A, not applicable.

Finally, the fifth section of the monthly report includes information about program member experiences and outcomes (Fig. 4). Telephonic wellness coaching outcomes include achieving blood pressure and cholesterol targets, stress level reduction, increased physical activity, improvement in hemoglobin A1c levels, tobacco cessation, and weight loss. The types of calls that program members place to the AtYourBest customer service center also are tracked in this section.

OPEN IN VIEWER

FIG. 4.

Example metrics for program member experiences and outcomes, monthly AtYourBest wellness program report, September 2015. BP, blood pressure; Not AYB, not in the AtYourBest program; NOVU, marketing technology platform.

The weekly report

The weekly AtYourBest report is an abbreviated version of the monthly report. The first section includes data on Activated Members based on participation within each of the 4 components. The second section is identical to the first section but includes data on those with Ongoing Engagement. Each section provides the number engaged during the last 4 weeks as well as summary metrics for the current month and year to date. Also included are metrics for the percent who were Activated Members or with Ongoing Engagement based on the number eligible. Figure 5 provides data from one of the weekly reports. Note that gym membership for those with Ongoing Engagement is not included. Gym membership for Activated Members is determined through the enrollment process, which can be tracked weekly, while ongoing participation data are only available on a monthly basis.

OPEN IN VIEWER

FIG. 5.

Example metrics for Weekly AtYourBest wellness program report, December 2015.

The monthly report

In September 2015, about 650,000 individuals were eligible to participate in AtYourBest. The demographic comparisons are shown in Figure 3. Individuals who are younger than 75 years of age and women were more likely to engage in the program. In contrast, individuals with HCC scores >2, indicating they were likely to have double the average health care expenditures, were less likely to engage.

Figure 1, containing the summary section of the monthly report, shows 6 metrics on track, 1 at risk, and 3 that are not on track. Program management used this report to identify several items needing attention. For example, based on the percent of Activated Members compared with the number eligible, program participation was found not to be on track. Also, participation among Activated Members is lower than desired in the online portal, telephonic wellness coaching, and the in-market events. Using this report, program management focused resources to increase the number of Activated Members and their participation in these 3 components. As a result, these items were back on track in subsequent reports.

In the Program Activities and Interventions section, 2 items were not performing as well as desired and will be discussed in more detail next.

First, ∼80% of AtYourBest participants used the online portal fewer than 3 times. Noting this low usage, program management made revisions to enhance the participants' experience with this offering. First, it was decided that the portal views were too busy and had too much information, making it hard for participants to focus on 1 particular topic. As a result, both the portal content and design were simplified to make the online experience less confusing and more friendly to users. In addition, participants were asked to provide their zip code of residence as a first filter to identify local in-market events, whereas previous searches first identified events inside and outside of the participant's zip code, and then additional searching had to be accomplished to identify events within the desired zip code.

It also was noted that the telephonic wellness coaching was underutilized. Additional investigation revealed that it was often difficult for the coaches to reach participants by telephone. As a result, an automated interactive voice response (IVR) telephone component was added. An IVR call is made within a couple days of the telephone appointment with the wellness coach to remind the participant of the pending appointment. Then, should the wellness coach fail to contact the participant at the scheduled time, a second IVR call is initiated to notify the participant of the missed appointment and notify him or her of a new appointment time.

As the AtYourBest program matures, the monthly reports provide feedback to program management regarding which of the 4 offerings are popular, and conversely, which ones are not well utilized. The managers of the 4 offerings meet each month to review monthly results. If the monthly reports suggest that there are business risks or opportunities, managers use this forum to develop an action plan for the risk or opportunity. For example, telephonic wellness coaching, diabetes lifestyle, and heart health were the most popular topics, so program management may be able to identify learnings from these topics to improve interest in the fall avoidance and stress reduction topics. Similarly, learnings from the in-market events for exercise may help improve the smoking cessation events.

The weekly report

The December 2015 week-by-week trends are shown in Figure 2. Other than the number of Activated Members who are using the online portal, the numbers demonstrate week-by-week variability. For example, 87 Activated Members used the gym in the first week of December; this dropped to 57 in the second week, and rebounded back to 80 in the third week. All values were down in the fourth week, demonstrating less activity over the Christmas holiday. Looking at the year-to-date data, it appears that telephonic wellness coaching is the most popular among Activated Members and those with Ongoing Engagement.

Although the monthly reports were used to better manage the 4 program offerings, the weekly reports help to gauge program engagement and scoping. Program management uses the engagement metrics from the weekly report to determine the effectiveness of marketing campaigns to increase engagement. This is accomplished by looking for changes in engagement based on the previous week's marketing campaign. Overall program success is measured by the engagement rates, which reflect the program's popularity. A national program is only warranted if engagement rates reflect that the program is popular.