Long-Term Results Evaluation in Medical Management Programs

Introduction

T he evaluation of multiyear programs in medical management in general and disease management in particular is affected by the time period involved and the way in which results are calculated. As the optimal program effect is exhausted for a given population under a medical management delivery approach, total impact appears to “plateau” (eg, quality indicators often reach and seem to peak at a stable value, utilization begins to be maintained within a constant range). This is often mistaken by less sophisticated observers of these programs as exhaustion of effect or “no effect,” as opposed to optimization of impact and sustained improvement. This concern is particularly a focus of nonclinical managers who demand year-over-year demonstrated savings and return on investment for these programs.

In a related observation, programs often also seem to have their maximum impact on newly-identified, high-cost/risk, or “fresh” members such as new employees, causing managers to wonder whether programs should only be applied to these subpopulations, while a lower cost “surveillance” program is applied to the general population or elsewhere. The general argument follows that aggressive medical management programs should be reserved for “high acuity” patients, for whom high-cost exposure is most susceptible to savings interventions and/or to newly enrolled members for whom the impact is clearly visible in unmanaged populations. This has emerged as a counter position to the more traditional “population based” expenditure of energies in disease management and medical management programs on a continuous, albeit focused, basis across entire populations.

General Analytics

To first consider whether a program has reached an optimal state, one needs to determine whether the program is having an effect at all. At the population level, actuaries are able to predict the relative cost trajectory of medical costs over time, which in the recent past has been an upward sloping line (Fig. 1). (The line in fact shows lots of variation within both short and longer periods of time, but for descriptive purposes we will refer here to a straight line.)

OPEN IN VIEWER

FIG. 1.

A trended baseline. Typically, baseline trends in care costs have been portrayed as linear trends, and costs extrapolated into the future for cost predictions. In fact, costs often move erratically in response to demographic mix, trends in medical care, and medical management and policy determinations, among others. In most cost predictions, however, these changes are noted as caveats for projected care cost trends and a linear assumption is made anyway.

A significant body of work has been developed around estimating costs and calculating impact of medical management programs in the absence of a control group. ¹ The impact of a cost savings program typically occurs in one of 2 ways, operating alone or in combination. The first is a model commonly known as “bend the trend.” In this approach, evidence of changes in the slope of the medical care cost trend line is sought. Here, costs on the absolute level are typically gradually reduced and the actual cost of health care services begins to change to a less steep slope than the predicted trend, flattened (rarely even to no trend or, even more rarely, decreasing trends). The difference in cost between the projected and actual costs is thus measured savings (Fig. 2).

OPEN IN VIEWER

FIG. 2.

“Bending the Trend” is the stated goal for most medical management initiatives, because continuously rising health care costs are a given. It is in fact rare for programs to stop or even reverse a trend for a variety of reasons, but even reducing the health care cost trend to the consumer price index growth would be considered a major victory by most initiative owners.

On a year-over-year comparative basis, annual savings over time typically decrease as the cost curve plateaus and shows persistent effects on slowing down the trend line. In mathematical terms, the first derivative of the cost slope goes to a low number or zero. This makes sense because year-over-year cost reductions in the same percentage would illogically result in zero costs within a short period of time (Fig. 3).

OPEN IN VIEWER

FIG. 3.

As medical costs are computed for each year in a category under management, year-over-year amounts are often noted to diminish as the potential for savings is wrung from the population under management. It is often mistakenly assumed that no year-over-year savings implies “no effect” when the effect is in fact being applied against what was once a high utilization baseline. In the absence of a control group, this strength of a program is often debated.

Analysts working in the field of continuous quality improvement do understand that continuous improvement efforts to save costs or improve quality have limits and argue for some type of graded curve for demonstrating impact, such as requiring less impact as compliance with evidence-based guidelines exceeds 90% or 95%. Flattening or plateauing of effect clearly does not indicate lack of effectiveness if the above patterns appear, only exhaustion of the optimal extractable benefit from a given intervention paradigm. ² The goal of most medical management executives is to continue to capture the effect by introducing new techniques while preventing recidivism of cost issues.

Understanding the significance of a plateau on total impact is important. Although the annual savings thus decreases over time, estimation of accumulated cost savings over time can continue to grow compared to the baseline (Fig. 4).

OPEN IN VIEWER

FIG. 4.

Even in the presence of low year-over-year savings, the effect of a program may continue to deliver sustained costs. In the diagram above, we see what a curve might look like if a purchaser obtained a fixed price discount on a product that was being used at a constant level. Year-over-year savings would not occur after the first year, but accumulated savings would increase annually.

As noted, these curves generally are neither straight lines nor simple curves, and are notoriously unstable over multiyear contract periods, demonstrating both inherent variability of the costs themselves and the unknown effects of numerous health care cost drivers and emerging technologies. It is thus quite difficult, absent a real control group, to identify point estimates of savings rather than general impact as described as outlined above.

A second pattern seen with aggressive utilization management or medical policy interventions is a cost reduction step function. Simply by denying payment for a benefit such as obesity surgery, infertility benefits, or magnetic resonance imaging scans can produce a drop in overall costs without necessarily changing the slope in health care spending. Critics typically point to the absence of change in slope as “no change” in health care spending whereas more astute observers again note the real savings over time from shifting rather than bending the trend (Fig. 5).

OPEN IN VIEWER

FIG. 5.

Shifting the Trend. Programs that are implemented to control utilization, but not cost growth, may show curve shifts. For example, a program that adds advanced radiology precertification and rigorous provider credentialing may significantly reduce the total number of procedures or scans, but not affect the overall annual growth in cost at the procedure level. Clearly, savings accrue in the absence of a change in trend.

After medical management programs are initiated in general populations, it is not unusual to see an initial effect followed by stabilization of the care cost trend pattern. Medical managers have seen shifts in medical care costs, for example, after efforts in managed care were put in place.

The introduction of hospital utilization control practices in the 1990s—when commercial hospital bed days per 1000 decreased from the 700–900 range to the 150 range.

As one might expect, the combination of strategies and multiple interventions may leave cost curves of mixed types and create some blending of effects.

Medical Management Provider Strategy

In theory, the summed amounts of savings should permit comparison of competing programs. This is rarely possible due to a variety of unfortunate facts including the following:

lack of standard methods

Each component is typically evaluated on its own, each project manager measuring impact in his or her own way, often aimed at demonstrating effectiveness. Methods vary by medical management program, which of necessity must account for such factors as the particular nuances of program design and patients enrolled, among others.

There are some general truths, however, for evaluating the effectiveness of programs over the longer term. These would include the following:

The sum of all component savings for each of the medical management programs should be the same or less than the projected total population savings using actuarial methods.

Once additional impact plateaus on an annual basis and “sustained impact” is noted, additional impact is likely only with further paradigm shift changes. This might occur with the addition of more aggressive utilization management, and benefit design changes, among others. Paradigm shifts in medical management strategies need to be incorporated on a regular basis if ongoing cost savings are to be realized. Some examples of paradigm shift changes are outlined in Table 1.

There should not be an expectation that a given program should show significant cost reductions each year when compared to a prior year for any particular method into an indefinite future. If you think about it, this makes good sense; otherwise costs would eventually become zero (Fig. 6).

OPEN IN VIEWER

FIG. 6.

Effects of paradigm changes on cost curves. The above schematic represents the manner in which paradigm shift changes can impact medical management curves. An initial program may “bend the trend” to a lower slope which, left alone, might continue at the new rate or revert eventually to a similar slope at a lower level. A second initiative applied has a similar impact on the outcome of the first initiative. Whether the combination of 2 efforts are completely additive or simply synergistic remains unknown until the results are isolated.

Long-term analyses and the experience of medical management planners recognize that there are several key tactical decisions in maintaining medical management health care cost reduction programs.

Multidisciplinary program analyses should be added to basic accounting analyses and be expanded to include:

clinical quality improvement and adherence to evidence-based guidelines

We do know that turning off price- and demand-sensitive controls can result in a rebound of health care costs. Historically, medical managers point to circumstances in which the retirement of a program resulted in recidivism in care cost issues, such as removal of certificate of need programs, ^4,5 the elimination of utilization management controls for elective procedures in high variation areas, and during recent conversion from gatekeeper to preferred provider organization payments for physician-based medical services.

As an aside, it also makes sense that turning off programs for which no effect is ever anticipated or noted could be a cost savings from an administrative perspective (eg, in requiring precertification for treatment of open fractures, for which the requested services are always approved).

Given both the tendency of a medical management program impact to stabilize over time and the potential for “relapse” if programs are rotated, it is clear that the appropriate long-term approach to health care cost reduction is a combination of known effective programs focusing on areas of the population for whom they are most likely to have an effect, coupled with surveillance of those portions of the population who have either stabilized or are at low risk.

There is an industry movement at this time to focus high-cost intervention programs only on individuals likely to benefit, such as disease management programs that focus only on high-risk employees, and benefit penalties for at-risk individuals who are not enrolled in programs. Purchasers of these services are a bit anxious, however, because a highly focused approach only targets a small number of individuals, and the surveillance and management of the larger group may be accordingly weakened in the high-risk areas where they may have greatest impact.

It is also worth noting that there are some legal discrimination issues and risk from too highly focused an approach. It should be noted that these approaches are very difficult to evaluate from an impact perspective as both the lack of comparison groups and a significant selection bias often result in impressive outcomes that cannot be scientifically confirmed or disproved. For example, a program might note that senior citizens who participate regularly in gymnasium exercise appear to have lower health care costs. The stated implication may be that enrolling all or as many senior citizens as possible in a vendor's gymnasium-related programs may reduce health care costs overall when significant issues of selection bias may be solely driving the effect.

Options for Long-Term Management of Programs

What courses of action are open to a manager who oversees medical management programs? Consider doing nothing if you have a program in place that initially had an impact and your annual cost savings seem to be reaching a steady state. The logic here is that the savings captured from the baseline are continuing to add on year over year from the efforts of the program. Continue to measure year-over-year savings, but also note the change from the amount of projected baseline amounts. If the baseline is too far in the past (eg, 5 years), consider resetting the baseline to several years ago (eg, 3 years) as a reasonable compromise. This logic would argue that you still need state policemen along the interstate even though the numbers of speeders is less than before you increased patrols. A steady approach would also be reasonable if benchmarks for your population, properly adjusted, are better than average or a larger “book of business.” Alternatively, you could even back off on efforts and redeploy resources in the same area with a different focus, or in other areas of need.

Consider eliminating programs that seem to have optimized their impact or exhausted their effectiveness but keep the analytic unit in place to confirm that costs don't start to go back up again. Use the same analytic methods you use now to confirm this. Then, take the difference in management costs and invest in working on another area of need, such as creating an oncology program and eliminating the asthma program. The analogy here would divert some of the state police who pick up speeders to do driving-while-under-the-influence screening while the handful of officers left behind maintain the highways and watch for increased speeders in the absence of patrols.

If you believe there is a return for your efforts, consider leaving the existing program in place and adding components or new programs that will change the overall effectiveness (Table 1). Current model changes that are believed to be useful include the following:

Use high-acuity programs for new and chronic high-need patients in high-cost diseases and case management

Consider switching vendors on the assumption that someone else could do a better job. This seems less smart unless you are convinced that the program vendor is doing a poor job from an operational perspective. Note also that switching costs may be higher than you think, because of such factors as employees needing to be reeducated and additional setup costs being required. Look at the number of individuals enrolled, and absolute values of evidence-based guidelines using standard methods such as Healthcare Effectiveness Data and Information Set, among others. Why call in the National Guard if the state police are doing a decent if unimaginative job?

In any event, one common mistake that should be avoided is the perceived need to change multiple things simultaneously. Doing this eliminates the ability to ever evaluate what you have done.

Conclusion

Individuals performing long-term evaluation of medical management programs in general and disease management programs in particular need to concern themselves with temporal aspects of program optimization and analysis design. Recognition of common patterns of outcome effects and their meaning are important to prevent inappropriate conclusions by nonmedical financial analysts of these programs.