Evaluating Program Effectiveness Using the Regression Point Displacement Design

Abstract

Most health care initiatives are evaluated using observational study designs in lieu of random–ized controlled trials (RCT) due primarily to resource limitations. However, although obser–vational studies are less expensive to implement and evaluate, they are also more problematic in determining causality than the RCT. This trade off is most apparent in the initial planning stage of program development. An RCT is generally preferred though the cost of implementing a pilot program using the RCT might outstrip the potential benefit if the desired results are not obtained. This article describes a simple quasi–experimental model called the regression point displacement (RPD) design, which compares the prepost results of a single or multiple treat–ment groups to that of a control population. This design has shown great potential in evalu–ating health care pilot programs or demonstra–tion projects–especially those that are community based–due to its relative ease of implementation and low cost of analysis

Keywords

quasi–experimental regression point displacement internal validity

Get full access to this article

View all access options for this article.

References

Campbell, D. T. , & Stanley, J. C. (1966). Experimental and quasi–experimental designs for research. Chicago: Rand McNally.

Cunningham, P. J. (2006). Medicaid/SCHIP cuts and hospital emergency department use. Health Affairs (Millwood),25, 237–247.

Jollis, J. G. , Ancukiewicz, M. , Delong, E. R. , Pryor, D. B. , Muhlbaier, L. H. , & Mark, D. B. (1993). Discordance of databases designed for claims payment versus clinical information systems: Implications for outcomes research. Annals of Internal Medicine,119, 844–850.

Linden, A. , Adams, J. , & Roberts, N. (2003). An assessment of the total population approach for evaluating disease management program effectiveness. Disease Management,6, 93–102.

Linden, A. , Adams, J. , & Roberts, N. (2004a). Using an empirical method for establishing clincal outcome targets in disease management programs. Disease Management,7, 93–101.

Linden, A. , Adams, J. , & Roberts, N. (2004b). The generalizability of disease management program results: Getting from here to there. Managed Care Interface,17, 38–45.

Linden, A. , & Roberts, N. (2005). A user's guide to the disease management literature: Recommendations for reporting and assessing program outcomes. American Journal of Managed Care,11, 81–90.

Linden, A. , Adams, J. , & Roberts, N. (2006). Evaluating disease management program effectiveness: An introduction to the regression–discontinuity design.Journal of Evaluation in Clinical Practice,12(2), 124–131.

Shadish, S. R. , Cook, T. D. , & Campbell, D. T. (2002). Experimental and quasi–experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.

10.

Trochim, W. (1984). Research design for program evaluation: The regression–discontinuity approach. Beverly Hill, CA: Sage.

11.

Trochim, W.M.K. (1990). The regression discontinuity design. In L. B. Sechrest, E. Perrin, & J. Bunker (Eds.),Research methodology: Strengthening causal interpretations of nonexperimental data AHCPR Conference Proceedings, PHS 90–3545(pp. 119–139). Rockville, MD: Agency for Health Care Policy and Research.

12.

Trochim, W.M.K., & Campbell, D. T. (1996). The regression point displacement design for evaluating community–based pilot programs and demonstration projects. Unpublished manuscript. Retrieved October 20, 2005, from http://www.socialresearchmethods.net/research/RPD/RPD.pdf

13.

Trochim, W. , & Campbell, D. (1999). Design for community–based demonstration projects. In D. T. Campbell & M. J. Russo (Eds.),Social experimentation (pp. 313–337). Thousand Oaks, CA: Sage.

14.

Trochim, W.M.K. (2001). The research methods knowledge base (2nd ed.) Cincinnati, OH: Atomic Dog.