Both the evidence-based policy and performance management movements aim to improve government effectiveness by developing and utilizing a more rigorous base of information and scientific evidence to guide decisions about program design, funding, implementation, and management. In practice, however, differences and tensions between these movements—such as their methods and standards for assembling and analyzing data, and the strategic timing and use of this information to influence policy and hold public managers accountable for performance—could limit their success. Using cases and empirical studies, this article considers questions about what should count as evidence, how it should be communicated, who should judge the quality and reliability of evidence and performance information, and how to achieve a balance between processes that produce rigorous information for decision making and those that foster democratic governance and accountability. Recommendations are made for improving government effectiveness by using more rigorous information in decision making, along with acknowledgment of the limitations and risks associated with such efforts.
Bakalar, N. (2006, July 27). One-two punch for 12 steps? Wisconsin State Journal, p. A7.
2.
Barnow, B., & Gubits, D.B. (2002). Review of recent pilot, demonstration, research and evaluation initiatives to assist in the implementation of programs under the workforce investment Act. Chapter 5 of the strategic plan for pilots, demonstrations, research, and evaluations, 2002-2007. Washington, DC: U.S. Department of Labor.
3.
Bevan, G., & Hood, C. (2006). What's measured is what matters: Targets and gaming in the English public health care system. Public Administration , 84, 517-538.
4.
Bloom, D., & Michalopoulos, C. (2001). How welfare and work policies affect employment and income: A synthesis of research. New York: Manpower Demonstration Research.
5.
Burghardt, J., Schochet, P.Z., McConnell, S., Johnson, T., Gritz, R.M., Glazerman, S., et al. (2001). Does Job Corps work? (Mathematica Policy Research Rep. No. 8140-530). Princeton, NJ: Mathematica Policy Research.
6.
Callahan, R.F., & Gilbert, G.R. (2005). End-user satisfaction and design features of public agencies. American Review of Public Administration, 35, 57-73.
7.
Churchill, G.A., & Surprenant, C. (1982). An investigation into the determinants of customer satisfaction. Journal of Marketing Research, 19, 491-504.
8.
Coalition for Evidence-Based Policy. (2002 , November). Bringing evidence-driven progress to education. Report of the Coalition for Evidence-Based Policy. Washington, DC: The Council for Excellence in Government.
9.
Cook, P.J., & Ludwig, J. (2006). Aiming for evidence-based gun policy. Journal of Policy Analysis and Management, 25, 691-735.
10.
Council for Excellence in Government. (n.d.). The coalition for evidence-based policy. Retrieved April 20, 2007, from http://coexgov.securesites.net/index.php?keyword=a432fbc34d71c7
11.
Courty, P., Heinrich, C.J., & Marschke, G.R. (2005). Setting the standard in performance measurement systems. International Public Management Journal, 8, 321-347.
12.
Courty, P., & Marschke, G. (2004). An empirical investigation of gaming responses to performance incentives. Journal of Labor Economics, 22, 23-56.
13.
Dillon, S. (2006, September 28). As 2 bushes try to fix schools, tools differ. The New York Times, p. 1.
14.
Dopson, S., Locock, L., Chambers, D., & Gabbay, J. (2001). Implementation of evidence-based medicine: Evaluation of the PACE programme. Journal of Health Services Research and Policy, 6, 23-31.
15.
Dyke, A., Heinrich, C.J., Mueser, P., & Troske, K. (2006). The effects of welfare-to-work program activities on labor market outcomes. Journal of Labor Economics, 24, 567-607.
16.
Ferri, M., Amato, L., & Davoli, M. (2006). Alcoholics Anonymous and other 12-step programmes for alcohol dependence. The Cochrane Library, 3, 1-22.
17.
Furstenberg, F. (2003). Teenage childbearing as a public issue and private concern. Annual Review of Sociology, 29, 23—39.
18.
Furstenberg, F. (2006, September 14). Destinies of the disadvantaged: Teenage childbearing and public policy. Institute for Research on Poverty Seminar, University of Wisconsin—Madison.
19.
Gilmour, J.B., & Lewis, D.E. (2006). Does performance budgeting work? An examination of the Office of Management and Budget's PART Scores. Public Administration Review, 66, 742-752.
20.
Gormley, W. (2003, September). Using organizational report cards. Presented at the National Public Management Research Conference , Georgetown University, Washington, DC.
21.
Greenberg, D.H., Cebulla, A., & Bouchet, S. (2005). Report on a meta-analysis of welfare-to-work program outcomes (Discussion Paper No. 1312-05). Institute for Research on Poverty, Madison, Wisconsin.
22.
Greenberg, D.H., Michalopoulos, C., & Robins, P.K. (2006). Do experimental and nonexperimental evaluations give different answers about the effectiveness of government-funded training programs? Journal of Policy Analysis and Management, 25, 523-552.
23.
Heckman, J.J., Heinrich, C.J., & Smith, J.A. (2002). The performance of performance standards. Journal of Human Resources, 37, 778-811.
24.
Heckman, J.J., & Smith, J.A. (1995). Assessing the case for social experiments. Journal of Economic Perspectives, 9, 85-110.
25.
Heinrich, C.J. (2004). Improving public-sector performance management: One step forward, two steps back? Public Finance and Management , 4, 317-351.
26.
Heinrich, C.J. (2007). False or fitting recognition? The use of high performance bonuses in motivating organizational achievements. Journal of Policy Analysis and Management, 26, 281-304.
27.
Heinrich, C.J., & Choi, Y. (in press). Performance-based contracting in social welfare programs . American Review of Public Administration.
28.
Institute for Research on Poverty. (n.d.). The Institute for Research on Poverty: A history, 1966-2005. Retrieved April 20, 2007, from http://www.irp.wisc.edu/aboutirp/history.htm#intros
29.
Jacob, B.A. (2005). Accountability, incentives, and behavior: Evidence from school reform in Chicago. Journal of Public Economics , 89, 761-796.
30.
Jacob, B.A., & Levitt, S.D. (2003). Rotten apples: An investigation of the prevalence and predictors of teacher cheating. Quarterly Journal of Economics , CXVIII, 843-877.
31.
Kelly, J.M. (2005). The dilemma of the unsatisfied customer in a market model of public administration. Public Administration Review , 65, 76-84.
32.
Koretz, D. (2002). Limitations in the use of achievement tests as measures of educators' productivity. Journal of Human Resources , 37, 752-777.
33.
Madison, J. (1865). Letter to W. T. Barry, August 4, 1822. In P. R. Fendall (Ed.), Letters and other writings of James Madison, published by order of Congress (Vol. 3, p. 276). Philadelphia: Lippincott.
34.
Marston, G., & Watts, R. (2003). Tampering with the evidence: A critical appraisal of evidence-based policy making. Drawing Board: An Australian Review of Public Affairs, 3, 143-163.
35.
Maynard, R.A. (2006). Evidence-based decision making: What will it take for the decision makers to care? Journal of Policy Analysis and Management, 25, 249-265.
36.
Pawson, R. (2001). Evidence-based policy: In search of a method. ESRC UK Centre for Evidence-based Policy and Practice. Working Paper 3, University of London.
37.
Pear, R. (2006, February 21). A.M.A. to develop measure of quality of medical care. The New York Times. Retrieved April 20, 2007, from http://www.nytimes.com/2006/02/21/politics/21docs.html?ex=1298178000&en=40a08fff80197428&ei=5088&partner=rssnyt&emc=rss
38.
Radin, B. (1998). The Government Performance and Results Act: Hydra-headed monster or effective policy tool? Public Administration Review , 58, 307-316.
39.
Radin, B. (2000). The Government Performance and Results Act and the tradition of federal management reform: Square pegs in round holes? Journal of Public Administration Research and Theory, 10, 11-35.
40.
Radin, B. (2006). Challenging the performance movement. Washington, DC: Georgetown University Press.
41.
Rosenstock, L., & Lee, L.J. (2002). Attacks on science: The risks to evidence-based policy. American Journal of Public Health, 92, 14-18.
42.
Rycroft-Malone, J., Seers, K., Titchen, A., Harvey, G., Kitson, A., & McCormack, B. (2004). What counts as evidence in evidence-based practice ? Journal of Advanced Nursing, 47, 81-90.
43.
Sanderson, I. (2002). Evaluation, policy learning, and evidence-based policy making. Public Administration, 80, 1-22.
44.
Sanderson, I. (2003). Is it “what works” that matters? Evaluation and evidence-based policy making. Research Papers in Education , 18, 331-345.
45.
Sanger, M.B. (2003). The welfare marketplace: Privatization and welfare reform. Washington, DC: Brookings Institution.
46.
Schochet, P.Z. (2006). Does Job Corps work? Impact and benefit-cost findings from the National Job Corps Study. Working paper, Mathematica Policy Research. Princeton, NJ: Mathematica Policy Research.
47.
Shulock, N. (1999). The paradox of policy analysis: If it is not used, why do we produce so much of it? Journal of Policy Analysis and Management, 18, 226-244.
48.
Smith, J.A., Whalley, A., & Wilcox, N. (2006). Are program participants good evaluators? Working paper, University of Michigan.
49.
Thorn, C.A., & Meyer, R.H. (2006). Longitudinal data systems to support data-informed decision making: A tri-state partnership between Michigan, Minnesota, and Wisconsin. Report of the Wisconsin Center for Education Research.
50.
Trutko, J., Barnow, B., Farrell, M., & Glosser, A. (2005, March). Earnings replacement outcomes for dislocated workers: Extent of variation and factors accounting for variation in earnings replacement outcomes across state and local workforce investment boards. Final Report to the U.S. Department of Labor Employment and Training Administration.
51.
U.K. Cabinet Office. (1999). Modernising government. London: Stationery Office. Retrieved September 10, 2006, from http://www.archive.official-documents.co.uk/document/cm43/4310/4310.htm
52.
U.S. Department of Education. (2002). What works clearinghouse. Retrieved April 11, 2007, from http://www.whatworks.ed.gov/
53.
U.S. Government Accountability Office. (2005 ). Performance budgeting: PART focuses attention on program performance, but more can be done to engage Congress (GAO Report #06-28). Washington, DC: Author.
54.
Van Ryzin, G. (2004). Expectations, performance, and citizen satisfaction with urban services. Journal of Policy Analysis and Management , 23, 433-448.
55.
Vaughn, R.G. (2000). Transparency—The mechanisms: Open government and accountability. Issues of Democracy, U.S. Department of State Volume, 5(2), 13-20.
56.
Walshe, K., & Rundall, T.G. (2001). Evidence-based management: From theory to practice in health care. Milbank Quarterly, 79, 429-457.
57.
Wiseman, M. (2004). The high performance bonus. Unpublished manuscript, The George Washington University.
58.
Workshop on Statistical Methods for Performance Analysis, Cassino, Italy. (2006). Workshop on statistical methods for performance analysis final meeting of the PRIN-PIE: National Research Project on Multivariate Statistical and Visualization Methods to analize, to summarize, to evaluate performance indicators. Retrieved August 19, 2006, from http://www.prinpie.unisa.it/smap/
59.
Young, K., Ashby, D., Boaz, A., & Grayson, L. (2002). Social science and the evidence-based policy movement . Social Policy and Society, 1, 215-224.
