Developing Performance Metrics for Science and Technology Programs: The Case of the Manufacturing Extension Partnership Program

Abstract

This article examines the development, implications, and limitations of a series of performance measures to gauge the success of individual Manufacturing Extension Partnership (MEP) centers. The measures represent a shift in how MEP judges and evaluates center performance and a challenge in terms of how they are used, how they are interpreted, and how they are limited. The MEPis an important example of innovative public programs and consists of a public-private, performance-based partnership that seeks to improve the productivity, competitiveness, and technological capabilities of America’s manufacturers, particularly small firms. The article makes two important contributions: (a) a comprehensive performance-management approach can be developed with a focus on program outcomes that are linked to long-term impacts and are not just stand-alone process measures or stand-alone outcome measures and (b) it is possible to develop valid and reliable measures for technology-focused economic development programs that can be used to report on and manage performance.

Get full access to this article

View all access options for this article.

References

Affholter, D. (1994). Outcome monitoring. In J. S. Wholey, H. P. Hatry,& K. Newcomer (Eds.), Handbook of practical program evaluation(pp. 96-118). San Francisco: Jossey-Bass.

Behn, R. D. (1994). Here comes performance assessment—and it might even be good for you. In A. Teich, S. Nelson, & C. McEnaney (Eds.), Science and technology policy yearbook—1994(pp. 257-264). Washington, DC: American Association for the Advancement of Science.

Chapman, R. E. (1998). Using Data Envelopment Analysis to assess performance of Manufacturing Extension Centers. Gaithersburg, MD: National Institute of Standards and Technology, Office of Applied Economics.

Feller, I. (1997). Federal and state government roles in science and technology. Economic Development Quarterly, 11(4), 283-295.

Feller, I. (2002). Performance measurement redux. American Journal of Evaluation, 23(4), 435-452.

Georghiou, L.,& Roessner, D. (2000). Evaluating technology programs: Tools and methods. Research Policy, 29(4-5), 657-678.

Hatry, H. P. (1999). Performance measurement: Getting results. Washington, DC: Urban Institute Press.

Hatry, H. P., Fall, M., Singer, T. O., & Liner, E. B. (1990). Monitoring the outcomes of economic development programs. Washington, DC: Urban Institute Press.

Heinrich, C. J. (1999). Do government bureaucrats make effective use of performance management information? Journal of Public Administration and Theory, 9(3), 363-393.

10.

Jarmin, R. S. (1999). Evaluating the impact of manufacturing extension on productivity growth. Journal of Policy Analysis and Management, 18, 99-119.

11.

Jarmin, R. S. (2001). The impact of the MEP on plant productivity(Draft paper). Washington, DC: U.S. Census Bureau, Center for Economic Studies.

12.

Kettl, D. F. (1997). The global revolution in public management: Driving themes, missing links. Journal of Policy Analysis and Management, 16(3), 446-462.

13.

National Institute of Standards and Technology, Manufacturing Extension Partnership. (1998). MEP criteria for performance excellence. Gaithersburg, MD: Author.

14.

National Institute of Standards and Technology, Manufacturing Extension Partnership. (1999). Report to Congress—The NIST Manufacturing Extension Partnership: A network for success and a review of the results and evaluation process. Gaithersburg, MD: Author.

15.

National Institute of Standards and Technology, Manufacturing Extension Partnership. (2002). The Manufacturing Extension Partnership: Delivering measurable returns to its clients. Gaithersburg, MD: Author.

16.

National Research Council. (1993). Learning to change: Opportunities to improve the performance of smallermanufacturers. Washington, DC: National Academy Press.

17.

Newcomer, K. (1997). Using performance measurement to improve programs. In K. Newcomer (Ed.), Using performance measurement to improve public and nonprofit programs(pp. 5-14). San Francisco: Jossey-Bass.

18.

Office of Technology Policy, U.S. Department of Commerce. (2000). Federal role in support of technology-based economic development: Report of a roundtable. Washington, DC: Author.

19.

The Omnibus Trade and Competitiveness Act of 1998, 100-418, 5121 (15 U.S.C. 278k).

20.

Osborne, D.,& Gaebler, T. (1992). Reinventing government: How the entrepreneurial spirit is transforming the public sector. Reading, MA: Addison-Wesley.

21.

Porter, M. (1990). The competitive advantage of nations. New York: Free Press.

22.

Scheirer, M. A. (2000). Getting more “bang” for your performance measures buck. American Journal of Evaluation, 21(2), 139-149.

23.

U.S. Census Bureau. (2000). 1998 annual survey of manufacturers: Statistics for industry groups and industries. Suitland, MD: Author.

24.

Voytek, K. P. (2001, October). Measuring the effectiveness of economic development programs: The Manufacturing Extension Partnership’s effect on the productivity of small manufacturers. Paper presented at George Washington University’s Institute of Public Policy, Washington, DC.

25.

Wholey, J. (1977). Evaluability assessment. In L. Rutman (Ed.), Evaluation research methods: A basic guide(pp. 41-56). Beverly Hills, CA: Sage.