This action research study addresses the link between building evaluation capacity and enhancing evaluation use for learning. The author shares her experiences and reports on the evidence collected from a set of self-evaluation capacity-building interventions that she implemented. Using a mixed-method approach, the author first determined that the organization lacked the prerequisites to learn from evaluation. She then implemented self-evaluation capacity-building interventions, accompanied by a real-time study of how the participants changed attitudinally, cognitively, and behaviorally. Results indicate the potential of these interventions to facilitate an individual’s ability to learn from evaluation and underscore the need for supportive organizational contexts, structures, and processes if evaluation use for learning is to occur throughout the organization.
Airasian, P., Gullickson, A., Hahn, L., & Farland, D. (1995). Teacher self-evaluation: The literature in perspective. Kalamazoo, MI: The Evaluation Center.
2.
Alkin, M. C. (Ed.). (1990). Debates on evaluation. Newbury Park, CA: Sage.
3.
Alkin, M. C., & Taut, S. (2003). Unbundling evaluation use. Studies in Educational Evaluation, 29, 1-12.
4.
Buhren, C. G., Killus, D., & Müller, S. (1998). Wege und Methoden der Selbstevaluation. Ein praktischer Leitfaden für Schulen [Approaches and methods of self-evaluation. A practical guide for schools]. Dortmund, Germany: Institut für Schulentwicklung.
5.
Ciarlo, J. A. (1981). Utilizing evaluation: Concepts and measurement techniques. Beverly Hills, CA: Sage.
6.
Cousins, J. B. (2003). Utilization effects of participatory evaluation. In T. Kellaghan & D. L. Stufflebeam (Eds.), International handbook of educational evaluation (pp. 245-266). Boston: Kluwer.
7.
Cousins, J. B., & Earl, L. M. (1995). Participatory evaluation in education: Studies in evaluation use and organizational learning. Washington, DC: Falmer.
8.
Cousins, J. B., Goh, S., Clark, S., & Lee, L. (2004). Integrating evaluative enquiry into the organizational culture: A review and synthesis of the knowledge base. Canadian Journal of Program Evaluation, 19, 99-141.
9.
Cousins, J. B., Goh, S., & Lee, L. (2003). The learning capacity of schools as a function of evaluative inquiry: An empirical study. Paper presented at the Learning Conference, Institute of Education, University of London.
10.
Department for Education and Skills, United Kingdom. (2004). A new relationship with schools: Improving performance through school self-evaluation. Retrieved January 15, 2005, from http://www.ofsted.gov.uk/publications/index.cfm?fuseaction=pubs.displayfile&id=3862&type=pdf
11.
Deutsche Gesellschaft für Evaluation. (2004). Empfehlungen zur Anwendung der Evaluationsstandards der DeGEval im Handlungsfeld der Selbstevaluation [Recommendations on the application of the DeGEval evaluation standards in the context of self-evaluation]. Retrieved August 15, 2004, from http://www.degeval.de/cms_calimero/index.php?class=seite&id=9078
12.
Dillman, D. (2000). Mail and internet surveys: A tailored design method. New York: Wiley.
13.
Direktion für Entwicklungszusammenarbeit und Humanitäre Hilfe. (1995). Wegweiser zur Selbstevaluation [Guidelines for self-evaluation]. Retrieved August 15, 2004, from http://www.deza.admin.ch
14.
Durisch, P. (2003, March). What conditions are required for an optimal internalization of evaluation? Paper presented at the meeting of Partners in Development Evaluation: Learning and Accountability, French Ministry of Economy, Finance, and Industry and the DAC Working Party on Aid Evaluation, Paris, France.
15.
Engel, P. G., & Carlsson, C. (2002, September). Enhancing learning through evaluation: Approaches, dilemmas and some possible ways forward. Paper presented at the Conference of the European Evaluation Society, Sevilla, Spain.
16.
Forss, K., Cracknell, B., & Samset, K. (1994). Can evaluation help an organization to learn? Evaluation Review, 18, 574-591.
17.
Forss, K., Rebien, C., & Carlsson, J. (2002). Process use of evaluation: Types of use that precede lessons learned and feedback. Evaluation, 8, 29-45.
18.
Fukuda-Parr, S., Lopes, C., & Malik, K. (Eds.). (2002). Capacity for development. New solutions to old problems. New York: UNDP, Earthscan. Retrieved November 15, 2003, from http://capacity.undp.org/books/book1.htm
19.
Greene, J. (1988). Stakeholder participation and utilization in program evaluation. Evaluation Review, 12, 91-116.
20.
Heiner, M. (Ed.). (1998). Experimentierende Evaluation. Ansätze zur Entwicklung lernender Organisationen [Experimenting evaluation. Approaches for the development of learning organizations]. Weinheim, Germany: Juventa.
21.
Knox, A. B. (1992). Comparative perspectives on professionals’ ways of learning. New Directions for Adult and Continuing Education, 55, 97-108.
22.
Lindblom, C., & Cohen, D. (1979). Usable knowledge. New Haven, CT: Yale University Press.
23.
McTaggart, R. (Ed.). (1997). Participatory action research. Albany, NY: State University of New York Press.
24.
Morgan, P. (2002). Some observations and lessons in capacity building. In United Nations (Ed.), Capacity building supported by the U.N.: Some evaluations and some lessons (chap. 2). Retrieved November 15, 2003, from http://www.un.org/esa/coordination/Chpt2.PDF
25.
Morris Baskett, H. K., & Marsick, V. J. (1992). Confronting new understandings about professional learning and change. New Directions for Adult and Continuing Education, 55, 7-15.
26.
Moser, H. (2003). Selbstevaluation. Einführung für Schulen und andere soziale Institutionen [Self-evaluation. Introduction for schools and other social institutions] (2nd ed.). Zürich, Switzerland: Pestalozzianum.
27.
Nevo, D. (1995). School-based evaluation: A dialogue for school improvement. Oxford, UK: Pergamon.
28.
Organization for Economic Cooperation and Development/Development Assistance Committee. (2001). Evaluation feedback for effective learning and accountability. Paris: OECD.
29.
Organization for Economic Cooperation and Development/Development Assistance Committee. (2003). Glossary of key terms in evaluation and results based management. Paris: OECD.
30.
Osgood, C. E., Suci, G. J., & Tannenbaum, P. H. (1967). The measurement of meaning. Urbana: University of Illinois Press.
31.
Owen, J. (2003). Evaluation culture: A definition and analysis of its development within organizations. Evaluation Journal of Australasia, 3, 43-47.
32.
Patton, M. Q. (1997). Utilization-focused evaluation: The new century text (3rd ed.). Thousand Oaks, CA: Sage.
33.
Patton, M. Q. (1998). Discovering process use. Evaluation, 4, 225-233.
34.
Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage.
35.
Power, M. (1997). The audit society: Rituals of verification. Oxford: University Press.
36.
Preskill, H., & Caracelli, V. (1997). Current and developing conceptions of use: Evaluation use TIG survey results. Evaluation Practice, 18, 209-225.
37.
Preskill, H., & Torres, R. (1999). Evaluative inquiry for learning in organizations. Thousand Oaks, CA: Sage.
38.
Preskill, H., & Torres, R. (2001). The readiness for organizational learning and evaluation instrument (ROLE). In D. Russ-Eft & H. Preskill (Eds.), Evaluation in organizations: A systematic approach to enhancing learning, performance, and change. New York: Basic Books.
39.
Preskill, H., Zuckerman, B., & Matthews, B. (2003). An exploratory study of process use: Findings and implications for future research. American Journal of Evaluation, 24, 423-442.
40.
Robinson, T., & Cousins, B. (2004). Internal participatory evaluation as an organizational learning system: A longitudinal case study. Studies in Educational Evaluation, 30, 1-22.
41.
Schön, D. A. (1987). Educating the reflective practitioner. San Francisco: Jossey-Bass.
42.
Schwandt, T. A. (2005). The centrality of practice to evaluation. American Journal of Evaluation, 26, 95-105.
43.
Shulha, L. M., & Cousins, J. B. (1997). Evaluation use: Theory, research and practice since 1986. Evaluation Practice, 18, 195-208.
44.
Solomon, M., & Chowdhury, A. (2002). Knowledge to action: Evaluation for learning in a multi-organisational global partnership. Development in Practice, 12(3/4), 346-354.
45.
Stockdill, S. H., Baizerman, M., & Compton, D. W. (2002). Toward a definition of the ECB process: A conversation with the ECB literature. In D. W. Compton, M. Baizerman, & S. H. Stockdill (Eds.), The art, craft, and science of evaluation capacity building: New directions for evaluation (Vol. 93, pp. 7-25). San Francisco: Jossey-Bass.
46.
Swedish International Development Agency. (1999). Are evaluations useful? Cases from Swedish Development Cooperation. SIDA Studies in Evaluation, 99/01. Retrieved November 15, 2003, from http://www.sida.se/Sida/articles/11000-11099/11091/STUD99-1.pdf
47.
Taut, S. (2004). Experiences with self-evaluation in the United Nations Evaluation Group. Paris: UNESCO, Internal Oversight Service.
48.
Taut, S. (2005). Evaluation use for learning in an international development cooperation organization: An empirical study of process use and capacity building in self-evaluation. Unpublished doctoral dissertation, University of California, Los Angeles.
49.
Torres, R. (1994). Linking individual and organizational learning: The internalization and externalization of evaluation. Evaluation and Program Planning, 17, 327-337.
50.
Torres, R., & Preskill, H. (2001). Evaluation and organizational learning: Past, present, and future. American Journal of Evaluation, 22, 387-395.
51.
Turnbull, B. (1999). The mediating effect of participation efficacy on evaluation use. Evaluation and Program Planning, 22, 131-140.
52.
UK Evaluation Society (n.d.). Guidelines for self-evaluation. Retrieved July 15, 2006, from http://www.evaluation.org.uk/Pub_library/Good_Practice.htm%23self
53.
Weaver, L., & Cousins, J. B. (2004). Unpacking the participatory process. Journal of MultiDisciplinary Evaluation, 1, 19-40.
