Abstract
In the United States, Master of Public Administration Program (MPA) accreditations come through fulfilling the Network of Schools of Public Policy, Affairs, and administration (NASPAA) accreditation standards. In 2009, NASPAA made some significant revisions to its accreditation standards. One of the major revisions is the requirement for programs to focus on outcome assessments. MPA programs who have gone through the revised 2009 (NASPAA ) accreditation standards and those who are currently preparing for their upcoming accreditations or reaccreditations will most probably agree that while developing student learning outcomes (SLOs) is a difficult task, assessing those (the learning outcomes) is a much more challenging endeavor. This article describes Northern Kentucky University’s MPA program’s journey in using the capstone course for assessing the program’s SLOs, recent changes in the requirements of the portfolio to better align with the program’s SLOs, use of Benjamin Bloom’s taxonomy of cognitive learning domain as the framework to document evidence of SLOs from the portfolios, some of the initial data on assessment that came out of the last two years of capstone offerings, and the ongoing challenges of utilizing the capstone course for such assessment purpose.
Introduction
The NASPAA (Network of Schools of Public Policy, Affairs, and Administration) is ‘the global standard in public service education. It is the membership organization of graduate education programs in public policy, public affairs, public administration, and public and nonprofit management. Its 285 members – located across the U.S. and in 14 countries aroundthe globe – award MPA, Master of Public Policy, Master of Public Affairs, and similar degrees. NASPAA is the recognized accreditor of master's degree programs in these fields’ (http://www.naspaa.org/about_naspaa/naspaa.asp). NASPAA accreditation recognizes that a Master’s program in public affairs has undertaken a rigorous process of peer review conducted by the Commission on Peer Review and Accreditation (COPRA).
Over the years, NASPAA has made revisions to accreditation standards and expectations. The most recent of these revisions occurred in 2009. In contrast to the previous focus on mission and goals accomplishment, the revised standards require Master’s of Public Administration (MPA) programs to develop student learning outcomes (SLOs) consistent with the programs’ missions and to systematically collect assessment data or information on the SLOs. Programs that went through these revised accreditation standards and those currently preparing for their upcoming accreditations or re-accreditations will most probably agree that while developing such SLOs is a difficult task, assessing those learning outcomes is a much more challenging endeavor.
Some of the common processes that MPA programs utilize to gather such assessment data are student and alumni surveys, pre-tests and post-tests, course exams and assignments, and graduating students’ exit interviews. Other specific sources of assessment data include alumni placements, internship experiences, MPA portfolios, and the capstone projects. While on the surface, they all may appear to be easy to implement, the reality is that developing criteria or an appropriate framework to sort and collect information or data from these sources – that specifically relate to SLOs – is difficult and challenging. This is particularly true for the MPA portfolios and the capstone projects, since the former usually entails self-reflection on the part of the student, while the latter usually is a completion of an applied project.
The MPA program’s curriculum at Northern Kentucky University (NKU) includes a capstone course that students have to successfully complete during their last nine credit hours of coursework. The course requires students to develop a portfolio and complete an applied project. The program currently uses both these assignments to systematically gather assessment information and data on the program’s SLOs. The program also uses a variety of other assessment tools such as student and alumni surveys, focus group interviews, and employer surveys.
This article describes the program’s journey in using the capstone course for assessing SLOs; recent changes in the requirements of the portfolio to better align with these SLOs; use of Benjamin Bloom’s taxonomy of cognitive learning domain as the framework to document evidence of SLOs from the portfolios; some of the initial data and information on assessment that came out of the last two years of capstone offerings; and the ongoing challenges of utilizing the capstone course for such assessment purpose.
Capstone course across MPA programs
Most MPA program curricula offer a culminating experience for their students, typically during their final semester of coursework. The goals of the experience are usually to: (1) provide an opportunity to students to apply their learning to a project (such as a research or an applied project), and (2) gain feedback on the program’s quality and impact. This experience generally takes the form of completing a practicum paper, writing and defending a thesis, taking a comprehensive exam and/or completing the capstone course. For example, MPA programs at University of Georgia and Eastern Kentucky University require students to take and pass a comprehensive exam, while programs at Florida Gulf Coast and University of Kentucky require completion of a capstone project. Currently, few MPA programs have the thesis requirement.
The capstone course in MPA programs, generally speaking, requires students to complete projects (usually applied), thereby providing opportunities to students to apply the knowledge, skills and competencies that they gained through their MPA experiences. As Roberts and Pavlak (2008) explain, the key value of the capstone course is the bridging of theory–practice gap. Some of these projects are real/live projects involving problem-solving situations or applications of analytical skills. These capstone projects serve several other purposes, such as the following: the applied projects help students develop research and management skills (Whitaker, 2004); projects that have a service learning component lead to an increase in civic engagement (Bernstein, et al., 2003). Such projects also benefit the agencies served and help develop networking between the program and the agencies.
Capstone course’s requirements and processes vary across MPA programs. For example, the capstone projects at the University of North Carolina Chapel Hill are reviewed and approved by a committee of three members; in contrast students at Indiana State University complete their capstone projects under the guidance of the instructor teaching the capstone course. In several MPA programs (e.g. University of South Carolina, University of Missouri) students work in teams to complete their capstone projects. The capstone project at Northern Michigan University’s MPA program is part of a course (PS 592) that also includes a written and an oral exam, and a portfolio submission The capstone course at New York University’s MPA program covers an academic year (i.e. two full semesters). The program also solicits client-based projects through an open Requests for Proposals process. Faculty and students can also suggest potential capstone clients. Prospective clients receive e-mail messages and postcards announcing that the school is accepting proposals, and are referred to a website that outlines proposal guidelines.
The capstone course at Northern Kentucky University
The MPA program at Northern Kentucky University (NKU) started in 1987. During that early stage, the graduation options for the program were a thesis, an internship, or completion of a second area of concentration. Students opting for an internship or an additional area of concentration also took comprehensive written and oral examinations administered each spring on a pass/fail basis. Within a short few years, the program found that several students opted for, but did not complete, the thesis. An initial analysis also revealed that only 5 students from the first 100 graduates sought to further their education (three Juris Doctors, 2 PhDs) and only 1 of them wrote a thesis. Despite advice by MPA faculty, marginal students were using the thesis option to avoid the comprehensive exams. The MPA faculty and students also thought that the comprehensive examination process needed reform. Area employers and alumni also pushed for adding employment development to the graduation process. So, in 1998 faculty decided to eliminate the thesis option and replaced the existing graduation options to one graduation model: internship (3 hours), portfolio (3 hours) and comprehensive exams (1 credit hour). For the comprehensive exams, students were initially examined on every course in the program through both written and oral exams. Later on, the faculty decided to use the written exam to cover only the core classes and oral exams for borderline cases (i.e. for students who barely passed the written exam). In the year 2000, the faculty revisited the use of the comprehensive exam. An assessment of the comprehensive examinations experience indicated that it was becoming a duplication of grading efforts in core classes. So, in 2001, the program made another significant change in its graduation requirements. The comprehensive exam and the internship requirements were eliminated; instead a new course called the MPA capstone course was created. The program since then has been using the capstone course to provide students a culminating learning experience.
The capstone course has two major assignments: (1) submission of a portfolio, which includes students’ self-reflective essays on their learning experiences with the MPA program, and (2) completion of an applied project, which students can choose to work on in teams or individually.
The capstone projects include a wide range of methodologies including surveys, literature review, in-depth interviews, and focus groups. Examples of recent capstone projects include: conducting citizen satisfaction surveys, developing fundraising plans and strategies for nonprofits, developing a volunteer development handbook, conducting cost–benefit analyses of city projects, writing and submitting grants, conducting program evaluations, and developing risk management programs for cities and other localities. Most of these projects were requests from area government and nonprofit agencies; students also have the option to develop their own projects to demonstrate their learning. Since 2006, the MPA program has been requesting a voluntary donation from agencies for the completion of their requested projects. The donation amount has varied from USD$500 to USD$1,500 and goes directly to support student learning activities through the MPA Foundation account.
The course is a pass/fail course; however, passing the course requires at least a B grade, which is clearly stated in the course syllabus. Each of the assignments is graded, and students are required to receive at least a B grade to pass this course.
Capstone course and program assessment
Erwin (1991) defined assessment as ‘“the process of defining, selecting, designing, collecting analyzing, interpreting, and using information to increase students’ learning and development’ (p. 15). He also mentions the following four reasons for doing assessment: political (i.e. what is the value of higher education), economic (i.e. what are the costs and benefits of higher education), educational (i.e. curricular reform), and societal (i.e. accountability).
Even though MPA programs are now required to systematically collect assessment data due to the revised accreditation standards, one can validly assume that programs have been doing that anyway through their use of a variety of surveys (e.g. student, alumni), exit interviews, faculty feedback on student performances and so on. Such assessment data is, undoubtedly, very useful for program development. One of the NASPAA’s reasons for requiring MPA programs to engage in assessment activities was the very same reason why these programs were already engaged in the process: assessment improves student learning and guides program development.
However, the accreditation requirements for outcome assessment did provide an impetus and an opportunity for MPA programs to revisit their current assessment processes and strategies in regards to their effectiveness or usefulness, and to explore new options. As Fitzpatrick and Miller-Stevens explained, ‘“establishing outcomes, discussing them, and measuring them provide program administrators and faculty with information – information not only to debate and discuss, but also to help guide program changes and progress’ (2009: 18).
Assessment methods could be direct or indirect. Direct assessment methods are measures of student learning that require students to display their actual knowledge and skills (rather than report what they think their knowledge and skills are). Because direct assessment methods tap into students’ actual learning (rather than perceptions of learning), they are often seen as the preferred type of assessment. The MPA capstone project is a direct measure of student learning. Indirect assessment methods, on the other hand, involve students providing feedback on their learning. Indirect methods of assessment include surveys, focus group interviews, and in-depth interviews. The MPA portfolio, specifically the self-reflective essay, is an example of an indirect method of assessment.
The following sections explain the program’s experience and efforts in utilizing the capstone portfolios and projects to systematically collect information on student learning outcomes and some of the challenges it faced and is still facing in doing so.
Capstone portfolio: background and changes implemented in fall 2012 class
The MPA portfolio assignment requires students to write a self-reflective essay (10–12 pages) on their learning experiences with the MPA program. Students can choose any seven courses that they have completed to write the essays. Students submit their essays in a binder; the binder also includes copies of the graded exams and assignments of these seven courses and the students’ professional resumes.
These reflective essays have provided valuable feedback and program outcome assessment information and data. However, faculty also realized that since students’ reflective papers were narratives, it was somewhat difficult to extract assessment information from these narratives – information that specifically related to the program’s learning outcomes. For example, a student in his or her narrative could write at length about the learning experience in the public policy course, but the content of the discussion may not relate to any of the program’s SLOs. During the 2012 program’s reaccreditation process, the site visit team praised the program’s use of the capstone course, but also encouraged the faculty to find ways to more effectively utilize the portfolios for systematically collecting and documenting program assessment data and feedback.
With that goal in mind, the instructor teaching the capstone course in fall 2012 (the first time offered after the site visit) made some revisions to the portfolio assignment. The revisions required students to write the self-reflective narrative in terms of the program’s learning outcomes. The following is the syllabus statement on this assignment: Student will write a reflective essay (10–12 typed pages and double spaced) that will include the following components: (a) an introduction summarizing the student’s reasons and/or goals for joining the MPA program, and (b) a discussion of student’s learning experience with the MPA program in regard to the program’s core learning outcomes and the learning outcomes of his or her chosen area of concentration. The discussion will be based on and/or supported by papers, projects, reports, exams and other assignments that the student completed in at least seven of his or her MPA courses. These can be a mix of MPA core courses and area of concentration courses.
So, the new requirement (tying the student’s learning experience with the program’s learning outcomes) not only gave students a structure for the narrative, but also opened up a process to systematically gain relevant outcome assessment feedback.
The instructor also developed and used the rubric in Table 1 to grade the portfolio narratives.Students found this rubric very useful as they compose their portfolio essays.
Portfolio narratives: grading rubric.
Following are excerpts from several students’ portfolio essays as illustrations of their reflective learning experiences in relation to the program’s SLOs:
My work in PAD 611 (Budgeting Techniques & Analysis) enabled me to step inside the mindset of a government agency faced with deteriorating funding options and an increasing list of desired items to determine how to navigate these challenges. One of the most important things I learned from Organizational Theory & Behavior was the importance of effective leadership in developing a professional climate in the workplace. I believe this course and the Executive Leadership course helped me develop a good set of effective leadership skills and competencies. . My first class project, a class presentation on Organizations as Organisms, called “Organizisms!” was to me, a fascinating look at how Gateway Community & Technical College is structured as a “matrix species” internally, with many departments and divisions starting to work collaboratively toward a common goals of higher retention rates. The project also illustrated how Gateway adapted to and interacted with its environment, including our students, community, and competitors in the field of post-secondary education. I am thrilled to have the opportunity to examine deeply the structure and culture of my own organization and understand the management and leadership aspects of it.
My research paper assignment for PAD 682 (Terrorism) allowed me to tackle the issue of counter-terrorism by evaluating how effective international treaties have been at curbing terrorist attacks and determining which types of treaties had a greater impact. My participation in drafting a cost–benefit analysis of early childhood education in Census Track 505 during PAD 603 not only taught me about using cost–benefit analysis to evaluate policy options and suggest alternatives, but also allowed me to communicate to community stakeholders the impact and outcomes that implementation of early childhood education could have upon children in the community.
The PAD 616 Data Analysis research paper allowed me to study a large sample of a diverse population’s opinions on allowing same-sex relationships to be a part of sex education in public schools.… Not only did this analysis revealed diversity among acceptance of same-sex relationships, but research leading up to my analysis revealed other characteristics of diversity in the population that I was surveying, which included, among others, how often an individual participates in religious activities, education levels, political party affiliation, and sexual orientation. One of the biggest contributions that the program has given me is that I have continued to improve on my writing. I can recall my homeland security course, where a professor threatened to fail every submitted assignment for not following the proper citations and paper formatting guidelines or even my research methods course, where each paper submission was antagonizing because I knew in a short amount of time, that paper would be fraught with changes. Yet, despite these short-lived frustrations, I recognize and respect the professors who taught me to be mindful of my own writing so that in the future, I can prepare the best possible version of my research. With the lessons learned here and on the merits of my writing, I have worked with a city government to develop the written content for their website and composed grants for a nonprofit.
Administrative Ethics was another class that allowed me to reflect upon the work that I had done in the past, coordinating after-school programs for at-risk youth in rural Western Massachusetts as well as the work I currently do with at-risk adults in an urban community college. I knew that the easy answer to why these students were at risk were location and economy. Through my ethics coursework I examined some other circumstances, namely, the administrative culture in public schools, and how structures of schools and districts can affect the level of achievement in their students.… I delved into the ethics of decisions made by Michelle Rhee, who at the time was Washington D.C.’s Chancellor of Public Schools. She has recently come into the public eye for firing 241 teachers and administrators in the district, including the principal at the school her children attended. A national argument ensued about education reform and specifically merit pay for educators. For me, this topic brought up the question of whether using fear and power is the best way to make teachers more effective in the classrooms. When I write cover letters, I think about the organization’s mission and how my education has prepared me for this particular position, whether it is volunteer management skills I have acquired or the importance of the oft quoted maxim, ‘Nonprofits are about relationships’.
With my group project, I learned to cope with dissents among the group, and finding consensus. I learned that it is important to understand where each member’s strongest skills lie and how to maximize on those individual skills to create a good final product. This program really emphasizes collaboration. I have had at least one group project every semester. At first, I bristled at the thought, but in my last semester, I find myself in two voluntary group projects. I have learned to work better with people, when to choose my battles and when to be assertive, and the strengths of group work.
As Fitzpatrick and Miller-Stevens (2009) explain, a rubric contains two important components: (1) it specifies the criteria that will be used to assess performance, and (2) it describes the performance levels for each criterion.
The rubric in Table 1 thus helped the instructor to grade the narratives and provided a framework for students to use in writing their essays. As one can tell, the rubric focused on students’ writing abilities in terms of organizing the discussion around the learning outcomes. It also provided good information on student learning outcomes. However, it did not capture by itself the levels or stages of students’ learning outcomes. For example, is the student’s learning experience more at the knowledge level or at the application level? This assessment is crucial since a student learning outcome refers to a particular level of knowledge, skills, and abilities that a student has attained at the end (or as a result) of his or her engagement in a particular set of collegiate experiences (Ewell, 2001). In addition to assessing the student’s writing ability in completing the assignment, as mentioned before, these portfolios were also used to gain insights into the program’s learning outcomes. So, the instructor needed another framework to sort and identify student learning outcomes at various levels. The instructor’s intention was to develop that framework in a way that its criteria would also align with varying levels of a student’s development of the cognitive domain.
The cognitive domain was used since it involved knowledge and the development of intellectual skills relevant to higher education such as the MPA experience. ‘Cognitive developmental objectives are descriptions of higher-order thinking skills, including critical-thinking skills’ (Erwin, 1991: 39). The instructor decided to use Benjamin S. Bloom’s (1956) taxonomy of cognitive learning domain as the framework to structure and document the program’s SLOs. The taxonomy was developed by a group of academic and educational experts, under the leadership of Bloom, and hence the name (Bloom’s taxonomy).
Bloom’s work has been extensively cited and used by curriculum planners, administrators, researchers and classroom teachers at all levels of education. It is considered as a classical theory or model whose application is timeless. Bloom et al. (1956) identified three different learning objectives: cognitive (mental skills), affective (growth in feelings and emotional areas) and psychomotor (manual or physical skills). Each of these domains was categorized into different levels of higher order. An important premise of Bloom’s taxonomy is that each category (or level) must be mastered before progressing to the next.
Table 2 outlines the different categories (or levels) of Bloom’s cognitive domain and explains them using the MPA research methods course as a case in point.
Benjamin Bloom’s cognitive domain: illustration with the research methods course.
As a side note, some scholars have made some revisions to this basic model. One example of such work is the one completed by Anderson et al. (2001), who changed the rank order of synthesis and evaluation by putting evaluation before synthesis. The debate on which one should precede the other still continues.
Since a degree program level’s student learning outcomes represent the knowledge and skills that we hope graduates to possess, it is likely that at least some of a degree program’s outcomes will reflect what is called “higher order thinking skills”, rather than just basic learning. The Application, Analysis, Synthesis, and Evaluation of Bloom’s taxonomy are usually considered to reflect higher-order thinking skills.
The above classification thus provided the instructor a framework to assess and document student learning from simple learning to complex or higher levels of learning. At the end of the semester, the instructor reviewed each of the student’s portfolio narratives and used this framework to assess and document his or her learnings on the program’s five core learning outcomes and the student’s chosen area of concentration’s learning outcomes. (see Appendix A). The instructor, basically, identified excerpts from students’ essays that related to one or more of these cognitive skills. So, for each student the instructor completed five forms (one for each of the program’s learning outcomes) and an additional number of forms depending on the student’s chosen area of concentration.
For example, one of the students (STUDENT A) while describing, in the portfolio narrative, his or her learning experience with the human resource management class, stated the following: ‘I learned that FEMA is an open system which must adapt to environmental condition while meeting internal needs. I also learned how politics can influence administrative decisions and the process’. As per Bloom’s framework, this statement indicates the student’s learning at the Comprehension level. The same student, while reflecting back on his or her experience with the organizational theory & behavior class, noted the following: We were required to select an organization that we currently or had previously worked for and analyze its organizational culture. In doing so, I was able to reflect on one particular nonprofit organization that I had worked for and truly see it through this new knowledge learned. This was enlightening to me because prior to taking this course, I had not seen nor actually realized some of the greatest strengths of the organization, and also its greatest weaknesses. In hindsight, I feel that these were areas that I should have recognized.
Using Bloom’s categories of cognitive domain, this statement indicates the student learning at the Analysis level.
Appendix B includes an example of two forms that the instructor completed that documented this student’s learning outcomes on the MPA program’s learning outcomes 1 and 3.
Capstone projects
As mentioned before, NKU’s capstone course requires students to complete a major applied project. Most of these projects are requests for assistance from a variety of area agencies (nonprofits and government). Students work under the guidance of the course instructor to complete their projects. Over the years, the scope and quality of these capstone projects have provided valuable information to the program faculty on students’ abilities to demonstrate their knowledge, skills, and abilities. In addition, the quality of the completed projects gave good indications about areas that needed to be strengthened throughout the curriculum or in a particular course.
The program requires students to meet with the instructor the semester prior to enrolling in the capstone course. The reason for this is to ensure that students have approved projects before they enroll in the class, and thereby have sufficient time to complete their projects within a semester. Students are required to submit their draft capstone project papers three weeks before the final date of submission. If the draft papers do not demonstrate mastery of the subject matter and/or the required analytical abilities, then students are asked to revise and resubmit the papers. Throughout the semester, the instructor meets one to one with each student and guides them in completing his or her project, so in most cases the final revisions required are minor.
The capstone project papers are evaluated on content and writing quality (40–50 points; 50 points if the agency feedback does not apply), presentation (10 points) and feedback/evaluation from the agency representative (as applicable, 10 points). The instructor uses a rubric to review and grade these project papers (Table 3).
Capstone project paper: grading rubric.
The quality of the completed projects (e.g. grants applications, employee manuals) provides valuable feedback on the program’s curriculum and its coverage. For example, during one semester several students needed intensive guidance on the use of Statistical Package for Social Sciences (SPSS) to complete their team capstone project. The instructor teaching the capstone course realized after talking to the students that the instructor teaching the data analysis course was not covering SPSS comprehensively. That feedback was used to change the coverage of SPSS in later offerings of the data analysis course.
As mentioned before, agency representatives with whom the student or students maintain contact while completing the capstone projects are requested to complete a simple evaluation form (Appendix C).
Following are excerpts from comments that three agency representatives noted on the evaluation forms for three capstone projects from the fall 2012 capstone class.
*Comment on writing: While overall, I think the writing was good, I’m “old school” when it comes to editing and proofing. I noticed errors that were a result of lack of proofreading and Word spell check. The work represented by this Capstone is very helpful and exceeds all expectations on my part. The paper is an excellent and valuable collection of information that can be used by any nonprofit that wants to capitalize on collaboration and technology. When I met with (STUDENT) and (STUDENT), I sensed some frustration about feeling they had just begun to scratch the surface on their research into technology and collaboration. Thus, their research could lead to additional related projects and research. The paper is, in many respects, the culmination of numerous meetings between (STUDENT) and me (as well as others) as we discussed shared interests and visions about women and nonprofits in our region. I hope this paper is a springboard for other Capstones following the research presented. The findings and recommendations reflect what Xtraordinary Women (XWomen) wanted to know about local collaboratives and technology and how the findings could be used to reinvent XWomen as an organization that strengthens and works on behalf of local nonprofits serving women, i.e. the backbone support organization.
Both students did an excellent job. It was clear that they worked very well in a team environment. They listened to our needs and geared the project to meet those needs and more. We asked for specific information and they gave us that as well as much needed and very practical additional information. The presentation was very clear, both orally and written, and it was obvious that they were very knowledgeable when answering the questions thrown at them by 15 plus staff. These students communicated well and attended every scheduled meeting. They were flexible when needed and considerate of others’ time. They took on additional workload in inputting and re-coding the survey as necessary. They offered suggestions and recommendations to increase effectiveness and reduce our work load. I can’t say enough about their professionalism and commitment in working on this project. They provided extremely useful, accurate and complete documents that will be used for program development, program evaluation and grant writing.
As one can expect, agency representatives’ feedback and comments provided additional and valuable information on the program’s outcomes and impacts.
Initially, I think these three students had some issues or faced some difficulties in getting connected to our contact person, but after a week or so things moved very smoothly. I am very pleased with the quality of the final report. The students presented at our April City Council meeting and answered all the questions very competently and confidently. I want to assure you and your students that our Council and Mayor are committed to publish the major findings of this survey in the City’s website, and that we will meet soon to thoroughly review the findings and explore ways to better serve our residents. Considering our budget crisis, we really appreciate your students completing this significant project with no financial support from us. I assume the students also received a first-hand experience in dealing with city officials.
Exit interview – another assessment tool
The program, since 2010, has also been using an exit survey that all capstone students are required to complete. Rather than making the survey an all-inclusive and time consuming one for students to complete, the program decided to design the survey to focus on one of the learning outcomes with the goal to eventually have all the outcomes covered in seven years. So, the last two years, the exit survey was designed to focus on assessing students’ learning on the program’s SLO 3 – the ability to analyze, synthesize, think critically, solve problems and make decisions. The program faculty chose that outcome since, at that time, the program was collecting outcome assessment data and completing the entire cycle of assessment on that outcome for the self-study report.
Table 4 reports two years’ data (fall 2010, spring 2011, fall 2011, spring 2012; N = 52) from the exit surveys, focusing on SLO 3.
Exit interview findings.
The survey also includes an open-ended section for students’ suggestions for enhancing the program and their learning experience. Following is a listing of those open-ended comments: More focus on counseling – academic and career. Potential class to add: labor relations, public finance, land use, planning and zoning. Integrate electives from NKU’s business school. I felt the program did a good job at teaching us the use of the mechanics of writing as indicated by the strongly agree X above. More critical thinking, as found in classes like policy analysis and data analysis could not hurt. I feel the MPA program has helped me to improve my skills with public speaking, writing, researching, data gathering and analyzing. Despite Dr. — placing an emphasis on the writing of a memo in PAD —, I did not find memo writing discussed or encouraged in many other classes. I would encourage the program to place more of an emphasis on how to make a good presentation along with writing memos. Opportunities for presenting work in front of fellow students were very beneficial for my current position. Every class did a fairly good job in terms of public speaking opportunities. However, memos and other forms of professional correspondence were only implemented in two classes. I feel the benefit of the MPA program is the practicality of the coursework. I wish more emphasis would be placed on that and not as much traditional coursework. This program has been The depth and difficulty of the program in total has been
Some reflections on recent experience
The new structure of the portfolio undoubtedly gave insightful information on the program’s SLOs. The structure also assisted students in reflecting back on their learning experiences in regard to the program’s learning outcomes. Faculty found these reflective essays more informative in understanding students’ perspectives on the program’s strengths and weaknesses. However, for the instructor, reviewing and identifying relevant statements or information appropriate for the different levels of cognitive skills was somewhat time consuming. As Janesick (2001) pointed out, “Constructing portfolios and assessing them takes time, effort, and dedication to the task” (p. 12). The capstone class enrollments vary around 15 to 20 students, so using Bloom’s taxonomy to review and document each of these students learning outcomes would entail an extensive time commitment on the part of the instructor. The other issue that the instructor faced was arriving at an overall assessment of students’ learning outcomes that could inform future planning and revisions of the program. The instructor discussed some of the findings at one of the MPA meetings, and the faculty decided that it would be useful and beneficial to continue using this process and structure. However, considering the workload that it might impose on the instructor teaching the capstone course, it was agreed that the MPA director (who usually teaches the capstone course) would complete the assessment forms during the summer when things were somewhat less busy in the MPA office. The MPA director receives a summer stipend for managing the program during that time.
It was also decided that, next year, at one of the fall MPA meetings (probably at the MPA retreat), faculty would review all the completed assessment forms and explore ways to effectively and efficiently utilize the information to arrive at a meaningful assessment of students’ learning. While the results will not have the kind of statistical validity or reliability that would make the findings statistically significant, they will certainly be useful in the way this process intends – to give faculty members meaningful information on student learning outcomes and thereby guide future curriculum revisions.
As for the capstone projects, the requirement to meet with the instructor prior to the semester taking capstone was not included in the curriculum until fall 2013, as a pre-requisite for registering in the capstone class. Until that semester, the instructor would e-mail students, sometimes multiple times, asking them to meet with her before enrolling in the class and emphasize the importance of having the project approved before the semester. Since it was not a mandate or a formal curriculum requirement, a few students chose not to meet with the instructor or would meet a few days before classes begin; they therefore struggled to complete the project within a semester. To avoid this situation, faculty during one of their 2013 spring program meetings approved a curriculum change that now requires students to have an approved capstone project before they can register in the capstone course.
Conclusion
Several decades ago, Bob Cunningham (1977) emphasized the importance of experiential learning for effective MPA education. As he explained, ‘students learn better when they are actively involved in discovering for themselves the information or theory or significant questions’ (pp. 219–220). Capstone courses certainly do provide students an experiential learning experience. NKU’s MPA capstone course offers students an opportunity not only to work and complete an applied project to demonstrate their learning experiences with the program, but also opens up for them a variety of external networking opportunities through their interactions with agency staff. Students are, however, more prepared for this capstone experience if service learning and applied projects are embedded throughout the MPA curriculum. As Schon (1983) points out, ‘reflective practice cannot come from a magic, final experience, but must be built on a much larger collection of learning experiences’ (p. 93). At NKU, fortunately, several of the MPA courses (e.g. nonprofit management, human resource management, and volunteer management) include service learning and civic engagement projects. Students also find the experience very valuable, as exemplified in the following comment noted by a student on his evaluation of the capstone course: What was no nice about the capstone class and the capstone project was that everything I have learned in this program came together as a whole or completed experience. I cannot think of any other better way to complete my experience with this program.
There is no doubt that a well-designed capstone course could provide great opportunities for students to apply their MPA learning experiences into major projects and to get connected with agencies or prospective employers. Our experience with NKU’s MPA program demonstrated that the course could also be effectively utilized to gather systematic outcome assessment data. While the path has not been an easy one, our experience did show that the course offers promising options and opportunities for such a use.
Footnotes
Conflict of interest
None declared.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.