Doing Assessment Well: Advances for Undergraduate Psychology Programs and Psychology Educators

Curricular and Cocurricular Maps for Psychology

A strong starting point for assessment involves curricular mapping. Curricular mapping is an assessment approach aimed at collecting curriculum-related information that highlights core skills and content. A completed map serves as a guide for faculty members, department chairs, and other administrators who want to record what psychology knowledge and skills have been taught to students and to plan what material will be taught in the future (or at each stage) in the psychology major (i.e., in introductory, intermediate, and advanced courses).

A good curricular map should illustrate and drive assessments that occur at multiple cross-sectional points (e.g., Maki, 2010; Stanny, 2015). Both summative and formative assessments should be represented in the map (e.g., Broadbent et al., 2018), just as there is a working assumption that assessment measures may be adopted or adapted from other groups or resources. Summative assessments are familiar to any student or, for that matter, any faculty member, as they are exemplified by relatively high-stakes performance measures such as routine tests on course material as well as final exams. Term papers, exam scores, and scores on standard rubrics fit the summative pattern as well. In effect, summative assessments are designed to provide summary evidence of what a student has retained where material in a given course unit or course is concerned. Summative assessments are by turns used to praise performance or to penalize it, but, in any case, these assessments rarely include the opportunity to revise or upgrade the quality of finished work.

In contrast, formative assessments are more subtle and gradual, providing incremental measures of student learning. They are mostly low- or medium-stakes measures, so their goal is not to necessarily rank or provide a final word regarding a student’s performance but to illustrate what a student has learned (or is still learning) about a given skill arena (say, learning APA-style writing for drafting empirical reports of or literature reviews in psychology) and where there remains room for future growth (e.g., a student may know how to cite journal articles using APA style but not chapters from edited volumes). Formative assessments help to shape future learning, as they alert both the student and the instructor about what is being learned and to what degree—and also what knowledge or skills are yet to be realized. These assessments are particularly useful in helping psychology teachers recognize where students are struggling; in an experimental psychology course, for example, lab reports might reveal that between-subjects designs are understood well but that within-subjects designs are not. Such evidence helps the instructor target areas of lecture or course readings that prove to be difficult for students, perhaps by crafting new ways to present challenging information or creating new exercises for learning and then displaying desired skills.

Table 2 shows a sample curriculum map for psychology-related undergraduate programs that are pursuing academic program assessment using the Guidelines for the Undergraduate Psychology Major 2.0 (APA, 2013), which is a set of expectations for learning and performance by undergraduate students who are majoring in psychology. This sample illustrates what a curriculum map might look like for the core areas found in most psychology curricula by each of the five learning goals found in the Guidelines 2.0 (see the top portion of Table 2). Any department or program could indicate the courses that fall into each of the five goals. Note that the numbers provided in Table 2 are sample numbers—a given program should complete the grid or design its own based on the contents of particular courses and its assessment needs.

OPEN IN VIEWER

Table 2.

Recommended sample curriculum map identifying core content areas and Guidelines 2.0.

Note. Goal and objectives are introduced (I) to all students (formative—as a program goal and for student development). Goals and objectives are developed and reinforced (R; formative—as a program goal and for student development). Goals and objectives are advanced and students gain mastery and experience (E; tends to be more summative—as a program goal and for student development). Δ = general education related.

* Ethics and diversity issues are infused throughout the guidelines.

** Core psychology content areas (Halpern, 2010).

Psychology educators also need to recognize that students learn both in and outside the college classroom. Although most student learning outcomes are likely to be documented in formal coursework (whether in face-to-face, hybrid, or online venues), many outcomes can also be assessed in cocurricular and extracurricular programs (e.g., student organizations, clubs, campus activities). ¹ Thus, for instance, students might experience many outcomes related to functioning well with diverse groups of people through achieving a leadership role in Psi Chi, the international 4-year undergraduate honor society in psychology.

Table 3 illustrates a cocurricular map with typical contributions from a variety of contexts related to some of the five goals in the Guidelines 2.0 (APA, 2013). A given psychology program should create a matrix illustrating whatever cocurricular activities are available that can be assessed to support its educational contributions to the psychology major.

OPEN IN VIEWER

Table 3.

Recommended sample curriculum map identifying cocurricular areas and the Guidelines 2.0.

Note. All faculty members (full time and adjuncts) should be aware of these goals and routinely assess their students’ level of awareness. Goals should be explicitly communicated to students along with the identified student learning outcomes. Goal and objectives are introduced (I) to all students (formative—as a program goal and for student development). Goals and objectives are developed and reinforced (R; formative—as a program goal and for student development). Goals and objectives are advanced and students gain mastery and experience (E; tends to be more summative—as a program goal and for student development).

We now turn to some essential arenas where promoting high-quality assessment activities in psychology can take place.

Curricular Structure Considerations

Undergraduate psychology programs should craft curricula that provide students with both a sense of the educational breadth and depth of the discipline (e.g., Halpern, 2010). The likely minimum number of courses for a workable psychology major is nine courses, although many programs can and do require additional psychology courses for graduation (for more detail, see Dunn et al., 2010). To this end, students should take courses at the introductory, intermediate, and advanced levels, all of which should be infused with scientific methods, diversity issues, and consideration of ethical decision making (APA, 2013).

The introductory level primarily targets introductory or general psychology, the overview course for both prospective majors, those seeking general education credit, and, more recently, premed students. Intermediate courses include at least one course in research methods and one in statistics/data analysis as well as exposure to one course offering in each of psychology’s core areas: biopsychology, cognitive/learning, social/personality, developmental, and clinical (e.g., Dunn et al., 2010). Advanced courses include some seminar or capstone experience (to be discussed in detail shortly), one designed to cover a specific topic while also allowing students to call upon the knowledge and skills they acquired earlier in the major, integrating them together in the service of some course-related project. Ideally, students should take at least one applied course (e.g., internship, industrial/organizational psychology, health psychology) and an elective course in any area of their own choosing (but selected in collaboration with their academic advisor; e.g., Dunn et al., 2010).

Course Sequencing, Levels, and Assessment

Even in a major with what appear to be minimal requirements, some thought must be given to course sequencing and course levels as well as their link to assessment issues. Sequencing refers to the fact that some courses should be completed (as prerequisites) before other courses can be taken. For example, before taking a statistics course, it will benefit students to be aware of the myriad areas that now comprise the field by first completing introductory psychology. Once the first class and a statistics class are completed, a student should learn about research methods before tackling the courses in the field’s four core areas. In other words, the knowledge and skills acquired in these three foundational courses build upon one another and should make learning about the diverse approaches to study the core areas more understandable.

Course sequencing also entails recognizing the importance of course levels; that is, courses offered earlier in the psychology major should provide more basic information and skills, while those in the middle and end of the curriculum should be more challenging, containing advanced material that assumes student understanding of earlier subject matter, as well as the acquisition of particular skills (e.g., research design skills in methods and interpretative skills in statistics/data analysis). Effective programs, then, require that students enroll in courses in a sequence where higher course levels represent increasing rigor. Less effective programs allow students to enroll in courses out of sequence, which means that instructors cannot assume particular knowledge as they teach (e.g., a course instructor in cognitive psychology cannot assume all enrolled students have completed research methods, so the instructor will need to teach in depth about the nature of between-subjects and within-subjects research designs rather than simply recapping the basic distinction between them). Indeed, a particular problem in programs that lack sequencing and levels is that many students postpone taking research methods and statistics as long as possible, primarily because these courses are seen as difficult and technical. Waiting until one’s senior year to complete these or other “difficult” courses is problematic for student learning and course design and delivery, as it undermines the ability to run a quality curriculum and potentially increases the risk that graduating on time will be delayed.

Naturally, the presence of course sequencing and levels can make assessment efforts much easier for a department or academic program, as faculty members discussed and planned how courses build upon one another, so that the transmission of content and skills occurs and becomes integrated within the whole curriculum. More basic assessments of general knowledge, for example, would likely occur in the introductory courses, whereas measuring critical thinking and scientific reasoning would occur later when students have acquired sufficient skills for asking and answering advanced questions about human thought and behavior.

The absence of sequencing and levels means that psychology students with varying levels of psychological literacy, or students’ ability to use psychological science to meet personal needs and societal goals (Cranney & Dunn, 2011; McGovern et al., 2010), are likely to be enrolled in any given course. Assessing what students know—and when they know it—under these circumstances is very difficult because students who have had one or two psychology courses can nonetheless enroll in a core course or even a seminar where their performance may suffer because they lack sufficient prior learning experiences that promote particular skill acquisition, application, and retention. And at the same time, their presence may slow the learning pace of students who have already completed several psychology courses and are on track where their psychological literacy is concerned.

Unfortunately, these structural problems are not uncommon at institutions that do not have a planned curriculum where a student can complete the major within 2.5–3 years (i.e., five or six semesters) or those where courses are offered in an unpredictable manner rather than on a rotating—and public—schedule (note that both situations pose hurdles for any student but perhaps especially transfer students). An advisor might suggest that a student complete the capstone course being offered the next semester in lieu of finishing intermediate offerings simply because the former is available. It is important to remember that assessment efforts are not merely to measure what students have learned but to also help determine when students should ideally learn particular material or apply the skills they have already acquired in meaningful ways.

We now turn to a discussion of courses and course types within the psychology major.

Introductory Psychology

The introductory or general psychology course is important for two main reasons. First, it is the only course that many students take in the field, which means that emphasis on psychology as a science is paramount. Moreover, estimates suggest that between 1.2 and 1.6 million undergraduate students per year enroll in introductory psychology (Gurung et al., 2016; Steuer & Ham, 2008). Providing students with an accurate portrayal of the current state of the discipline is an important way to dispel common myths (e.g., “all psychologists are clinicians”). Second, it is the first course in the psychology major; hence, it sets the stage—as well as students’ expectations—for the courses that follow. As the discipline has expanded over the last few decades, the amount of material in most introductory psychology texts has grown as well. New material rarely supplants older or classic content, which means teaching the first course effectively is an increasing challenge for many educators. There are a variety of ways to teach this fundamental course (e.g., Dunn & Chew, 2006), though ideally, instructors should promote scientific foundations, review the five domains or pillars of knowledge (biological, cognitive, developmental, social/personality, and mental and physical health), as well as cross-cutting themes relevant to all domains (sociocultural diversity, ethics, variations in human functioning, and applications; see APA, 2014; Gurung et al., 2016).

For these reasons, assessment is an important concern for teaching the introductory psychology course. Teachers and psychology administrators want assurances that students are learning material that will help them in intermediate psychology courses and the major as a whole. In effect, are knowledge and skills learned early on being applied later? Those disciplines and majors that depend on introductory psychology are also important stakeholders. Students in neuroscience and in premed programs, for example, need to learn content from the introductory course for their course work and the Medical College Admission Test entrance exam (see Hong, 2012), respectively. Particular assessment targets on quizzes or exams can include key points reviewed in lecture and discussion, and targeted short answer or essay questions can be designed to evaluate the acquisition and understanding of key concepts. Instructors also can embed desired learning outcome questions in relevant exams or a course’s final exam.

Midrange and Advanced Courses

What about midrange and advanced courses? By midrange courses, we refer generally to topical courses covering the core areas of psychology, such as stand-alone courses on personality, infancy and childhood, cognitive psychology, and so on. Depending on the curriculum’s structure, research methods and statistics courses (which we already advocated should be taken by psychology majors shortly after the introductory course) can also appear here. Advanced courses, which should not be confused with capstone and seminar courses (we discuss those in detail below), could be a special topics course that builds on a core course (e.g., after completing social psychology, a student might want to take a course focused on prejudice and discrimination or one exploring helping and prosocial behavior). An advanced course might also be a required or an elective offering designed to provide students with more breadth and depth, such as an advanced methods or research design course or even a second course in statistics and data analysis.

Assessment opportunities in midrange and advanced course can go well beyond content knowledge. Indeed, the ideal situation is when assessments focus on skill development tied to a particular course. Consider three examples. In a life-span development course, students could focus on communication by creating a professional-looking poster providing advice on coping with some of the cognitive and physical changes tied to aging. As author and expert, the students would be assessed on the quality (aesthetics, legibility) of the digital poster, including any images and graphs, APA-style citations, and the clarity (no jargon) and scientific accuracy of the guidance it provides to the target audience (older adults and their children). Meanwhile, students in an industrial/organizational psychology course could distill course material into a list of recommendations in an op-ed piece on how to maintain a balance between work life and home life aimed at recent and not-so-recent college graduates. Finally, in an advanced course in research design, students could be asked to create descriptions of experiments and quasi-experiments that actually illustrate design flaws covered in class, readings, and discussions. Having students create intentional design shortcomings, such as failing to counterbalance stimuli in a multifactorial experiment, and then explaining the problem is an alternative and thoughtful approach to searching for flaws in an instructor-created research design (for other ideas, see APA Project Assessment).

Capstone Courses or Seminars

In contrast to the introductory course and midrange and advanced courses, capstone courses are still more involved affairs where assessment is concerned. Capstone or seminar courses are meant to be an ending but highly integrative experience for students (e.g., Hauhart & Grahe, 2015). In psychology, a capstone course is an opportunity for students to demonstrate that they have achieved the learning goals associated with the major and its content and that they have mastered the skills linked to the discipline (APA, 2013; see also, Dunn & McCarthy, 2010). Capstone courses usually tie together themes learned in earlier courses in the psychology curriculum, allowing students to complete some focused project—perhaps a complex one undertaken collaboratively by a team of students, such as an elaborate controlled experiment or an applied quasi-experiment tackling some real-world problem on campus or in the local community.

Any assessment of capstone experiences should be designed to measure abilities tied to advanced psychology students, such as the scientific inquiry and critical thinking skills, ethical decision making, communication, and professional development skills found in the APA Guidelines 2.0 (APA, 2013). The benefit of capstone course assessments is that psychology faculty members can verify which skills transfer from the classroom setting for use in research or applied venues as well as those that may need more pedagogical emphasis in the future. Additionally, of course, students can tell prospective employers or graduate programs or other stakeholders (e.g., parents) about the knowledge and skills they used in creating an actual project.

Capstone courses also represent an opportunity for students to display disciplinary competence acquired in the psychology major and linking into some assessable product outcomes (e.g., paper, project) or performance (e.g., classroom talk, conference presentation; e.g., Palomba & Banta, 1999). A product or a performance outcome can be useful when searching for a postbaccalaureate job or applying to graduate school in psychology or another field (e.g., Dunn & Halonen, 2020). As a result of careful examination of capstone outcomes and other assessable capstone features (see below), psychology departments can determine whether their graduates are competent or whether changes to the program curriculum are warranted. In a survey of 374 psychology programs at colleges and universities, Stoloff and colleagues (2010) found that 40% of programs required students to complete a capstone course experience (some programs offer only one capstone course, while other provide students with options, including selecting one course from a list [e.g., history and systems in psychology, topical seminar], doing a thesis project, or undertaking some other sustained effort).

Many activities used in psychology capstone courses are readily assessable; typically, students will be given a scoring or template rubric by their instructors in order to help them craft successful projects (see Rhodes, 2010; Stevens & Levi, 2005, for guidelines on rubric development). Example projects include the following.

Internships

Like capstone experiences, internships and related opportunities (e.g., externships, job shadowing, apprenticeships, volunteer venues) provide psychology majors with the opportunity to use their discipline-based knowledge and skills in real-world settings. Depending upon their nature and structure, internships often are sanctioned by psychology departments so that students can receive academic credit, remuneration, or a combination of both. Doing an internship enables many students to learn whether a particular career appeals to them (or not), thereby shaping their postgraduate plans. Internships also afford psychology departments with an opportunity to assess whether and to what degree students can transfer psychology-related skills from the classroom to the workplace, clinic, or other community or corporate setting. Ideas for what can and should be assessed can be found in the Guidelines for the Undergraduate Psychology Major: Version 2.0 (APA, 2013) as well as the Assessment Cyberguide (Pusateri et al., 2009). Perhaps the most straightforward approach is to have internship students write a reflective paper toward the end of their experience that indicates what particular psychology-based knowledge and skills they used, along with behaviorally based, descriptive examples to illustrate their points.

Beyond that, it is vital for students and their faculty advisors to receive feedback on the students’ performance at the internship site from their supervisor(s). Such expert opinion—really, constructive performance appraisal—should be given directly to the student by the supervisor in a face-to-face manner as well as in writing for faculty advisors. This appraisal feedback can be incorporated into the students’ internship grade along with written and oral reflections done by the student as part of the internship experience. Psychology programs should also routinely survey internship sites and supervisors regarding their satisfaction with students’ abilities in order to affirm or address how well the current major curriculum is inculcating the desired skills and the behavior and demeanor of students in the workplace. Some final assessment tool should not be ignored: Are any students invited back to sites for second internship experiences? Are any students recruited and hired as full-time employees after graduation?

Research Experiences and Honors Projects

Other hands-on experiences that nicely demonstrate a variety of student learning outcomes include research experiences and honors projects. Research experiences come in different forms. At the low end are research participation opportunities (either as course requirements or extra-credit options) embedded in the introductory psychology course. At many colleges and universities, students are required to take part in three or four psychology experiments where the goal is twofold—to help faculty and graduate students collect data for their research projects and to give undergraduates a “lived” sense of what psychological research is like through active participation. At the higher end of research experiences, students may join a faculty member’s lab group or research team. This option provides an in-depth learning experience coupled with readily assessable responsibilities for students, such as designing and running studies, recruiting participants, analyzing data, writing up results for presentation or publication, creating posters explaining the research in formal and informal settings, and possibly giving an oral presentation at a conference or an on-campus research day (Improving Undergraduate Research Taskforce, 2019). These activities represent high-impact practices that help students understand the discipline at a deeper level than is possible in the classroom (Kuh & Schneider, 2008). Any or all of the aforementioned learning activities are assessment worthy and can be used as evidence of meaningful learning and skills acquisition (i.e., content knowledge of research area, performance of research protocols, writing skills, speaking skills).

Similar goals can be met when well-prepared, advanced students are invited to conduct a solo research project—usually referred to as an honors or thesis project—during their senior year of college. The honors project is usually by invitation based on a student’s grade point average, and it typically lasts for two semesters. Under the supervision of a faculty member, the honors student conceives, designs, and executes an original research project—usually but not necessarily an empirical investigation. The student collects data, analyses it, and writes it up in the form of an honors thesis, which is then defended prior to the student’s graduation. This intensive experience, like the capstone course, allows a student to pull together the psychological knowledge and skills to complete a solid piece of research. This largely independent effort is effective preparation for graduate school or the workplace. Assessment opportunities abound at each step in the research process (i.e., formative assessments) or at its conclusion (i.e., projects receive a pass or fail summative evaluation). Assessable products include the content and quality of a written thesis and that of its accompanying oral defense before a panel of faculty members. To some extent, such solo undergraduate research experiences mimic the activities of graduate students, which means that inviting external experts to review these student efforts (and perhaps participate in any oral examinations) can validate the quality of the teaching and learning within the department or program (Stanny, 2015).

Graduate Placement: Tracking Where Students Go

One of the most straightforward and helpful assessments for both faculty members and students is to ascertain what paths students follow once they graduate from a psychology program. This information can be used to revisit a program’s curriculum or particular courses. If most students enter the workforce following graduation, then emphasizing the workforce readiness skills that students learn in the major may well be a good idea. Of course, there will always be a group of students who elect to pursue a graduate degree in psychology or another field, so particular tools that will help students matriculate into graduate study should also be assessed. At Moravian College, for example, an internal study found that most of the graduates entered the workforce for a time, but a few years following graduation, above 50% of the graduates gravitate toward acquiring some postgraduate degree in or outside of psychology (L. J. Toedter, personal communication, June 1, 2012).

Most graduate placement data are archival in nature. Simple ways to obtain this information include polling students about their plans during their final semester (i.e., aforementioned exit interview) or contacting the institution’s alumni office, which usually tracks where graduates are employed (e.g., types of positions taken by students, starting salaries). Admission rates to graduate schools or other professional programs (e.g., law schools, business schools) can be gathered in a similar manner. Such information (at least in aggregate form) can be shared on the program’s website to inform would-be students and their parents about postgraduate options and choices of former psychology majors. Of course, the actual graduation rate at the institution and that for the department also represents useful information that should be reported on the website as well (i.e., what percentage of matriculated students graduates within 4 years, 6 years, etc.).

Highlighting the career journey and biographies of recent graduates is also a way to draw in potential applicants who want salient, compelling evidence of the program’s student success. Some departments also post a map of the United States and with pushpins indicate where graduates have ended up. This feature is a simple but powerful illustration of postgraduate plans.

Routine APRs

Effective departments engage in assessment through periodic APRs, where an outside peer or peers read a department’s self-study document (e.g., Barak & Breier, 1990; Dunn et al., 2013); visit campus and meet with department constituents, students, and administrators; and then write an APR aimed at touting the department’s strengths while pointing to areas for future growth or remediation (see, e.g., McCarthy et al., 2015; Pusateri et al., 2004). APRs, which should be conducted every 5–7 years, help deans and provosts understand departments’ strengths and weaknesses as well as providing them with a perspective on how a program’s current constitution is viewed by informed outsiders who are aware of larger educational trends and pedagogy practices tied to the discipline. Some assessable elements that are worthy of inclusion in an internal review document include the following: how institutional mission informs department mission; course enrollment data; curricular quality, including course development, structure, and sequencing as well as the highlighting of particular student learning outcomes; number of majors, minors, and yearly graduation rates as well as student attrition; departmental and institutional retention figures; postgraduate paths of students (i.e., pursuing graduate degrees or employment); student performance on the graduate record examinations (GRE) and Psychology GRE; faculty scholarly publications and presentations (highlighting student coauthors and presenters); submitted as well as funded grants, teaching awards, national profiles, and aggregated teaching evaluations of full-time and adjunct faculty members (within department and cross-institution departmental comparisons); and a review of program resources, including departmental budget, physical plant/facilities, and staffing/administrative support.

Involve All Members of the Department or Program in the Assessment Effort

Collegial buy-in is an important feature of quality assessment efforts. Everyone needs to be convinced that academic assessment is a good idea. In ideal circumstances, all department members should have a hand in the effort of data collection or analysis as well as implementing any changes.

Close the Loop

Lots of academic departments collect assessment data for the purpose of filling out reports, either for the institution or an accrediting body. Such record keeping is necessary, but it does not really represent appropriate assessment until faculty use the data to make improvements to a course or a teaching practice. Change—however incremental or dramatic—is at the heart of assessment.

Select a Few Assessments to Conduct Each Year in the Department or Program

Avoid assessment overkill by implementing a handful of useful and important assessments each year (recall Table 1). Doing so will ensure the work needed is reasonable, as will be the analyses, which means making adjustments or changes is more likely to happen. Too much assessment is exhausting so that the participants view the activity as “busy work” and lose interest in doing a thorough job. Less really is more.

Treat High-Quality Assessment Work as not only a Service to the Department but Possibly a Form of Scholarship

In many departments and programs, assessment is a duty or a responsibility, one for which service credit is often given. For many faculty who become interested in quality assessment processes, it can also become a form of scholarship if they publish their assessment efforts in professional pedagogy journals or present them at teaching or administrative conferences. Indeed, assessment scholarship can often reignite the moribund research efforts of mid- to late-career faculty members.

Promote a Department or Program by Sharing Assessment Results

Upper-level administrators welcome good news from their academic units. If assessment efforts are enhancing a program’s effectiveness, colleagues should not hesitate to share their findings and resultant changes in teaching or course delivery with a dean or division head. Such assessment results can also be used for campus-wide reaccreditation purposes, a consequence that can bring the department further accolades and possibly additional resources.

To paraphrase the late Herbert Simon (see Spice, 2000), assessment is a secret weapon that psychology faculty and program administrators should use in order to reasonably assess the effects of teaching and other factors on students’ learning and related experiences. Just don’t keep a program’s assessment weapons a secret—use them in an ongoing way for continual improvement and renewal of the academic program.