Context Matters: Outcome Assessments for Associate’s Degree Psychology Programs

Assuring Quality

The attention to student learning outcome assessments is driven in part by the national and international need for quality assurance (European Training Foundation, 2013). In the United States, the Department of Education employs program outcomes as a way to assure program quality, holding accountable the institutions that offer the programs. The 2017 U.S. Department of Education, Current Practice of Recognized Accreditors: Student Achievement Standards (p. 1) recognizes:

Code of Federal Regulations (34 CFR 602.16) requires that accrediting agency (accreditor) standards address the quality of the institution or program in regard to success with respect to student achievement in relation to the institution’s mission, which may include different standards for different institutions or programs, as established by the institution, including, as appropriate, consideration of course completion, State licensing examination, and job placement rates.

Promoting High-Quality Teaching and Learning

The demand by institutional and program accreditors for program outcome assessments underscores their importance for quality assurance. However, there are other equally important uses for program outcomes. The APA (2007, 2013) Guidelines for the Undergraduate Psychology Major identified the importance of program outcome assessment in curriculum design, goal setting, and assessment planning. Simply put, these functions of program outcomes involve “setting clear expectations for student learning, aligning curricula with these expectations, assessing student attainment, and using assessment results to effect changes that promote better student learning” (APA, 2017, p. 1). This directive goes well beyond the summative program evaluation for institutional accreditors to ensure a continuous formative assessment and improvement process for both faculty and students.

Curriculum maps serve as an important tool in establishing program quality (Wijngaards-de Meij & Merx, 2018). These maps document how faculty introduce, reinforce, and promote mastery of disciplinary knowledge skills and abilities through the sequence of courses. For example, Wijngaards-de Meij & Merx observed that undergraduate students were methodologically unprepared to complete their senior theses. The students also appeared at a loss to explain where in the curriculum their methodological knowledge and skills would have been honed. When faculty completed a curriculum map of the methodological knowledge and skills offered in the program, both faculty and students gained a better understanding of when and how such knowledge and skills emerged.

The Guidelines for the Undergraduate Psychology Major (2013, p. 3) specify “reasonable departmental expectations for the undergraduate psychology major across different kinds of educational contexts.” They do so by defining broad goals, outcome objectives, and specific indicators, ¹ attempting not just to assess students learning outcomes but also to make transparent to students and faculty how a department’s curriculum promotes the outcomes. For example, a broad goal for bachelor’s students in psychology is to have a knowledge-base in the discipline for which an outcome objective is for them to describe concepts, principles, and overarching themes in psychology. Evidence of achieving the objective can be assessed by multiple specific indicators, including one that students demonstrate use and evaluation of theories/frameworks to explain and predict behavior. By design, the Guidelines were not prescriptive, that is, the principles were not meant to apply definitively to all programs in exactly the same way. The Guidelines affirmed the importance of tailoring the program outcomes to the context of the institutional mission, traditions, faculty, and students.

The APA articulated and published the first set of guidelines in 2007 to include 10 broad goals for the undergraduate major. In 2013, revised guidelines condensed the ten goals to five, including (a) knowledge base in psychology, (b) scientific inquiry and critical thinking, (c) ethical and social responsibility in a diverse world, (d) communication, and (e) professional development. More important to the focus of this article, the new Guidelines added foundational indicators to complement and augment the baccalaureate ones. The foundational indicators addressed content and skills that should be completed in the first set of courses in the major. These indicators were proposed to serve as program-level outcomes for the associate’s degree. Although the same broad goals and outcome objectives apply to students seeking either the associate’s or bachelor’s degree in psychology, the difference in the standards of performance lies in the sophistication of the indicators or the specific measured elements relevant to each degree. For example, associate’s and bachelor’s degree students should have the same goals (e.g., having a knowledge base in the psychology) and outcome objectives (e.g., describing concepts, principles, and overarching themes in psychology). However, unlike the bachelor-level indicator of students’ use and evaluation of theories/frameworks to explain and predict behavior, associate-level students are assessed for their use of basic psychological terminology, concepts, and theories in psychology to explain behavior and mental processes (see Table 1).

OPEN IN VIEWER

Table 1.

Content Goal for the APA Guidelines for the Undergraduate Major (2.0).

OutcomesStudents will:	Foundation IndicatorsStudents will:	Baccalaureate IndicatorsStudents will:
1.1 Describe key concepts, principles, and overarching themes in psychology	1.1a Use basic psychological terminology, concepts, and theories in psychology to explain behavior and mental processes	1.1A Use and evaluate theories to explain and predict behavior, including advantages and limitations in the selected frameworks
	1.1b Explain why psychology is a science with the primary objectives of describing, understanding, predicting, and controlling behavior and mental processes	1.1B Describe the complexity of the persistent questions that occupy psychologists’ attention
	1.1c Interpret behavior and mental processes at an appropriate level of complexity	1.1C Analyze the variability and continuity of behavior and mental processes within and across animal species
	1.1d Recognize the power of the content in shaping conclusions about individual behavior	1.1D Examine the sociocultural and international contexts that influence individual differences (e.g., personality traits, abilities) and address applicability of research findings across societal and cultural groups
	1.1e Identify fields other than psychology that address behavioral concerns	1.1E Compare and contrast the nature of psychology with other disciplines (e.g., biology, economics, political science), including identifying the potential contribution of psychology to interdisciplinary collaboration

Note. APA Guidelines 2.0 have common goals (e.g., knowledge of psychology) and outcomes (e.g., describe key concepts, principles, and overarching themes in psychology) but different baccalaureate and foundation indicators for bachelor- and associate-level students.

The alignment of the foundational and baccalaureate indicators in Guidelines 2.0 suggests that the foundational indicators can function as milestones for the bachelor’s degree. The same assessment goals and outcomes apply whether students are just beginning or completing their undergraduate study of psychology discipline. However, as they progress through the curriculum, and are assessed by foundational- then bachelor-level indicators, students are expected to demonstrate more advanced knowledge (conceptual, procedural, and metacognitive) and higher level cognitive processes (analyze, evaluate, and create) as defined in the revised Bloom’s taxonomy (Krathwohl, 2002).

Facilitating Transfer

Moving beyond quality assurance and improved teaching and learning, the outcomes for associate’s degree psychology program can also be used to promote student transfer. Students commonly transfer from an associate’s degree granting institution to a bachelor-granting one. According to the National Student Clearinghouse (2017a), 24.4% of all students who began college at 2-year public institutions in fall, 2008, transferred to a 4-year institution within 6 years. Within dual mission institutions that rate is even higher, with estimates of more than 60% proceeding to the bachelor’s degree (Millner, 2018). Indeed, the National Student Clearinghouse (2017b) further reports that 49% of all 2015–2016 graduates with a bachelor’s degree had enrolled at a 2-year public institution at some point in the previous 10 years.

The goal of promoting a seamless process for student transfer is facilitated if associate’s degree granting institutions align their program learning outcomes to those offered at the bachelor’s level (Carter et al., 2011); in the manner that foundational indicators align with and undergird baccalaureate indicators. The transferring institution can provide the quality assurance that the students have achieved associate-level program outcomes in a well-connected curriculum. With their prior learning assessed, a transfer student can then more easily identify courses that promote bachelor’s degree program outcomes.

Ease of transferability is central to the Tuning (Jones, 2012) and Degree Qualifications Profile (DQP; Jankowski et al., 2013) projects. In these popular reform initiatives, faculty specify competencies ranging from intellectual skills, applied and civic learning, and specialized and integrative knowledge that may be associated with lower division courses (e.g., those offered by community colleges) but are identified as necessary for students to achieve bachelor-level outcomes. We found a few examples of Tuning and DQP processes in psychology. Kinzie (2014) described a successful example of aligning outcomes between institutions to promote transfer between Georgia Perimeter College and Georgia State University. Faculty at both institutions used the original APA Guidelines, which GSU assessed extensively, to guide discussions about competencies for transfer within the DQP framework. Kinzie (2014, p. 8) reported that “the GSU and GPC psychology faculty learned more about where students have struggled in the transition from the two- to four-year degree program and, in particular, where challenges exist in the curriculum.”

However, this kind of collaborative coding of outcomes across degree programs is not how transfer often works. Typically, transfer involves a course-by-course review of the transferring student’s transcript based on catalog descriptions augmented with additional information. Carter et al. (2011) describes how to make such transfer evaluations more efficient by using transfer agreements, developing university transfer guides, and creating block transfer agreements of degree programs. Still, many institutional decisions to accept course or program transfers from a community college graduate depend on making equivalence judgments. Kennepohl (2016) noted that such judgments may be misleading at best and lead to transfer that is far from seamless. For example, consider the following two courses, both of which use the same textbook, have the same contact hours, and specify Introductory Psychology as a prerequisite.

Course A—Social Psychology (3 Credits): This course is a survey of the effects of social influences on the basic psychological processes of individuals. The course considers individuals in the context of their culture and society, the development of attitudes, and the impact of the group on individual behavior.

While initially appearing to be similar, Course A is a lower division (2000 level) offered by a community college in Utah and is not judged as equivalent to Course B, which is an upper division (3000 level) course offered by a dual-mission institution also in Utah. The lack of equivalence between lower and upper division courses is codified in Utah State System of Higher Education Regent policy (R470). However, paradoxically, the same policy allows for program-wide transfer so that a student who completes an associate’s degree must transfer with upper division status to any Utah 4-year institution without being required to take any lower division premajor courses at the receiving institution. Course and degree program transfer policies for all 50 states can be found at the website of the Educational Commission of the United States (2018).

The focus on transfer of courses and degrees could be enhanced by creating broadly accepted program learning outcomes, as seen in the Tuning and DQP projects. The transfer of the degree through the alignment of the program outcomes at the associate’s and bachelor’s level can lead to there being “more measureable, authentic, and collaborative approach for comparing students’ experiences” (Carter et al., 2011, p. 22). Receiving institutions can evaluate student readiness for upper division course work on the basis that assessing whether they achieved outcomes associated with lower division coursework in the discipline, irrespective of precisely the disciplinary coursework that was completed. This practice is consistent with many institutional uses of AP and College Level Examination Program (CLEP) tests to waive course requirements of entering 1st-year students. Many institutions and state systems are experimenting with Prior Learning Assessment (PLA; B. Klein-Collins, 2010, R. Klein-Collins & Hudson, 2017), defined as the evaluation of (typically adult) learners’ experiential learning for purposes of granting college credit, certification, or advanced standing toward further education or training. According to Klein-Collin and Hudson, PLA has been shown to save students money and improve graduation rates. PLA includes national standardized tests (such as AP and CLEP), challenge exams often offered by departments, individualized assessments (e.g., portfolio-based demonstrations), and relevant evaluated noncollege programs (e.g., transfer of military service).

Although no mention is made explicitly of transfer in Guidelines 2.0, foundation indicators assess whether students acquire “a psychological worldview on how they think about behavior” (p. 2). Consequently, these indicators may serve not only as program outcomes for the associate’s degree in psychology but as a successful vehicle for efficient transfer decisions. Put differently, students achieving foundational indictors demonstrate a necessary step for their path to achieve the bachelor’s outcomes.

Variability in Psychology Courses

The primary concern was whether the variability in the course offerings in associate’s degree programs made the foundational indicators irrelevant to programs that do not offer or cannot offer key courses in the curriculum. In one survey, most of the associate’s degree programs offering psychology courses in community colleges did not have psychology in their degree title (Johnson & Rudmann, 2004). For example, students earned a general associate’s degree or an associate’s degree in liberal studies. In a more recent survey (Norcross et al., 2016), 80% of associate’s degree programs offer only a handful of courses in common, including Introduction to Psychology, Developmental Psychology ² , Abnormal Psychology, and Social Psychology. Other courses, such as Statistics and Research Methods, were available at half or fewer of the reporting associate’s degree programs. Moreover, not every associate’s degree program catering to students interested in pursuing psychology is a full-fledged “psychology” program since many different kinds of configurations for degree programs can enroll students in psychology classes. For example, based on research by the second author, no associate’s degree programs specific to psychology appear to be offered by any Arkansas community colleges although they all offer between two and four psychology courses. The courses offered are ones described above and are typically required for other majors (teaching, early childhood education, nursing, etc.). Nonetheless, some students eager to major in psychology enroll in all the available psychology courses and seek to transfer to a 4-year Arkansas university.

A similar story emerges from the first author’s research in the state of Utah. Students earning an associate’s degree program in general studies at one community college may complete up to nine available psychology courses. One course is Psychology as a Science and Profession which, although not a methodology course per se, introduces students to the basic tenets of and connections between the scientific and professional practice of psychology (Amsel & Kay, 2008). Another program in the state offers an associate’s degree in social science and includes eight psychology courses although without any formal research methods class. This program is offered in a dual-mission university, so there is an easier path for these students to earn a bachelor’s degree in psychology. A third offers an associate’s degree in psychology with one required course (intro to psychology) and 12 additional credits taken from seven courses including Psychology as a Science and Profession. Arkansas and Utah are microcosms in the range of ways institutions offer a variety of psychology courses leading associate’s degree.

It remains an open question whether any of these programs can expect their students to achieve the foundational indicators as defined in Guidelines 2.0. On the one hand, evidence indicates that all students learn the discipline in the manner suggested in Guidelines 2.0, whereby increasing learning experiences in the discipline result in broad and incrementally deeper understandings, reflective of the relationship between the foundational and baccalaureate outcomes. Landrum et al. (2019) found that participants with no, some (introductory psychology), and extensive (senior majors) involvement in psychology show parallel differences in performance across multiple psychological assessments tapping similar areas as the Guideline 2.0 goals (including general psychology knowledge, research, ethics, and careers). Those assessments were also positively intercorrelated, further suggesting that elements of the discipline are learned in a broad, connected, and incremental manner rather than separately and in isolation. For example, students’ ability to apply psychological knowledge on one scenario was correlated with higher performance on assessments across a range of psychological knowledge and skills associated with the Guidelines goals.

However, Landrum et al. also established that some acquisitions measured at the foundational level may require key course prerequisites beyond introductory psychology. That is, research knowledge and skills may be necessary for students to acquire other knowledge and skills. For example, students’ ability to apply psychological knowledge to another scenario was associated with assessments of knowledge and skills about research methods. The authors concluded that a research methods course may serve as a gateway class for students to appreciate fully the depth and breadth of the discipline. Other research affirms that a methodology course may be foundational for other acquisitions (and presumably learning outcomes) in the curriculum (Betancur et al., 2019; Freng, 2020; Freng et al., 2011; Stoloff et al., 2010). The research methods course is important for students’ understanding of the discipline as a science (Amsel et al., 2011). Similarly, students adopt the methodological framework of their discipline after being exposed to a range of courses, including a methods course (Lehman & Nisbett, 1990). Specifically, psychology students’ probabilistic reasoning was more effective when compared to those in the natural sciences and humanities commensurate with the number of statistics courses the psychology students completed.

The role that research methods plays in the curriculum for students to achieve Guidelines 2.0 foundational outcomes prompts concern, given the number of associate’s degree programs that may not offer a methods course. If the impact of a methodology course is as profound as described, then failing to offer a methods class at the associate’s level may result in students not achieving all the foundational outcomes in Guidelines 2.0. As part of our research, we noted that some associate’s degree programs in psychology offered in community colleges are designed explicitly for transfer to a psychology bachelor’s program at a 4-year college. We have found such programs across the country from New England to California, and they notably include research methods as a requirement. The research methods course is identified in Guidelines 2.0 as “ideal” for foundational outcomes. “Ideally, foundation courses are likely to include an introductory psychology course and a methods course along with other lower-level requirements or electives that firmly establish the nature of the discipline” (p. 3). Perhaps the foundational indicators are best suited for associate’s degree psychology programs with a research methods requirement.

Multiple Functions of Psychology Courses

Further complicating assessment development at the associate’s degree program level is the multiple functions that many psychology courses serve in 2-year institutional contexts, including general education and service courses for professional programs. Many psychology courses may fulfill general education requirements, highlighting the acquisitions of soft or transferable skills in addition to the psychology-specific program-level outcomes. Focusing on the skills identified by general education outcomes is of particular importance to faculty at open-enrollment institutions, whose students are often underprepared for college. Two thirds of community colleges students require remediation in mathematics or English (Jaggars & Stacey, 2014). In California, 49% of community college students required remediation in both (Mejia et al., 2016). Remediating these students and promoting general education outcomes is a challenge for many community colleges. At commuter campuses, typically enrolling older students, there are fewer opportunities to support students’ academic engagement outside the classroom, which underscores the importance of introductory courses as a vehicle for helping students understand and embrace college culture and norms to enhance their success. Consequently, faculty at these institutions may be expected to focus on preparing students for and support their completion of general educational outcomes at the associate’s level, perhaps more than their colleagues in other institutions. Although arguments have been made about the alignment between general education and course- or program-level outcomes in psychology (Gurung et al., 2016; Jhangiani & Hardin, 2015; Strohmetz et al., 2015), these arguments do not always recognize the additional complexity of creating such alignments with underprepared students in community college contexts.

Besides their function as general education classes, the associate’s degree psychology courses are service courses for other professional programs at the associate’s level (e.g., nursing, early childhood education, etc.). The outcomes for these courses may emphasize the preparation of students for careers. That is, associate’s degree psychology classes, no matter their institutional context, may have program outcomes appropriate to the practical and professional needs for job seekers that one could argue might be even more important than milestone academic outcomes for transfer students.

Summarizing context factors, perhaps the foundational indicators are optimally suited for associate’s degree psychology programs designed for transfer. However, “transfer programs” are a minority of all associate’s degree programs enrolling students interested in studying psychology. This situation may be due in part to the various constraints on courses that community colleges can offer and the resource capacity of the institutions to offer them. This variability in psychology courses and requirements precludes assumptions being made about the psychology content and skills that constitute the associate’s degree curriculum.

Proposed Core Concepts for Associate’s Degree Programs

The team worked on identifying core concepts that were consistent with Guidelines 2.0 foundational indicators and can be acquired irrespective of the courses in which they enroll. Put differently, we focused on those conceptual underpinnings central to acquiring key beliefs, attitudes, skills, and knowledge about the discipline. We were motivated by the notion of students acquiring key ideas of the “psychological worldview,” described in Guidelines 2.0 as an outcome of the foundational courses. We were also compelled by findings suggesting that students exhibit conceptual resistance about adopting such a worldview, which provides a basis for personal, professional, and social applications that can be used to solve problems (cf. Cranney & Dunn, 2011, Hughes et al., 2013; Stanovich, 2014). We identified six interrelated core concepts that we believed would underlie a psychological worldview and for which instruction would have to address student resistance. Broadly, the six core concepts address the discipline as a science (1) for which analytic skills (2) are necessary to test theories (3) that are connected to and informed by a range of other disciplines (4) and can be applied to personal issues (5) and social concerns (6). We assumed that if students adopt these core concepts, they would be better able to achieve goals and objectives in Guidelines 2.0, even though students may not have the opportunities to explicitly learn details addressed in many of the indicators. Table 3 lists the core concepts.

OPEN IN VIEWER

Table 3.

Core Concepts and Their Functional Definitions.

Explain the nature and methods of psychology as a science	Justify psychology as a science where conclusions are supported by evidence Base inquiry on skepticism in contrast to other ways of knowing
Demonstrate quantitative literacy in psychological science	Use graphical, symbolic, and numerical methods to analyze, organize, and interpret behavioral phenomena Discriminate between association and causation Identify the types of evidence used to establish causation
Describe why theory is fundamental to psychological science	Identify the functions of theory to explain phenomena, guide research, support/modify /reject theory, and explore new phenomena Recognize theory supports a systematic method of investigation and interpretation
Integrate psychology within the broader interdisciplinary nature of science and behavioral science	Recognize that psychological science is interdisciplinary Identify the complex interdependencies between the biological, psychological, and sociocultural processes contributed by the other sciences
Apply psychological science to improve personal well-being, interpersonal collaboration, professional growth, and civic responsibility	Construct and recognize examples of relevant and practical applications of psychological findings to everyday life
Promote the value of psychological science for addressing community needs, societal problems, and advancing science as a whole	Describe the value of psychology in generating potential solutions to local, societal, global problems, and promoting personal welfare

The six core concepts separately address unique elements of the discipline but together capture the broad scope of a “psychological worldview” that is likely to be fostered in any given 2-year program. Indeed, we present this approach based on core concepts as an alternative to, not a replacement of, the foundational indicators in Guidelines 2.0. The core concepts approach is designed for those associate’s degree psychology programs that are not well aligned to the ideal program identified in the Guidelines as including methods courses. However, it is compatible with the goals and objectives outlined in Guidelines 2.0.

We further note that the six core concepts align well with proposed outcomes for introductory psychology that were recently developed by the APA Introductory Psychology Initiative (IPI, 2019). The IPI has been exploring optimal policies and practices in the teaching of introductory psychology that build from discussions about ways of conceptualizing the course (Gurung et al., 2016; Jhangiani & Hardin, 2015). We are struck that the outcomes framed by the Assessment Subcommittee of IPI propose a similar integrative approach as the proposed core concepts for associate’s degree psychology program. See Table 4 for the outcomes proposed by that group to address introductory psychology needs.

OPEN IN VIEWER

Table 4.

Proposed APA-IPI Student Learning Outcomes for Introductory Psychology.

Goal 1: Psychology Content: Identify basic concepts and research findings.

Define and explain basic psychological concepts.

Interpret research findings related to psychological concepts.

Apply psychological principles to personal growth and other aspects of everyday life.

Goal 2: Scientific Thinking: Conduct, analyze, or evaluate scientific investigations.

Draw logical and objective conclusions about behavior and mental processes from empirical evidence.

Describe the advantages and limitations of various research strategies.

Design, conduct, or evaluate psychological research.

Evaluate how psychological science can be used to counter unsubstantiated statements, opinions, or beliefs.

Goal 3: Key Themes: Provide examples of psychology’s integrative themes:

Psychological science relies on empirical evidence and adapts as new data develop.

Psychology explains general principles that govern behavior, while recognizing individual differences.

Psychological, biological, social, and cultural factors influence mental processes and behavior.

Our perceptions filter our experiences of the world through an imperfect personal lens.

Applying psychological principles can change our lives and communities in positive ways.

Ethical principles guide psychology research and practice.

Psychologists strive to promote respect for human diversity in its many forms.

Multiple Instruments Measure Associate’s Degree Learning Outcomes

The core concept approach offered for understanding the associate’s degree learning outcomes can be assessed using a variety of instruments discussed in this special edition and the SNAP website. In addition, the SNAP committee on associate’s degree assessment explored the possibility of creating new instruments designed to be used in a variety of program and institutional contexts. We reasoned that students armed with an understanding of the core concepts can not only achieve program outcomes (including Gen Ed) irrespective of institutional context, but also be competitive for relevant jobs and positioned for success in upper division, bachelor-level courses offered by 4-year colleges. Whether a single assessment can function to address all these measurement goals is an open question. For example, an instrument assessing specific knowledge, skills, experiences, and efficacy as advocated by Rudmann et al. (2016) than core concepts approach adopted here may better measure transfer success.

Two instruments are available (by emailing the first author) to assess students’ achievement of the core concepts. The first employs a multiple-choice format to assess each of the six core concepts. The items of the assessment have come from a variety of sources, including some created especially for this assessment and have been calibrated to a “developing,” “proficient,” or “foundational” level of achievement. The second instrument includes short answer and essay questions. Each question comes with single-point rubrics (Fluckiger, 2010), which are designed to promote standards of proficiency for particular questions. The single-point rubrics may only provide one point of assessment but does so for the multiple dimensions being assessed by the rubric. The single-point assessment technique promises to make assessing the responses to the short answer and essay questions easy, reliable, and consistent.

We invite colleagues to collaborate on providing analysis of the psychometrics of our and all other measures assessing program outcomes for associate degree psychology students. Additional work is also necessary to assure the concurrent validity of the instruments in being related to general education outcomes and their predictive validity in determining the future success of students in upper division psychology courses. The SNAP goal of abroad based set of assessments for all bachelor degree psychology students is also critically important for associate degree programs with their unique institutional context and characteristics.

Instruments based on the core concepts can also be used to assess students in other educational contexts and programs of study than an associate’s degree. For example, such instruments can be used as milestone assessments of students who completed 60 credits of a bachelor’s degree psychology program in a 4-year college. Like the community college students, the milestone use of the assessments would be related to the students’ completing their general education requirements. However, the measures can be used en passant with bachelor-seeking students to assess their progress. Another use of the instruments assessing core concepts would be to assess whether students minoring in psychology have acquired a deep and conceptually rich understanding of the discipline. These students would be taking a limited number of psychology courses, but they are likely to be upper division and without general education designation. Finally, the instruments would be appropriate for students in interdisciplinary programs that include an emphasis in psychology. Like psychology minors, these students are likely to enroll in upper division psychology courses that are not general education designated.