Unlocking Learning

Welcome to the Journal of Management Education (JME) special issue on Assessment in Management Education. We have talked about the importance of this topic for years and are happy to present a special issue to help our readers think critically about the “what, why, and how” of assessment. Assessment is a reality of life for management educators, but the ideal assessment plan can be quite elusive. Historically we can identify both effective and ineffective uses of assessment.

For a positive example, we turn to Alverno College (Alverno College Faculty, 1994; Mulvey, 1992). In 1973, Alverno implemented an ability-based curriculum that employs competency assessment and faculty feedback to assess students’ performance instead of traditional testing. They focused on eight abilities, including critical thinking, social interaction, and communication, that faculty considered the foundation of liberal arts education. Instead of assessing students at the end of the program, Alverno students begin the assessment process as soon as they enroll in the College. Thus, their assessment process is not a single point in time but spans students’ entire college career. In addition, students are assessed on abilities in the context of real-life situations using specified criteria, and then they receive individualized feedback they can use for further development. These assessments are also integrated into course work and tied to a real-world professional work context (e.g., the integrated competence seminar in the middle of each program).

The purpose of Alverno’s ability-based curriculum seems focused and explicit:

One of our primary concerns is the promotion of growth and change of our students. We use assessment not just to predict, but also to diagnose and prescribe developmental measures. Instead of being primarily a capstone experience which measures acquired ability, assessment is a formative part of the learning process from beginning to end. (Georgine Loacker as quoted in “Alverno and the Assessment Center Movement,” 1983, p. 3)

Alverno has thought carefully about the purpose of their assessment process and has implemented an integrated and systematic process into their curriculum.

The Alverno approach might not have emerged without the seminal idea that was promulgated in the early 1970s. In 1973, David McClelland called for testing for competence rather than intelligence. Prior to McClelland’s suggestion to use a different assessment process to test job applicants, the prevailing practice was to use a variation of the intelligence test to assess job candidates. For instance, during World War II, when the U.S. military needed an efficient way to place their enormous number of the recruits, they adopted the IQ (Intelligence Quotient) test to match recruits with roles (Flancher, 1985; Sternberg, 1994). McClelland’s focus on competencies not only helped organizations select the “right” person with the right skill set for a specific position but also allowed the organization to identify a specific set of competencies needed in the job and then to develop those competencies in current employees

An assessment tool that we have used with success in our Executive Leadership and Organizational Change master’s program is Kolb’s Learning Style Inventory. The Learning Style Inventory is based on Kolb’s (1984) experiential learning model that tries to measure different learning styles. The learning style assessment is designed not to identify one form of learning style as superior over other forms but instead to highlight how each learning style offers both strength and weakness. By being aware of one’s own learning style, one is able to adapt better to different learning situations. In addition, understanding the different learning styles of others can help resolve differences and enhance teamwork.

Unfortunately, assessment tools are not always used as intended. For example, PISA (Programme for International Student Assessment) testing for 15-year high school students, which is sponsored by the OECD (Organisation for Economic Co-operation and Development), was intended to be a tool to identify the strengths and developmental needs of high school students across countries. Students from Finland have been ranked high on the comparison among countries. Since ranking high on the test, this small country has become the model for other countries, and now every other country (including the United States) is trying to copy Finland’s ways without seeming to consider any cultural or historical differences among their countries. In addition, similar to the U.S’s No Child Left Behind initiative, many countries are now simply focused on test scores instead of helping student develop the knowledge and skills necessary for future academic and life endeavors beyond high school (Ravitch, 2010; Ripley, 2013).

Assessment can be effective, but it can also be replete with detours, potholes, and obstacles. For instance, The Wall Street Journal reported that about 200 colleges in the United States are planning to use a new assessment for graduating students that “could prove to be more important to their future than final exams” (Belkin, 2013). This new Collegiate Learning Assessment is supposed to assess the critical thinking skills of graduating students and is proposed as an alternative to traditional grades that have failed to assess these skills accurately. This new assessment seems to be aligned with changes in business given that many corporations such as Procter and Gamble and General Mills have already been using their own assessments to screen their applicants instead of relying on traditional grades. Google is also relying less on applicants’ GPA in its hiring practices for “they have shown to have little correlation with job success” (Belkin, 2013) However, if many schools blindly follow and start using this assessment, then there is a strong possibility that this assessment might end up being another form of standardized tests such as SAT, GRE, or GMAT that have not yet demonstrated any predictive validity when it comes to forecasting one’s future success in employment (Gropper, 2007; Sternberg & Williams, 1997; Wright & Palmer, 1997).

Even the grandfather of all assessment, the IQ test, has not been immune to these potential pitfalls. The IQ test was originally devised by Alfred Binet in 1905 to identify disadvantaged children in the school system (Deary, 2001). When the U.S. immigration agency used the IQ test at Ellis Island for the sorting of immigrants, we see a use of the assessment that was far removed from the original purpose. Despite the cultural and socioeconomic bias of the current IQ test, its use has proliferated, and it is still being defended (Herrnstein & Murray, 1994). These examples are somewhat ironic in a sense that Binet himself thought that the IQ test was limited in scope and should not be used to indicate the wholeness of intelligence (Gould, 1996). This might be a symptom of our overreliance of assessment as a convenient way to predict certain outcomes (Gerst, 2013; Lemann, 1999; Perrin, 1998)

The AACSB (Association to Advance Collegiate Schools of Business), similar to other accreditation agencies, is now requiring its accredited schools to engage in an assessment process they have named assurance of learning. Although in theory this type of assessment and closing the loop make perfect sense, there is a question as to whether schools are merely complying with the requirement in order to pass the accreditation requirement or whether faculty and administration of business schools have developed an assessment program that helps improve students’ learning and the business school curriculum (Kilpatrick, Dean, & Kilpatrick, 2008; Navarro, 2008). Thus, it is not clear whether any given accredited or aspiring school is merely interested in compliance or is using assessment for real improvement in student learning.

As seen, assessment can produce either positive or negative outcomes. If done appropriately, assessment can have a huge positive impact on outcomes. On the other hand, if done inappropriately, assessment could result in unwanted consequences. Assessment as a concept remains neutral, but what we do with the assessment can affect its overall effectiveness. Therefore, what we are trying to measure (construct), why are we assessing (purpose), and how are we approaching and implementing assessment (methodology) can determine assessment’s effectiveness. As seen in the examples in this introduction, it is easy to get distracted by environmental or situational pressures, causing us to lose sight of these three questions: what, why, and how.

Figure 1 suggests graphically a holistic and systematic approach assessment that can make the assessment process more effective. The intent of the special issue is to help us pause and reflect about what, why, and how we are using our assessment in our lives, and what impact assessment is having on management education as well as administration, faculty, and students.

OPEN IN VIEWER

Figure 1.

A model for optimizing assessment effectiveness.

One of the questions we ask is “What should we be assessing?” For college educators, we might start with the following: What is it that we are hoping students will learn during their time at college? Traditionally, many instructors develop student learning outcomes for our courses and programs, but what types of outcomes do we include? As such, educational assessment design begins with the end in mind by identifying learning objectives for the unit, course, or program. We identify what we are hoping our students will achieve during a given learning experience. For many years, Bloom’s taxonomy has been used to shape the level of expected student accomplishment (cf. Kreitzer & Madaus, 1994; Marzano & Kendall, 2007). In a lower division course, student might be expected to achieve the first two levels, knowledge and comprehension. An upper division course might push students to progress to the application and analysis levels, wheras a capstone course might challenge students to synthesize and evaluate what they are learning.

Once learning objectives have been identified, instructors will traditionally determine how to create an integrated class environment that allows them to assess how well students are moving toward the achievement of the learning goals and how far they have come by the end of the course or program. Additional questions such as how can a learning environment be provided where students can succeed in meeting the course objectives, and how can lectures, teamwork, experiential activities, homework, tests, and assignments throughout the course be best combined, become important.

Another important question to ask is “Why is the assessment being conducted?” What is the purpose of the assessment? Are we assessing so that we can help students understand how far they have progressed toward specific learning outcomes? Are we assessing so that we know how much students have learned by the end of a course or program? Are we assessing so that we can show our accreditation bodies that we are meeting their requirements?

The ways instructors answer these questions can help us understand their approaches to assessment. If instructors want to assess how much students have learned about a specific topic, they may use an exam. If instructors want to know whether students can apply a concept to solve a problem, they may ask students to analyze a case study. If instructors want to know whether students can effectively communicate their ideas in a written format, they may ask students to write a short paper. Assessment can generate grades, a performance measurement, but assessment can also used to help students understand how far they have progressed in meeting their learning goals and in identifying areas of needed improvement: that is, summative versus formative assessment. Program review requires assessment to determine whether a program of interest continues to meet the goals of faculty, students, and employers. However, the somewhat recent requirement of accrediting bodies at both the discipline and university levels has accelerated the formalization and reporting of assessment activity. For colleges of business, AACSB requires an assurance of learning program. For universities, regional accrediting bodies such as the Southern Association of Colleges and Schools and North Central Associate of Colleges and Schools require assessments at the university level, including the general education curriculum. The question of purpose remains an important consideration.

The reason for doing assessment often influences the approach we take to conducting assessment, the “how to do assessment” question. Instructors doing assessment in their own classes may use formative assessment tools to see whether students are grasping the material in any given class or unit. These assessments commonly take the form of an informal quiz during class or a 1-minute writing assessment (Angelo & Cross, 1993) where students spend the last minute of class writing down any questions they still have about the class topic. At the same time, summative assessment such as tests may be given to determine the level to which each student has achieved the course learning goals. The question of which type of assessments should be used or which are most appropriate might be related to the purpose (Black & William, 2005).

In summary, we can differentiate between formative and summative assessment, task-specific versus generic assessment, and developmental versus evaluative assessment depending on the purpose of assessment. We might assess students’ learning to understand how and to what extent students are learning throughout the course or program so that we might make adjustments in our teaching and work individually with the students who are falling behind. We might also assess knowledge, skills, and attitudes at the end of a course or program to see how well students have mastered learning objectives on completion of the learning experience.

Several questions must also be answered with respect to how assessment is completed. What level will be assessed? When will assessment take place? How many students will be assessed? What tools will we use to assess learning? Assessment might occur at the individual level, the team level, the program level, the college level, and the university level. Assessment might take place during every class, at the end of a unit, at the end of a semester, at the end of a program, every 2 years, and so on. We can assess every student or a sampling of students, every course or a sampling of courses, every program or a sampling of programs.

With all of the questions to be considered when conducting assessment or developing a program of assessment, it is easy to be overwhelmed and wonder where and how to begin. In this special issue, we present a variety of approaches to assessment that will be informative for readers struggling with these questions.