The Challenge of Assessing Creative Problem Solving in Client-Based Marketing Development Projects

Abstract

Creativity and marketing imagination are essential core competencies for marketers. Therefore, higher marketing education emphasizes creativity in several ways. However, assessing creativity and creative problem solving is challenging and tools for this purpose have not been developed in the context of marketing education. To address this gap, we analyze the use of Structure of the Observed Learning Outcome (SOLO) taxonomy as an assessment tool in creative problem solving and in doing so develop an assessment framework by contextualizing SOLO taxonomy for client-based marketing development projects. We first introduce earlier literature on assessment and SOLO taxonomy. We then describe a client-based marketing course on creative problem solving. We use SOLO taxonomy to analyze the outcomes and development processes of six student projects and in doing so develop an assessment framework for creative problem solving. Finally, we give suggestions for teachers on how to enhance creative problem solving and its assessment in marketing classrooms.

Keywords

creative problem solving assessment client-based projects innovative teaching methods experiential learning techniques

Marketing is a creative process that creates new products and services, develops pricing and channel strategies, and uses branding to enhance the customer relationship. In addition, marketing communications use a wide range of creative solutions. Marketing involves doing things differently, combining old things with new ones, innovating, surprising, and delighting. Indeed, marketing is closely linked to creative thought and imagination (Levitt, 1983). As Titus (2007) emphasizes, Levitt’s ideas hold significance for marketers today and “marketing professionals are likely to find themselves under greater pressure to identify and produce new breakthrough products, services or marketing concepts.” Thus, creativity and innovativeness are core competencies for marketers.

Creativity is the ability to give birth to something that is both novel (such as original, unexpected, or surprising) and appropriate (useful or adaptive in terms of the task constraints; Stenberg & Lubart, 1999). Within marketing, creativity is defined as a problem-solving process that aims to produce products, services, and marketing initiatives that are unique to the marketplace and create value for the customer (Titus, 2000, 2007). A creative original idea is not enough—for it to have value, the idea must also be useful and actionable (Amabile, 1998) and implementable (Levitt, 1963/2002). In order to create value in business, creative problem solving needs to combine both creative and critical thinking. Creative thinking is expansive, imaginative, and unconstrained thinking that is associated with exploration and idea generation. Critical thinking in turn is focused, logical, realistic, and practical thinking (Nickerson, 1999). This is why marketing educators need to aim to enhance both the creative and critical thinking skills of their students.

Creative problem solving and the ability to invent new products and customer experiences should also be emphasized in the teaching of marketing (Vogel, Cagan, & Boatwright, 2005). Creativity has already been recognized as an important learning objective and tool in marketing curriculums, and many creative methods are used in marketing education even though there are still concerns about whether marketing faculties formally encourage, recognize, develop, and reward creativity in marketing students (e.g., Lunsford, 1990; McCorkle, Payan, Reardon, & Kling, 2007; Ramocki, 1996; Strauss, 2011; Titus, 2000, 2007). However, no systematic efforts to assess creativity or creative problem solving in a marketing education context have been published to date, even though their importance has been noted (McCorkle et al., 2007).

There are two different facets to assessing creativity: assessing the creativity in students themselves as a personal trait and assessing the creativity of their work (Elton, 2005). In this article, the emphasis is on assessing the creativity of the students’ work. However, assessing creativity and creative problem solving is challenging and tools for this purpose have not been developed in the context of marketing education. Especially when teaching creative problem solving in real-life business cases, the assessment practices need to support the fuzzy, complex nature of such cases, where the outcome is uncertain, there are neither specific correct answers nor the possibility of quick closure, and all the answers are context dependent. Thus, assessment practice needs to focus on the relational, contextual, and uncertain aspects leading to open-ended rather than closed answers.

Constructive alignment implies that the intended learning outcomes, teaching and learning activities, and assessment need to be aligned together (Biggs, 1996). It is argued that what and how students learn depends greatly on how they think they will be assessed (Biggs & Tang, 2007). Thus, assessment practices should send the right signals to students about what they should be learning and how they should be learning it. The SOLO (Structure of the Observed Learning Outcome) taxonomy provides one possible framework for evaluating problem-solving cases. It differentiates the different levels of understanding and how they are present in students’ solutions (Biggs & Collins, 1982; Biggs & Tang, 2007).

Therefore, we analyze the use of SOLO taxonomy as an assessment tool in creative problem solving and in doing so develop an assessment framework by contextualizing SOLO taxonomy for client-based marketing development projects.

The article proceeds as follows. First, we introduce earlier literature on assessment and SOLO taxonomy. Second, we describe a client-based marketing course on creative problem solving. Third, we use SOLO to analyze six student projects and then develop an assessment framework by contextualizing the SOLO taxonomy for creative problem solving. Finally, we give suggestions for teachers on how to enhance creative problem solving and its assessment in marketing classrooms.

Assessing Learning Outcomes

Assessment is critical to any academic work. Different assessment strategies all aim at finding out the knowledge and skills that the student may or may not have (Jenkins, 2010). The students’ learning depends on how they think they are assessed. This is why the assessment criteria need to be communicated openly so that the students focus on the correct aspects in their learning (Biggs & Tang, 2007). Different tools, such as the use of an assessment grid with explicitly specified assessment criteria, are argued to raise the quality of assessment and thus assist students to focus on the most important things (Rust, Price, & O’Donovan, 2003).

Both formative and summative assessments are critically important for education (Taras, 2008). Formative assessment aims to guide and enhance students’ learning during the learning process (Brown, Bull, & Pendlebury, 1997). Students are provided with information about the gap between their current knowledge and skills and the desired outcome in order to improve the learning of individual students and to improve the teaching itself (Biggs & Tang, 2007). However, Wingate (2010) notes that formative feedback in learning can only have a positive effect if students actually make use of the feedback to improve their performance. Highly motivated students are ready to do this, but students whose motivation is low may not consider revising their work based on feedback and may even become less motivated if they receive overly critical feedback during their learning. Self-evaluation and peer evaluation are valuable tools that can be incorporated in teaching and learning activities to promote formative assessment. Indeed, formative assessment and feedback should be used to empower students as self-regulated learners (Nicol & Macfarlane-Dick, 2006).

Summative assessment aims at grading the students at the end of a course; it is most often considered the final grade. The aim is to inform students how well they have learned what they were supposed to have learned (Biggs & Tang, 2007). Summative assessment is used to ensure that the student has sufficient knowledge and skills in order to continue his or her studies and later to enter working life; it is thus important for accreditation (Taras, 2008). As formative and summative assessments have different goals, both are needed. It is most important when aiming to ensure that students learn from their mistakes—noticing and admitting a mistake should not be punished, but rather students should be encouraged to engage in trial-and-error processes and learn from them. However, making a mistake and learning from it is different from not making any effort at all.

Biggs and Collins (1982) developed the SOLO taxonomy, which describes how the learner’s performance grows in complexity when mastering academic tasks. It is a method that encourages higher cognitive outcomes (Leung, 2000). They themselves provide numerous examples on how to apply the SOLO taxonomy in, for example, teaching history, mathematics, English, geography, and modern languages (Biggs & Collins, 1982; Biggs & Tang, 2007). It has also been applied earlier in, for example, measuring the design performance of students of design and technology (Leung, 2000), engineering (Lundberg, 2004), mental health (Chan, Tsui, Chan, & Hong, 2002), and accounting (Lucas & Mladenovic, 2009). However, it has not been applied in assessing creativity and creative problem solving, even though creativity has been considered important (Biggs & Tang, 2007; Elton, 2005).

Biggs and Collins (1982) distinguish five levels of understanding, each of which is characterized by capacity, relating operation, consistency, and closure. Capacity refers to the amount of working memory or attention span required at different levels. One needs to think more and about several aspects at the same time in order to reach higher levels of understanding. Relating operation refers to the way in which the problem and the solution interrelate. At the lower levels, there is no logical interrelation; instead, there may be denial or tautology, where the student clearly misses the point. Higher levels correctly draw a general conclusion from particular instances using either one or several cues. The highest levels introduce new abstract principles not given in the problem.

Consistency and closure refer to two opposing needs of the learner: one is the need to come to a conclusion and the other is to make consistent conclusions so that there are no contradictions in logical argumentation. The greater the learner’s need to come to quick closure, the lesser he or she will use the data. A quick closure usually leads to a higher probability that the outcome is inconsistent with the original data. Lower levels of understanding are often high in quick closure and low in consistency, whereas higher levels emphasize consistency and may even require closure to be left open (fuzzy, complex situations). These all affect how the student can structure the answer and show an understanding of a given matter. The five levels of understanding are prestructural, unistructural, multistructural, relational, and extended abstract (Biggs & Collins, 1982).

At the prestructural level, the student has not done the task properly or has not understood the point. The capacity used is minimal, and the problem and response may be confused. The answer has no logical relationship to the problem because of an inability to comprehend, tautology, or idiosyncratic relevance. The student may reach closure without understanding the problem. For example, a student may produce a quick answer based on his or her earlier experiences without understanding the context of the problem. At prestructural level, the student cannot be given a passing grade on the assignment (Biggs & Collins, 1982; Leung, 2000; Lucas & Mladenovic, 2009; Lundberg, 2004).

At the unistructural level, the student picks up on and uses one or a few aspects of the task, and thus his or her understanding is nominal. The student can make generalizations based on one aspect or experience. He or she reaches a quick closure that oversimplifies the issue; the student might jump to conclusions and be inconsistent in his or her answer. For example, the student may rely on one or few shallow customer interviews without understanding the differences between different customer groups (Biggs & Collins, 1982; Leung, 2000; Lucas & Mladenovic, 2009; Lundberg, 2004).

At the multistructural level, the student has learned several aspects of the task; however, he or she treats these aspects separately and can make generalizations based on these independent aspects. Understanding involves knowing about different things; the student may describe and list them, but may not be able to relate them to each other. There begins to be a feel of consistency, but closure is selective and immature. For example, the student may list several ideas for new services, but treats them independently, does not link them, and cannot construct a coherent concept out of them (Biggs & Collins, 1982; Leung, 2000; Lucas & Mladenovic, 2009; Lundberg, 2004)

At the relational level, the student integrates the different aspects into a coherent whole, where each part contributes to the overall meaning. The student identifies and uses most or all of the relevant data, and resolves conflicts using a relating concept that applies to the given context. This integration leads to a firm conclusion that applies in a given context; however, inconsistencies may occur outside the context. For example, the student reaches the level of a coherent whole and is able to form a holistic marketing concept where different elements add value to the concept (Biggs & Collins, 1982; Leung, 2000; Lucas & Mladenovic, 2009; Lundberg, 2004)

At the extended abstract level, the student reconceptualizes the integrated whole at the relational level and at a higher level of abstraction. The reconceptualization enables generalization to a new topic or area, or turning it reflexively on oneself. Understanding is seen as involving metacognition, thinking about thinking. The student may question basic assumptions, present counterexamples, and provide new data that did not form part of the original problem that was posed. A firm closure is often seen to be inappropriate, and conclusions are held open or qualified to allow logically possible alternatives. For example, the student can integrate new original, unexpected, or surprising elements such as technological innovations into the integrated whole or provide different concepts for different strategic goals (Biggs & Collins, 1982; Leung, 2000; Lucas & Mladenovic, 2009; Lundberg, 2004).

Assessment of problem-solving activity varies depending on the level and challenge of the problems. On the one end, there are specific or standard questions that can be solved by using conventional paradigms, thus calling for relational answers. However, even then, an extended abstract response may be reached by originality and conciseness (Biggs & Tang, 2007). On the other end, there are fuzzy and complex problems that have no definite correct answers, only better or worse ones. In such cases the quality of the answer is dependent on the context and the teacher has to decide on assessment criteria, such as degree of originality, “elegance,” or holistic understanding of value for different parties. This is the case in assessing creativity (Biggs & Tang, 2007).

The practical outcomes of creative problem solving in marketing are also context-dependent and relational. Indeed, professional knowledge in marketing is complex, unstable, unique, and includes possible value conflicts (Schön, 1983). The characteristics of marketing knowledge set requirements for the teaching of practical marketing. Teachers should aim at enabling students to reach the extended abstract level in their studies. Originality and uniqueness are important aspects of assessing creativity. The outcome must also be applicable and appropriate so that it can provide value (Biggs & Tang, 2007). Furthermore, in our client-based marketing development projects, we enhance creative problem solving throughout the course. Thus, the assessment should not only concentrate on the final outcome or outcomes but also on the process of how the outcome has been reached.

Enabling Creative Problem Solving: A Description of the Case Course

The context for this article is a master’s level course called “Concept Factory” in marketing at a business university of Vaasa in Finland. The marketing students attending the course are in their first year out of two. They have a bachelor’s degree before entering the master’s program. The learning focuses on creative problem solving in client-based marketing projects. The teaching in the course is based on experiential learning (Kolb, 1984) that emphasizes constructive teaching leading to deep learning in real-life contexts. During the past 6 years (2007-2012) the course has been run seven times, three times with marketing students and four times with cross-functional teams, first with designers and then with communication students. Twice, the course has also involved international collaboration with Umeå Institute of Design in Sweden.

The course runs for a fall semester. The students work intensively for 4 months. They meet approximately once a week in a class to discuss the projects with the instructor and with each other. Altogether they meet 11 times. Students are divided into teams and given real-life development challenges, such as brand building, product development, or service innovation. During the class time, the instructors organize workshops on different topics, such as innovative ways to gather customer information or how to analyze qualitative customer data. The student groups present their progress and the challenges facing their process, and then discuss these issues with the class. Drawing on the teacher’s background in business, the constructive method imitates the normal practices of business life more than traditional teaching by lecturing.

The development process has varied slightly over the years as the students have followed business planning, innovation, design, or integrated marketing communication processes depending on the client projects and real-life contexts. However, the students have had a systematic process to follow each year in order to diminish complexity and fuzziness. This process provides guidelines for critical, business-oriented thinking, but leaves freedom for creative thinking in different phases. Figure 1 presents an example of the different phases of a development process. For the project work the students are credited with 10 study points.

Figure 1.

The general development process for students to follow.

Engaging Students in Creative Processes

In earlier courses of “Concept Factory,” creativity merely involved implicit expectations whereas later courses have emphasized creativity by teaching creative methods and bringing in designers or other professionals to challenge the students’ thinking processes. Over the years, creativity has been taken up more explicitly and promoted in every phase of the development process. For example, students have been encouraged to use different ideation methods, engage in user observation, use innovative research techniques to gain customer insight, run customer workshops, develop customer profiles, conduct professional interviews, transfer ideas from one industry to another, carry out benchmarking, use multiple ways to test and evaluate their first concepts, and use pictures and other visual storytelling aids.

A creative thinking process involves task motivation, disciplinary knowledge, cognitive flexibility, and serendipity, all of which involve uncertainty (Titus, 2007). Many individuals are not motivated enough to tackle creative problems, as it takes a lot of time, energy, and effort. In short, it requires persistence and task motivation (Collins & Amabile, 1999; Lehrer, 2012; Titus, 2007). The absence of motivation can lead individuals to give up on their creative efforts and fail to come up with valuable creative outputs. Motivation can be either intrinsic or extrinsic. Intrinsic motivation means engaging with a case primarily for its own sake, because the student finds it interesting, exciting, and personally challenging. Extrinsic motivation in turn involves a situation where students work on a task to earn some reward, for example, study points and high grades (Collins & Amabile, 1999).

In our course, we try to affect the extrinsic motivation directly by choosing interesting and challenging real-life cases, by showing the process of creative problem solving, and by explaining what to do in order to pass with different grades. We try to affect intrinsic motivation indirectly (Collins & Amabile, 1999), as from experience we have noticed that students often find intrinsic motivation after digging into real-life cases and getting customer insight. At that point they seem to gain ownership of the project and knowledge, and the ownership seems to have a positive effect on their intrinsic motivation. Indeed, Stenberg and Lubart (1999) note that creativity does not only require motivation but may also generate it.

Individuals operating in a particular industry, discipline, or area of expertise gain a significant amount of explicit, implicit, and tacit knowledge over time. It is believed that heightened levels of disciplinary knowledge increase the likelihood of producing creative breakthroughs (Titus, 2007). However, it is argued that the biggest problems we need to solve require the expertise of people from different disciplines (Lehrer, 2012). In our course, we enhance disciplinary knowledge in two ways. First, there is business and marketing knowledge that is partly gained through earlier studies. Second, there is case knowledge, that is, knowledge about the industry in which the development project is being carried out. We enhance the contextual understanding by bringing in professionals from the industry and by coaching the students to use different tools to analyze the case and interview industry professionals.

Individuals who consistently use alternative approaches or pathways to solve difficult problems are said to be cognitively flexible in their thinking (Titus, 2007). In other words, they can think outside the box. The opposite of cognitive flexibility is functional fixedness, which means the inability to break free from conventional problem-solving pathways (German & Barrett, 2005). We enhance cognitive flexibility by means such as introducing creative methods and problem-solving methods, helping students combine ideas from different contexts, and assembling cross-functional or culturally diverse teams. We encourage the transfer of knowledge gained from different contexts such as hobbies, part-time jobs, or traveling to the task at hand.

Serendipity refers to the fact that creative success is difficult to plan. In fact, many scientific discoveries, such as penicillin, Viagra, or the microwave oven have seemingly occurred by accident (Titus, 2007). Even though we cannot plan a lucky coincidence, we discuss the role of serendipity in different cases and make this facet more explicit for the students.

The most important aspect of a creative process is that it occurs within an environment of uncertainty. Hence, there are no guarantees that any given process will produce a creative solution or marketing breakthrough (Titus, 2007). In fact, there are barriers to expressing creativity such as fear of failure, fear of doing something different, and fear of taking risks (Anderson, 2006). The students might find it difficult and challenging to bear the uncertainty, as many of them want to make safe decisions in order to pass the course with good or satisfactory marks. Jumping into unknown territory, going to the wild side by letting go of the usual and safe, requires courage and boldness. The spontaneous “letting go” can not only be very frightening but also be an extremely valuable source of creativity (Lehrer, 2012, p. 89). Moreover, the school represents a difficult context for teachers to convince students to spend time and energy to tackle difficult marketing problems without the certainty of reaching a creative and valuable solution (Titus, 2007).

We try to make it easier for students to tolerate uncertainty by planning the development process so that it is less fuzzy and more systematic, creating an atmosphere where students feel secure in expressing their ideas and letting themselves engage in trial-and-error processes, by openly discussing the feeling of uncertainty, and by motivating the students to challenge themselves. For the students, it is valuable to see that other students or student groups are tackling similar kinds of problems. We enhance the knowledge sharing by organizing meetings, discussions, and presentation possibilities for the students.

Table 1 describes the course outline and timetable from the fall of 2012. Next, we will turn to the assessment of creative problem solving in learning outcomes.

Table 1.

An Example of the Course Outline and Timetable (Fall Semester 2012).

Number of week/day/time	Teacher prepares/lectures/workshops	Student activity	Student assignment outside class time
	Negotiate with the case organization and plan the course content, outline, methods, and activities
	Write a syllabus
	Write a business-like project brief for the students
	Prepare self-test materials for the students
Week 38/Tuesday/9-12 (3 hours)	Break the ice—this course is different! Present the introduction to the course	Introduction to the course and working methods, individual learning style test, social communication test, setting the teams, team-building activities	Find background information about the client organization. Prepare questions for the client
	Give individual essay topics to the students (survey, observation, (n)ethnography, interview, visual aids and collages, and focus groups)	Introduction to the client organization, project brief	Start preparing your individual essay on one method of customer inquiry
Week 39/Tuesday/9-12	Introduction to methods to enrich problem formulation (mind map, what if . . ., 5W + H), tools for project planning (e.g., Michanek & Breiler, 2009; Tassoul, 2009)	Visiting the case organization	Finalize the preliminary project plan and submit by Friday. You will receive feedback next Tuesday
		Presentation from the client and interviews	Prepare your individual essay on one method of customer inquiry
		Problem-setting exercises
		How to write a project plan?
Week 40/Tuesday/9-12	Check the project goals and how they fit the project brief. Negotiate with the students and client if necessary	Workshop—gaining customer insight	Discuss with your group what you learned about different methods of gaining customer insight. How would you gain insight for your project? Decide and write a research plan, present it next time. Start digging! (You can always get feedback and guidance by visiting the teacher/s.)
	Vote for the best poster—have a small present (candy) for the winning group	Prepare a poster about your essay topic with others
		Present a poster with the walkabout method
Book a time	Give feedback about the project plans to each group	Feedback
Week 41/Tuesday/9-12	Feedback on research plans and analyzing methods	Present your plan to gain customer insight!	Continue working on gaining customer insight—prepare a presentation for the next time
	Serve coffee to everyone to make the guest lecture more special	How to become a convincing presenter! —A special guest lecture!
Week 42/Tuesday/9-14 (5 hours)	With each group, guide discussion about what has been found out, the validity and reliability of the information, and how all the groups can benefit from each other! Try to create an atmosphere that encourages knowledge sharing, not competition between groups	Knowledge sharing! Present your customer research!
	Presentation about the use of customer profiles	Customer profiling workshop. Presentation about customer profiles in product development and how to prepare them
	Introduce several ideation methods and practice them (me–we–us, dreaming, brainstorming, negative brainstorming, using imaginary places, situations or roles, challenging the self-evident, random word technique, using pictorial cards in ideation; e.g., Michanek & Breiler, 2009; Tassoul, 2009)	From scattered information to holistic concepts—ideation methods and customer-centered concept development
Week 43		Exam week, no class. Voluntary guidance sessions	Continue working on ideation methods and concept development
Week 44/Tuesday/9-12	Give feedback on each concept. Encourage critical thinking by questioning. Avoid direct suggestions	Present three strong concepts	Familiarize yourself with idea evaluation methods and choose how you will evaluate your three concepts. Evaluate them and modify if needed
	Present idea evaluation methods (3i—influence, importance & imaginative, six thinking hats, SWOT, point). (e.g., Michanek & Breiler, 2009; Tassoul, 2009)	Idea evaluation methods	Work on the creative communication plan for your concepts (what you want to say, how you will say it)
			Look through the syllabus again and check all the things you need to do before the end of the course. Reschedule your project plan, if necessary
Week 45/Tuesday/9-12	Discuss concept testing	Present preliminary creative plans for your concepts, explain and motivate your visual choices	Plan and test your concepts. Modify, combine, and develop the final concept
Week 46/Tuesday/9-12	By now the groups are getting tired—motivate and encourage them to push till the end	Present your test results. Get feedback from your peers. Be prepared to modify your work!	Work with action and media plan + budget
Week 47/Tuesday/9-12		Present your action plan, media plan, and budget	Work on your final presentation. What makes your group stand out in the class?
Week 48/Tuesday/9-12		Extra time. Final presentation rehearsal on Friday or next week Monday	Send a teaser about your concept to the members of the jury
Week 49/Tuesday/9-12	Prepare the jury beforehand	“Dancing with the Stars”—final show
	Create an exciting but motivating atmosphere. Have a present for the winning team	Enjoy!
	Serve coffee to make the session more business-like and special
Week 50			Write an evaluation about the project work with your group. Be critical!
			Finalize the final client report. Hand it in by the end of Week 50
Week 51			Hand in the individual learning diaries
Week 52
Weeks 1-2	Read and assess the assignments (final report and all the presentations). Write a review for each student group. Wish them well
	Collect feedback and develop the course further

Analyzing Creative Problem Solving in the Student Teamwork

The data for the study consist of a client-based project that was run in the fall semester. The client for the course was an open-air museum called Stundars in the countryside of western Finland. The museum wanted to attract more families with children to visit their grounds not only once but several times during the summer. The students received a two-page brief explaining the current situation of the museum, the objective of the student work, and resource constraints. Thus, the starting point for all the groups was the same. The course followed the outline presented earlier. The students came up with six distinct final concepts that they both presented orally and reported in writing.

Altogether 25 students participated in the course, of whom 14 were marketing students and 11 communication and language students. There were six cross-functional teams and thus six final reports. The students presented their work several times during the course. The teacher took notes during the presentations and provided feedback to the teams. These notes together with the presentation material serve as data along with the final reports. Furthermore, each student wrote an individual learning diary during the course in order to develop their metacognitive and learning-to-learn skills. The final assessment was based 20% on the outcome, 60% on the process, and 20% on the communication skills and presentation. The learning diaries were assessed and credited for two study points separately.

These six student projects were analyzed in terms of their creative outcome and process. Our interest focuses on what kinds of creative solutions the students came up with and how they reached those outcomes. Thus, our interest lies in both the outcome and the process. The unit of analysis is a teamwork-based group assignment; thus, we do not analyze individual students. The usefulness of the outcome for the client was evaluated by the client. However, we used the individual learning diaries to understand the process of development work.

The outcome was analyzed in terms of its novelty to the market and the targeted customer group (high, medium, low), usefulness to the client (high, medium, low), and the structure of the concept (SOLO levels, the simplicity or complexity of the presented solution, and how well the solution fit the problem). The development process was analyzed by using the elements of the SOLO taxonomy, namely capacity (how much time and energy was used for the solution, both thinking and doing, creative and critical), relating operation (how well the students acquired new data and used them in constructing their solution), and consistency and closure (how the group dealt with inconsistencies and uncertainty and whether they took their time or aimed to close quickly). To demonstrate the analysis, a detailed description of two final concepts and the evaluation of the outcome and development processes are presented in Table 2. Table 3 summarizes the evaluation of the all six different concepts.

Table 2.

Description and Evaluation of the Two Exemplary Cases.

Name of the concept	Description of the final concept	Evaluation of the outcome	Evaluation of the process
Concept 1. “Genuine life”	The final concept integrates several new activities and services in the area. The concept is built on three ideas: being natural, close, and easy to access. It includes both activities targeted at the expected target group and new activities to attract new potential target groups that would generate substantially more income for the museum. The final concept enables wide cooperation between different actors in the region and thus provides potential value for many different parties.	The structure of the outcome is a coherent, integrated whole that provides novelty, usefulness, and value for several parties, namely the client, potential customers, and other service providers in the area, such as farmers. The concept has unexpected potential to generate new business opportunities. The outcome was clearly developed from the students’ own ideas, which they honed on the basis of feedback from the teachers.	In the development process, the team worked eagerly and was motivated to finish the project in an exemplary manner. The development process was explorative, deep, rich, and holistic. The process included sourcing customer data from multiple sources and using that data as bases of idea generation and evaluation. For example, the students designed a little pig mascot that the museum can use both in its marketing and on its premises. The group was brave and able to “let go” in the creative process, welcoming new ideas, testing different ideas, and working hard toward a coherent marketing concept. At the same time they were able to tackle inconsistencies and aim for coherent closure. In their final presentation they suggested one concept that fit the resource limitations given by the client, but left some options open so that a different option can be implemented if the client is interested in further developing the concept. Their use of capacity was very high and the process showed evidence of trial and error, where students both kept their minds open to new ideas but at the same time tried to make sense of the new data and the client’s expectations. This case represents the extended abstract level of the creative problem-solving process.
Concept 6. “TimeAdventure!”	TimeAdventure! is a concept that combines a new cafeteria and Internet-based game for little visitors. The player travels through time and learns how things were done in the past. Afterward the children can dress up in costumes like the ones in the game and walk through the museum area. While the children play the game, the parents can relax and enjoy coffee and special local delicacies. The main focus is to relax, to have a break in the cafeteria.	The idea of combining a larger new cafe and Internet play area is modern but unrealistic for the open-air museum. However, the student group successfully implemented business thinking in a museum context. Unfortunately, their ideas are not well tested, but rely mostly on their own evaluation. The level of novelty is low in the case of both the cafe and the game (the museum already has one on its website). However, the group did well in combining these with the idea of getting children to play outside the cafeteria as well. The structure of the outcome is multistructural. The students combined different independent aspects but were not able to form a coherent whole out of them.	In the development process, the team was able to complete all the assignments. However, their task motivation and capacity were low. The teachers had several discussions with the group to clarify the level of intended learning. Their customer insight was at a shallow level, as they conducted only two interviews. The team used only some tools for creative thinking and few evaluation methods. Their strength was one creative student who was able to present ideas and advertisements that fit the concept, even though the concept itself was not very well structured. The team had a few ideas at the beginning but the students did not put themselves through the creative and critical thinking processes. The process faced difficulties in the data-gathering phases and after that the group had difficulties in focusing on problem solving. The team worked on one idea for a long time and clearly aimed for a quick closure. Eventually they were able to build a concept, but its premises were not well grounded. Their final presentation surprised everyone as it was mostly a well-done video presentation.

Table 3.

Analyzing Creativity in Client-Based Marketing Development Projects.

	Outcome and structure	Capacity and task motivation	Relating operation	Consistency and closure	Assessment (Structure of the Observed Learning Outcomes, SOLO)
Case 1: Genuine life	Novelty: medium	Very high	High amount of new data collected for use in problem solving. Able to bring new insights. Able to use them in problem solving.	Unhurried closure, worked with different options and inconsistencies.	Extended abstract
	Usefulness: very high
	Structure: coherent whole with unexpected enlargements
Case 2: For the whole family!	Novelty: low	High	High amount of new data acquired and used in problem solving. Medium amount used in a solution.	Aimed for closure but worked until they found a coherent whole with a clear core idea.	Relational
	Usefulness: high
	Structure: coherent whole, neat package
Case 3: A moment to myself	Novelty: medium	Very high	High amount of new data collected for use in problem solving. Medium amount used in problem solving.	Unhurried closure, worked with different options and inconsistencies.	Relational/extended abstract
	Usefulness: medium
	Structure: coherent whole, neat package
Case 4: Our [name]	Novelty: medium	High	Medium amount of new data acquired.	Aimed for closure but worked until they found a coherent whole with a clear core idea.	Relational
	Usefulness: medium
	Structure: coherent whole
Case 5: Book your own puzzle!	Novelty: very high	Medium-Low	Low amount of new data acquired.	Aimed for quick closure but worked with inconsistencies.	Unistructural/multistructural
	Usefulness: low
	Structure: multistructural
Case 6: TimeAdventure!	Novelty: low	Low	Really low amount of new data acquired. Used in problem solving, few cues.	Aimed for quick closure, difficulties with working with inconsistencies.	Unistructural/multistructural
	Usefulness: low
	Structure: multistructural

All the student teams came up with case solutions that either provided novelty or were useful for the client. The levels of novelty were not very high; the teams mostly reached a medium level of novelty while still providing value for the client. It seems to be difficult to balance between high novelty, high usefulness, and a holistic concept. The team that clearly strived for high novelty lacked contextual understanding. The teams that strived for high usefulness for the client had to compromise in order to account for the expectation of high novelty. However, we do not think that the students should be punished for striving for usefulness and medium level of novelty. If they understand the contextual demands and can provide novel ideas in a holistic manner, they have reached the relational level of understanding. Those who can combine all have clearly reached the level of extended abstract understanding.

Most student teams reached the multistructural or relational levels. The project brief suggested that the students should work toward an integrated marketing concept with practical phases such as action plans and budgets. Thus, it guided the students to aim for closure and maybe even neglect the extended abstract level. Indeed, the teacher needs to be careful on how to formulate the problem at the beginning of the course. The multistructural and relational levels also serve as a watershed between different solutions. At the multistructural level, the students have not yet reached a holistic understanding of their construct, but on the relational level they have. Indeed, it seems clear that we should aim for the relational level.

The extended abstract level requires the students to question parts of the project brief or to clearly take up a challenge that has not been asked directly. Reaching out for the extended abstract level requires a great deal of courage from the students. In order to push the students further to reach the extended abstract level, the project brief and the problem could be left more open, thereby giving students more latitude. However, the project might then lose something else, such as practical marketing math. Indeed, the teacher needs to balance between these matters.

None of the cases exemplify the prestructural level, either in terms of the outcome or the process. None of the student groups lacked motivation to complete the task. All of them put their mind to the development process and produced real solutions for the client’s problem. Also, all the groups were active and completed the tasks required. However, their performance level during task completion and how well they were able to use that information in the development process differed between the teams.

The evaluation of capacity is difficult if it is restricted only to working memory, inside the student’s head. Students work hard not only in the classroom but outside it as well. The teacher needs to assess what she or he sees and the use of capacity should be evaluated through different tasks in terms of what the students have done and how well. Also, the teacher can sense how far the students have gone out of their comfort zone to work and think in an area of uncertainty. That is why it is suggested that the assessment framework should combine capacity with task motivation.

The level of the process was not constant throughout the project. Indeed, there were differences between different phases and different teams. There were phases in which the groups worked harder and some in which their motivation faltered. Some teams worked really hard at the beginning and served as examples for the rest of the teams. Their example built up social pressure for the others in a positive manner. Some became tired and demotivated and did not do well in some parts, but were able to pull it all together in the end. In our case, the project ran for quite a long time, 4 months, and all the students attended other courses at the same time. Sometimes, teamwork challenges also affected the level of working. Thus, the level of the process might not remain the same during the whole process and there can be imbalance between the phases.

Also, earlier it has been noticed that determining the actual level can be difficult, as it may be somewhere in between the defined levels. Biggs and Collins (1982) identify transformational in-between levels and Chan et al. (2002) include several sublevels, increasing the number of levels to eight. However, we suggest that trying to assess several phases through several aspects at several levels may result in an overly complicated assessment framework. In the end, the teacher should also strive for a concise assessment that both develops students as individuals and treats them fairly.

Table 4 presents the framework for assessing creative problem solving in client-based projects. It is based on assessing both the outcome and the process. The outcome emphasizes the novelty, usefulness, and the coherence (structure) of the solution. The process emphasizes students’ capacity and task motivation, the use of relating operations, and students’ need to aim for consistency and closure.

Table 4.

An Assessment Framework for Assessing Creative Problem Solving in Client-Based Marketing Development Projects.

		Process
Level of understanding	Outcome	Capacity	Relating operation	Consistency and closure
Extended abstract	Innovative, even surprising outcome that generates value for different parties and even adds new novel elements. The structure of the solution builds on a coherent holistic whole.	An integrated and holistic process where the students use creative methods, tools, and techniques in different phases. They show boldness by working in an area of uncertainty, out of their comfort zone.	Several ideas combined, even more abstract aspects combined into the solution.	Unhurried closure, working with inconsistencies, and keeping several options open at an abstract level. Relating options to each other and presenting one of them as only one possible solution.
Relational	A coherent and holistic solution, where different aspects are in balance with each other. The ideas are novel and context-specific. They create value for the market and are useful and applicable for the client.	A clear process in which many new ideas are evaluated and tested. Creative thinking and explicit methods are used in several phases of the process and the choice of methods is contextual. Pushing the limit, working in uncertainty.	The students are able to combine several ideas into a more abstract whole in different phases of the process.	Unhurried closure, working with inconsistencies, and keeping several options open at a practical level. Working in uncertainty. Presents coherent closure.
Multistructural	A solution that includes several novel, useful, and valuable factors, but does not achieve a coherent holistic whole. Some aspects may not fit the whole.	The process is based on few ideas, but the creative techniques are not used to gather ideas or the choices are not motivated or there are no clear criteria for choosing ideas. The ideation phase or visual presentation may be emphasized over critical evaluation.	The students have several ideas that are contextually relevant, but do not connect them to each other. In some cases, there might be a lack of focus or a core idea that would help to conceptualize the whole.	Aiming for unhurried closure, working with inconsistencies, finding it hard to work in a state of uncertainty. Closes before reaching a coherent whole.
Unistructural	A solution that includes some novel, useful, and valuable factors. Based on one or few ideas, lacking relativity and context dependency.	Weak capacity and task motivation. May be based on one idea that has been obtained from the customer or expert. Much of the time, the students seek to find support for their own idea rather than critically reviewing it. Students do not leave their comfort zone.	Students rely on one or few ideas that may not be contextualized for the client.	Aiming for quick closure, difficulties in working with inconsistencies and bearing the uncertainty.
Prestructural	A solution that has no real novel or useful ideas or which does not fit the market or client’s context.	No real capacity used in the process. Lack of task motivation. No creative techniques used, no boldness to leave the comfort zone.	No cues outside customer brief.	Quick closure.

Discussion: Enhancing Creative Problem Solving and Its Assessment

Higher education generally should develop knowledge that is organized structurally at the relational or extended abstract level. These levels mean not only knowing different concepts or facts but also understanding their relationships and contextual constraints. Applying marketing knowledge requires an understanding of several factors at the same time while still being able to solve problems. Thus, marketing education also needs to aim at the relational and extended abstract levels of understanding.

In a marketing classroom, we rarely see real-world marketing breakthroughs (Titus, 2007) or Creativity with “a capital C.” However, for marketers, creativity with a “lowercase c” may be just as important. Creativity in marketing is an ability to think differently and to combine creative unconstrained thinking with analytical, business-oriented critical thinking. It is an ability to understand holistic concepts and how to create value both to the customers and to the company. Sometimes the deep understanding of customer feelings and fantasies can produce minor changes to the current marketing and still result in market success.

Teaching and assessment of creativity and creative problem solving should focus on not only the outcome but the whole process. For the students it is an extremely valuable experience to feel that they are in a state of uncertainty but are still able to accomplish a solid solution by working hard. Indeed, successful experiences build confidence (Nickerson, 1999). However, some students need rules and examples, encouragement, and motivation to be able to push through the uncertainty of not knowing what to do next. Thus, the course structure should both give a structure for the development process and leave freedom for the students (see also, e.g., Nickerson, 1999). The example we have given in this article balances between logical and analytical critical thinking (a path to follow) and unconstrained creative thinking (creative methods and freedom within phases).

Many of the business courses at our university are based on lectures, discussion, exams, and/or group work. They aim at developing the students’ knowledge and analytical thinking skills. Attending a course that involves creative problem solving is simultaneously inspiring and frightening for many students. Many of them do not know how to start working. When the students are introduced to tools for creative thinking in different phases of the development projects, they are more eager to use them. Creative tools may be introduced in several ways, such as teaching some of them, asking the students to dig into some of them, bringing in professionals from creative fields, or structuring the course around multifunctional and/or international teams. We have introduced such creative tools in the problem formulation, customer insight, idea generation, and evaluation phases (see Table 1). The students become familiar with the way that this course is taught and by the end of the course they harness their own creativity by making advertisements and professional final presentations.

The background development process is used to assess the student work in different phases. The teacher needs to decide on the background process and how open or closed it is. The process may be different in different client projects, depending on whether the challenge involves, for example, business development, concept development, brand building, or event marketing. In our case we used a process for concept development and structured the course based on that process. It gave structure for the students while leaving room for creativity. A less organized process provides even more latitude, while a more organized process gives more structure. The background process and the course structure also help the teacher provide formative feedback for the students and thus help them in reaching higher levels of the taxonomy. The process also helps the teacher to conclude the summative feedback.

The assessment framework developed and contextualized in this article aims to make assessment easier for the teacher and more open and understandable for the students. In this way the students understand where they should focus the most effort. The assessment framework and its use should be discussed with the students to make sure the students understand the criteria (Rust et al., 2003). The assessment of creativity is often just one part of assessment; other aspects must be assessed as well, such as teamwork skills, communication skills, and critical reflection of own learning.

Assessment is a challenging task. It is subjective in nature, which poses many challenges. For example, there are cases where the teacher likes a certain group and its work and really wants to give these students a good grade, even if they cannot reach the relational level. On the other hand, one of the groups might be arrogant—although the students in the group produce great work, they get on the teacher’s nerves by questioning him or her and working in unexpected ways. The teacher needs to learn to tolerate the unforeseen and let the students follow their own creative paths. Also, the use of an assessment framework may lighten the burden and help the teacher to maintain objectivity.

Limitations, Future Research Directions, and Conclusions

While developing the assessment framework, we have emphasized the teamwork of the students, that is, group creativity. We have not taken into account the differences in creativity between people, creativity as a personal trait, nor have we aimed to assess individual creativity. However, from the analysis, it can be seen how students with more creative personalities can make a great contribution to the teamwork. This difference opens a new path for future research—the study of the role of individual creativity within group work.

Also, we have looked into the SOLO taxonomy from a master’s level creative problem-solving activity point of view, where the aim has been to develop coherent marketing concepts in a client-based project. However, the SOLO taxonomy can be contextualized to different marketing topics. Indeed, the different levels of understanding can be seen in the bachelor’s level as well. Also, marketing educators can find the framework useful to be applied in different teaching methods such as case method.

Creativity and creative problem solving are the seeds for business and thus need be taught in marketing curriculums. Also, creative problem solving needs to be assessed because the assessment practices signal to the students what they need to learn. The constructive alignment of teaching is also strengthened. However, assessing creativity and creative problem solving is a challenging task, and there have been no assessment frameworks that have been contextualized for creative problem solving in marketing education.

We have analyzed and contextualized SOLO taxonomy for assessing creative problem solving in client-based marketing development projects. We have combined both creative unconstrained thinking with business-oriented critical thinking in different phases of development projects. When planning for marketing courses we should aim at relational and extended abstract levels of understanding. The assessment framework provides a tool for teachers to plan and structure courses, to openly discuss the assessment criteria with the students, and to assess the learning.

When assessing creative problem solving, it is not enough to assess the outcome of the project but also the process, that is, how the students have reached the outcome. Even though in real life, after graduation, the outcome itself may be the most important goal, for learning purposes the process, the route map, becomes very important. The process can be enhanced to spur students to reach new heights in creativity. The process is where the teacher sees how the students’ creative and critical thinking have evolved and how they have reached their conclusions. Thus, the assessment framework includes both the outcome and the process.

Teaching creative problem solving in marketing is challenging—but at the same time inspiring and fun. Young students may have the craziest ideas that challenge the client organizations’ thinking. By letting these ideas flower, we can both challenge the old way of thinking and aim at new ways to innovate.

Footnotes

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

References

Amabile

(1998). How to kill creativity. Harvard Business Review, September-October, 77-87.

Anderson

(2006). Building confidence in creativity: MBA students. Marketing Education Review, 16(1), 91-96.

Biggs

(1996). Enhancing teaching through constructive alignment. Higher Education, 32, 347-364.

Biggs

Tang

(2007). Teaching for quality learning at university (3rd ed.). Berkshire, England: Society for Research in Higher Education and Open University Press.

Biggs

J. B.

Collins

K. F.

(1982). Evaluating the quality of learning: The SOLO taxonomy. New York, NY: Academic Press.

Brown

Bull

Pendlebury

(1997). Assessing student learning in higher education. London, England: Routledge.

Chan

C. C.

Tsui

M. S.

Hong

J. H.

(2002). Applying the Structure of the Observed Learning Outcomes (SOLO) taxonomy on student’s learning outcomes: An empirical study. Assessment & Evaluation in Higher Education, 27, 511-527.

Collins

M. A.

Amabile

T. M.

(1999). Motivation and creativity. In Stenberg

R. J.

(Ed.), Handbook of creativity (pp. 297-312). New York, NY: Cambridge University Press.

Elton

(2005). Designing assessment for creativity: An imaginative curriculum guide. Retrieved from http://www.heacademy.ac.uk

10.

German

T. P.

Barrett

H. C.

(2005). Functional fixedness in a technologically sparse culture. Psychological Science, 16, 1-5.

11.

Jenkins

J. O.

(2010). A multi-faceted formative assessment approach: Better recognising the learning needs of students. Assessment & Evaluation in Higher Education, 35, 565-576.

12.

Kolb

D. A.

(1984). Experiential learning: Experience as a source of learning and development. Engelwood Cliffs, NJ: Prentice Hall.

13.

Lehrer

(2012). Imagine: How creativity works. New York, NY: Houghton Mifflin Harcourt.

14.

Leung

C. F.

(2000). Assessment for learning: Using SOLO taxonomy to measure design performance of design & technology students. International Journal of Technology and Design Education, 10, 149-161.

15.

Levitt

(1983). The marketing imagination. New York, NY: Free Press.

16.

Levitt

(2002). Creativity is not enough. Harvard Business Review, August, 137-144. (Original work published 1963)

17.

Lucas

Mladenovic

(2009). The identification of variation in students’ understandings of disciplinary concepts: The application of the SOLO taxonomy within introductory accounting. Higher Education, 58, 257-283.

18.

Lundberg

(2004). Student and teacher experiences of assessing different levels of understanding. Assessment & Evaluation in Higher Education, 29, 323-333.

19.

Lunsford

D. A.

(1990). Developing creative problem solving skills in marketing case analysis. Marketing Education Review, 1(3), 62-69.

20.

McCorkle

D. E.

Payan

J. M.

Reardon

Kling

N. D.

(2007). Perceptions and reality. Creativity in the marketing classroom. Journal of Marketing Education, 29, 254-261.

21.

Michanek

Breiler

(2009). The idea agent: A handbook on creative processes. Malmö, Sweden: Idélaboratoriet.

22.

Nickerson

R. S.

(1999). Enhancing creativity. In Stenberg

R. J.

(Ed.), Handbook of creativity (pp. 392-430). New York, NY: Cambridge University Press.

23.

Nicol

D. J.

Macfarlane-Dick

(2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31, 199-218.

24.

Ramocki

S. P.

(1996). Developing creative marketing graduates. Marketing Education Review, 6(1), 47-53.

25.

Rust

Price

O’Donovan

(2003). Improving students’ learning by developing their understanding of assessment criteria and processes. Assessment & Evaluation in Higher Education, 28, 147-164.

26.

Schön

D. A.

(1983). The reflective practitioner: How professionals think in action. London, England: Ashgate.

27.

Stenberg

R. J.

Lubart

T. I.

(1999). The concept of creativity: Prospects and paradigms. In Stenberg

R. J.

(Ed.), Handbook of creativity (pp. 3-15). New York, NY: Cambridge University Press.

28.

Strauss

(2011). Marketing capstone models The Apprentice television show with client-sponsored projects. Journal of Marketing Education, 33, 312-325.

29.

Taras

(2008). Summative and formative assessment: Perceptions and realities. Active Learning in Higher Education, 9, 172-192.

30.

Tassoul

(2009). Creative facilitation. Delft, Netherlands: VSSD.

31.

Titus

P. A.

(2000). Marketing and creative problem-solving process. Journal of Marketing Education, 22, 225-235.

32.

Titus

P. A.

(2007). Applied creativity: The creative marketing breakthrough model. Journal of Marketing Education, 29, 262-272.

33.

Vogel

C. M.

Cagan

Boatwright

(2005). The design of things to come. How ordinary people create extraordinary products. Upper Saddle River, NJ: Wharton School.

34.

Wingate

(2010). The impact of formative feedback on the development of academic writing. Assessment & Evaluation in Higher Education, 35, 519-533.