Exploring Undergraduate Attainment of Sustainability Competencies

Introduction

With the emergence of sustainability as an academic field has come a focus on defining the competencies that will enable graduates to promote sustainability. ¹ Sustainability competencies can be defined as the “complexes of knowledge, skills, and attitudes that enable successful task performance and problem solving with respect to real-world sustainability problems, challenges, and opportunities” (p. 204). ² Addressing complex sustainability challenges will require a unique set of competencies; without identifying them, our ability to ensure that new graduates can meet the needs of a changing world is limited.^3,4 A focus on competencies has value for sustainability students, educational programs, and prospective employers. In higher education generally, students may not feel equipped to describe the knowledge, attitudes, and skills they have developed through a degree program. ⁵ Prospective employers may not understand what knowledge and skills can be expected of recent graduates. ⁵ These challenges are perhaps greater for new fields, such as sustainability. As such, elucidating and communicating sustainability competencies is of the utmost importance.

There have been multiple attempts to solicit, summarize, and describe a set of sustainability competencies.^2,6,7 Scholars are beginning to coalesce around sustainability competencies named by Wiek, Withycombe, and Redman ² (systems thinking, anticipatory, normative, strategic, and interpersonal). (See Table 1.) Since the development of their framework, several others have further adapted and refined similar sets of sustainability competencies.^7–10 For example, Glasser and Hirsh ¹¹ suggest that existing competencies are not grounded in knowledge about the state of the planet; nor are they likely to foster the type of transformative change needed to address complex sustainability challenges.

Similarly, a wide range of assessment techniques have been used to measure sustainability competencies: course evaluation comments, ¹² student self-assessments, ⁴ student responses to questions about case studies, ¹³ focus group discussions, ³ quantitative self-reflection questionnaires, ¹⁴ and assessments of student actions compared to self-reflections. ¹⁵ These assessments have focused on different sustainability competencies, making comparison difficult. Furthermore, few efforts have been made to compare different assessment methods with the same group of students.

The authors are particularly interested in the discussion of what competencies graduates of sustainability programs should develop and how to assess them because of a new Sustainability Dual Major (SDM) at the University of New Hampshire, a public research university located in New England. The SDM was established in January 2016, with the stated mission to prepare students to develop real-world solutions to environmental, economic, and social challenges. Students must pair their dual major in sustainability with a primary major from any department on campus.

SDM students take three core courses in sustainability (introduction, methods, and capstone) and five electives from a pre-approved list of courses offered from departments throughout the university. The introduction class explores the definitions and history of the sustainability concept and uses a sustainability lens to understand grand challenges such as climate, energy, and food systems. The methods course focuses on how stakeholder groups collaborate to solve sustainability challenges and includes field-based case study exploration. The capstone course requires students to work in interdisciplinary teams with external mentors on projects associated with ongoing efforts to address local sustainability challenges. There are currently 65 approved electives from disciplines across campus, and students must choose at least one natural science elective and one social science or humanities elective. The objectives of this research are threefold: 1.) to explore the degree to which the project-based capstone course advances students' sustainability competencies; 2.) to identify the extent to which graduating students demonstrate sustainability key competencies; and 3.) to compare and assess two methods of measuring student mastery of sustainability competencies.

Methods

A case study is an appropriate methodological approach when the study focuses on a phenomenon situated in a real-life context. ¹⁶ Because the operationalization of sustainability competencies is context dependent, shaped by the social, political, and environmental realities of the place in which they are enacted, case studies are especially relevant to exploring sustainability education efforts. ¹⁷ In order to address the third research objective—to compare and assess different methods of measuring student mastery of sustainability competencies—this case study draws on both a quantitative multiple-choice survey and a qualitative analysis of student blog assignments.

Results

For the survey, students correctly answered 64 percent (score=13.4 out of 21) and 61 percent (score=12.7) of the questions pre- and post-course, respectively. Scores were not significantly different pre- and post-course (p=0.37). Pre-course, students performed most competently in problem solving and worst in anticipatory. (See Table 2.) Post-course, anticipatory competency remained the lowest-scoring competency, whereas normative replaced problem solving as the highest-scoring competency. Before and after the course, the top three competencies were normative, strategic, and problem solving; the lowest were systems and anticipatory competency.

In their blog posts, students demonstrated the highest mastery of the interpersonal and systems thinking competencies, and lowest mastery of the anticipatory competency. (See Table 3.) The following quotes are representative of the average score for each competency.

Systems thinking had an average score of 2.4. An example of a quote that was scored as a 3 comes from a student reflecting on an elective that focused on systems analysis:

Systems […have] points of pressure that require perturbation, chaos and disruption to change the entire system.…Although some people may not support China in their decision to not responsibly continue to recycle plastic…we needed the change, otherwise it will never leave the production cycle.…So, in order to improve our waste minimization, purchasing decisions will be made in favor of compostable plastics.…In systems thinking this is known as chaos followed by reorganization.

Anticipatory competency had the lowest level of mastery, with an average score of 0.7 across all students. An example is a sentence coded as a 1 in which the student simply mentioned a relevant concept: “The … project … analyzed the ways in which coastal NH would be harmed by sea level rise and coastal storms in the near future.”

Normative competency received an average score of 1.6. A representative quote coded as a 2 comes from a student who is discussing landowner actions in coastal New Hampshire:

Finding a balance between enjoyment and conservation is one of the struggles within sustainability… destroying the dunes to build a fire pit is an obviously ill-advised idea, but other people think otherwise. This just exemplifies the different perspectives that everybody has.

Similarly, strategic competency had an average score of 1.5. A student describing an environmental economics elective received a score of 2 for implying related concepts without explicit discussion of specific ideas:

We also performed a mock UN summit in our labs.…By having each student represent a certain country or climate stance, we learned the ins and outs of global climate studies and negotiations, while also experiencing first-hand why it is so difficult to make serious changes to current environmental and economic models.

Interpersonal competency achieved an average score of 2.6. One student's quote, coded as a 3, describes a field trip during which the class learned about citizen science:

I enjoyed this field trip because it taught me how people from different backgrounds can work together toward a purpose-driven goal. … Our class concluded that transdisciplinary data collection is an integral part of sustainability because it involves the greater public, thus narrowing the knowledge gap between citizens and researchers, and because it creates an environment where citizens know they are making a contribution to science. Achieving sustainability inherently requires collective efforts from individuals from all disciplines, all age groups, and all socioeconomic backgrounds.

In addition to student statements coded to each of the competencies, two additional themes emerged: knowledge of grand challenges and sustainability in practice. The statements reflecting knowledge of grand challenges were those that demonstrated more (or less) understanding of the facts, opinions, complexities, and strategies related to specific complex problems, such as climate change, waste, water quality and quantity, poverty, biodiversity loss, and food systems. While occasionally these statements could be coded within other competencies (e.g., the systems or normative aspects of the problems), there was no explicit place to note the degree to which the students understood the sustainability problems writ large.

Additionally, some students discussed their newfound ability to act based on their acquired knowledge and skills; one student called this “sustainability in practice.” These statements focused on applying knowledge and skills to action in real-life situations, on their potential roles (personally or professionally) in tackling sustainability challenges, and on their ability to persevere despite frustrations inherent in working on complex issues. These comments could often be coded as indicating ultimate mastery within each of the specific competencies. For example, one student discussed a semester-long group project in which they learned the perseverance needed to work with others on sustainability challenges:

It was extremely difficult and at times seemed impossible. It taught me how to work with a group and how to overcome obstacles. Although it was far from perfect, it was a nice feeling to work with so many people on one common cause, just like CAW [Coastal Adaptation Workgroup] does. It was the hands-on experience that sustainability work requires.

This student was indicating a perceived ability to practice interpersonal competence in an on-the-ground work situation.

Discussion

In this case study, there was no significant difference in the students' competency scores from pre- to post-capstone course. While it is possible that students did not improve mastery of any competencies during their capstone course, it is doubtful that is the sole explanation based on the authors' interactions with this group of students throughout the semester. It is more likely that the survey questions were not fine-tuned enough to capture smaller increases in competency development over a single semester. The survey might be more appropriate to use at the programmatic level, to test students at the start of the first introductory-level course and then again at the end of their capstone experience.

Both quantitative surveys and qualitative analysis of blog posts indicated that students had not mastered anticipatory competency by graduation. This result mirrors the findings of Trencher and colleagues, who found low development of anticipatory competency across 14 graduate-level programs in sustainability ¹⁸ globally. For the other competencies, the level of mastery was dependent on the method by which they were assessed. These results suggest that the method of assessment can significantly influence results and demonstrate the need to focus more efforts on developing accurate and precise competency assessment tools.

The discrepancy in rankings between the survey and blogs is likely attributable to multiple factors. One of the more interesting possibilities is that the survey and the blog analysis tested for different levels of competency mastery. For example, the survey questions related to normative competence focused on simple conceptual understanding (e.g., the meaning of intergenerational equity), whereas for the blogs, students were expected to demonstrate conceptual understanding as well as critical analysis to receive high scores for normative competence; for example: explicitly recognizing multiple values at play, identifying trade-offs across ethical dimensions, or empowering marginalized voices. Thus, it was easier to score well in normative competency in the survey than in the blog analysis. The work of Wiek et al. merits further study as it explores the degree to which sustainability competencies should be mastered at novice, intermediate, and advanced levels, and then tailoring assessment methods to match desired outcomes. ¹⁹

In comparing the two assessment methods, neither seems ideal for measuring student mastery of competencies. Testing for mastery in a multiple-choice format requires specific operationalizations of the competencies with associated terms and concepts. For example, the survey questions utilized some terminology that was not familiar to the students (e.g., distinguishing between coherent, plausible, probable, and desirable scenarios in the anticipatory competency questions). Using such specific terminology in standard competency tests could lead to faculty teaching to the exam. If the core sustainability courses do not teach these terms, are students ill-prepared to assess possible future scenarios? The authors do not believe so, but pose this question to readers: Should we better articulate what specific knowledge, skills, and actions should be known to demonstrate each competency?

In addition, qualitative coding of blog entries was less than ideal. Because the blog post assignments did not ask students to reflect on concepts related to each competency, the lack of discussion about a competency (e.g., anticipatory competency) may mean that those concepts were not particularly noteworthy in a particular class, rather than that students had poor understanding of the concepts.

On the other hand, the open-ended nature of the blogs did allow two themes to emerge: 1.) sustainability grand challenges, and 2.) sustainability in practice. Students should have knowledge of the major sustainability grand challenges such as climate change, food systems, water quality and quantity, waste, and poverty. Accurate knowledge of the problems, positions, and possibilities of the challenges themselves is foundational to effective sustainability action. Indeed, it is plausible that a student could demonstrate mastery of the competencies described in Table 1 without grounding in any of the sustainability grand challenges. While some of the early literature on sustainability capabilities and competencies includes knowledge of sustainability debates and issues, it appears that these concepts have become less prominent over time.^6,20 More recently, some scholars, such as Glasser and Hirsh ¹¹ and Mintz and Tal ¹ have suggested a competency termed “knowledge of state of the planet,” i.e., knowledge of at least one major sustainability issue that underlies sustainability education. Knowledge of grand challenges is not a competency in itself, but a foundational knowledge that cuts across all sustainability competencies and includes the biophysical, social, and political context of the complex challenges within which students will be working.

Second, due to the unique use-inspired nature of sustainability science, students must be able to demonstrate what one student called, “sustainability in practice.” Presumably, ultimate mastery of each competency culminates in action. Thus, additional focus should be placed on determining how assessment methods can measure the ability of students to act for sustainability. Competency assessment often stops at the conceptual level (including within this study), failing to analyze the association between conceptual mastery of each competency and the ability to act for sustainability.

Implications and Conclusion

This study reports on a modest assessment of competency achievement of Sustainability Dual Majors at the University of New Hampshire. There are three primary implications of this research. First, there is a growing movement to better define competencies for sustainability education programs. While competencies should remain ambiguous enough to be shaped to the context in which they are being taught, the authors contend that there should also be attempts to better define the subtopics' relevant concepts and appropriate levels of mastery for each competence. Related, the knowledge of sustainability grand challenges should be made more explicit as foundational knowledge in the sustainability competencies literature. If students do not have accurate knowledge about the challenges on which they will be working, their ability to act competently will be limited. While sustainability scholars may already perceive this to be important knowledge, if it is not explicitly stated in the competency literature, it may not be assessed, and it can lose its importance.

Second, additional comparative studies of assessment methods should be conducted to better understand the appropriate use of each one. Which tools are sensitive enough to be used at the level of courses versus programs? The authors suggest specifically exploring self-assessment. An approach from the field of critical pedagogy, self-assessment also has the virtue of being a competency or professional skill in itself. ⁴ Galt et al. provide an example of implementing this approach using a detailed, ungraded self-assessment rubric and graded reflective essays to evaluate students' learning. ⁴ Despite not being provided with a self-reflection guide in their own courses, SDM students often volunteered some self-assessment in their blog posts. Many students' self-reflections showed that they at least perceived to have improved their competencies through the course, especially related to the practice of interpersonal competencies. Self-assessment questions could be framed at the programmatic level as well as at the course level.

Assessing mastery of sustainability competencies provides a window into larger questions about how, as educators in the field of sustainability, we can adapt our approaches and continuously improve our efforts to provide students with sustainability knowledge, skills, attitudes, and values. Ensuring that our students are set up for success in their careers and lives as promoters of sustainability may be the most powerful contribution we make to the field.