Student Perceptions in Homeland Security and Emergency Management Education: Experiential Learning Survey

Review of the Literature

Students in HSEM programs serve at all levels of government and business, managing and responding to incidents around the United States. Disastrous incidents such as the 9/11 terrorist attacks, Hurricane Katrina, and others have shown the potential consequences of failures of judgment, decision-making, and capability (Collins & Peerbolte, 2011; McCreight, 2014; Walker, 2006). Several evaluations of U.S. emergency manager professional concerns as well as higher education HSEM programs have assessed critical thinking skills, curriculum, and methodologies. These have highlighted the need for the inclusion of elements such as critical thinking, analysis of complex situations, assessment of problems, and collaborative problem solving, all essential elements of experiential learning in education (Collins & Peerbolte, 2011; McCreight, 2009, 2014; Stewart & Vocino, 2013).

As Klein and Riordan (2011) noted in their analysis of a professional development for teachers that involved experiential learning, Knowles’ (1980) discussion of andragogy as problem-centered learning for adults aligns with Kolb’s active experimentation within learning. Applying concepts and previous experience through real-world applications in the learning process enables adult learners to leverage their previous knowledge and experiences while focusing on the particular problems they face in their professional journeys. Assessing andragogy within fields such as management, criminal justice, and law enforcement, Chan (2010) described the opportunity for instructors to target student needs and interests by identifying real-world problems that facilitate instruction centered around student objectives for learning that build their competencies and skills for the relevant profession. An andragogical approach aligns well with HSEM higher education, with its inclusion of learner engagement in the process, focus on real-world problems applicable to learners, and inclusion of experience in constructing knowledge while developing the ability to apply needed skills (Chan, 2010).

Researchers have called for greater insight to the connections between theory and practice for these learning experiences, including the context and environment of student experiences along with a conceptual understanding of experiential learning and its applications to education (Mullins, 2014; Seaman, Brown, & Quay, 2017). This includes the psychometric tradition within experiential learning described by Seaman et al, which considers new potential methodologies and theories for examining and explaining learning in an experiential context. Others have looked for a neurological perspective on experiential education, incorporating the prioritization of learning and experiences for models that integrate reflection, and expansion of understanding in experiential learning with student engagement as well as context in new models (Schenck & Cruickshank, 2015). These concepts provide an impetus for researchers in experiential education to identify potential tools for the evaluation of experiential learning within expanded quantitative and qualitative research, more deeply examining the philosophical underpinnings and outcomes of the student learning process (Roberts, 2002; Schenck & Cruickshank, 2015).

HSEM programs face the challenge of engaging students in a variety of learning contexts while enabling this theory-to-practice transition as well as knowledge and skill gains. Experiential learning, with its emphasis on authentic learning exercises through real-world problems, is generally assessed as ideal for complex fields with ill-structured problems such as HSEM (Koury et al., 2009; Zavrel, 2015). Through context and reflection, experiential learning methods enable students to consider information within a larger framework of contingencies and effects, apply decision-making skills, and engage critical thinking (Prince & Felder, 2006; Yadav, Vinh, Shaver, Meckl, & Firebaugh, 2014). In a U.S. study of nationwide academic institutions offering emergency management programs, Darlington (2008) noted that HSEM graduate programs needed to include real-world problem solving integrated into their curricula. Similarly, McCreight (2009, 2014) pointed out that it was essential for HSEM education to include experiential practices that parallel the complex real-world contingencies that students must manage as HSEM professionals.

Kolb’s experiential learning theory forms the foundation for the experiential learning opportunities commonly employed in HSEM higher education to meet this need of higher-ordered thinking. For example, Kapucu and Knox (2011) concluded that most national HSEM programs include experiential opportunities in the curriculum, with projects as varied as developing organizational policy to designing and executing emergency exercises. This aligns with the goals of HSEM education by connecting theoretical knowledge to the application of real-world context in the practical domain.

Purpose

The purpose of this research was to apply the ELS tool as a pilot study in determining student perceptions of the value and contribution of experiential learning methods in HSEM higher education. It was intended to measure student perceptions of the usefulness and application of experiential learning while also examining potential relevance and its contribution of an authentic learning environment to student experiences within the studied HSEM program. By piloting the use of the ELS in HSEM education, this research further sought to assess its potential use for a broader application to quantitative research in experiential and HSEM education.

Method

This quantitative study examined student perceptions of experiential learning in a graduate HSEM capstone course at a major research university’s Master of Arts in Homeland Security program. The face-to-face course included multiple experiential opportunities that all course students participated in, including an emergency management functional exercise, case-based learning, and simulation/role-playing. The descriptive study used a census sampling strategy, with surveys available to and completed by all students in the course. The ELS, developed and tested for reliability and validity in examining experiential education for social work students, was used with the permission of the authors (Clem et al., 2014).

After approval by the university Institutional Review Board, the ELS was provided in an anonymous format to course participants (university name withheld for confidentiality reasons). The researcher was not the course instructor. Participants were all graduate students associated with the Master of Arts in Homeland Security program (N = 33; 40% female). Other specific demographic information, such as the number of years’ experience in the HSEM field, was not collected for this study, a limitation of the results.

Clem et al. (2014) developed and tested the ELS as a tool to expand the assessment of experiential learning in the field of social work higher education. The 7-point Likert-type-style survey with 28 questions evaluates student perceptions of experiential learning. The four measures included in the ELS are environmental authenticity, active learning, relevance, and utility, each important measures of effectiveness for experiential learning in conveying the knowledge and skills needed for complex fields such as HSEM (Clem et al., 2014). The ELS was intended by the developers to support instructional design in social work higher education, but it also has the potential to fill the larger need for quantitative tools for broadening experiential learning research in other fields such as HSEM.

Results

The results of this study represent the four scales of the ELS, environmental authenticity, active learning, relevance, and utility, and are shown using the median, the measure of central tendency most appropriate for the ordinal measures of the survey (Muijs, 2011). The ELS itself is a Likert-type-style survey, with answers ranging from strongly disagree to strongly agree. An exact determination of participant perceptions of the distance between each scale item could not be determined for student responses due to the nature of the ELS data. While there is debate within the statistics and research community as to the ordinal/interval nature of the Likert-type-style scale, educational research as well as statistics guidance such as Creswell (2012) and Muijs (2011) refer to scales such as this as ordinal data and recommend the median as the measure of central tendency. This recommendation as well as the fact that this research that did not include participant group comparisons made the median the most suitable measure for analysis.

Scales are measured by summing the individual responses within each category (Clem et al., 2014) and may also be interpreted as individual queries. When scoring, questions 3, 9, 15, 23, and 27 were reverse coded for consistency with a rubric that has a high level of perceived value matching high scores for the learning experience. The results of the survey information are reported in Table 1. Scales of environmental authenticity, active learning, relevance, and utility each yielded high scores, indicating high student perceptions of the value of experiential learning within the course context. While the range for some answers indicate some variability, the high median scores indicate a consistent high perception of the measured indicators, as well as a global perceived value to experiential learning in this context.

OPEN IN VIEWER

Table 1.

Scale Results for HSEM Student Perceptions of Experiential Learning as Measured by the ELS.

	Median	Range
The setting where I learn helps me understand the material better	6	6
I expect real-world problems to come up during this learning experience	7	5
The environment I learn in does not enhance the learning experience_RECODE	7	6
The learning experience requires me to interact with people other than students and teachers	6	6
I expect to return to an environment similar to the one where this learning experience occurs	6	5
ENVIRONMENT AUTHENTICITY SCALE (potential 0-35)	32
I am stimulated by what I am learning	6	2
The learning experience requires me to do more than just listen	6	4
The learning experience is presented to me in a challenging way	6	3
I find this learning experience boring_RECODE	7	4
I feel like I am an active part of the learning experience	6	4
The learning experience requires me to really think about the information.	6	4
I am emotionally invested in this experience	6	3
ACTIVE LEARNING SCALE (potential 0-49)	43
I care about the information I am being taught	7	2
The learning experience makes sense to me	7	2
This learning experience has nothing to do with me_RECODE	7	5
This learning experience is enjoyable to me	6	4
I can identify with the learning experience	6	4
This learning experience is applicable to me and my interests	6	3
My educator encourages me to share my ideas and past experiences	6	5
This learning experience falls in line with my interests	7	2
I can think of tangible ways to put this learning experience into future practice	6	3
RELEVANCE SCALE (potential 0-63)	58
This learning experience will help me do my job better	6	3
This learning experience will not be useful to me in the future_RECODE	6	6
I will continue to use what I am being taught after this learning experience has ended	6	3
I can see value in this learning experience	7	3
I believe this learning experience has prepared me for other experiences	6	4
I can see myself using this learning experience in the future	7	5
I doubt I will ever use this learning experience again_RECODE	7	5
UTILITY SCALE (potential 0-49)	45

Note. HSEM = Homeland Security and Emergency Management; ELS = experiential learning survey.

The ELS was originally developed for the evaluation of experiential learning with social work students, so analyses of internal consistency reliability for this application in HSEM higher education were performed for each of the four scales using Cronbach’s alpha. These results, shown in Table 2, reveal an overall Cronbach’s alpha of .947, which exceeds the threshold for internal consistency reliability. The lower alpha coefficient of the environmental authenticity scale (.688) was similar to the original study’s slightly lower alpha score of .680 for this scale (Clem et al., 2014, p. 497). The internal consistency reliability exceeded the threshold for active learning, relevance, and utility, indicating a strong internal consistency for the ELS application.

OPEN IN VIEWER

Table 2.

Reliability Analyses.

Overall ELS	.947
Environmental authenticity	.688
Active	.898
Relevance	.917
Utility	.837

Note. ELS = experiential learning survey.

Subscale correlations were also performed for the ELS subscales of environmental authenticity, active learning, relevance, and utility (Table 3). Considering the potential correlation of environmental authenticity and relevance to student learning, a goal of experiential learning, Spearman’s rho was calculated. This revealed a strong positive correlation between environmental authenticity and relevance, which was statistically significant, rs = 0.592 and p < .001. The strongest correlation for the subscales in the ELS in this HSEM survey was between active learning and relevance (0.881) and relevance and utility (0.732). With a statistical significance of p < .05, each subscale met the minimum for a statistically significant relationship with the others. Spearman’s rho was used for correlation instead of factor analysis due to difficulties presented in data analysis resulting from the overall small sample size of this pilot study.

OPEN IN VIEWER

Table 3.

Subscale Correlations for ELS.

Subscale	Environment	Active	Relevance	Utility
Environment
Active learning	.528
Relevance	.592	.881
Utility	.600	.613	.732

Note. ELS = experiential learning survey.

Discussion

The ELS is a Likert-type-style 28-question survey that asks students to measure their own perception of experiential learning’s value in a particular setting. In this application, graduate HSEM students were surveyed as part of a capstone course which included several different experiential activities. As designed by Clem et al. (2014), the higher the score within the subscales, as well as the subscales combined, indicates a higher perceived value for experiential learning. This study’s results indicate that students perceived a high overall value for experiential learning, suggesting that they believed the experience was relevant to real-world applications and would be useful in future experiences. The ELS survey results also showed student interest in the activity and the learning process, which corresponds with established research on experiential learning (Mesny, 2013), HSEM experiential learning (Cozine, 2015), and previous applications of the ELS instrument (Clem et al., 2014; Hefley & Thouin, 2016).

The ELS was originally developed and tested for use in evaluating experiential learning for social work students (Clem et al., 2014) and has been further tested with Agile project management students (Hefley & Thouin, 2016). Considering its application for HSEM students, the internal consistency reliability was evaluated and exceeded the threshold for active learning, relevance, and utility, indicating a strong internal consistency. The environmental authenticity approached but did not meet the threshold for internal consistency reliability. However, the results of this survey were similar to that of the original developers and indicate that it may be useful to re-evaluate this portion of the ELS for further development with other student populations.

Furthermore, the ELS results for each subscale strongly correlate to one another, with environmental authenticity, active learning, relevance, and utility each showing a statistically significant correlation. The ELS results indicate the success of the capstone’s experiential learning activities for HSEM education aspirations, enabling students to consider contingencies while applying decision-making skills. As an evaluative tool for quantitative research in experiential learning, the ELS has also now been tested in two fields other than the original social work premise and has supported the original developer’s findings in each (Clem et al., 2014; Hefley & Thouin, 2016).

This study was a pilot, and the results are limited in application by the small size of the sample (n = 33). While the results of this specific study are indicative of the potential use of experiential education in graduate HSEM programs, it was a limited pilot and is not necessarily directly comparable or generalizable to other programs. Beyond that, the ELS has limitations in its application for HSEM higher education, as it does not include student perceptions of the application of concepts in problem solving, or account for specific types of experiential learning and potential subsequent differences in student perceptions. This pilot study of an HSEM program also does not necessarily indicate the applicability of the ELS to the evaluations of other disciplines.

Implications of Research

Opportunities exist for the development and use of quantitative assessment tools to expand the research-based information available to HSEM program developers and instructors. Advancing the field and serving students requires moving beyond anecdotal evidence and experience to evaluate learning practices which contribute to the student learning process. As an instrument, the ELS is a tool for evaluating students’ perceptions of experiential learning as an approach in disciplines including HSEM higher education. Systematically examining the utility and results of instructional practices through the use of instruments like the ELS in conjunction with more extensive indicators of student experiences provides an opportunity to enhance and improve teaching practices in disciplines incorporating experiential learning. The ELS results in this pilot study indicated positive student perceptions of experiential learning in HSEM higher education. On a larger scale, the ELS can provide insight to faculty developing and revising curriculum and practices. In addition, further potential applications of the ELS to different academic fields would expand the application and validity of the four subscales to other disciplines for quantitative research applications.

As a tool, the ELS should provide a useful mechanism for further exploration of experiential learning practices in HSEM higher education when combined with other qualitative research. By piloting the application of the ELS in HSEM education, it also supports the potential application of the ELS to other academic fields. The ELS presents an applicable and accessible instrument warranting further research applications for the evaluation of student perceptions of experiential learning frameworks in higher education.