Developing a Survey of Transformative Learning Outcomes and Processes Based on Theoretical Principles

Abstract

The purpose of this research was to develop an inclusive evaluation of “transformative learning theory” that encompassed varied perspectives of transformative learning. We constructed a validated quantitative survey to assess the potential outcomes and processes of how transformative learning may be experienced by college-educated adults. Based on a review of the rational/cognitive, extrarational, and social/emancipatory perspectives of transformation learning theory, the survey reflects the assumptions underlying these perspectives through survey items and allows the survey to be used in multiple contexts both inside and outside the formal classroom. Survey development included a comprehensive review of the literature, external review by experts in adult education, focus groups for clarification of the items, the calculation of interitem correlations for each scale and cross-scale correlations, and the calculation of Cronbach’s α reliability coefficients. This survey has the potential to advance the study of transformative learning by being inclusive of several existing theoretical perspectives that have common outcomes.

Keywords

transformation personal transformation transformative education survey research assessment of transformative learning

Over the last 35 years, transformative learning theory has been investigated extensively in a variety of settings, significant life experiences, and across a range of populations. Most research on the process and outcomes of transformative learning has been qualitative in nature and has relied on retrospective interviews as a means of data collection. The prevalent philosophical orientation to research has been rooted in constructivism, where “research constructs an understanding of the phenomenon of interest from the perspectives of those who experienced it” (Merriam & Kim, 2012, p. 58). However, recently there have been several emerging trends in research designs in the study of transformative learning, including a greater sophistication of qualitative designs (narrative inquiry, autoethnography, case study), longitudinal studies, action research, and mixed methods (Taylor & Snyder, 2012).

The study of transformative learning appears to be at a turning point. The research has emphasized basic interpretive methodologies conducted with small groups of participants using inductive thematic analysis (Taylor & Snyder, 2012). There appears to be commonly accepted outcomes of transformative learning across definitions and perspectives with different ways of reaching those outcomes—cognitive, intuitive, arts based, and ideology critique (Brookfield, 2000; Dirkx, 2012; Lawrence, 2012). We see these perspectives as complementary ways of working toward common transformative outcomes, such as greater openness and inclusion. It is this understanding that led us to this project; that is, of finding a way to bring together the perspectives on transformative learning in a meaningful way and exploring a way of assessing both the common outcomes and the variety of processes for reaching those outcomes.

The intent of the article is to share the development of a reliable and valid self-report survey for assessing transformative learning that begins to offer clarity to the outcomes of transformative learning and its relationship to the processes of transformation. We discuss the process of conceptualizing and operationalizing transformative learning theory, discuss what we have learned from the development of the survey, and provide an overview of work to be completed. The merging of various perspectives related to transformative learning has the potential to help to address a major concern raised by Brookfield (2000, p. 139) and others about the “misuse of the word transformation to refer to any instance in which reflection leads to a deeper, more nuanced understanding of assumptions.”

Thinking through the nature of a self-report survey led us to see transformative learning as a distinct form of learning and teaching in relationship to other adult learning theories (Cranton & Taylor, 2012; Newman, 2012). Practically, a survey instrument offers educators and scholars the means to determine more accurately what strategies have the potential to foster transformative learning.

Theoretical Framework

The diverse theoretical perspectives about transformative learning bring a rich complexity to our understandings of transformation but also may have the tendency to encourage the categorization of perspectives; that is, to attempt to create distinct points of view on what transformative learning is. In our survey development, we hoped to bridge these perspectives, following the suggestion from Cranton and Taylor (2012, p. 3): “A more unified theory allows us to continue to speak of transformative learning while maintaining the diversity of approaches that are so important to the complexity of the field of adult education.” This means that the process of transformative learning may vary according to context and those involved; however, we see the outcome for all conceptions to be similar—that is, developing a more inclusive, discriminating, and permeable worldview (Mezirow, 1991).

Operating from this assumption, the items on this survey reflect and include three dominant conceptions of transformative learning (Cranton, 2006). One is the cognitive/rational perspective (Mezirow, 1991) that emphasizes rationality, critical reflection, and ideal conditions for discourse. This is a constructivist and universal view of learning, explaining a process of constructing and appropriating new or revised interpretations of the meaning of one’s experience with a goal of greater personal autonomy and independence. Criticisms of this model have centered predominantly on its minimal attention to other ways of knowing, overemphasis on rationality, and lack of attention to power and social change.

The second perspective has been called an extrarational perspective (Dirkx, 1998; Lawrence, 2012; Tisdell, 2000); it emphasizes the emotive, imaginal, spiritual, and arts-based facets of learning, those that reach beyond rationality. Dirkx (2001) extends the work of Boyd and Myers (1988) who followed a Jungian approach to describe how symbols, images, and archetypes play a role in personal discernment and illumination as individuals work with unconscious content. This view of transformative learning recognizes personal, intuitive, and imaginative ways of knowing that lead to individuation (the development of the self as separate from, but integrated with, the collective of humanity). Other theorists and writers go beyond the rational in different ways; for example, Tisdell describes transformation as a spiritual process, and Lawrence focuses on arts-based learning.

The third is the social critique perspective (Brookfield, 2012; Freire, 1970) that emphasizes ideological critique, unveiling oppression, and social action in the context of transformative learning. This view is about developing an “ontological vocation” (Freire, 1970, p. 12); a theory of existence, which views people as subjects, not objects, who are constantly reflecting and acting on the transformation of their world so it can become a more equitable place for all to live. Its goal is social transformation by “demythzing” reality, where the oppressed develop critical consciousness. Four broad concepts/methods are central to fostering emancipatory transformative learning: (a) centrality of critical reflection with a purpose of rediscovering power and helping learners develop an awareness of agency to transform society and their own reality, (b) a liberating approach to teaching couched in “acts of cognition not in the transferral of information” (p. 67), (c) a “problem-posing” (p. 70) and dialogical methodology, and (d) a horizontal student–teacher relationship where the teacher works on equal footing with the students. This view emphasizes social transformation over personal change.

We derived the outcomes and process items for the survey directly from these theoretical conceptualizations and obtained expert reviews from those scholars working within each conceptualization (see Survey Development section). This allowed us to operationalize transformative learning in a flexible way that maintained the diversity of approaches.

Research on Transformative Learning Surveys

In beginning of this project, the first task involved recognizing and reviewing the growing application of surveys and questionnaires related to transformative learning (Brock, 2010; Morris & Faulk, 2007; Stevens, Gerber, & Hendra, 2010). The most popular survey, the Learning Activities Survey Questionnaire (King, 2009a), has seen applications in a variety of contexts over the last decade (e.g., Brock, 2010; Glisczinski, 2007; Hodge, 2010; Johnson & Nelson, 2010; King, 1999, 2000, 2003, 2004, 2009a, 2009b). Its purpose is to identify “whether adult learners have had a perspective transformation in relation to their educational experience; and if so, determining what learning activities have contributed to it” (King, 2009a, p. 14). Although originally developed for research in higher education, it has been continually modified for use in a variety of other educational settings (e.g., adult English as a Second Language Learning [ESL], grief, educational technology, professional development, competency-based management training). Despite the wide application of this instrument, validity and reliability have not been established to capture the process and outcome of transformative learning. More specifically, the survey has not been operationalized to determine the psychological constructs of transformative learning. King’s survey predominantly uses a reconstruction of the terminology found in the theory of transformative learning (e.g., Mezirow) and seems to overlook the extensive qualitative research available on “how people typically make meaning of transformative experiences” (Taylor & Snyder, 2012, p. 47). As a result, it does not address alternative orientations of transformative learning (e.g., extrarational, emancipatory). Most significantly, even though the Learning Activities Survey has been reviewed by experts for face validity, there is no statistical evidence substantiating its reliability or content validity.

Another approach to assessing transformative learning, although found among fewer studies, has been the application of established instruments to identify a particular aspect of change. For example, Cragg, Plotnikoff, Hugo, and Casey (2001), who studied the professional resocialization of RN to BSN, used the Professional Values Scales (Jacobsen & Sabritt, 1983) to capture change in professional attitude as students completed a baccalaureate degree via distance education. Other studies include the Frommelt Attitude Toward Care of the Dying (Mallory, 2003) Scale and the Ethics of Health Care survey (Goldie, Schwartz, & Morrison, 2005) that were used to “measure significant change in perspectives about palliative care among medical and nursing students” (Taylor, 2007, p. 177). The challenge for these studies is the particularistic nature of their focus. For example, the Frommelt Scale was exclusively about change in empathy and did not draw implications of their findings on transformative learning theory explicitly. However, it is important to note that these tools and scales could offer the means to both identify individuals who have experienced a change in perspective based on criteria characteristic of transformative learning and potentially to identify essential components (e.g., support, critical reflection, perspective transformation) inherent to a particular change event. These studies provide a background for the development of a comprehensive survey of transformative learning.

Issues Associated With Quantifying Transformative Learning Theory

Survey development serves two complementary purposes (Merriam & Simpson, 2000). First, survey development involves quantitative analyses of large numbers of responses from participants. Second, the end product, the survey, can provide feedback to individuals on the extent and nature of their transformative learning as well as feedback on whether transformative learning was fostered in a particular group. In collecting aggregate survey data that rely upon self-report in response to statements related to different theoretical perspectives on transformative learning, we are able to determine patterns that occur in the outcomes and processes of transformative learning. The results gathered from the initial survey are a starting point for further investigation into how individuals and groups process transformative learning experiences.

Additionally, we wanted to include qualitative elements to investigate more about participants’ transformative experiences and the kind of changes they observed that may be missed through quantitative methods alone. By collecting individual stories of transformation experiences in the open-ended questions, we are able to analyze the themes in the types of transformation. We will also be able to later track whether certain transformative experiences lead more toward one outcome or process than another.

The current understanding of research paradigms comes from Habermas’s (1971) three kinds of human interests and knowledge—technical, practical (renamed communicative by Mezirow, 1991), and emancipatory. Ewert (1991), in a comprehensive analysis of Habermas’s work on our understanding of research and evaluation in education, outlines the ways in which research paradigms are derived from these different kinds of knowledge. In his analysis, he relies on Mezirow’s earlier work. Most recent research method books refer to three primary paradigms—empirical–analytical (also called positivist) for acquiring technical knowledge, interpretive for acquiring practical or communicative knowledge, and critical for acquiring emancipatory knowledge (e.g., Glesne, 2011).

Because the data in this research are largely quantitative, one could argue we are working from the empirical–analytical paradigm. In another way, we are working in the interpretive paradigm through self-report. Our intent is to further the understanding of individuals and groups about their processes of learning using their responses to perceptions of personal transformation. It is also reasonable to imagine that the survey could be used in a critical paradigm, for example, in a participatory action research project. Our hope is to develop a survey that is a synthesis of the theories and reflective of how people typically process transformative experiences.

Survey Development

The development of the survey to assess the outcomes and process of transformative learning began with an exploration of theoretical perspectives. The research team (authors of this article) met regularly for 6 months to develop potential items that were derived directly from theory. We took statements, definitions, and descriptions from the literature related to each theoretical perspective and crafted items to reflect the descriptions and definitions. We created a survey with two primary parts (a) outcomes of transformative learning and (b) processes of transformative learning that transfer across the theoretical perspectives in the survey (Table 1).

Table 1.

Summary of Outcomes and Processes Developed in the Survey.

Outcomes	Processes
Acting differently Having a deeper self-awareness Having more open perspectives Experiencing a deep shift in worldview	Cognitive/rational Critical reflection Action Experience Disorienting dilemma Discourse Beyond rational/extrarational Arts based Dialogue Emotional Imaginal Spiritual Soul work Social critique Ideology critique Unveiling oppression Empowerment Social action

Outcomes

Processes

Acting differently

Having a deeper self-awareness

Having more open perspectives

Experiencing a deep shift in worldview

Cognitive/rational

Critical reflection

Action

Experience

Disorienting dilemma

Discourse

Beyond rational/extrarational

Arts based

Dialogue

Emotional

Imaginal

Spiritual

Soul work

Social critique

Ideology critique

Unveiling oppression

Empowerment

Social action

Each of these outcomes can be found in more than one theoretical perspective. For example, Mezirow’s (2003, pp. 58–59) cognitive/rational perspective has this as a basic definition, “transformative learning is learning that transforms problematic frames of references … to make them more inclusive, discriminating, open, reflective, and emotionally open to change.” He comments that transformed frames of reference are what guide action. Dirkx’s (2012) bases his understanding of transformative learning on depth psychology, especially a Jungian approach, and often refers to it as nurturing soul or soul work. As such, he is interested in the psyche and bringing the unconscious to consciousness (increasing self-awareness and developing more open perspectives). Brookfield (2012, pp. 131–132) cautions against the overuse of the terms “transformative learning” and “sliding into an unproblematized focus on the self.” He frames his position on transformative learning with critical theory (learning to challenge dominant ideology, uncover power, and contest hegemony). The outcome of this perspective is also a deep shift in worldview, as is the case with both the cognitive and extrarational perspectives.

As we have mentioned, the theoretical perspectives described previously can be grouped into three processes. For the cognitive, rational process, five scales were developed to represent this process: critical reflection, action, experience, disorienting dilemma, and discourse. The extrarational process is comprised of six subscales, namely arts-based learning, dialogue with others, emotional reactions, imaginal learning, spiritual learning, and soul work. The social critique includes four subscales, namely ideology critique, unveiling oppression, empowerment, and social action.

These outcomes, processes, and their subcategories were formed into a survey with 19 scales. To establish construct validity, the first version of the survey was sent to 20 known experts in the area of transformative learning—people who have published extensively, done research related to transformative learning, and/or contributed to theory development. We deliberately chose reviewers who worked primarily within one or more of the theoretical perspectives we were using for the survey development. We asked them to comment on the degree to which the items were true to theory. Ten reviewers responded to this request. Their comments were compiled, and their suggestions for change were considered and/or integrated into the survey.

The second version of the survey was created by adding a 4-point Likert-type scale (mostly disagree = 1 to mostly agree = 4), randomly ordering the items and adding a short section on demographics. We added two open-ended questions to encourage participants to focus on a specific context in which their transformative learning may have occurred. The next stage of survey development involved two focus group meetings with members of the university community (staff, students, and faculty) for usability and comprehension. Focus groups were important to include, as research demonstrates a survey is only as useful as its comprehension by the participant (Lutz, Kneipp, & Means, 2009; O’Donnell, Lutfey, Marceau, & McKinlay, 2007). Participants completed the survey and then discussed the questions: Were there items you didn’t understand? Were there questions where you didn’t know how to respond? Were the directions for the survey clear? One researcher facilitated the focus group, while another researcher took extensive notes on the discussion. We used the results from the focus groups to revise the items and prepare the third version of the survey for pilot testing. Preliminary descriptive statistics and interitem correlations were calculated, and these results contributed to the revisions made to the items.

Pilot Testing of the Survey

In this section, we describe the readability of the survey, the participants, their responses to the open-ended questions, descriptive statistics for each scale, interitem correlations for each scale, and cross-scale correlations.

Participants

Of the 136 adults who participated in the pilot study, 21 were men, 95 were women, and 20 who did not respond to the question on gender. The majority of the respondents ranged between the ages of 24 and 64, with one person <24, and nine being >65. Twenty-seven people had a bachelor’s degree and 83 a graduate degree, reflecting the context and nature of our recruitment of participants (mostly through Canadian and U.S. graduate program listservs and word of mouth). Seven people had completed high school or some college and university. The majority of respondents (88%) gave their race as White; and 12% described their race as Black, Latino, Asian, or mixed race. Fifty-three people listed the United States or American as their nationality; 37 people gave Canadian as their nationality. The remaining respondents were spread across a wide variety of nationalities with no more than one or two for each of the nationalities. The majority of the participants (91%) described their employment as “professional.”

Responses to Open-Ended Questions

At the beginning of the survey, we asked the pilot study participants to respond to two open-ended questions. The purpose of these questions was to focus people on a specific life-changing event (transformative learning experience) that they could consider as they rated each of the statements. As survey development continues, we hope to create a checklist from the themes we find in the open-ended responses and substitute this checklist for the open-ended questions. This will allow us to conduct cross-tabulations of the nature of people’s experiences with their responses to the survey items.

The first question asked participants to focus on one specific life-changing event:

Before you turn the page to start the survey, please think about one specific life changing experience. This should be an event that altered your life in a deep and fundamental way. Take a few minutes to describe that experience here. When did it happen? Who was involved? Where did it happen?

The most frequently listed event was the death of a loved one or loved ones (n = 27). One respondent said, “My husband passed away completely unexpectedly at age 40 of a heart attack while playing hockey.” Other events were life-threatening illness of self or loved one (n = 16), divorce or separation (n = 13), educational experiences (n = 13), birth of a child (n = 12), loss of job (n = 7), living outside country or culture, immigration (n = 7), and love/marriage (n = 5). The majority of the narratives were descriptive, with these as examples:

A life changing experience in my life was the death of my father. He was diagnosed with cancer, lived 4 short months and passed away.… The night that he passed away, there was a thunderstorm that came through town. After the storm there was a parting of the sky, almost tunnel-like where the sun shown through brightly as if God were calling him to heaven. I have been wavering about what might be ahead for us when we die, and now I am sure that there is life after death.

I sobered up with the help of AA 25 years ago.

My daughter was diagnosed with a life-threatening form of childhood cancer called neuroblastoma. This happened 2 months after our son was born.

I lost my career job in 2000 (was fired) as a chemist at age 50 and I knew I would not be able to be hired again. It was the single most feared moment of my life, and I had few backup skills to seek new employment. After 2 years of underemployment, I decided that the best thing to do was to go back to school.

The second open-ended question asked people to respond to: “In what ways did this event change your life?” The most frequently reported changes were changed family life (n = 15), changed my perspective as an educator/learner (n = 14), changed my lifestyle/my worldview (n = 12), left job, changed job, took job (n = 8), new life, new friendships (n = 8), faced with raising children alone (n = 7), life is precious, greater appreciation of life, love life (n = 7), and every aspect of my life (n = 5). For both of these questions, many participants responded at length, providing several paragraphs telling their stories and describing their experiences. For example, a death in the family of one respondent led to the response of “how precious life is and that your whole life can be changed in an instant. It also brought me closer to people who also knew him in a very deep way.” Other responses included:

[On ending a relationship]

I had a difficult time leaving because I felt so responsible for his well-being. I used to blame him for all my problems, then I recognized my own issues and took responsibility to focus on my own well-being. I still struggle, but I am working on myself one day at a time.

[After life-threatening medical event]

I redefined my idea of risk. I was in a job that I was afraid to leave because of ‘my comfort zone.’ After this experience, my entire idea of what was risky or not changed. If I could almost die and live through this medical event, then changing jobs was a very small risk that I could take. I resigned my position while recuperating at home from surgery and another opportunity appeared that was one of the best positions I’ve ever had.

[On being a support for a dying patient]

I learned that it was an amazing privilege that a nurse has to share such moments with people: in death, in life, in sickness, and in dying, one can be healed. Healing transcends the body. I learned to love my profession, and I learned not to fear death. I learned the depth of the human soul, and how anger can define us, and stunt our life. I witnessed the grace of forgiveness. I learned to spend time with the sick. I want to teach these stories to other nurses.

[A new relationship]

I became more adventurous and outgoing than I had been in my life. I came out of my shell and learned to love life a little more.

Many respondents wrote extensive descriptions of their experiences. When we derived the categories for the process scales, we based this on theoretical perspectives and the research literature on transformative learning. The descriptions of how respondents experienced the process of transformative learning confirmed our original choices of scales and supplied evidence of content validity.

Descriptive Statistics of Survey Items

Descriptive statistics were calculated for each of the original 110 survey items. We review some of the highlights and point out those statistics that were influential in reviewing and revising the survey. With the exception of 1 item, all of the outcomes items had means of greater than 3.0 on a 4-point scale and standard deviations less than 1. The mean of one scale was only slightly less than 3.0. This means that those people responding to the survey indicated that they had experienced a life-changing event—events that involved acting differently, increased self-awareness, increased openness, and a shift in worldview. We expected that this would be the case, given that many of the participants were drawn from graduate programs and alumni of graduate programs in adult education where transformative learning experiences are often reported (e.g., Stevens-Long, Schapiro, & McClintock, 2012).

For the process items based on three different perspectives (cognitive/rational, beyond rational, and social critique), there was considerably more variation in the descriptive statistics. Some items had means less than 3.0, but most were in the range of 2.5–3.0 (with the lowest mean being 1.9 on a 4-point scale). Many items had high standard deviations (1.1, 1.2, or 1.3), indicating that people were responding to the items on both ends of the scale. This was particularly the case in the “beyond rational” realm and especially with the scales related to arts-based transformative learning, spiritual transformative learning, and soul work.

Interitem Correlations by Scale

Interitem correlations with scales are used to determine whether the items on each scale are related to each other, as they should be if they are measuring the same concept. Although most researchers use Pearson’s r correlations with data from a Likert-type scale, we wanted to err on the side of caution. Pearson’s r correlations are based on the assumption that the data are on an interval scale. We chose to use Spearman’s ρ for this calculation, as our rating scale had 4 points, and we did not feel we could be confident in assuming equal intervals between the points on the scale. Spearman’s ρ is suitable for data on an ordinal scale, which only assumes that the data can be ordered, and not that the intervals between the data points are equal. Table 2 shows sample outcome and process items.

Table 2.

Sample Outcome and Process Items.

Sample Outcome Items	I have experienced a deep shift in the way I see some things in the world It would be impossible for me to go back to being the way I once was
Sample process items	Encountering a disorienting event leads me to see myself in a different way (Disorienting Dilemma) I call upon a higher power to help me get through a difficult situation (Spiritual) During a social change, I challenge what I see and hear on television, in print, and on the Internet (Ideology Critique) I seriously question my beliefs and actions (Critical Reflection)

With only one exception, all of the items in each of the “outcome” scales correlated significantly with each other. Among the “beyond rational” process scales, all items correlated significantly and strongly with each other for the arts-based, support/dialogue, emotions, imaginal/soul work, and spiritual scales. Among the cognitive scales, action, critical reflection, discourse, and experience all had 1 or 2 items that did not correlate well with the other items. We were able to eliminate some items, and we revised the wording of others. The items on the disorienting dilemma scale correlated significantly and highly with each other. In the social critique grouping, the items for social action and ideology critique correlated significantly and well with each other. The scales for emancipation and social action each had 1 or 2 items that did not correlate well. Because we had extra items for these scales, we were able to simply delete those that were not functioning.

Cross-Scale Correlations

Cross-scale correlations were calculated using Pearson’s r. In this case, since each scale combined the ratings of several items, we were able to make the assumption of equal intervals among the data points with confidence. This assumption was supported by the descriptive statistics for each of the scales. The four outcome scales correlated highly with each other, with correlations ranging from .59 to .77. We expected these scales to be related as they represented different facets of the possible outcomes of transformative learning. In general, the process scales were correlated with each other in predictable ways. The beyond rational scales correlated between .29 and .73 with each other. The cognitive scales correlated between .27 and .70 with each other. The social critique scales correlated between .43 and .71 with each other.

What is perhaps more interesting, and also a test of validity, is to consider the scales that did not correlate with each other. A few examples will illustrate this. “Cognitive action” did not correlate with “support.” Since cognitive action is a scale within the cognitive/rational perspective, and support is within the extrarational perspective, it makes sense that these two scales were not correlated. Similarly, “disorienting dilemma” from the cognitive/rational perspective did not correlate with “spirituality” or “support” from the extrarational perspective. “Discourse” did not correlate with “dialogue” or with “support.” Even though one would expect discourse and dialogue to be similar, we worded the items according to the theoretical perspectives, where they are quite dissimilar. Discourse involves a cognitive process of weighing evidence and making rational judgments, whereas dialogue and support are more relational in nature. “Unveiling oppression” from the social critique perspective did not correlate with “spirituality” or “support” from the extrarational perspective or “disorienting dilemma” from the cognitive/rational perspective. Similarly, “ideology critique” from the social critique perspective did not correlate with “dialogue,” “support,” or “spirituality” from the extrarational perspective or “disorienting dilemma” from the cognitive/rational perspective. These findings gave us confidence in the distinctions we made between the different strands of the theoretical framework.

Cronbach’s α Reliability Analysis

Finally, in the pilot study, we calculated Cronbach’s α, a measure of internal consistency reliability. This statistic is an overall item correlation where the values range between 0 and 1, with .7 being acceptable, that is, 70% of the variance in the equally weighted composite is due to the common factor among the tests (Cronbach, 1951). In other words, it shows how closely related a set of items are as a group. For scales that are used as research tools to compare groups, α values of .7–.8 are regarded as satisfactory (Bland & Altman, 1997; DeVillis, 1991). As can be seen in Table 3, all of the Cronbach’s α coefficients are good, with the exceptions of the “soul work” scale and the “experience” scale, both of which could be called “moderate.” At this point, we were already aware of the problematic nature of the soul work scale (see Survey Development section), and we knew that some items in the “experience” scale did not correlate well with the other items based on our interitem correlation analysis.

Table 3.

Cronbach’s α Reliability on Pilot Study Data.

Scale	M	SD	α
Acting differently	19.1	3.4	.79
Self-awareness	20.5	2.9	.75
Openness	19.4	3.3	.78
Shift in worldview	21.1	2.9	.76
Arts based	10.4	4.6	.90
Dialogue	15.9	3.0	.80
Emotions	16.5	2.7	.77
Imaginal	11.5	2.9	.78
Spiritual	13.4	4.9	.90
Support	15.8	3.4	.88
Soul work	18.0	2.9	.52
Action	15.5	2.6	.70
Critical reflection	15.5	2.9	.73
Disorienting dilemma	15.5	3.9	.84
Discourse	16.7	2.1	.54
Experience	16.6	2.2	.65
Empowerment	17.6	3.3	.63
Social action	16.7	4.0	.77
Unveiling oppression	15.4	2.9	.67
Ideology critique	15.7	3.1	.77

Note. SD = standard deviation.

Revisions Based on the Pilot Study Data

Most scales in the pilot study version of the survey included additional survey questions, which were not included in the final version. This was done deliberately, so that we would be able to eliminate items that did not correlate well with each other within scales (according to Cronbach’s α) or otherwise seemed unclear. In our revisions to the survey based on the pilot study analysis, we deleted the items that were not functioning as they should, as described subsequently. This is a typical strategy for survey development, as problematic items are deleted from the analysis, with the purpose of improving Cronbach’s α. Detailed information about the calculation and interpretation of Cronbach’s α can be found in a classic publication by DeVillis (1991).

Initially, as a separate scale, soul work had a number of weak correlations, so we consulted with Dirkx’s (2012) description of soul work and images and realized that if we combined those two scales (soul work and imaginal), we would have a more reliable new scale. Support and dialogue were merged because the items across those two scales were highly correlated and conceptually the items were about the same thing—getting support from others through dialogue. The moderate Cronbach’s α for “discourse” was somewhat of a surprise, but we addressed it by removing the items that did not correlate well with the other items. The discourse items were initially based on Mezirow’s ideal conditions of discourse, taken from Habermas’s work. The language of those ideal conditions was not in line with the language used in the rest of the survey and was likely confusing for respondents.

Testing the survey with focus groups helped to confirm the comprehension and readability of the survey. In addition, the readability of the instrument, assessed by using the Microsoft Office Word software program (MAC 2011, Version 14.3.0), determined that the article was written at 6.3 grade level according to the Flesch–Kincaid Readability Scale. However, it is important to note, the survey was tested only in the United States and Canada and was not tested cross-culturally. At this stage, the survey is limited to college-educated students as that comprised our pilot study participants. As we move forward in the survey development process, additional participants with varied educational and cultural backgrounds will be included.

Implications for Theory, Research, and Practice

From this initial survey development and data analysis of 136 responses, we have learned that we can distinguish between the three major perspectives of transformative learning theory, and within those perspectives, the various strands that form the process of transformative learning. In terms of the outcomes of transformative learning, the cross-scale correlations demonstrate that the outcomes are strongly related to each other, yet unique enough that they can be defined separately. The process strands identified within each major perspective similarly are confirmed as separate processes but related to each other. This was determined by examining the correlations within each scale and the correlations of individual items across scales. Items correlated more highly within scales (with some of the exceptions described previously) than across scales; at the same time, the cross-scale correlations demonstrated the overall connections between the various strands.

Theoretically, this survey begins to bring clarity to the consequences of transformative learning and its relationship to the process of transformation. It also begins to move the study of transformative learning toward a perspective that is more inclusive of multiple interpretations of transformative learning (cognitive, extrarational, and social critique). Through the development of this survey, boundaries start to emerge related to transformative learning as a distinct form of learning in relationship to other adult learning theories (e.g., Newman, 2012). Even though the survey was built upon theoretical foundations in the literature, the limitations of this evaluation are that these qualitative theories were translated to quantitative form to perform measurements with tools and techniques that appear to produce numerical and binary answers. We understand this limitation, but realize that to move the field forward, we need to begin to place boundaries on the amorphous term “transformative learning” in order to collaboratively develop surveys and tools of discovery, subject to future revision as our understanding becomes clearer and refined. In relation to both theory development and practical applications, a person taking the survey will receive a score on each scale by combining their responses to the items representing the scale. The outcome scores indicate the degree to which the person has engaged in transformative learning in general; the process scores indicate the probable processes a person goes through during a revision of perspectives. Practitioners will be able to describe the extent to which a specific group, in the context of a course or program, engages in transformative learning. It becomes a tool that captures change, but also offers learners/educators, in a participatory fashion or individually, a means to deconstruct a transformation as it unfolds within an educational experience. It provides a framework that practitioners and learners can use to continually evaluate themselves as they attempt to foster transformative learning. From the point of view of theory development, if we can show which learning processes lead to which kinds of outcomes, we can further the unification of the various perspectives on transformative learning theory.

Similarly, in terms of research, an educator or researcher who wants to assess the extent to which transformative learning occurs can use the survey alone or in conjunction with other data collection techniques such as interviews or storytelling. The survey includes open-ended questions, but these could easily be elaborated on in a mixed-methods research design where participants tell the story of their transformative experience.

The next step for this survey is to sample a large group of individuals in a variety of settings, circumstances, educational backgrounds, and positionalities. This will allow us to further establish the reliability and validity of the survey. In addition, it would also be important to use this survey with participants who have recently shared a similar transformative event (e.g., graduating from higher education, significant health recovery, intercultural experience, social movement). Using both broad data collection along with purposeful sampling could strengthen survey significantly and provide an opportunity to better understand the kinds of events that lead to transformative experiences. The survey will be used in context-specific settings (e.g., a particular program) as well as in a broad-based fashion (e.g., similar to any learning styles inventory). In applying the survey to additional contexts, we are expecting that the survey items themselves will not need to be revised, thus retaining established validity and reliability of the survey items. However, the two open-ended questions at the survey introduction can be modified to reflect the application to different contexts without affecting the cohesion of the survey. It will be exciting to see how this survey will build upon existing theories of transformative learning to create an inclusive definition of transformative learning.

Footnotes

Authors’ Note

An abstract of this paper was presented (oral) at the International Conference on Transformative Learning, San Francisco, CA, November 1–4, 2012.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Bland

J. M.

Altman

D. G.

(1997). Statistic notes: Cronbach’s alpha. British Medical Journal, 314, 572–573.

Boyd

R. D.

Myers

J. B.

(1988). Transformative education. International Journal of Lifelong Education, 7, 261–284.

Brock

(2010). Measuring the importance of precursor steps to transformative learning. Adult Education Quarterly, 60, 122–142.

Brookfield

(2000). Transformative learning as ideology critique. In Mezirow

& Associates (Eds.), Learning as transformation: Critical perspectives on a theory in progress (pp. 125–148). San Francisco, CA: Jossey-Bass.

Brookfield

(2012). Critical theory and transformative learning. In Taylor

E. W.

Cranton

(Eds.), Handbook of transformative learning theory: Theory, research and practice (pp. 131–146). San Francisco, CA: Jossey-Bass.

Cragg

C. E.

Plotnikoff

R. C.

Hugo

Casey

(2001). Perspective transformation in RN-to-BSN distance education. Journal of Nursing Education, 40, 317–322.

Cranton

(2006). Understanding and promoting transformative learning (2nd ed.). San Francisco, CA: Jossey-Bass.

Cranton

Taylor

E. W.

(2012). Transformative learning theory: Seeking a more unified theory. In Taylor

E. W.

Cranton

(Eds.), Handbook of transformative learning theory: Theory, research and practice (pp. 3–20). San Francisco, CA: Jossey-Bass.

Cronbach

L. J.

(1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.

10.

DeVillis

R. F.

(1991). Scale development: Theory and applications. Newbury Park, CA: Sage.

11.

Dirkx

(1998). Transformative learning theory in the practice of adult education: An overview. PAACE Journal of Lifelong Learning, 7, 1–14.

12.

Dirkx

(2001). Images, transformative learning, and the work of soul. Adult learning, 12, 15–16.

13.

Dirkx

(2012). Nurturing soul work: A Jungian approach to transformative learning. In Taylor

E. W.

Cranton

(Eds.), Handbook of transformative learning theory: Theory, research and practice (pp. 116–130). San Francisco, CA: Jossey-Bass.

14.

Ewert

G. D.

(1991). Habermas and education: A comprehensive overview of the influence of Habermas in educational literature. Review of Educational Research, 61, 345–378.

15.

Freire

(1970). Pedagogy of the oppressed. New York, NY: Continuum.

16.

Glesne

(2011). Becoming qualitative researchers (4th ed.). Boston, MA: Pearson Education.

17.

Glisczinski

(2007). Transformative higher education: A meaningful degree of understanding. Journal of Transformative Education, 5, 317–328.

18.

Goldie

Schwartz

Morrison

(2005). Whose information is it anyway? Informing a 12-year patient of her terminal prognosis. Journal of Medical Ethics, 31, 427–434.

19.

Habermas

(1971). Knowledge and human interests. Boston, MA: Beacon Press.

20.

Hodge

(2010). Trainers and transformation: Facilitating the ‘dark side’ of vocational learning. International Journal of Training Research, 8, 53–62.

21.

Jacobsen

Sabritt

(1983). RN baccalaureate education: A process-product evaluation. Lexington: University of Kentucky.

22.

Johnson

S. M.

Nelson

B. M.

(2010). Above and beyond the syllabus: Transformation in an adult, foreign language classroom. Language Awareness, 19, 35–50.

23.

King

K. P.

(1999). Unleashing technology in the classroom: What adult basic education teachers and organizations need to know. Adult Basic Education, 9, 162–175.

24.

King

K. P.

(2000). The adult ESL experience: Facilitating perspective transformation in the classroom. Adult Basic Education, 10, 69–89.

25.

King

K. P.

(2003). Understanding adult learners amidst societal crisis: Learning and grief in tandem. Journal of Continuing and Higher Education, 51, 13–23.

26.

King

K. P.

(2004). Both sides now: Examining transformative learning and professional development of educators. Innovative Higher Education, 29, 155–174.

27.

King

(2009a). The handbook of the evolving research of transformative learning based on the learning activities survey. Charlotte, NC: Information Age Publishing.

28.

King

(2009b). Workplace performance—PLUS: Empowerment and voice through professional development and democratic process in health care training. Performance Improvement Quarterly, 21, 55–74.

29.

Lawrence

(2012). Transformative learning through artistic expression: Getting out of our heads. In Taylor

E. W.

Cranton

(Eds.), Handbook of transformative learning theory: Theory, research and practice (pp. 471–485). San Francisco, CA: Jossey-Bass.

30.

Lutz

B. J.

Kneipp

Means

(2009). Developing a health screening questionnaire for women in welfare transition programs in the United States. Qualitative Health Research, 19, 105–115.

31.

Mallory

J. L.

(2003). The impact of a palliative care educational component on attitudes toward care of they dying in undergraduate nursing. Journal of Professional Nursing, 19, 305–312.

32.

Merriam

S. B.

Kim

(2012). Studying transformative learning: What methodology? In Taylor

E. W.

Cranton

(Eds.), Handbook of transformative learning theory: Theory, research and practice (pp. 56–72). San Francisco, CA: Jossey-Bass.

33.

Merriam

S. B.

Simpson

E. L.

(2000). A guide to research for educators and trainers of adults (2nd ed.). Malabar, FL: Kreiger.

34.

Mezirow

(1991). Transformative dimensions of adult learning. San Francisco, CA: Jossey-Bass.

35.

Mezirow

(2003). Transformative learning as discourse. Journal of Transformative Education, 1, 58–63.

36.

Morris

Faulk

(2007). Perspective transformation: Enhancing the development of professionalism in RN-to-BSN students. Journal of Nursing Education, 46, 445–451.

37.

Newman

(2012). Calling transformative learning into question: Some mutinous thoughts. Adult Education Quarterly, 62, 36–55.

38.

O’Donnell

A. B.

Lutfey

K. E.

Marceau

L. D.

McKinlay

J. B.

(2007). Decision making using focus groups to improve the validity of cross-national survey research: A study of physicians. Qualitative Health Research, 17, 971–981.

39.

Stevens

Gerber

Hendra

(2010). Transformational learning through prior learning assessment. Adult Education Quarterly, 60, 377–404.

40.

Stevens-Long

Schapiro

S. A.

McClintock

(2012). Passionate scholars: Transformative learning in doctoral education. Adult Education Quarterly, 62, 180–198.

41.

Taylor

E. W.

(2007). An update of transformative learning theory: A critical review of the empirical research (1999-2005). International Journal of Lifelong Education, 26, 173–191.

42.

Taylor

E. W.

Snyder

M. J.

(2012). A critical review of research on transformative learning theory, 2006-2010. In Taylor

E. W.

Cranton

(Eds.), Handbook of transformative learning theory: Theory, research and practice (pp. 37–55). San Francisco, CA: Jossey-Bass.

43.

Tisdell

E. J.

(2000). Spirituality and emancipatory adult education in women adult educators for social change. Adult Education Quarterly, 44, 34–42.