The Effects of Reflective Activities on Skill Adaptation in a Work-Related Instrumental Learning Setting

Abstract

In work-related instrumental learning contexts, the role of reflective activities is unclear. Kolb’s experiential learning theory and Mezirow’s transformative learning theory predict skill adaptation as an outcome. This prediction was tested by manipulating reflective activities and assessing participants’ response and error rates during novel applications of an instrumentally learned skill (skill adaptation). Participants were randomly assigned to three conditions (interference, reflection, or critical reflection) according to blocking variables: gender, age, and reflective propensity. Participants then completed behavioral skills training with embedded reflective activities. Afterward, participants completed a novel application task. Analyses of variance did not reveal differences in response or error rates among conditions or an interaction between reflective activity and reflective propensity on response rate. An interaction was found between reflective activity and reflective propensity on error rate: In the critical reflection condition, nonreflective learners had higher error rates than reflective learners. Potential explanations and implications for research and practice are offered.

Keywords

reflection critical reflection instrumental learning work-related learning

A defining feature of adult work-related education is skills-based learning. Common in professions with complex procedures (e.g., surgeons, pilots, plumbers), skills-based approaches ensure stakeholder well-being by requiring neophytes to demonstrate precise, predefined skills and knowledge. As adults test this knowledge, they use hypothetical deductive reasoning to envision potential actions and their outcomes. This process is often termed instrumental learning (Cranton, 1996; Habermas, 1971; Mezirow, 2000), a form of adult learning underlying various demonstrable work-related skills, including designing automobiles, constructing bridges, diagnosing diseases, filling teeth, and forecasting the weather (Mezirow, 2009).

To facilitate this process in formal settings, educators often use reflective activities. But how these activities affect instrumental learning is unclear. Claims to their efficacy remain largely unsubstantiated. For instance, Kolb’s (1984) experiential learning theory and Mezirow’s (2000) transformative learning theory predict skill adaptation as a potential outcome in instrumental settings, yet each provides no empirical support. This study aims to test these claims to determine how reflective activities affect skill adaptation in adult work-related instrumental learning.

Reflection and Critical Reflection

Researchers may struggle to evaluate reflective activities’ effect on instrumental learning because the processes that they are designed to evoke (i.e., reflection and critical reflection) remain difficult to define. Cornford (2002) attributes this to divergent epistemological and theoretical positions, which Mackintosh (1998) contends result in ambiguous, confusing terminology of little help to practitioners or researchers. Some embrace this ambiguity, arguing that these processes’ philosophical foundations are inapplicable to quantitative, empirical inquiry (Cranton, 2000; Duke & Appleton, 2000). Despite such claims, researchers refer to reflection and critical reflection in specific ways, particularly when discussing attempts to occasion their referents using structured pedagogical events, hereafter referred to as reflective activities.

Numerous variants of reflection appear in the literature (e.g., reflective thinking, critical thinking, reflectivity, reflexivity, and self-reflection). Although researchers may claim that meaningful differences exist, it is beyond this study’s scope to informatively discuss each. This study focuses on the most prominent variants: reflection and critical reflection. Space dictates a pithy overview of each form’s salient characteristics.

Reflection: Themes and an Emergent Functional Definition

Several themes emerge from the literature on reflection. Some researchers consider reflection a process of problem solving and meaning construction whereby learners evaluate alternative courses of action against past experience (Dewey, 1938; Kolb, 1984; Moon, 1999; Van Manen, 1991). Schön (1983, 1987) noted that professionals do this intuitively while in the immediate experience (reflection in action) and while removed from that experience (reflection on action). Others consider reflection a process of retrospectively examining experience for myriad purposes beyond problem solving, such as gaining awareness, understanding, and appreciation (Boud, 2001; Boud, Keogh, & Walker, 1985; Burke, Scheuer, & Meredith, 2007; Mezirow, 1998).

Using both themes, one can suitably operationalize reflection for this study and other work-related learning contexts. Moon’s (1999) definition addresses working adults’ needs to problem solve in novel contexts in relevant and meaningful ways: “Reflection is a mental process with purpose and/or outcome in which manipulation of meaning is applied to relatively complicated or unstructured ideas in learning or to problems for which there is no obvious solution” (p. 161). For instance, learners may encounter problems that they are unable to solve but able to identify successful outcomes for: A scrambled Rubik’s Cube offers an easily identifiable endpoint but a maddeningly difficult means of getting there. When learners imagine how their known actions affect new problems, they change these actions’ previous functions and manipulate their meaning (see Dennett, 1995, for a discussion of meaning as function). Consequently, they find relevant, meaningful solutions to novel problems.

Critical Reflection: Themes and an Emergent Functional Definition

Most researchers describe critical reflection as a process of identifying, analyzing, and questioning assumptions underlying how one sees the world (Merriam, Caffarella, & Baumgartner, 2007). Throughout, learners may engage in ideology critique (Kreber, 2010) to consider how power, culture, and dominant narratives influence their feelings, beliefs, and assumptions (Brookfield, 2000b). Critical reflection departs from reflection in that it requires learners to assess what is being reflected on (Mezirow, 1998) rather than examine actions or potential actions in light of accepted principles (Silverman & Casazza, 2000).

Two disagreements emerge from the literature. First, several researchers argue that critical reflection is a collaborative and interactive process (Brookfield, 2000b; Cranton, 1996; West, 2010), whereas others contend that it can occur outside a discursive group (Fook, 2010; Mezirow, 1998). Second, traditionalists consider critical reflection a rational process (Mezirow, 1990, 1998), whereas some reconceptualize it as an emotional (Brookfield, 2000b; Fook & Askeland, 2007) or “extra-rational” (Dirkx, 2013) activity.

For some, critical reflection’s emancipatory focus may appear unrelated to instrumental learning. Mezirow (2000), however, has outlined critical reflection’s function and form in instrumental settings. When learners critically reflect on assumptions in task-oriented problem solving, they engage in objective reframing. Learners use narrative critical reflection to assess the validity of knowledge and skills and action critical reflection to examine the assumptions underlying how a problem is defined. Here, learners critically reflect on the content, process, or premise of the problem to increasingly adapt or modify skills and knowledge to immediate contexts. Although Mezirow does not specifically mention adaptability, his examples are illustrative. When discussing beginning teachers’ use of critical reflection while learning how to assign grades, Mezirow suggests that teachers critically reflect to determine value for different learning artifacts (content reflection) and question whether these artifacts are representative of a particular student’s abilities (process reflection). The implication is that teachers who critically reflect on the content and process of this procedure will increasingly adapt and generalize their grading skills to a specific context.

Acknowledging these views, one can operationalize critical reflection as a dialogic and emotionally laden process of identifying, analyzing, and challenging assumptions underlying prior learning. Learners may again encounter problems that they are currently unable to solve. When examining their beliefs surrounding these problems, they identify how those beliefs formed and how each influences their approach to problem solving. Consequently, if learners discover their beliefs invalid, they may consider previously rejected alternatives. With new insights, learners approach problem solving differently, changing the function (i.e., meaning) of potential solutions and the problems themselves.

Reflective Activities and Instrumental Learning Outcomes

Reflective activities are often discussed with another adult learning process: communicative learning. When adults engage in communicative learning, they analogize from familiar to unfamiliar and infer explanations using analogic-abductive reasoning. This commonly occurs when learners estimate what others mean when communicating abstract concepts such as values, feelings, ideals, and moral decisions (Mezirow, 1990). Researchers in communicative contexts have demonstrated various benefits from reflective activities—for example, improved self-awareness, emotional support, professionalism, collegiality, organizational learning, and informed practice (Fook & Gardner, 2007); enhanced critical thinking in complex situations (Brookfield, 2000a); advanced affective development (Hill, 2005); increased self-esteem (Heinrich, 1992); and improved political and social emancipation (Taylor, 2001).

Few researchers, however, have identified how reflective activities affect instrumental learning outcomes. Empirical findings are inconclusive and limited. Researchers have demonstrated that question-based reflective activities heighten engineering design quality (Wetstein & Hacker, 2004), improve learning transfer (Bannert, 2006), and, when coupled with feedback, increase multiple-choice examination scores (Van den Boom, Paas, & Van Merrienboer, 2007). Others, however, have revealed that one-on-one reflective questioning (Grez, Valcke, & Roozen, 2009), reflective teaching methods (Lowe & Kerr, 1998), and reflective verbalizations in e-learning contexts (Bannert, 2006) do not affect formal skills-based assessment. Several systematic reviews have found no empirical support for reflective activities in skills-based settings (Cornford, 2002; Mann, Gordon, & MacLeod, 2009; Ruth-Sahd, 2003). Furthermore, there appear to be no published empirical studies demonstrating a relationship between critical reflection activities and instrumental learning. Numerous disciplines have requested additional research: physician education (Mamede, Schmidt, & Cesar Penaforte, 2008; Mamede, Schmidt, & Rikers, 2006), K-12 teacher education (Borko, Michalec, Timmons, & Siddle, 1997; Cornford, 2002), adult and postsecondary professional education (Malkki & Lindblom-Ylanne, 2012; Warhurst, 2008), nursing education (Mackintosh, 1998; Mann et al., 2009; Ruth-Sahd, 2003), career and technical education (Roessger, 2012), and continuing professional development (Roessger, 2013).

The paucity of studies in this area does not stem from deficient theoretical direction. As noted, transformative learning theory claims that learners who critically reflect on instrumental learning will better adapt skills and knowledge to novel contexts (see Mezirow, 2000). Experiential learning theory argues that learners who reflect on immediate experiences will create new theories for action and demonstrate increased “adaptive flexibility” (Kolb, 1984, p. 213). Additionally, experiential learning theory maintains that learning style may affect reflective learning. Individuals with divergent (introverted/feeling) and assimilative (introverted/intuitive) styles use reflection more often than those with accommodative (extroverted/sensing) and convergent (extraverted/thinking) styles. One can conceptualize learning style along a reflective continuum by considering divergent and assimilative learners as high reflective propensity learners and accommodative and convergent learners as low reflective propensity learners. High reflective propensity learners should excel in settings that promote varieties of reflection, while low reflective propensity learners should not.

Purpose of the Study

This study experimentally examined how reflective activities affect skill adaptation in an adult work-related instrumental learning setting. Two reflective activity foci were examined: reflection and critical reflection. Interlocking concrete paver installation was chosen as this study’s focus because it represents an authentic skill in beginner and experienced hardscape professional courses for adults. Skill adaptation outcomes (see the Dependent Measures section) were defined as a participant’s response rate on a novel task (i.e., number of pavers installed in 25 minutes) and error rate (i.e., number of errors committed per paver installed).

Research Questions

Four questions were posed:

Question 1: How do reflective activities affect the response rate of a participant when applying an instrumentally learned skill in a novel way?

Question 2: How do reflective activities affect the error rate of a participant when applying an instrumentally learned skill in a novel way?

Question 3: Is there an interaction effect between reflective activity and reflective propensity (i.e., preference for reflective learning) on response rate?

Question 4: Is there an interaction effect between reflective activity and reflective propensity on error rate?

Hypotheses

Eight hypotheses were proposed:

Hypothesis 1a: Participants in activities with a reflection focus will have higher response rates on novel application tasks than participants in critical reflection and nonreflection activities.

Hypothesis 1b: Participants in activities with a critical reflection focus will have higher response rates on novel application tasks than those in nonreflective activities but lower response rates than those in activities with a reflection focus.

Hypothesis 2a: Participants in activities with a reflection focus will have lower error rates on novel application tasks than participants in critical reflection and nonreflection activities.

Hypothesis 2b: Participants in activities with a critical reflection focus will have lower error rates on novel application tasks than those in nonreflective activities but higher than participants in activities with a reflection focus.

Hypothesis 3a: Activities with a reflection focus will affect the response rates of high and low reflective propensity participants differently.

Hypothesis 3b: Activities with a critical reflection focus will affect the response rates of high and low reflective propensity participants differently.

Hypothesis 4a: Activities with a reflection focus will affect the error rates of high and low reflective propensity participants differently.

Hypothesis 4b: Activities with a critical reflection focus will affect the error rates of high and low reflective propensity participants differently.

Method

Participants and Placement

Participants were adult students from an urban Midwestern 2-year technical college. Its population is approximately 30% low income, with an ethnic/racial demographic of 53% White, 28% African American, 14% Hispanic, 4% Asian American, and 1% Native American / Alaska Native. The gender distribution is 51% female and 49% male. Average student age is 27, and 65% of the student population is 25 or older. A convenience sample (n = 42) comprised student respondents to campuswide advertisements. The 36 male and 6 female participants were distributed across three predefined age categories: 24-29 years, 14.3%; 30-39 years, 28.6%; and 40+ years, 57.1%.

There were four selection criteria: Participants must be 24 years or older, hold a valid student ID from the college, hold a high school diploma or GED, and have no prior experience installing concrete or clay pavers. Participants received $40 for completing the study.

Procedures

Participants were assigned conditions via a matched random assignment procedure. Three blocking variables equated conditions: gender, age, and reflective propensity. Gender was categorized as male or female. Age was categorized as: 24-29, 30-39, and 40+. Reflective propensity was categorized according to the Kolb Learning Styles Inventory 3.0, which evaluates a learner’s tendency toward one of four learning styles: diverging (introverted/feeling), assimilating (introverted/intuition), converging (extraverted/thinking), and accommodating (extraverted/sensation). Divergent and assimilative learners are thought to use reflection more often in learning than accommodative and convergent learners (Kolb, 1984). Divergent and assimilative learners were therefore classified as high reflective propensity, and convergent and accommodative learners, low reflective propensity. The internal validity and reliability of the inventory have been repeatedly demonstrated (Kayes, 2005).

Once categorized, a participant was matched with two other participants sharing equivalent categories (e.g., male, 24-29 years old, high reflective propensity). Each member of a three-participant block was randomly assigned to a different 14-participant condition.

Course Design

Learners participated individually in a 50-minute course on how to install a 90-degree concrete paver walkway (see Figure 1). The course design followed a behavioral skills training model (Ward-Horner & Sturmey, 2012) based on four phases: instruction, modeling, practice, and feedback. Instruction and modeling phases were combined into one 25-minute block, practice and feedback phases into another. The instruction and modeling phases were prerecorded and presented on a 13-in. laptop computer for consistent delivery. This featured the instructor explaining the steps of paver installation while demonstrating the project that learners would later undertake. Immediately following, learners individually participated in 15-minute reflective activities that varied by condition. The control condition featured no reflective activity (interference task); the reflection condition, a reflective activity with a reflection focus; and the critical reflection condition, a reflective activity with a critical reflection focus. For a visual model of the procedural sequence for each condition, see Figure 2.

Figure 1.

Ninety-degree herringbone pattern.

Figure 2.

Visual model of procedural sequence for each condition (experimental design).

Following the first reflective activity, all learners individually participated in the 25-minute practice-and-feedback phase. Participants were asked to install a 25-square-foot, 90-degree herringbone-patterned paver walkway using the methods discussed and demonstrated earlier. Practice occurred within a prepared area on a 66- × 66-in. platform constructed from tongue and groove oriented–strand board subfloor panels with honeycombed plastic backing. A 58- × 58-in. frame constructed from PVC paver-edge restraints was fastened to the platform using three-quarter-inch bolts. Within the frame, 1 in. of screeded, washed concrete sand was distributed. One pallet of 4- × 8-in. Holland pavers rested adjacent to the frame. Precut pavers were stacked alongside these pavers. Participants were told that their performance would not be assessed and that the activity’s purpose was to gain experience. The researcher, unaware of participants’ assigned conditions, provided verbal feedback.

Feedback consisted of praise and correction. When performance conformed to instruction, the researcher verbally identified the behavior and stated, “Good job,” “That’s how it’s done,” or “Excellent work.” When an error occurred, the researcher verbally identified it and restated the applicable best practice. If the participant expressed confusion, the researcher modeled the behavior. At the activity’s conclusion, learners individually participated in a second 15-minute reflective activity that varied by condition.

Interference condition

Participants in the interference condition performed a task to prevent reflection from occurring. This condition substituted for a neutral (free choice) condition because some learners reflect independently, a confounding variable that may have eliminated group differences despite effective activities, thereby increasing the likelihood of a type 2 error. A reading-aloud procedure was used (Mulatti, Peressotti, Job, Saunders, & Coltheart, 2012). This procedure is thought to compel learner attention, thus interfering with the construction of further explicit knowledge. Participants were asked to read aloud for 15 minutes from the Scott, Foresman Handbook for Writers (Hairston & Ruszkiewicz, 1996)—a text selected for its unrelated subject matter. Participants read from a chapter on forming logical arguments and detecting fallacies. The research assistant stated “Continue” if participants paused from speaking for greater than 2 seconds. Concurrently, the research assistant scribed inconsequential notes to prevent demand characteristics (interpretations of the study’s purpose and alteration of behavior to align with those interpretations).

Reflection condition

Learners in the reflection condition participated in reflective dialogue (with reflection focus) with a research assistant. Cranton (1994, 2006) has claimed that educators can stimulate content and process reflection in instrumental learning contexts by asking suitable reflective questions. A series of questions was designed to incorporate the problem-solving and meaning-making nature of reflection. These questions were divided equally between two foci: the content related to the problem and the process of problem solving. Adhering to Mezirow (1998) and Silverman and Casazza’s (2000) reflection/critical reflection distinction, these questions did not attempt to identify and/or challenge hidden assumptions. Rather, they asked participants to examine actions and potential actions under accepted principles.

Questions consisted of 12 content and 12 process reflection inquiries divided into two groupings (see Figure 3). Grouping 1 was delivered during the first reflective activity, and it focused on content and potential problem solving related to instruction and modeling phases. Grouping 2 was delivered during the second reflective activity, and it focused on content and problem solving related to practice and feedback phases. Questions were delivered in paired sequences: A content reflection question always preceded a specific process reflection question. Participants were told there were no right or wrong answers. Short responses (≤1 sentence) were followed with probe questions: “Can you explain that further?” “What do you mean by that?” and “Can you give me an example?” The second reflective activity interval began with Grouping 2 questions regardless of Grouping 1’s completion. If either grouping was completed early, the research assistant asked participants to elaborate on previous answers.

Figure 3.

Questions and sequence for reflective activity with reflection focus.

Critical reflection condition

Learners in the critical reflection condition participated in reflective dialogue (with a critical reflection focus) with a research assistant. A series of questions was adapted from Cranton’s (1994, 2006) critical reflection examples, which Cranton extends as a way to facilitate content and process critical reflection (see Figure 4). Cranton cautions that these examples not oversimplify these processes; instead, she suggests that they aid in developing further strategies to encourage specific types of critical reflection. Accordingly, these examples served as models from which critical reflective questions were developed for this study’s focus.

Figure 4.

Questions and sequence for reflective activity with critical reflection focus.

Questions consisted of 12 content and 12 process reflection inquiries categorized according to Mezirow’s (1990) original three critical reflective foci: psychological, sociological, and epistemic meaning schemes. Questions were delivered sequentially in paired sequences: A content reflection question always preceded a specific process reflection question. Participants were again told that there were no right or wrong answers. The research assistant followed short responses (≤1 sentence) with follow-up questions (discussed previously). The second reflective activity interval began with the content reflection question immediately following the last discussed two-question sequence. If a participant completed all 24 questions before the second reflective interval’s conclusion, she or he was asked to elaborate on previous answers.

Novel application task

Following behavioral skills training and reflective activities, participants individually completed a novel application task. Learners were asked to install a 58- × 58-in. 45-degree herringbone paver walkway (see Figure 5) in a prepared area as previously described. A 14- × 14-in. photographic model featuring a 45-degree herringbone patterned walkway was displayed, and participants were told that it was a good example of the new pattern. The researcher then asked participants to complete the task as quickly as possible, with as few errors as possible, observing the methods and techniques discussed earlier. No further guidance, instructions, or feedback were given. The researcher then began timing the task. After 25 minutes, the activity was stopped, and participants were debriefed, thanked, and dismissed from the study.

Figure 5.

Forty-five-degree herringbone pattern.

Dependent Measures

To minimize outcome subjectivity, novel application tasks were quantitatively assessed via two data categories: number of pavers installed per 25-minute interval (response rate) and number of errors produced per paver installed (error rate). Errors consisted of a paver joint exceeding 1/8 in., an upside-down paver, a chipped or cracked paver, or a paver deviating from the 45-degree herringbone pattern. A participant’s error rate was her or his cumulative error total divided by the total number of pavers installed. Number of pavers installed per 25-minute interval was counted by visual inspection, so too were errors. Paver joints exceeding industry standard tolerance were verified with an electronic caliper, with accuracy to ±0.001 in.

Analysis

One-way analysis of variance (ANOVA) was conducted to evaluate mean differences between groups’ response and error rates. One test was performed per dependent variable. To identify interaction effects between reflective activity and reflective propensity on response and error rates, two-way ANOVA was conducted. Interaction effects were interpreted with tests for simple main effects to identify mean differences in each reflective activity condition of the two-factor data matrix. A power test based on an effect size of 0.5, an alpha level of 0.05, and a power of 0.80 (see Creswell, 2005) dictated the study sample (n = 42).

A multivariate ANOVA was avoided because the research questions did not require investigating dependent variable interactions (the multivariate ANOVA’s primary function). This also avoided a more complicated design with increased ambiguity between independent and dependent variables.

Results

Hypotheses 1a and 1b: Reflective Activity and Response Rate

Hypothesis 1a stated that participants in activities with a reflection focus would have higher response rates on novel application tasks than participants in critical reflection and nonreflection activities. A one-way ANOVA indicated that response rates did not differ between groups, F(2, 39) = 0.603, p = .552, η² = .031. Levene’s test indicated that the assumption of equal variances was met (F = 0.050, p = .952). Shapiro-Wilk’s test indicated that conditions were normally distributed: interference (p = .740), reflection (p = .500), and critical reflection (p = .182). The mean, standard deviations, and 95% confidence intervals are presented in Table 1. Hypothesis 1a was not supported. Neither was Hypothesis 1b: Participants in critical reflective activities did not have higher response rates than those in the nonreflection condition or lower response rates than those in the reflection condition.

Table 1.

Number of Paver Installed per 25 Minutes (Response Rate).

	Interference	Reflection	Critical reflection
M	86.214	73.214	77.571
SD	31.911	31.259	32.458
95% CI	[67.8, 104.6]	[55.2, 91.3]	[58.8, 96.3]

Note. CI = confidence interval.

Hypotheses 2a and 2b: Reflective Activity and Error Rate

Hypothesis 2a stated that participants in activities with a reflection focus would have lower error rates on novel application tasks than participants in critical reflection and nonreflection activities. A one-way ANOVA indicated that error rates did not differ between groups, F(2, 39) = 0.715, p = .495, η² = .036 (see Table 2). Assumptions of equal variance (F = 1.472, p = .242) and normal distribution were met: interference (p = .881), reflection (p = .132), and critical reflection (p = .229). Hypothesis 2a was not supported. Neither was Hypothesis 2b: Participants in critical reflection activities did not have lower error rates than those in the nonreflective condition but higher error rates than those in the reflective condition.

Table 2.

Number of Errors Committed per Paver Installed (Error Rate).

	Interference	Reflection	Critical reflection
M	0.439	0.362	0.363
SD	0.134	0.239	0.200
95% CI	[0.36, 0.52]	[0.22, 0.50]	[0.25, 0.48]

Note. CI = confidence interval.

Hypotheses 3a and 3b: Interaction on Response Rate

Hypothesis 3a stated that activities with a reflection focus would affect response rates of high and low reflective propensity learners differently. A two-way ANOVA indicated no main effect of reflective propensity, F(2, 39) = 2.243, p = .143, η² = .059, or interaction effect between reflective activity and reflective propensity, F(2, 39) = 1.353, p = .271, η² = .070 (see Table 3). Assumptions of equal variance were met (F = 0.947, p = .463). Hypothesis 3a was not supported. Neither was Hypothesis 3b: Reflective activities with a critical reflection focus did not affect the response rates of high and low reflective propensity learners differently.

Table 3.

Response Rates by Reflective Propensity and Reflective Activity Condition.

	Reflective propensity
	Reflective			Nonreflective
Condition	n	M (SD)	95% CI	n	M (SD)	95% CI
Interference	7	67.9 (26.7)	[44.0, 91.7]	7	104.6 (26.6)	[80.7,128.4]
Reflection	7	71.7 (24.3)	[47.9, 95.6]	7	74.7 (39.0)	[50.9, 98.6]
Critical reflection	7	75.9 (27.5)	[52.0, 99.7]	7	79.3 (27.5)	[55.4, 103.1]
Total	21	71.8 (29.3)	[58.0, 85.6]	21	86.2 (32.8)	[72.4, 100.0]

Note. CI = confidence interval.

Hypotheses 4a and 4b: Interaction on Error Rate

Hypothesis 4a stated that activities with a reflection focus would affect error rates of high and low reflective propensity learners differently. A two-way ANOVA indicated no main effect of reflective propensity, F(2, 39) = 0.733, p =.398, η² = .017 (see Table 4). Equal variances were confirmed (F = 1.561, p = .196). There was, however, an interaction between reflective activity and reflective propensity on error rate, F(2, 39) = 3.251, p = .050, η² = .153. Simple main effects post hoc analysis revealed no differences in error rate between high and low reflective propensity learners in the reflection condition, F(1, 36) = 0.061, p = .806, η² = .002, but differences within the critical reflection condition, F(1, 36) = 6.501, p = .015, η² = .153 (see Figure 6). Specifically, participants with a high reflective propensity had error rates 0.254 points lower than those with a low reflective propensity. Hypothesis 4b was supported.

Table 4.

Error Rates by Reflective Propensity and Reflective Activity Condition.

	Reflective propensity
	Reflective			Nonreflective
Condition	n	M (SD)	95% CI	n	M (SD)	95% CI
Interference	7	.480 (.13)	[.34, .62]	7	.398 (.13)	[.26, .54]
Reflection	7	.375 (.12)	[.23, .52]	7	.350 (.33)	[.21, .49]
Critical reflection	7	.236 (.14)	[.09, .38]	7	.490 (.18)	[.35, .63]
Total	21	.364 (.16)	[.28, .45]	21	.413 (.23)	[.33, .50]

Note. CI = confidence interval.

Figure 6.

Interaction of reflective activity condition and learning style on error rate.

Discussion

This study’s first two research questions asked if reflective activities affect learners’ response and error rates on novel applications of instrumentally learned skills. No overall rate differences attributable to reflective activities were found. When interpreting these findings, researchers should consider the interaction between reflective propensity and reflective activity, which illustrated how high and low reflective propensity learners respond diametrically to critical reflection, thereby neutralizing its overall effect. Previous empirical research showing no overall effect supports these findings (see Bannert, 2006; Grez et al., 2009; Lowe & Kerr, 1998). Only Wetzstein and Hacker’s (2004) findings of higher overall quality engineering designs in reflective activity groups appear contradictory. However, these stem from subjective quality assessments without interrater reliability measures. Possibly, judges assessed qualities outside the instrumental domain (e.g., aesthetic appeal).

This study’s initial findings may inform calls for reflective activities in construction education courses (see Boyd, 2012; Hayles & Holdsworth, 2008; Selman & Westcott, 2005). Educators seeking to improve overall construction skills learning and adaptability may find more effective alternatives. Even considering this study’s interaction effect—which revealed critical reflection’s polarizing effect on performance—it remains questionable to responsibly advocate for critical reflection where skills-based competencies affect stakeholders’ welfare. If some learners’ performances markedly improve while others worsen, are such activities effective and equitable strategies for bettering overall instrumental learning? Advocates may still value critical reflection activities for attaining different outcomes (e.g., altered habits of mind, enhanced collegiality, and improved political and social emancipation), but these were not this investigation’s focus.

This study’s last two research questions asked if learners’ reflective propensity interacts with reflective activities to affect response and error rate. Neither reflection nor critical reflection activities affected response rates of high or low reflective propensity learners differently. Additionally, no differences were found between high and low reflective propensity learners’ error rates within the reflection condition. Critical reflection activities, however, did affect high and low reflective propensity learners’ error rates differently. Specifically, high reflective propensity learners within this condition had the lowest error rates of any group and low reflective propensity learners, the highest.

Although this finding is unsupported by previous empirical research, it confirms theoretical claims. Kolb’s (1984) experiential learning theory, for instance, asserts that assimilative and divergent learners prefer reflective learning and therefore should excel in settings that promote varieties of reflection. Mezirow’s (2000) transformative learning theory, although not addressing reflective propensity directly, claims that critical reflection in skills-based learning can improve adaptive performance. This study’s findings with high reflective propensity learners are consistent with this assertion, particularly when it is augmented using recent claims asserting learning style’s influence on transformative learning experiences (see Kiely, 2005). These findings may help researchers understand the direction of this claim.

One potential explanation for high reflective propensity learners’ improved performance originates in contextualist epistemology. Fox (2006) has identified two varieties of contextualism relevant to learning: descriptive contextualism and functional contextualism. Each underlies a well-established theory describing learning as a process of relating new experience to previous experience (see Novak, 1998, for what he terms human constructivism, a theory based in descriptive contextualist epistemology, and Hayes, Barnes-Holmes, & Roche, 2001, for relational frame theory, a theory grounded in functional contextualist epistemology). Dewey (1933), writing before either was articulated, described such relations as an essential characteristic of reflection for creating meaning.

If one assumes that forming such connections is a learned skill, then it follows that some learners may have more advanced repertoires than others. These learners may benefit more from critical reflective activities that ask learners to relate novel skills (e.g., paver installation) to familiar experiences. Consider several of this study’s critical reflection questions: “(6a) What was the perception of laying pavers in your home community?” “(9a) What have you read or heard about laying pavers?” “(11a) What is your favorite way of learning a manual skill?” Activities focusing on reflection, rather than critical reflection, may not similarly affect error rate, because they emphasize the immediate methods, applications, and problems in the novel task. In this study’s reflection condition, for instance, 2 of 12 questions asked learners to connect paver-setting skills and knowledge with previous experiences. In the critical reflection condition, however, such questions were prevalent (9 of 12). Reflective learners in the reflection condition were therefore given fewer opportunities to relate novel experiences to familiar ones. Consequently, opportunities for drawing on models and methods from previous experience and for creating meaning were reduced, and performance was unaffected.

Cognitive psychology offers one potential explanation for nonreflective learners’ poor performance following critical reflection activities. Processing efficiency theory (Eysenck & Calvo, 1992) suggests that capacity of the working memory system is finite and that additional demands placed on this system during learning affect performance. Anxiety behavior, one such demand, is thought to reduce storage and processing capacity for concurrent tasks. Nonreflective learners may engage in anxiety behavior because they lack repertoires for critical reflection and perceive themselves unable to fulfill their classmates’ or instructor’s expectations. Alternatively, they may be proficient in critical reflection but find it ineffective and grow irritated when asked to participate in it. Additionally, critical reflection’s focus (i.e., hegemony, cultural norms, societal assumptions, personal biases) may contribute to anxiety behavior (see Fook & Askeland, 2007). That is, nonreflective learners may prefer avoiding these topics and may engage in anxiety behavior when restricted from doing so. Processing efficiency theory suggests that, in each instance, anxiety behavior interferes with performance and learning suffers.

These findings and explanations inform research showing that some reflective activities negatively affect learner performance. Broyles, Epler, and Waknine (2011) and Niessen and Volmer (2010) found that as reflective tasks increase demands on learners (potentially growing the number of relations that learners form), performance suffers. Neither study, however, controlled for nor considered reflective propensity. Reflective activities having a “higher cognitive load” or deemed “more intense” may seemingly affect aggregate data because learning styles are not partitioned in the analysis. Furthermore, reflective propensity, reflective activity, and task may interact. Wetzstein and Hacker (2004) concluded that dialogue-specific reflective verbalizations (e.g., those from this study) improve performance for complex problems but worsen it for simple and predominantly nonverbal tasks.

Future Research

This study differs from others in how it categorized its dependent measures: response and error rate. Written assessment scores—common dependent measures in previous studies—are easily transformed to error rate and likely represent effective performance. Response rate, however, is not traditionally measured in written assessments and likely represents a different quality—efficient performance. This study’s findings on response rate, then, add to the literature on reflective activities and instrumental learning outcomes. Researchers may wish to examine the relationship between reflective activities and specific performance indicators (e.g., response and error rate) rather than general terms (e.g., “instrumental learning outcomes”). Additionally, this study addresses recent calls to examine how reflective activities influence adult motor learning (see Roessger, 2012). By replicating it in other trades, researchers can begin illuminating this line of inquiry.

Researchers may also wish to explore how experienced learners respond to reflective activities in instrumental learning. This study focused on novice learners. Daley (1999) has noted that novices and experts use different learning strategies in work-related learning. It remains unclear how reflective activities affect experts’ skill adaptations. Do nonreflective experts experience similar performance declines following critical reflection activities? Do reflective experts exhibit similar improvements? Might some professions select for reflective learners and thereby eliminate practical concerns over critical reflection’s polarizing effects? Additionally, researchers may consider how course duration affects instrumental learning outcomes. This study featured 50 minutes of instruction and 30 minutes of reflective activities—a duration indicative of workshops and short courses in continuing professional education. Might longer exposure to reflective activities improve adaptive competencies? Might learners need to acclimate to reflection and critical reflection?

Last, researchers may consider how other learning activities compare with critical reflection. Do alternatives affect overall performance changes in instrumental learning? Do they avoid nonreflective learners’ performance declines? To gain answers, one could replicate the current study substituting different learning activities for the interference and reflection conditions. Researchers could then examine how different activities and reflective propensity affect response and error rates.

Implications

Considering how poorly nonreflective learners performed in the critical reflection condition, some may conclude that educators should teach this group how to critically reflect. This study’s findings do not support this conclusion. The Kolb Learning Styles Inventory 3.0 did not measure learners’ ability to reflect—rather, their preference or propensity to reflect. Nonreflective learners may be as capable of or as skilled at critically reflecting as others but, for myriad reasons, find it unnecessary.

Educators in work-related instrumental learning settings may be best served allowing learners to choose between critical reflection and other empirically supported activities. This position supports principles of universal design for learning, which advocate addressing learner differences through multiple methods of content engagement (see Rose & Strangman, 2007). Reflective learners, then, can participate in critical reflection and improve adaptive competencies, while nonreflective learners can participate in alternative activities and avoid performance declines. Some educators may hesitate in advocating for nonreflective approaches—reflection has, in fact, been called the defining quality of adult learning (Mezirow, 1991)—but numerous studies empirically demonstrate their effectiveness in select contexts. Karpicke and Blunt (2011), for instance, found that retrieval practice (i.e., recalling salient information following study) produced significantly better performance on short-answer tests than elaborative concept mapping, an activity advocated by numerous adult educators (see Daley et al., 2010). A follow-up experiment replicated these findings despite learners consistently predicting that concept mapping would yield better results. Bucklin, Dickinson, and Brethower (2000) found that fluency training (i.e., practice for both speed and accuracy) produced higher response rates, better accuracy, and less deterioration of accuracy in skilled performance than accuracy training (i.e., practice for accuracy alone). A comprehensive review is outside this paper’s scope, but these examples demonstrate effective, available alternatives to critical reflection. Offering nonreflective learners choices may be the most effective way to help them attain their goals.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biography

Kevin M. Roessger is an assistant professor of adult education and training at Seattle University. He teaches graduate courses in all areas of adult workplace learning. His research interests include: behavior analytic approaches to skills training, deliberate practice, reflective actviities in skills-based learning, and reflection as a natural phenomenon.

References

Bannert

(2006). Effects of reflection prompts when learning with hypermedia. Journal of Educational Computing Research, 35, 359-375.

Borko

Michalec

Timmons

Siddle

. (1997). Student teaching portfolios: A tool for promoting reflective practice. Journal of Teacher Education, 48, 345-357.

Boud

(2001). Using journal writing to enhance reflective practice. New Directions for Adult and Continuing Education, 90, 9-18.

Boud

Keogh

Walker

(1985). Promoting reflection in learning: A model. In Boud

Keogh

Walker

(Eds.), Reflection: Turning experience into learning (pp. 18-40). London, UK: Kogan Page.

Boyd

(2012). Critically learning from practice in construction education. In Sulbaran

(Ed.), Forty-eighth ASC annual international conference proceedings. Hattiesburg, MS: Associated Schools of Construction. Retrieved from http://ascpro.ascweb.org/chair/paper/CEGE175002012.pdf

Brookfield

S. A.

(2000a). The concept of critically reflective practice. In Wilson

A. L.

Hayes

E. R.

(Eds.), Handbook of adult and continuing education (pp. 110-126). San Francisco, CA: Jossey-Bass.

Brookfield

S. A.

(2000b). Transformative learning as ideology critique. In Mezirow

& Assoc. (Eds.), Learning as transformation (pp. 125-148). San Francisco, CA: Jossey-Bass.

Broyles

T. W.

Epler

C. M.

Waknine

J. W.

(2011). A case study examining the impact of cognitive load on reflection and pre-service teachers’ transfer of specific teaching behaviors. Career and Technical Education Research, 36, 49-66.

Bucklin

B. R.

Dickinson

A. M.

Brethower

D. M.

(2000). A comparison of the effects of fluency training and accuracy training on application and retention. Performance Improvement Quarterly, 13, 140-163.

10.

Burke

M. J.

Scheuer

M. L.

Meredith

R. J.

(2007). A dialogical approach to skill development: The case of safety skills. Human Resource Management Review, 17, 235-250.

11.

Cornford

I. R.

(2002). Reflective teaching: Empirical research findings and some implications for teacher education. Journal of Vocational Education and Training, 54, 219-235.

12.

Cranton

(1994). Understanding and promoting transformative learning. San Francisco, CA: Jossey-Bass.

13.

Cranton

(1996). Professional development as transformative learning: New perspectives for teachers of adults. San Francisco, CA: Jossey-Bass.

14.

Cranton

(2000). Individual differences and transformative learning. In Mezirow

(Ed.), Learning as transformation: Critical perspectives on a theory in progress (pp. 181-204). San Francisco, CA: Jossey-Bass.

15.

Cranton

(2006). Understanding and promoting transformative learning (2nd ed.). San Francisco, CA: Jossey-Bass.

16.

Creswell

J. W.

(2005). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (2nd ed.). New York: Pearson.

17.

Daley

B. J.

(1999). Novice to expert: An exploration of how professionals learn. Adult Education Quarterly, 49, 133-147.

18.

Daley

B. J.

Conceição

S. C.

Mina

Altman

B. A.

Baldor

Brown

(2010). Integrative literature review: Concept mapping: A strategy to support the development of practice, research, and theory within human resources development. Human Resources Development Review, 9, 357-384.

19.

Dennett

D. C.

(1995). Darwin’s dangerous idea: Evolution and the meanings of life. New York, NY: Touchstone.

20.

Dewey

(1933). How we think: A restatement of the relation of reflective thinking to the educative process (2nd ed.). New York, NY: Heath.

21.

Dewey

(1938). Experience and education. New York, NY: Touchstone.

22.

Dirkx

J. M.

(2013, October). Re-Imaging Adult Education: The Role of Emotional-Laden Experience. Keynote speech presented at the Western Region Research Conference on the Education of Adults, Seattle, WA.

23.

Duke

Appleton

(2000). The use of reflection in a palliative care program: A qualitative study of the development of reflective skills over an academic year. Journal of Advanced Nursing, 32, 1557-1568.

24.

Eysenck

M. W.

Calvo

M. G.

(1992). Anxiety and performance: The processing efficiency theory. Cognition and Emotion, 6, 409-434.

25.

Fook

(2010). Beyond reflective: Reworking the ‘critical’ in critical reflection. In Bradbury

Frost

Kilminster

Zukas

(Eds.), Beyond reflective practice: New approaches to professional lifelong learning (pp. 37-51). New York, NY: Routledge.

26.

Fook

Askeland

G. A.

(2007). Challenges of critical reflection: “Nothing ventured, nothing gained.” Social Work Education: The International Journal, 26, 520-533.

27.

Fook

Gardner

(2007). Practicing critical reflection: A resource handbook. Maidenhead, UK: Open University Press.

28.

Fox

E. J.

(2006). Constructing a pragmatic science of learning and instruction with functional contextualism. Educational Technology Research and Development, 54, 5-36.

29.

Grez

L. D.

Valcke

Roozen

(2009). The impact of goal orientation, self-reflection and personal characteristics on the acquisition of oral presentation skills. European Journal of Psychology of Education, 24, 293-306.

30.

Habermas

(1971). Knowledge and human interests. Boston, MA: Beacon Press.

31.

Hairston

Ruszkiewicz

J. J.

(1996). The Scott, Foresman handbook for writers (4th ed.). New York, NY: HarperCollins.

32.

Hayes

S. C.

Barnes-Holmes

Roche

(Eds.). (2001). Relational frame theory: A post-Skinnerian account of human language and cognition. New York, NY: Plenum Press.

33.

Hayles

C. S.

Holdsworth

S. E.

(2008). Curriculum change for sustainability. Journal for Education in the Built Environment, 3, 25-48. Retrieved from http://cebe.cf.ac.uk/jebe/pdf/Hayles&Holdsworth3%281%29.pdf

34.

Heinrich

K. T.

(1992). The intimate dialogue: Journal writing by students. Nurse Educator, 17(6), 17-21.

35.

Hill

(2005). Reflection as a professional development strategy during organizational change. Reflective Practice, 6, 213-220.

36.

Karpicke

J. D.

Blunt

J. R.

(2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 331, 772-775.

37.

Kayes

D. C.

(2005). Internal validity and reliability of Kolb’s Learning Style Inventory version 3 (1999). Journal of Business and Psychology, 20(2), 249-257.

38.

Kiely

(2005). A transformative learning model for service-learning: A longitudinal case study. Michigan Journal of Community Service Learning, 12, 5-22.

39.

Kolb

D. A.

(1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall.

40.

Kreber

(2010). Critical reflection and transformative learning. In Taylor

E. W.

Cranton

(Eds.), The handbook of transformational learning: Theory, research, and practice (pp. 323-341). San Francisco, CA: Wiley.

41.

Lowe

P. B.

Kerr

(1998). Learning by reflection: The effect on educational outcomes. Journal of Advanced Nursing, 27, 1030-1033.

42.

Mackintosh

(1998). Reflection: A flawed strategy for the nursing profession. Nurse Education Today, 18, 553-557.

43.

Malkki

Lindblom-Ylanne

(2012). From reflection to action? Barriers and bridges between higher education teachers’ thoughts and actions. Studies in Higher Education, 37, 33-50.

44.

Mamede

Schmidt

H. G.

Cesar Penaforte

(2008). Effects of reflective practice on the accuracy of medical diagnoses. Medical Education, 42, 468-475.

45.

Mamede

Schmidt

H. G.

Rikers

(2006). Diagnostic errors and reflective practice in medicine. Journal of Evaluation in Clinical Practice, 13, 138-145.

46.

Mann

Gordon

MacLeod

(2009). Reflection and reflective practice in health professions education: A systematic review. Advances in Health Science Education, 14, 595-621.

47.

Merriam

S. B.

Caffarella

R. S.

Baumgartner

L. M.

(2007). Learning in adulthood: A comprehensive guide (3rd ed.). San Francisco, CA: Jossey-Bass.

48.

Mezirow

(1990). How critical reflection triggers transformative learning. In Mezirow

& Associates (Eds.), Fostering critical reflection in adulthood (pp. 1-20). San Francisco, CA: Jossey-Bass.

49.

Mezirow

(1998). On critical reflection. Adult Education Quarterly, 48, 185-199.

50.

Mezirow

(2000). Learning to think like an adult: Core concepts of transformation theory. In Mezirow

& Associates (Eds.), Learning as transformation (pp. 3-33). San Francisco, CA: Jossey-Bass.

51.

Mezirow

(2009) An overview of transformative learning. In Illeris

(Ed.), Contemporary theories of learning: Learning theorists . . . in their own words (pp. 90-105). New York, NY: Routledge.

52.

Moon

(1999). Reflection in learning and professional development: Theory and practice. Sterling, VA: Kogan Page.

53.

Mulatti

Peressotti

Job

Saunders

Coltheart

(2012). Reading aloud: The cumulative lexical interference effect. Psychonomic Bulletin and Review, 19, 662-667.

54.

Niessen

Volmer

(2010). Adaptation to increased work autonomy: The role of task reflection. European Journal of Work and Organizational Psychology, 19, 442-460.

55.

Novak

J. D.

(1998). Learning, creating, and using knowledge; Concept maps as facilitative tools in schools and corporations. Mahwah, NJ: Erlbaum.

56.

Roessger

K. M.

(2012). Toward an interdisciplinary perspective: A review of adult learning frameworks and theoretical models of motor learning. Adult Education Quarterly, 62, 371-382.

57.

Roessger

K. M.

(2013). But does it work? Reflective activities, learning outcomes, and instrumental learning in continuing professional development. Journal of Education and Work. Advance online publication. doi:10.1080/13639080.2013.805186

58.

Rose

D. H.

Strangman

(2007). Universal design for learning: Meeting the challenge of individual learning differences through a neurocognitive perspective. Universal Access in the Information Society, 5, 381-391.

59.

Ruth-Sahd

L. A.

(2003). Reflective practice: A critical analysis of data-based studies and implications for nursing education. Journal of Nursing Education, 42, 488-497.

60.

Schön

D. A.

(1983). The reflective practitioner: How professionals think in action. New York, NY: Basic Books.

61.

Schön

D. A.

(1987). Educating the reflective practitioner. San Francisco, CA: Jossey-Bass.

62.

Selman

Westcott

(2005). Interprofessional issues in construction education. In Tommelein

I. D.

(Ed.), Proceedings of the Construction Research Congress 2005: Broadening perspectives. Reston, VA: American Society of Civil Engineers. Retrieved from http://ascelibrary.org/doi/pdf/10.1061/40754%28183%2944

63.

Silverman

Casazza

(2000). Learning and development: Making connections to enhance teaching. San Francisco, CA: Jossey-Bass.

64.

Taylor

(2001). Identifying and transforming dysfunctional nurse-nurse relationships through reflective practice and action research. International Journal of Nursing Practice, 7, 406-413.

65.

Van den Boom

Paas

Van Merrienboer

J. J. G.

(2007). Effects of elicited reflections combined with tutor or peer feedback on self-regulated learning and learning outcomes. Learning and Instruction, 17, 532-548.

66.

Van Manen

(1991). Reflectivity and pedagogical moment: The normativity of pedagogical thinking and acting. Journal of Curriculum Studies, 23, 507-536.

67.

Ward-Horner

Sturmey

(2012). Component analysis of behavior skills training in functional analysis. Behavioral Interventions, 27, 75-92.

68.

Warhurst

(2008). Reflections on reflective learning in professional formation. Studies in the Education of Adults, 40, 176-191.

69.

West

(2010). Really reflective practice: Auto/biographical research and struggles for a critical reflexivity. In Bradbury

Frost

Kilminster

Zukas

(Eds.), Beyond reflective practice: New approaches to professional lifelong learning (pp. 66-80). New York, NY: Routledge.

70.

Wetzstein

Hacker

(2004). Reflective verbalization improves solutions: The effects of question-based reflection in design problem solving. Applied Cognitive Psychology, 18, 145-156.