Abstract
As an assignment in their course on worldwide religions, a group of Swedish High School pupils followed 12 biblical rules for two weeks, while another group from the same school just imagined the experience. Groups were asked to reflect and write down either how it was (experience) or how it would have been (imagine) to follow the rules. By applying a semantic test, based on a Latent Semantic Analysis generated representation of the statements, we first found that the semantic representations of the written reflections differed between the experience and imagine groups, and between gender. Analysis of word frequency count suggests that the group that followed the rules were more likely to use words related to their task in their reflections, while the group that imagined the experience generated words related to themself and problems. The results suggest that the consequences of learning by experience might culminate in greater student engagement.
Keywords
Kolb’s Experiential Learning Theory defines learning as ‘the process whereby knowledge is created through the transformation of experience. Knowledge results from the combination of grasping and transforming experience’ (Kolb, 1984, p. 41). Concrete experiences are the basis for observations and reflections, which in turn, are assimilated into abstract concepts from which new decisions can be drawn (Kolb, Boyatzis, & Mainemelis, 2000). Thus, the learner needs not only to experience something in order to learn, the experience needs to be reflected upon if learning is to take place. Reflecting about an actual experience should then be different from a theoretical or imagined experience of the same phenomena.
We designed a two-week high school assigment that could be performed either by experience or by just imagining how it would be to perform the assignment for the same period of time. The assignment was embedded within a course on Worldwide Religions and pupils were asked to reflect and write down their thoughts about how it was, or how it would have been, to complete the assignement. We found this specific course of special interest to test our hypothesis—Sweden is one of few countries in which pupils learn about worldwide religions at school—the aim of the course is to ask pupils to ‘reflect on other peoples’ different ways of thinking about life, beliefs, and ethics’ (Skolverket, 2009). Moreover, relating to others is essential for adolescents’ well-being (Magen, 1998).
In order to investigate pupils’ acquired knowledge in both versions of the assignment we used a computational method for quantifying semantic content of words called Latent Semantic Analysis (LSA). This method is based on algorithms stemming from computational linguistics where a high dimensional semantic representation of words can be generated from co-occurrence of words in massively-large text corpora (Landauer, 2008). The context in which words are typically present has a meaning that is similar to the meaning of the words (Landauer & Dumais, 1997). Applied to a text corpus, it produces a high dimensional semantic space, in which each word is represented as a vector in this space. The basic idea is that co-occurrences of words produce information about the semantic meaning of the words.
We used the semantic quantifications of written reflections to test statistically if the reflections differed beteween participans that lived the experienced and those who only imagined it. We also investigated whether the semantic content differed between texts generated by boys and by girls. In summary, we outlined and tested statistical differences among the words that are more frequently used in the two versions of the assignment (experience vs. imagine), and by gender (boys vs. girls).
Method
Participants and procedure
A class of Swedish high school pupils in the South of Sweden was asked to participate in ‘Living Biblically’, which is a free-choice-assignment in their course on worldwide religions. The assignment was inspired by the experience of A. J. Jacobs who wrote of his experience when he decided to follow biblical rules for a year (Jacobs, 2008). The rules were selected by the pupils’ religion teacher after brief communication with Jacobs.
Parents were informed of the nature of the assignment and questions and concerns were directed to the teacher, who was blind to the actual hypothesis. Fifteen pupils (8 boys, 7 girls, 17-years-old; 43% of the class) chose to participate in the experience version of the assignment. Participants were instructed to strive to follow 12 rules (see Supplemental Materials) for a period of two weeks and to write an internet dairy in which they reflected about their daily experiences when following the rules. At the end of the two weeks, participants wrote their final reflection, which was used in the analysis.
Another class, in the same school, was also asked to participate in ‘Living Biblically’ as part of their course on worldwide religions. However, these pupils were instructed only to read the 12 rules and to reflect on how it would have been to follow these rules for two weeks. A total of eight pupils (1 boy, 7 girls, 17-years-old; 73% of the class) participated in this imagine version of the assignment. Pupils who chose not to participate were assigned another task during the same time span.
Statistical treatment
Latent Semantic Analysis (LSA)
The texts generated by the subjects were not sufficiently large to construct a high quality representation; thus, a semantic representation of Swedish words was generated using a collected corpus based on 100k Swedish news articles (150 Mb), collected from the 100 largest Swedish newspapers during 2007 (see Garcia & Sikström, 2012). The result of this analysis is a semantic quantification of the 100,000 most frequent words in the Swedish corpus, where each word is described by a 100 dimensional vector, where each dimension (bounded by −1 and +1) represent a semantic feature of the words. An intuitive and qualitative understanding of what they represent can be obtained generalizing from words that are either high or low in a certain dimension; however, such interpretation could be difficult to make. This representation is typically more comprehensible by investigating words with similar values on all dimensions that typically are semantically similar as judged by humans, or are synonyms. Each reflection was summarized in the semantic representations by simply adding the semantic vectors representing all words in each participant’s own written text. The resulting vector was normalized to a length of one.
We investigated whether the semantic representation of the written reflections was statistically different between conditions (experience vs. imagine) and gender. This was done by first creating a data-generated semantic scale between the conditions; each condition was first summarized into one semantic vector, where the to-be-measured semantic statement were removed (i.e. the semantic scale is slightly differently for each statement), and then normalizing the vectors to the length of one. Then the differences between these two vectors were calculated, and the resulting difference vector was also normalized to the length of one.
The semantic scale is then defined as the semantic distance between the differences vector and the vector representing the removed statement—where the semantic distance is the defined as the dot product (sum of the multiplication of dimension) between the vectors. This resulted in a one value (spanning between +1 and −1) for each statement (where the method above was repeated for each statement), representing how similar one statement is to either of the conditions. A semantic test was then performed by subjecting a standard t-test on these values to investigate whether the two conditions differ. (For a detailed description of semantic test see Arvidsson, Werbart, & Sikström, 2011; see also Garcia and Sikström, 2012 for a related method.)
Frequency analysis
We were interested in analysing whether the number of occurrence of some of the words was over-represented in one data set compared to the other data set. The standard method for such frequency analysis is to conduct a chi-square test for easy unique word, and correct the p-value for multiple comparisons. In total there were 13,064 words in the experience dataset, 10,718 words in the imagine dataset, and 2,585 unique words in both datasets. A frequency vector was generated consisting of the number of occurrences of each unique word in the experience and imagine datasets, respectively. For each unique word a chi-square test was conducted, where the input consisted of four numbers: The frequencies of the word in the two datasets, and the number of remaining words in the two datasets. These four values were applied to 2 x 2 chi-square matrix, which tests for independency between the numbers (i.e. a significant value indicates that a word is more likely than chance to be placed in one category over the other). This procedure was repeated for each unique word, and the resulting p-values were corrected for multiple comparisons using the Bonferroni method (i.e. multiplying each p-value with the number of unique words). Words that were significant (at the 0.05 level) are presented below. The same statistical treatment was applied to the text generated by boys (7,752 words) and girls (16,030 words), which together included 2,585 unique words.
Results and discussion
Semantic test
The semantic representation differed between the assignments (p = 0.0052) and gender (p = 0.0008) as tested by t-tests applied to the semantic dimensions representing assignments or gender respectively. Participants who strived to follow the rules generated a significantly different semantic representation than participants who just imagined doing so. As explained by Gilbert (2007), we humans’ lack accuracy when forecasting affective experience of imagined experiences. This might explain the differences in semantic content between written reflections from each condition. The results also suggest that reflections generated by boys differ in semantic content to those generated by girls. This specific result should be interpreted cautiously due to unequal gender distribution between conditions.
Frequency analysis
The results show that pupils who followed the rules for two weeks where more prone to use the terms: Rules (chi-square = 36.32), they (chi-square = 33.30), follow (chi-square = 24.14), and assignment (chi-square = 18.68). Pupils who imagined the experience were prone to use the terms: I (chi-square = 21.00), problem (chi-square = 19.12), and talk (chi-square = 18.29). When individuals are immersed in a task they tend to use I at very low levels (Pennebaker, 2011), as pupils who lived the experience seem to do. These pupils were prone to use the pronoun they; suggesting they took note of others’ perspective (Pennebaker, 2011). Conversely, imagining the experience drew adolescents’ attention to their own struggles—as suggested by the frequency of the words I and problem. We suggest that pupils who lived the experience were more engaged in the assignment—thereby explaining the co-occurrence of words related to the task. Pupils imagining the experience were more self-conscious in their reflections. The recurrent use of I, for instance, distinguish real vs. fabricated stories (Pennebaker, 2011).
Boys’ reflections contained the following significant words: Rules (chi-square = 54.28), they (chi-square = 29.15), sat (chi-square = 28.97), these (chi-square = 27.03), woman (chi-square = 24.83), religion (chi-square = 23.06), menstruating (chi-square = 20.69), bible (chi-square = 18.62), and you (chi-square = 18.51). Reflections by girls contained only one significant word: had (chi-square = 24.22). Boys appear focused on the rules and women-related-words, perhaps because one of the rules was directed specifically to women. The recurrent use of the verb had among girls suggests that girls were prone to refer to more active and dynamic events in their reflections (Pennebaker, 2011).
Implications
By using a relatively new method for quantifying written reflections of high school pupils, we have demonstrated that reflections of lived and imagined learning experiences at school differ from each other in important ways. That reflecting upon learning experiences is important, is indeed, a recurrent observation in the literature. For example, learning-journals are a way for teachers to help pupils to reflect on a school topic and to foster self-regulated learning (e.g. Nückles, Hübner, & Renkl, 2009). Learning journals are expected to help pupils to ‘articulate what they found personally interesting and important and how the new information relates to what they already know about the subject’ (Schmidt, Maier, & Nückles, in press, p. 3). Thus, learning-journals are a way to reflect upon concrete experiences, to assimilate these experiences, and to draw new conclusions as suggested by Kolb’s Experiential Learning Theory (Kolb et al., 2000). Nevertheless, our study suggests that these reflections might be different for pupils who have lived an experience compared to those who only had been asked to reflect upon an imaginary experience. Moreover, the method used here (i.e. LSA) opens new possibilities for objectively studying the content of learning-journals, to supplement the study of outcome from the experience of writing the journal.
A concrete scenario, which school psychologist and teachers might find useful, is to apply the study design presented here to the school environment—but within-subjects instead of between-subjects. That is, first let pupils reflect on phenomena that they only have theoretical knowledge about; second, the teachers can develop an assignment around this phenomenon in which the same pupils participate and write a learning-journal. As suggested by the results presented here, the reflections of the same individual should differ in semantic content. The teacher and psychologist can then use the written reflections to help pupils to become more self-aware of their changes in attitudes towards the studied phenomenon.
Conclusion
Reflections generated by pupils differ in semantic content depending on whether they experienced or imagined an assignment. Pupils living an experience were focused on the task, while pupils imagining the experience seemed more focused on their own feelings and struggles. We suggest that the consequences of learning-by-experience might culminate in greater levels of student engagement and perhaps even influence adolescents to take greater account of the others’ perspective.
‘Nothing we ever imagined is beyond our powers, only beyond our present self-knowledge’ Theodore Roszak.
Footnotes
Note
Author biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
