Embodiment in knots of sense-making between learning physical and virtual configurations

Abstract

The purpose of this study was to explore the implications of students’ and teacher’s creative configuration in the physical and virtual world. This analysis will be performed in a secondary school. Adopting a socio-material perspective, this paper focuses on embodiment configuration features for sense-making via new technology mediation. The context of this study was a secondary school who were adopting an immersive 3 D virtual world in different teaching and learning subjects. Selected episodes from video-recordings of two types of sessions mediated by a virtual world – online and in the classroom – were analysed. The analytical framework of this paper draws on the complex and creative configurations of the body in both the physical and virtual setting. Our results highlight the creative ways in which the arrangement of teacher and student bodies acted as a mediational instrument between real and virtual settings.

Keywords

Sense-making embodiment secondary school virtual world configuration creativity

Introduction

A Virtual World (VW) has proven to be rather special due to its interactive quality and for the possibilities of seeing, moving, touching and doing. Advances in Virtual Reality (VR) becomes a multisensory experience that immerses users to an extent not previously thought possible. The body becomes present in virtual space, meaning that the device is adapting to the body while the body is adapting to the device. Taking into consideration this specific technology, this paper aims to analyse the implications of the body’s configuration features in the sense-making process, specifically in the physical and virtual learning context. In the next section, we will briefly discuss our conceptualization of embodiment for the present investigation. We will then describe the context, methods and findings of a case study involving a VW teaching session in a French secondary school. Finally, we will discuss the findings and conclude them.

Embodiment between a real and virtual dimension

The concept of the embodiment has gained popularity, particularly in light of new forms of interaction and engagement with emergent technologies. This perspective offers rich reflections for conceptualising the relationship between the physical actions of the body and meaning-making processes (Cheville, 2006; Dourish, 2004).

The physical body is on one side of the frame of reference, the side in which all of our experiences take place; while on the side, our experiences emerge from our sensory perceptions. So, it is the main link between the mind and the world (Lakoff and Johnson, 1999). Embodied interaction claims that any action is constituted by a complex arrangement of multiple semiotic fields (e.g. gestures, the body, language), that are deployed simultaneously and influence each other. Goodwin (2000) wrote that the notion of embodiment encompasses orientation, gesture and intonation, among other physical aspects of interaction. The moment-to-moment arrangement of these semiotic fields is called a contextual configuration (Goodwin, 2000). The contextual configuration changes during the action; new semiotic fields can be brought in and old ones treated as irrelevant, thus, adapting with the news proposes with the news proposed. Human activity can, therefore, be understood through the contextualised experience of a body-environment system, and not as the result of representations of the world disconnected from a context.

Within the virtual dimension, the body becomes central to all interactions that manipulate any available information. Transportation in a virtual setting is an experience of cognitive, emotional imagination that evolves into a narrative (Green et al., 2004). In such systems, media interface development aims to provide users with fully immersive experiences, with the ultimate goal of making the virtual experience something natural and pleasant for the user. With the use of different sensory devices, the body plays a central role by becoming progressively extended into the virtual dimension. It is, therefore, challenging to determine how the body enters into the real and virtual setting configurations, or how it gains significance about the activity. Also, the body needs to be communicated with differently in this ‘hybrid’ setting of the virtual and real worlds. Here, the concept of multi-activity is mobilized to indicate the struggle of coordinating interaction between teacher and students with their respective co-present and 'virtual' body (Mondada, 2011).

Discursive framing of embodiment: Knots of sense-making between real and virtual

The virtual setting could be, at the same time, a communicative, educational and ludic setting. We consider the context of an activity as the outcome of a continuous process of social construction engaged in by its participants, who according to the activity of the moment, embody meaning in the spaces and artefacts around them (Cole, 1996; Duranti and Goodwin, 1992). The term interaction indicates the complex construction of meaning that human beings undertake when acting in a space, be it simulated or not. This strict connotation between virtual and material is metaphorically called ‘knots’, to show the interweaving of the aspects considered: body versus avatar, classroom versus VW dimension and material versus online resources and tools. The materialities that co-present and virtual bodies encounter, manipulate, utilise and transform in the course of their activities promote a new and extended multimodal analysis of social interaction (Mondada, 2019).

When considering this new modality, it is interesting to focus on how the participants make sense of their environment and learning activity through the continued reference to their real and virtual body – something discursively negotiated in a specific space and time setting. Bodies, language and objects make sense in this respect as they are meaningfully assembled in the production of actions as well as in their interpretations (Mandada, 2019). Meaning emerges as a feature of action in situ within the arrangements of bodies, objects, places and environments, along with silent or talkative practices. Klein et al. (2006) briefly defined sensemaking as a motivated, continuous effort to understand connections among people, places, and events to anticipate and act effectively. Within this process, language has been conceptualised as an intersubjective phenomenon, constitutive of human sensemaking (Salvatore and Venuleo, 2013). Research on developing an embodied analysis of sensemaking is less developed, especially in a learning context mediated by a virtual dimension. Nevertheless, it is interesting to see how the embodied dimension of the real and virtual body of the participants in a specific framing of space-time, contributes to the sense-making process through a discursive negotiation.

The study: Learning activities mediated by VW

Research questions

Our main research questions for this study are as follows:

Which kind of creative configurations of student and teacher interactions emerge in a hybrid setting mediated by VW?

How are creative configurations of student and teacher interactions discursively negotiated in a hybrid setting mediation by VW?

To reach this aim, we analysed teacher and student’s embodiment configuration features that emerged in the classroom and online learning settings (that were mediated by VW), and how they were discursively negotiated.

Methods

Context and data collection: VW in a French secondary school

This study is part of an extended project that took place in a secondary school in the south-east of France (Gadille and Impedovo, 2019, 2020). This school was part of a technological experiment in-line with the Digital Plan for Education launched in 2015 by the French government. The main aims of the project were: to propose a scientific study about the technological and social conditions following the appropriation of VW; to support (in the territory of the academy via a collaborative network) the sharing of knowledge resulting from the joint work of researchers and teachers; to create a training guide for trainers or teachers wishing to get involved in this field.

Participatory and collaborative processes were developed between teachers, researchers, ICT staff and institutional partners, to achieve a critical change of practices; this would be done through a continuous process of planning, action, observation, reflection and re-planning (McIntyre, 2008). The methodological approach of the project is in-line with design-based research and used scaffolding process to empower the participants with the technology proposed.

The VW proposed is technologically advanced for the context, especially as the participants had only been introduced to the tablet in the previous year, and had not experienced a VW. Indeed, the students and teachers used a screen-based system and an avatar-based system (rather than goggles, haptic technologies and so on). At the beginning of the project, all teachers and students involved in the VW attended an initial one-hour training, where they learned some basic notions of VW. The introducing of VW to participants in the pedagogical scenario was planned and adopted by the teachers; a tailored scenario was then co-developed by them, alongside the ITC support (of the school) and the research team. All the data has been made anonymous and the children’s families were consulted and asked for consent before video recording. The collection of data consists of ethnographic observations (with note-taking and photos), full video-recordings of classroom lessons (45-minutes duration), as well as the automatic recording of log data and virtual activities. The main data was catalogued (Heath et al., 2010) and organised in collections using Nvivo, software specifically developed for qualitative text and video analysis.

Data analysis

This study was developed for a participative observation of student and teacher interactions within teaching settings mediated by VW. To meet the aims of this paper, our main data sources were the audio-video recordings; the other data sources were primarily used to clarify and enrich our interpretation of the collaborative student and teacher activity. We analysed six video recordings in-depth based on two learning conditions that we were exploring: the first being in the classroom using the VW, and the second being the use of VW only. The six videos used consisted of four video recording sessions in the classroom; this involved the use of a VW about Mathematics, French and English – the setting was a sixth-grade classroom composed of 22 students. Two video recordings consisted of online VW sessions in Math. These sessions involved four students in the fifth grade. All students were familiar with the VW before starting. See Table 1 for a summary of the video-recording sessions selected.

Table 1.

Selected teaching sessions.

Condition	Classroom	Lesson session
Teacher and students in the classroom using VW	6th grade – 22 students	1 Math
Teacher and students in the classroom using VW	6th grade – 22 students	3 Language
Teacher and students online using VW	5th grade – 4 students	3 Math

The video data analysis of the learning sessions (involving VW mediation in the classroom and online), started by reviewing closely each video to understand its structure and interactional order from the participants’ perspectives. Video supports the conceptualization of human action as an emergent and temporal phenomenon situated in its material environment, evidence that is supported by a careful transcription of recordings (Mondada, 2019). We, therefore, explored the videos, selecting relevant episodes for in-depth analysis.

The criterion of selection was detecting knots of sense-making, also discursively negotiated regarding real and virtual teacher and student body configurations. We repeated our searches in the data to find relevant clips based on the criteria above and the two conditions proposed: in VW classroom and online. The analytical framework of our study draws on iterative interaction analysis frameworks with a focus on embodied interaction (Erickson, 2006; Goodwin, 2000; Jordan and Henderson, 1995; Streeck et al., 2011; Theobald, 2012). Finally, six topical episodes (Linell, 2009) were selected to carry out a detailed analysis, the findings of which will be discussed in the next section.

Results

We identified three main creative arrangements of student-teacher interaction in the hybrid space mediated by VW. To support the explication of the corresponding dimensions, we will provide some micro-topical episodes with a narrative overview of the teaching session, transcription of excerpts and some screenshots of in the classroom, or VW, as an illustration.

Knot 1: Overlapping of real and virtual body boundaries

The knot, ‘Overlapping of real and virtual body boundaries’, gives evidence for the overlapping consistency of the real body in the classroom and the virtual avatar body in the VW, and the consequently real and online body boundaries discursive negotiation to reach a sense-making of the learning activity. To illustrate this knot, we analyse three micro topical episodes. The first two topical episodes are from the Math session in the classroom mediated by VW and the third is from one online Italian sessions.

In the first selected episode, Mathy, the Math teacher, used the VW to explore different geometric shapes. Mathy used his avatar to manipulate a cube in virtual space (Figure 1(a)) to show the students how to solve a Math problem. The students were sitting in a traditional position, watching the teacher in front of them while at the same time being confronted with the teacher avatar. The real and virtual body of the teacher were co-existent in front of them. Trying to clarify the instructions of the task, Mathy performed a continuous transition between the use of VW projected on a screen panel, and the use of the underlying light board to facilitate his explanation. There was a constant interruption of the flow of observations from the VW projection and the screen. Some student-avatars were lost and could not reach the teacher in the same VW space where the activity with the cube had to be performed. Mathy began to search for them and transport them with his avatar. After several minutes, the teacher reassembled the student-avatars close to his avatar, reproducing in the VW the social presence context of the classroom. While proposing some further oral explanations near the PC, Mathy saw on the projector his avatar surrounded by several avatar-students who jumped on him for fun (see Figure 1(b)).

Figure 1.

(a) The consistency of Mathy body and his avatar; (b) Students jump on Mathy-avatar.

This unexpected behaviour of the avatar-students left him exclaiming, “seriously” (see Extract 1).

Extract 1

Teacher: Ketty, I do not see you here! Who has been still lost? Ketty has been found, ok…. Who else got lost? Nina? Romain? Is it ok for you? [looking to the screen projector] Stop all of you to jump on me! [serious tone, students laugh and Mathy then, too]

In this sequence, we assisted in the overlapping of Mathy’s body and avatar, with the for-fun “attack” of the Mathy-avatar by the student’s avatars. The student avatars jumped on the teacher’s avatar, taking advantage of Mathy's absence from his avatar while he was talking to them. The teacher's normative regulation « Stop all of you to jump on me» is interpreted by the participants in the continuity between real space and virtual space and thus becomes, for this, funny; the teacher didn’t specify it to be “my avatar” but instead, “on me” that was accompanied by a serious tone. This attack, not at all accepted in the real setting where it could be severely punished, lead to a situation considered funny by the participants. The relationship is close as everyone laughed, including the teacher.

The second episode to illustrate the above dimension was in the continuity of the same Math session. On this occasion, the students were placed in a small group to work autonomously. During his activity in VW, a boy saw his avatar in a collision with another one, which caused the boy to laugh loudly (Figure 2(a)). The avatar collision created hilarity for the boy, who believed that the two avatars were overlapping or fused. He then quickly shared his experience with a girl next to him, and the other two girls and the other two girls opposite him (Figure 2(b)). He regained control of his avatar and commented on the physical appearance of it. See an extract from the group of three girls and one boy in Extract 2.

Figure 2.

(a) Overlapping of the two avatars; (b) Sharing to the group.

Extract 2

Boy: I'm about to push Patrice [Two avatars are near one with the others. He laughs].

Boy: Wait, I'm Patrice ! [the two avatars are superimposed one with the other and, therefore, he thinks that there has been a merge or that there is an exchange of avatars. He continues to laugh.]

Boy: Look at him. I’m Patrice. Look, look! [the boy call the attention of the girl next to him to share the situation. The two laugh and call the attention of the girls in the group, who joint in the observation. The two girls getting up from their seat and approaching the boy’s screen. They laugh too. The smash-up is close and he retakes the control of his avatar].

Boy: In the reality no…but in virtual in fact , I got full of muscles !

The overlap of the two avatars created a breaking situation that according to the participant needed to be shared. Indeed, the meeting of the two avatars – apparently fused for a few seconds – was recognised by the students as a moment of hilarity, probably due to the acquired “power” of being able to enter into a friend’s body. Patrice, the student of the collided avatar, is also working in the classroom. In the end, the boy stressed the reacquisition of his avatar, commenting that his avatar “got full of muscles”.

Finally, the third episode illustrating the knot of the overlapping of bodies was taken during the second video recording of the Italian language lesson. In this session, the teacher, Alice, had made a special configuration for the class; one student at a time went to the teacher's desk to use her avatar and talk to a mother language student-avatar connected directly from Italy, thanks to another arranged project. The image of the VW was projected onto the wall of the classroom using a projector; this way, all the students from their desks could follow and listen to what their peers discussed with the guest avatar. This special configuration was planned by the teacher to overcome her difficulty with using her avatar by letting the students manipulate it for her. In Figure 3, the teacher-avatar was used by a student while interacting.

Figure 3.

Italian language teacher-avatar manipulated by a student in the classroom.

In this episode, the VW was discovered by the students and teachers to be a space that extended the social interaction opportunities beyond that of the classroom, thereby allowing them to talk in real-life with other Italian students. At the same time, thanks to the configuration proposed by the teacher, the students gained a new sensory experience by entering materially into the avatar body of the teacher. Through this experience, they assumed new physical features such as gender, clothing and the status of the teacher. The other children watched the activity live on the wall of the classroom, observing the teacher-avatar as it was animated by a peer. The VW environment shared on the wall of the classroom gave the students the possibility to experience a new representation of the space, the teacher and the activity.

Knot 2: Multi-modality: Disruption and recall of the activity

The second knot proposed is the dimension “Multi-action: disruption and recall of modality”. It is illustrated here with three main topical episodes; two are from a French lesson in the classroom and the third is from an online Math lesson. We analysed them to show how the sense-making of the participant about the task performed was interconnected and co-developed through active multimodal engagement between the real and virtual body.

The first episode is from the French language session. The teacher proposed a group activity be performed in the classroom with the mediation of the VW space. The activity aimed to find the covers of some books and to find the correct answers for a list of questions given in a paper sheet. The teacher gave the instructions to the class, using deictic gestures such as pointing. She indicated how to fill in the sheet while going around the room many times to get the attention of all the students. The teacher then began to manipulate her avatar to show to the student how to find the resources (the book cover). Here the use of the gestures is addressed to move the attention from her, as the person speaking, to the projection on the screen where she is acting (Figure 4(b)). Moments later, her avatar fell (see Figure 4(c)). This extract is taken from her reaction.

Figure 4.

(a) Teacher use gestures to catch attention; (b) The teacher-avatar falls to the ground.

Extract 3

Teacher: Please, look at the screen if you like … [addressing to some students that are turned to watch her and not the screen]. Oh, I'm falling, do not worry, sorry, sorry, I'll go back [She is nervous and sound embarrassed]. Sometimes I fall and I imagine you too.

The teacher expressed slight embarrassment at her avatar falling, as she afterwards repeated “sorry” and looked for an explication “I guess you fall too”. Her competency in the use of VW was in progress; she had to adopt new gestures and practices that she had not yet fully appropriated.

In this episode, the teacher, Katy, showed her embodiment to be involved in the activity, while at the same time, she was engaging online with her avatar.

A second episode that illustrated this dimension was from the same session, right after the teacher had put the students to work on the activity. Indeed, the teacher proposed a team activity to perform with the mediation of the VW. Within the team, each student had to perform a certain role: there was the one responsible for the question sheet to share during the chat; the one responsible for the book and where to find the answer; finally, there was the student in charge of the writing the shared answer. The teacher’s aim was for the group to work only online and they were not supposed to talk. Technical issues arose meaning that everything has to be written in the local chat; some students were lost because they had to find the names of the members of their teams, while some students didn’t have the chat window open at all, thus, messages were disappearing regularly from the online chat. The teacher tried to manage the session and to help the students through technical problems. In this context, Paul moved his attention between the sheet (where the task had to be performed) and the screen, while at the same time he discussed with the researcher (Figure 4(a)) who had interrupted him, about his work (see Extract 4).

Extract 4

Researcher: Are they coming near to you ?

Boy: Yes, yes, they are all there now.

During his dialogue, Paul continued the discussion with the researcher without moving toward her, searching for the relevant information and trying to make sense of the task. His avatar and his body were engaged in the activity. This knot between the physical body and the avatar behaviour was also shown by other students; for example, in Figure 5(b) when one student manipulated his tablet during the surfing of his avatar, following the flying motion in the VW with his manipulation of the computer. The engagement was broken when the activity, for some reason, stopped working and needed the attention of the teacher to bring it back.

Figure 5.

(a) Between sheet and video screen; (b) Manipulation of the pc; (c) Calling attention with arms.

The rich multimodality expressed by the teacher in the first episode found a correspondence into the multimodality of the student’s actions during the session.

A second topical episode that illustrated the “Multi-action: disruption and recall” dimension was also part of the online Math session. The Math teacher from the school was connected online with four students at their homes. The session aimed to recall and review learning about angles and was designed to help the students to solve some exercises for the next classroom lesson. The VW space was composed of some exercises displayed in a large panel. The teacher started the lesson by inviting the four students to read the panels and to explore them. The three students zoomed to the panel without moving the avatar (Figure 6(a)). To the contrary, Lilly moved her avatar closer to the panel without zooming (Figure 6(b)), therefore, reproducing a common physical behaviour undertaken in real-life – walking to go closer.

Figure 6.

(a) Student-avatar in front of the panel; (b) Lilly moves closer to the panel.

In this context, the teacher addressed Lilly, who had already had connection problems when working at home. Lilly had activated different multimodality of action during the exercise to support her understanding of the task – as we can see in Extract 5.

Extract 5

– Teacher: Is it better there? Exercise 2 pas. 49. In front of you on the left.

– Lilly: I see it very badly.

– Teacher: And what are you seeing?

– Lilly: And, yes. Yes, I see it now.

– Teacher: Do you remember the subject of the sheet page 33? Go, Lilly, I am listing you. [waiting some seconds]. We have learned to call the angles and the nature of the angles. Do you remember the nature of the angles? [waiting some seconds]. What is the angle that we know most?

– Lilly: Straight angle.

– Teacher: Put your hands near to near as if we did [in the classroom] and if we stop, if I put one hand palm next to the other what happens?

– Lilly: There is no space

– Teacher: So?

– Lilly: It is zero. Straight angle.

– Teacher: Put your hands back on them and make a little space. If I continue to spread, we have which angle?

– Lilly: Acute.

– Teacher: To make good work, I propose to take the form of sheet page 33. Do you have everything you have available next to you?

– Lilly: Something, not all.

– Teacher: [waiting about 30 seconds]. Lilly, do you look or not?

– Lilly: Ok, it is well.

The online learning activity mediated by the VW was distributed between the online panel, the sheet of paper given out in the classroom (that the students had near to them), the teacher and student’s hands and bodies and the avatars. Indeed, the teacher’s instructions as to how they should perform the activity had a spatial connotation, for example, “In front of you on the left”, when referencing the avatar body, while at the same time she heavily referred to a real physical dimension with “I see it very badly; I am listening to you”. In addition to this, the teacher used their physical body to help the student understand the angle when stating “if I put one hand palm next to the other what happens”. Thus, the interplay goes on in a continuous interaction between the physical and virtual dimension through the avatar body and personal body – the online space and the virtual space. The multimodality is activated between the two dimensions to make sense of the learning activity, showing a creative activation of real and virtual resources.

Knot 3: Epistemic and social activation trough virtual and psychical exploration

The third and last knot identified was “Epistemic and Social activation trough virtual and psychical exploration”. This can be illustrated using two topical episodes: the first is from an English lesson held in the classroom, while the second came from an online VW Math session. We analysed them to show how the exploration of the physical and virtual environment is done by physical and virtual body actions – both guided by the need to give meaning to the activity.

The first episode is from the English lesson. The teacher who had only recently begun using VW. Students were distributed around the computer room; they had to follow the instructions written on the sheet that had been distributed to them to fill out. Several technical problems were encountered such as blocked avatars, panels that did not load and teleportation that did not work – many students got lost. The session did not work as the teacher had intended. That being said, from watching the video sessions, the students were engaged in the task and even tried to manage by themselves the resources available. For example, while a researcher was filming the students working, she decided to ask some questions to the students to understand what there were doing. A student answered without turning towards her as he kept navigating the avatar (Figure 7(a)). The student instantly took advantage of the opportunity to have an adult next to him so that he could have any information that would allow him to advance in his exploration, as is seen in Extract 8.

Figure 7.

(a) The boy in the VW interacting with the researcher; (b) teacher with two girls.

Extract 6

- Researcher: Do you see the panel or not?

- Boy: Yes … no indeed. What is “almost”?

- Researcher: The most important.

The student mobilized the resources around him to achieve his goal, demonstrating his commitment to the task and advancing in the virtual exploration thanks to the close physical resources.

In Figure 7(b), the English teacher, Carmen, and two students are involved in the English exercise. One student had called her. There was a problem with the English instruction panel. They tried to resolve the problem together with a new solution to the teacher’s surprise (see Extract 7).

Extract 7

Teacher: Normally you do this …

Girl 1: Why is it all fluorescent? Madame, could I watch on the screen of Lydia?

Teacher: You, watch from Lydia. And so you can complete: “Your favourite role is …”

Teacher [toward the researcher which is filming]: “ So what is incredible is that students don’t do the same as me. I said them to go to the right; you have to turn and click on … . they instead do this ….”

This extract shows how students found an autonomous solution to a technical problem after saying, “Madame, could I watch on the screen of Lydia?”. The teacher is surprised by the creativity of the students in the extract, who implement autonomous strategies concerning her modality and prescription as the “students don’t do the same as me”. In the educational field, one of the opportunities of virtual settings is the possibilities it offers to the user through active participation in the creation and development of their knowledge. Virtual exploration is led by the need to give meaning to an activity and to find solutions to perform it. The second episode that we used to illustrate this category is from the Math online course. The teacher tried to give four students some exercises that were posted on the virtual panel to finish individually before writing the answer in the chat with the teacher. The teacher moved the attention to another student called Nicolas. The teacher realised that he no longer see him in the VW space where they were working (see Extract 8).

Extract 8

– Teacher: Girls, I suggest you come and sit next to me. I put the following exercise on the panel (…). Is it ok for you Lilly?

– Lilly: No.

– Teacher: I turn the panel a little

– Lilly: A little to the left.

– Teacher: You tell me when you see well.

– Lilly: It is ok.

– Teacher: Sally, it is a course of math homework, can you leave us alone? [a student avatar enters the space. The avatar goes away]. You work and I will return to you. Nicolas, I do not see you anymore. Where is he?

- Lilly: He was seated there. He is coming back. [the connexion is weak]

– Teacher: Well, it's just that we do not see you

– Lilly: We do not see you

– Nicolas: I see myself. I am dressed in a football dress (laugh) …

– Teacher: We don’t see you.

– Nicolas: I am behind and I see.

– Teacher: Ah ok.

– Nicolas: How can I change dress?

– Teacher: Put the cross on the avatar, no?

All the participants are involved in the interactions. The position of Nicolas was negotiated by the others, for example “he is seated there” and then by himself. The student, who had lost the connection for some time, claimed to see himself – thus confirming his participation in the activity – and described himself to reinforce his presence by saying, “I am dressed in a football dress”. The episode was closed when Nicolas asked how to change his dress, probably considering it inappropriate for the working activity. Participants had the perception of sharing a physical space where they found themselves acting with other people in a social event with a code to respect (to join in the activity when it was required and to respond to the request),; therefore, reproducing the code of face to face communication in the VW. The avatar provided the perception of a reciprocal sense of presence and the interactions in real-time gave the perception of a shared experience.

Discussion

Configuration in this context refers to Charles Goodwin’s (2000) work denoting how participants employ different arrays of semiotic resources (configuration Goodwin s), to make sense of the activities at hand. “Knots” is used metaphorically to refer to how classroom versus VW, body versus avatar and material versus online tools, are tied together in the processes of meaning-making. The empirical analysis, based on the video recordings from classrooms, identified three knots of embodied configuration that appear tightly connected to the specific space-time setting: the overlapping of real and virtual bodies boundaries; multi-action: disruption and recall of the activity; epistemic and social activation trough virtual and psychical exploration. Our results highlight the dynamic ways in which the arrangement of body and material acted as mediational means between real and virtual settings. In the following we focus on each knot, searching for evidence that the “real” and “virtual worlds” and the physical and the avatar’s bodies are intermingled in various ways.

About the first knot “Overlapping of real and virtual bodies boundaries”; this knot shows that the virtual and the physical boundaries are discursively negotiated during the activity with the VW between teacher and students and between themselves. The reference to the virtual and in the material body help to support the understanding of the task. From the episodes proposed, the use of VW in the learning environment introduced a new form of spatial and physical proximity between teachers and students; something that can introduce new behaviour, as was seen in the episode of student avatars jumping on the avatar-teacher which is forbidden in the real-life.

The use of the avatar put into question the traditional authority of the teacher, someone usually embodied in posture, in older age and their social status - like in the episode of “entering” into the teacher’s avatar body. At the same time, the teachers were confronted with a student “avatar that looks aggressive, seductive and, anyway, adult”. The teacher-student relationship is, therefore, changed and is considered by students and teachers to be more symmetrical and participative. This change needs to be continually negotiated to define the boundaries between the teacher’s real and virtual bodies, and the student’s real and virtual bodies. Also, virtual interaction becomes an activator of social interactions in the classroom, relativizing the idea of full immersion in the virtual world without social and physical implications, as was shown by the accidental overlapping of two avatars. This was considered to be an exceptional situation by the student before he discussed and shared it with his peer. The avatars are experienced as an extension of the body for the teacher and the students as both discursively negotiated the powers, but also the limits and responsibilities. The physical and virtual boundaries are not defined by reality or the virtual but co-constructed (Hirst and Vadeboncoeur, 2006).

About the second knot “Multi-modality: disruption and recall of the activity”; this knot shows how the act of sense-making by the teacher and students concerning how the task should be performed is interconnected and co-developed through active multimodal engagement – through the material, virtual, and spatial manipulation. This is full of interruptions, resumptions, reformulation and negotiation attempts; each of them oriented to the different contemporary physical and virtual activities. The teacher’s body in the classroom was extended but also made vulnerable by the act of the avatar falling. This can also be seen in the episode where the teacher's gesture in the VW cannot be reproduced in the same way in real-life. The gesture as a form of embodied action for socially coordinated action and thought is thus, heavily context-dependent. So, in our opinion, the use of VW has to be supported by adequate linguistic and corporal orientations references. We, therefore, witness the breaking of the professional gesture that is traditionally performed in the classroom and instead, see the reconstruction of a new one in the VW, whereby the reconstructed gestural interaction helps to develop meaning (Goodwin, 1986).

The third knot, known as the “Epistemic and social activation trough virtual and psychical exploration” shows how the exploration of the environment is led by the need to give meaning to the activity – the video-game-like interface seems to invite a playful framing of the activity – and the teacher seems to be engaged in restoring/reframing the virtual world as a ‘serious’ arena for school work. The edutainment context (Charsky, 2010) of the VW activity allows the students to build an effective social and epistemic engagement in the task. The students explore the physical and virtual environment to determine how they will be able to act on (like activating the resource near the physical body to perform the virtual one), in a form of distributed cognition between physical and real (Hutchins, 1995). The students demonstrate just a few months after being introduced to the VW, how to develop different participation in tasks. For example, the ability to exercise autonomy, to activate the real and virtual resources embodied in learning places and in giving opportunities for physical interaction with the learning material and virtual resources (Kosmas et al., 2019). The use of VW requires a form of adaptation to the available resources, making explicit the reference to them whether they are real or virtual. The sense-making in action between the involved participant is not linear, but a continuous attempt to clarify the activity and the intersubjective understanding between the physical and material process.

From the three knots analysed, the virtual is lived no longer as an experience in itself, rather it is based on the frame of social meanings in which each individual can place it. In this sense that virtual dimension can be considered experiential, in which the perceptual component (visual, tactile, kinaesthetic) merges with interactivity. The virtual setting conceived becomes a creative communicative tool that acts not only among users but also between them and the environment in which they are immersed. Consequently, learning activities that use VW engage participants in a new creative embodied activation, mobilizing physical and virtual engagement, spaces and resources. The introduction of the VW puts a new non-linguistic, physical and visual learning dimension at the centre of the users’ experience, thus removing language as the dominant vector of the experience. In particular, all the episodes show activation of a deictic reference when giving instructions – the teachers alternate between speaking concerning the avatar and speaking about the students’ physical environment. This is evidence of the significant interweaving of virtual and physical ‘worlds’ in talk-in-interaction. Teachers and students are becoming aware through their experiences in the VW, that extended learning between online and in-presence also implies an embodiment framework as a unifying perspective – considering that all psychological processes are influenced by body morphology, sensory systems, motor systems, and emotions (Glenberg, 2010; Schubert and Semin, 2009).

The sense-making that emerged from the three knots deals with interactional issues connected with multi-activity – e.g. the issue of simultaneously interacting with co-present and ‘virtual’ students; consequently, teachers struggled to coordinate this hybrid interaction – understanding the tool itself, coordinating and communicating with students and avatars at the same time. To summarise, the results show how learning activities are extensively transformed when integrating VW technologies into classrooms, but also for online interaction between teachers and students. Sense-making is interconnected with the real and virtual body of the participants, often negotiated discursively.

Conclusions

The problem addressed in this manuscript concerns the creative implications of the implementation of VW in a classroom. The study aimed to explore in what ways a VW application could mediate between “the real” and “the virtual” through embodied and material configurations. The variety of the pedagogical script makes it difficult to grasp the richness of the data. The data proposed two different Language subject sessions in the classroom, and online in the Maths classroom. Also, although there were different subjects, the teachers dealt with the same technical difficulties, trying to improve their pedagogical dialogue with the VW. Indeed, the attempt to improve the script is continuously reformulated by the teachers involved, particularly those willing to improve the quality of their lessons by harnessing virtual mediation. The tension between the prescription of the curriculum and the possibility of technological solutions challenged their creativity. From the analysis proposed, we can consider that the use of a virtual space in the classroom can introduce changes through authentic tasks involving 3 D object manipulation, creativity and corporal mobility via avatars; which bring change to how the learner acquires, applies and constructs knowledge (Jung and Latchem, 2011).

Future follow-ups of this analysis will be able to explore new creative dimensions that could emerge in this hybrid context, just like the social norms of conduct reformulated in the interaction between students and teachers through the real and virtual educational context. Indeed, from the episode emerge the social norms of physical conduct where it seems as if the virtual world generates a need to renegotiate issues around tact, deference, social hierarchies, and student-student interaction (e.g. the issue of 'pushing'). In conclusion, participants have lived with VW experiential learning, thus giving them the possibility to interact in a new way with peers and teachers. New material and embodied configurations emerged within this particular, unfolding relational context of its use, and co-constructed between material and human agents. Real and virtual can effectively be creatively integrated to give space and to combine creativity, play and meaningful meaning.

Footnotes

Acknowledgements

We thank the secondary school teachers and students that have accepted to cooperate in the project. We thanks Immersive CoLab, the start-up that developed the VW platform and codesigned the educational artefacts.

Research involving human participants

Students and teachers are involved in the project. All the authorisation for video and sound was captured with informed consent from the parents of the children and was also collected from the teachers.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Maria Antonietta Impedovo

Martine Gadille

Author Biographies

Maria Antonietta Impedovo is Associate Professor in School of Education, Aix-Marseille University, France. Research topics: identity, professional development, collaboration.

Martine Gadille is Senior researcher at CNRS, Aix-Marseille University, France. Research topics: Technological innovation, creativity and dynamics of knowledge in the transformation of professionalities, organizations and territories.

References

Charsky

(2010) From edutainment to serious games: A change in the use of game characteristics. Games and Culture 5(2): 177–198.

Cheville

(2006) The bias of materiality in sociocultural research: Reconceiving embodiment. Mind, Culture, and Activity 13(1): 25–37.

Cole

(1996) Cultural Psychology: A Once and Future Discipline. Cambridge: Harvard University Press.

Dourish

(2004) Where the Action is: The Foundations of Embodied Interaction. Cambridge, MA: MIT Press.

Duranti

Goodwin

(1992) Rethinking Context: Language as an Interactive Phenomenon. Cambridge: Cambridge University Press.

Erickson

(2006) Definition and analysis of data from videotape: Some research procedures and their rationales. In: Green

Camilli

Elmore

(eds) Handbook of Complementary Methods in Education Research. Mahwah, NJ: Erlbaum, pp.177–205.

Gadille

Impedovo

M-A

(2019) School goes online with avatars: Extended learning in a secondary school. In: Lund

Niccolai

Lavoué

, et al. Conference proceedings. A wide lens: Combining embodied, enactive, extended, and embedded learning in collaborative settings 13th international conference on computer supported collaborative learning. 2. USA: International Society of the Learning Sciences (ISCL), pp.549–552.

Gadille

Impedovo

M-A

(2020) The embodiment dimension while learning and teaching in a virtual world. Journal Systemics, Cybernetics and Informatics 18(3): 1–8.

Glenberg

(2010) Embodiment as a unifying perspective for psychology. WIREs Cognitive Science 1(4): 586–596.

10.

Goodwin

(2000) Action and embodiment within situated human interaction. Journal of Pragmatics 32(10): 1489–1522.

11.

Green

Brock

Kaufman

(2004) Understanding media enjoyment: The role of transportation into narrative worlds. Communication Theory 14(4): 311–327.

12.

Heath

Hindmarsh

Luff

(2010) Video in Qualitative Research. London: Sage.

13.

Hirst

Vadeboncoeur

(2006) Patrolling the borders of otherness: Dis/placed identity positions for teachers and students in schooled spaces. Mind, Culture, and Activity 13(3): 205–227.

14.

Jordan

Henderson

(1995) Interaction analysis. Journal of the Learning Sciences 4(1): 39–103.

15.

Jung

Latchem

(2011) A model for education: Extended teaching spaces and extended learning spaces. British Journal of Educational Technology 42(1): 6–18.

16.

Klein

Moon

Hoffman

(2006) Making sense of sensemaking: Alternative perspectives. IEEE Intelligent Systems 21(4): 70–73.

17.

Kosmas

Ioannou

Zaphiris

(2019) Implementing embodied learning in the classroom: Effects on children’s memory and language skills. Educational Media International 56(1): 59–74.

18.

Lakoff

Johnson

(1999) Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought. New York: Basic Books.

19.

Linell

(2009) Rethinking Language, Mind, and World Dialogically: Interactional and Contextual Theories of Human Sense-Making. Charlotte, NC: Information Age Publishing.

20.

McIntyre

(2008) Participatory Action Research. London: Sage.

21.

Mondada

(2011) The interactional production of multiple spatialities within a participatory democracy meeting. Social Semiotics 21(2): 289–316.

22.

Mondada L (2019) Rethinking Bodies and Objects in Social Interaction: A Multimodal and Multisensorial Approach to Tasting. In: Kissmann U and van Loon J (eds) Discussing New Materialism. Wiesbaden: Springer VS. DOI: 10.1007/978-3-658-22300-7_6.

23.

Salvatore

Venuleo

(2013) Field and dynamic nature of sensemaking: Theoretical and methodological implications. Papers on Social Representations 22(2): 21.1–21.41.

24.

Schubert

Semin

(2009) Embodiment as a unifying perspective for psychology. European Journal of Social Psychology 39(7): 1135–1141.

25.

Streeck

Goodwin

LeBaron

(2011) Embodied Interaction: Language and Body in the Material World. Vol. 8. Cambridge: Cambridge University Press.

26.

Theobald

(2012) Video-stimulated accounts: Young children accounting for interactional matters in front of peers. Journal of Early Childhood Research 10(1): 32–50.