A diffractive approach to multimodal transcription: Materialising entanglements between humans and non-humans

Introduction

Posthuman and more-than-human ontologies are increasingly being used to re-conceptualise the social sciences, arts and humanities. The ‘post’ within posthumanism is not to signify ‘after the human’, but instead to de-centre the human (Kuby and Rowsell, 2017). It involves challenging traditional humanist research which positions humans as the central point of analysis (Spyrou, 2018). In this conceptualisation, humans are not centred when discussing agency, non-humans and materials are also recognised as having agency. Posthumanism places emphasis on the entangled relationships between concepts, humans, place, and materials (MacRae et al., 2018).

Exploring methods to empirically ground posthumanism is vital, as without empirical grounding it can have no real-world impact and therefore creates a theory/practice divide (Lenz Taguchi, 2010). Nonetheless, researchers have encountered difficulties in the translation of posthuman theories into empirical research methods (Fox and Alldred, 2022). Indeed, some academics argue against the use of methods (St. Pierre, 2024) as conventional humanist methods are incompatible with posthumanism, and it requires “a different empiricism that is not grounded in the human subject” (St. Pierre, 2016: 29). In this article, rather than reject method, I “open up the empirical to reworkings” (Barad, 2011: 148). I re-think the existing use of multimodal transcription in posthuman research (Marsh, 2017), considering how Barad’s (2007) diffraction methodology might make this practice less anthropocentric and more in line with a relational ontology. I explore how diffracting multimodal transcripts could help to empirically ground posthumanism and undo the theory/practice binary. My intent is not to be methodologically prescriptive, but to “place the understandings that are generated from different interdisciplinary practices in conversation with one another” (Barad, 2007: 92-93). It is fundamental that this diffractive approach to transcription is not viewed as an authoritative tool but, as a dynamic thinking pattern (Rautio, 2021), a practice to bring forth the new. Additionally, it should not be applied to a study which began with a traditional humanist research design —posthuman and/or more-than-human theories should shape the entire inquiry.

Posthumanism and multimodality are diverse fields which encompass a wide range of theories and approaches, and it would be impossible to explore them both fully within one article ¹ . Within posthumanism, I expound on diffraction and within multimodality I focus on multimodal transcription. My rationale to centre on diffraction and multimodal transcripts is because they provide a much-needed tool to carry out empirical posthuman research. Applying diffraction to multimodal transcription is significant as these methodological approaches have not been linked together in this way before. I begin by discussing existing research that combines the broader fields of multimodality and posthumanism. After that, I define the new materialist term diffraction. Next, I outline a small-scale study on pre-schoolers’ creativity with tablet computers which used multimodal methods, illustrating how the diffractive analysis of multimodal transcripts aids in empirically grounding posthuman theories. Finally, I discuss the ways in which diffractive analysis aligns with multimodal research methods and how linking these two methodological approaches contributes to knowledge.

There are multiple ways in which multimodality and the broader field of posthumanism intersect, which enables the transfer of methods and concepts between them. Indeed, several researchers have already begun to make links between multimodality and posthumanism (Hackett and Somerville, 2017; Marsh, 2017; Sakr, 2022; Wohlwend et al., 2017; Wohlwend and Thiel, 2019). For example, in her study on connected play, Marsh (2017) employed a tabular transcription method whereby the visuals, audio, action and proximity of both the human and non-organic matter were transcribed, coining this Multimodal (Intra) action Analysis (re-framing Norris's (2011) Multimodal Interaction Analysis). Intra-action is a new materialist term which contends that separate bodies do not exist before interactions but come into being in a process of intra-action. When bodies intra-act they do so in co-constitutive ways and are entangled in the process of meaning making (Barad, 2007). Marsh (2017) clearly demonstrated that multimodal transcripts are a useful method for analysing the entangled relationships between humans and non-humans.

Nevertheless, while previous links have been made between posthumanism and multimodality, there is yet to be an article which applies a diffractive approach to several multimodal transcripts of the same event. Furthermore, most empirical articles on diffractive analysis read texts or data together, rather than focusing on methods (Fox and Alldred, 2023). To address these gaps, I will concentrate on the potentials of applying a diffractive analysis to multimodal transcripts. Exploring a diffractive approach to multimodal transcripts is important as it can: provide an understanding that the reality enacted through research is partial, be used to de-centre the human and, support in empirically grounding posthumanism. It is important to note that while I have narrowed in on a diffractive approach to multimodal transcription this is not a complete representation of a diffractive analysis which involves reading a range of materials through one another to provide novel philosophical insights (De Freitas, 2017; Fox and Alldred, 2023).

[D]iffraction can be a metaphor for another kind of critical consciousness... one committed to making a difference and not to repeating the Sacred Image of the Same (Haraway, 1997: 16)

The root of diffraction is from dis, “apart,” and frangere, “to break”, it derives from physics, referring to the phenomenon when waves spread outwards as they move through or bend around objects (Barad, 2014). When I use the term diffraction, I am alluding to the new materialist framing of the term (Barad, 2007; Haraway, 1997). When new materialists diffract data, they break, cut or fracture it into multiple parts to show the different components, and re-construct them anew (Hickey-Moody and Willcox, 2019). During this process they explore the messiness and multiplicity of the data, investigating what new patterns unfold, examining how differences are made and considering what is excluded through their chosen research apparatus (Murris and Bozalek, 2019).

Challenging reflexivity: Diffracting the self

“There is no ‘I’ that exists outside of the diffraction pattern, observing it, telling its story… I am neither outside nor inside; ‘I’ am of the diffraction pattern. Or rather, this ‘I’ that is not ‘me’ alone and never was, that is always already multiply dispersed and diffracted throughout spacetime (mattering)” (Barad, 2007: 181).

One of diffractions crucial distinctions from traditional qualitative multimodal research methodology is its departure from reflexivity. The aim of reflexivity is to make qualitative research more scientific and objective through self-reflection on our biases (Chee, 2022). Diffraction challenges this centring of the human researcher, suggesting it is impossible to represent representation, asking the question: how can one objectively represent their own biases (Knafo, 2016)? The risk in reflexivity is creating a mirror of a mirror and that the bias of the researcher influences how they identify their bias. Additionally, if a researcher is in a constant state of becoming, how can they cut up their subject positions and know which aspects are relevant during the dissemination of their research ² (Chee, 2022)?

Diffraction does not claim that the researcher’s biases do not have an influence on the research process. Instead, it positions the researcher as one part of the research apparatus, without centring the self (Larsen and Schwennesen, 2024). Diffraction involves “an embodied engagement with the materiality of research data: a becoming-with the data as researcher” (Taguchi, 2012: 265). This “becoming with” approach moves away from representation which places the researcher in a hierarchy with their participants and acknowledges that a researcher’s position in relationship to their research is not static (Vagg, 2022). A diffractive reading is not simply a reading of a transcript as it is, but reading with the transcript, considering your affective and embodied encounter with it. In this process, both the researcher and their thinking are affected and being affected by each other which creates new lines of inquiry (Hultman and Lenz Taguchi, 2010).

Diffraction in practice: Reading multimodal transcripts through one another

In this section, I explore the potential of diffractively analysing multimodal transcripts. I outline three different multimodal transcription methods used on the same video and screen recorded data, including exemplars of each transcription approach (Figures 1–3). The transcripts were read through one another (diffractively) to see what differences made a difference. Reading-through involves understanding that boundary production is a material-discursive practice which configures and performs reality. Rather than reflecting on individual transcripts, comparing them against one another and creating a hierarchy, reading-through accounts for how the differences between specific practices matter (Barad, 2007).OPEN IN VIEWER

Figure 1.

Tabular transcript separated by entities.

OPEN IN VIEWER

Figure 2.

Tabular transcript separated by mode.

OPEN IN VIEWER

Figure 3.

Comic strip transcript.

Whereas traditional transcription involves turning speech into writing (Bezemer and Mavers, 2011), multimodal transcription explores methods to transcribe multiple modes of communication. In my study, multimodal transcription was initially completed using ELAN software, creating annotation tiers for both the humans and the tablet. Annotating gaze, gesture, speech, and action allowed me to analyse embodiment (Price and Jewitt, 2013). Embodiment helps to empirically ground posthumanism (Braidotti, 2019). Creating tiers for the tablet (vocalization, colour, movement) provided an opportunity to focus on the non-human and to think-with posthuman theories. A limitation of the ELAN transcript was that it did not allow for the simultaneous transcription of the screen recording alongside the parent recorded video. This was fine for this video in which the parent has filmed the screen throughout but, in some of the recordings, children moved around which made the screen not fully visible in parts.

During the process of annotation, I came to the realisation that this separation of the two data sets was not appropriate to explore the entangled nature of the intra-actions between child, family member, and tablet. When considering other software to complete the analysis, I was wary of choosing another that might restrict the way I wanted to transcribe and analyse my data through a posthuman lens. So, while there is software available which allows two images to be annotated simultaneously, I chose to complete this process manually using a word document. This method was time consuming but allowed for more flexibility to explore different ways to annotate without relying on the software’s restrictions and to ensure a focus on the entangled elements of play with the tablet. This meant that I did not encounter another issue such as that experienced while using my initial choice: ELAN. Creating my own transcripts manually, compelled me to think carefully about what I was choosing to include and exclude.

Applying a diffractive analysis involved creating a range of different transcripts and then reading these through one another to see what they include and exclude. While previous papers have compared a range of multimodal transcription methods (Bezemer and Mavers, 2011; Cowan, 2014) or read multimodal transcripts through theory (De Freitas and Sinclair, 2014), no researcher has read multiple transcripts of the same event through one another. In my analysis, I did not pit one transcript against another, instead diffraction provided detailed understandings of a range of viewpoints and how they built upon each other to create new understandings. Indeed, the very process of compiling transcripts of the same data in a range of formats is posthuman, as it forces me to diffract my lens and see how difference can make a difference.

Using the same data presented in different ways provided me with an opportunity to read each of the methods through one another. The process of diffraction and repeated engagement with the same data in different formats clearly revealed what different transcription methods include and exclude and which ones are suitable to think-with posthuman theories. Having different methods of data transcription “can be understood as an obstacle or an interference that overlaps with my embodied affective and theoretical thinking as researcher, causing me to read diffractively one through the other” (Hultman and Lenz Taguchi, 2010: 536). In the act of writing and viewing things in multiple formats, my relationship with the data changed both in affective and embodied ways.

The differences between transcripts and the difference this makes

“Envisioning difference differently – i.e. theorizing a different difference – leads to a thought-practice in which concepts are not abstraction from the world, but an active force of this world – and thus always/ already implicated in and concerned with world(ing): practicing and envisioning specific practices for this world” (Thiele, 2018: 203).

Reading theories/methods/data through one another and accounting for the relations of difference is a key part of a diffractive reading. In Figure 1, the differences include: placing image and text description together; and separating out the agents and positioning the non-human as an equal participant. The difference this makes is that it gets us to reflect on the agential cuts made during transduction and clearly positions the non-human on an equal playing field as the humans. Additionally, the process of scrutinising the image, thinking through it and describing it in words changed my relationship with the data as a researcher and helped me to de-centre my anthropocentric gaze. In Figure 2, the difference is that it separates out communication and interactional modes but positions all participants together. This transcript presents a more entangled view of the event; entities are not presented as being self-contained. In Figure 3, the difference is that it is a more accessible presentation of the data, meaning that the participants can experience and become with the data and diffract their analysis. This transcript does not position the researcher’s reading of the transcript above the participants and is non-hierarchical. In short, Figures 1–3 are different, and those differences provide an insight into the multiple realities present within the same data set. In a diffractive reading, we are looking at differences not to separate and compare opposing states, but instead to understand more about the connectivity, multiplicity, relationality and continuity of events. A diffractive analytic process is both creative and transformational (Vagg, 2022) disclosing a hidden reality that exists amongst the multiple realities being enacted (Lenz Taguchi and Palmer, 2013).

Reading transcripts through one another enabled me to read with methods, understanding that they were acting upon me as I read and created them (Lenz Taguchi and Palmer, 2013). Diffracting my lens involved recognising how, as a researcher, I am entangled with the data and that as I act upon the data the data acts upon me. Viewing the data in different formats allowed me to think differently. As I “experienced with” the data, I “became with” it — this generated new understanding. This was most evident when I was separating the data first by entity and then by mode and transducing images into words. In both cases, the process of re-making something anew, of transformation, forced me to think in a new way. Pushing myself to think in new ways and create multiple alternative methods to present the data made me attend to the numerous ways in which an event can be observed and illustrated. It compelled me to reflect on what I privilege in an observation, why this might be, and what this agential act reproduces and makes visible/invisible.

For instance, initially when watching my video data, I privileged social interaction between humans, and speech. If I consider why this might be, as an early childhood educator, I am trained in completing narrative observations of children which concentrate on the development and learning outcomes of human children. Narrative observations centre on speech and do not record multimodal interaction. The agential act of using narrative observation reproduces developmental perspectives on childhood. In focusing on humans, these observations re-establish the binary divide between the human subject and the non-human object. In a narrative observation, children are the subjects of observation, and the tablet is being acted upon, causing entanglements with non-humans to become invisible. Furthermore, the focus on speech makes multimodal interaction and embodied cognition invisible. In this article, I argue for the use of diffractive analysis on multimodal transcripts. This process is valuable as multimodal transcription makes multiple modes of interaction observable, and a diffractive analysis enabled me to give form to entanglements with non-humans, making visible things which are typically ignored in Early Childhood Studies. Therefore, applying a diffractive analysis to multimodal transcripts allowed me to think differently with the data, rather than duplicating accepted arguments within my field.

Applying a diffractive analysis to multimodal transcripts provided me with tools to work across (and re-conceptualise) the disciplines of multimodality and posthumanism. It enabled me to think-with a range of theoretical frames, cross disciplinary boundaries and to think-through disciplines. Diffraction does not place one discipline or theory in a hierarchy with another. Instead, it provokes new thinking through blurring disciplinary boundaries and reading theories against one other. Diffraction remains “attentive to important details of specialized arguments within a given field” (Barad, 2007: 25) without erasing them, comparing them or creating dualisms. This is valuable as specialised arguments and terms are frequently diluted down and weakened when they cross disciplinary boundaries.

Given the gap identified in the literature, I have focused here on applying diffraction to methods. However, diffractive analysis should also be used to read theory, data, texts and the researcher’s affective responses through one other. For instance, Hultman and Lenz Taguchi (2010) carried out a diffractive analysis of a photograph of a child in a sandbox. They considered the same photograph through a developmental lens, the sociology of childhood and finally through the posthuman theory of intra-action. They read these theories through one another coming to a more nuanced understanding of the agency of non-humans and how the sand acts upon the child as much as the child acts on the sand. In my study, I performed a similar diffractive analysis reading both socio-cultural and posthuman theories through one another, but rather than analysing a singular photograph, I also diffracted my methods of transcription. Creating a range of multimodal transcripts from the same data further diffracted my lens.

When reviewing different types of transcription methods to use in my study, I noticed that multimodal methods cannot, in their current form, materialise entanglement between non-humans, as they either centre human or non-human or place them in a hierarchy. Nevertheless, re-conceptualising these approaches offers exciting possibilities for capturing and transcribing intra-actions between humans and non-humans. Multimodal Inter-action Analysis (MIA) focuses on how space and artefacts mediate interaction between social actors (Jewitt, 2011). MIA places social actors (humans) in a hierarchal position to mediated tools (non-humans). Adopting key concepts and methods from MIA and altering it, placing social actors and mediated tools on a flat plane, provides a methodological approach to study intra-actions between humans and non-humans (Marsh, 2017). Being playful with form is an important part of re-working transcription so that it can materialise entanglements. For instance, De Freitas and Sinclair (2014) ‘playfully overturn’ written transcription by bracketing speech (rather than actions) and presenting these in sentences entangled with other modes.

How diffraction and multimodality align conceptually

I contend that multimodality and diffraction align conceptually in three significant ways. The first way which I argue that multimodal research and diffraction converge is in their claim that language has been placed in a hierarchy above other matter and/or communication modes. Multimodality challenges the idea that language (speech and writing) is central to interaction, asserting that both communication and representation encompass multiple modes (gaze, gesture, touch, image, proximity) (Dicks, 2019). Correspondingly, Barad (2003: 801), one of the key theorists behind diffraction, suggests that “language has been given too much power” and is positioned as more trustworthy than matter (Barla, 2023). This conceptual alignment was clearly illustrated in my study. Multimodal transcripts enabled me to de-centre human language and focus on the entanglements between humans and non-humans, empirically grounding posthuman research.

The second way in which multimodal research and diffraction overlap is through their recognition that the methodological choices a researcher makes enacts a partial representation of reality. Diffraction posits that we cannot discover a pre-defined truth about the world and that knowledge production is an agential act (Jenkins et al., 2021). Research is a product of the researchers chosen apparatus and, even studying the same phenomenon, results would be different if they used an alternative method (Uprichard and Dawney, 2019). In a similar vein, multimodal research recognises that multimodal transcripts are partial representations of an event and that they foreground specific elements from the data (Mavers, 2012). Any transcript is limited temporally, spatially and by the modes in which it is presented (Bezemer and Mavers, 2011) and different modes perform different communicative and representational functions (Jewitt, 2011). Through the acknowledgement of partial representation and agential cuts, multimodal and diffractive researchers both acknowledge that their methodological choices reduce accounts of reality (Flewitt et al., 2009; Haraway, 2004) and they take accountability for what is included and excluded in their research apparatuses (Bezemer and Jewitt, 2010; Bezemer and Mavers, 2011; Chee, 2022). In my research, I acknowledged that each multimodal transcript was only a partial representation of reality. Creating multiple multimodal transcripts and reading them diffractively through one another allowed me to see the multiplicities and hidden realities in my data that would have been lost from using just one method alone. Each transcript offered different insights into the event, making my overall analysis more nuanced. For instance, Figure 1 flattened out the hierarchy between human and non-human; Figure 2 revealed how all entities are entangled and act upon one another; and Figure 3 facilitated my participants to “become with” the data. This acknowledgement of the multiplicity of realities, challenges assumptions that human researchers can step back, separate themselves and describe a singular autonomous reality (Spyrou et al., 2018).

The third way in which diffractive analysis aligns with multimodal research is that they both involve repeated engagement with the same data during analysis. Multimodal analysis involves exploring a range of transcription methods (Cowan, 2014) and “different stages of analysis and presentation will require multiple transcriptions” (Goodwin, 2000: 161). Multimodal researchers engage in this repetitive process to analyse the data and to delineate the most appropriate method (and mode) of transcription. My process of diffracting multimodal transcripts expands upon this. A diffractive researcher reads disciplines through one another as a means of “luxuriating in the data, creating a space for analytical thinking and a launch pad for further development of ideas” (Mac Lure, 2013 in Sakr, 2022: 441). With diffraction, repeated engagement with theory, data and methods is not about comparing disciplines or getting to a “truth” but instead to explore differences, consider what is excluded and the significance of those exclusions (Larsen and Schwennesen, 2024). In particular, it considers the exclusion of non-humans, more-than-humans, and those that do not fit into the dominant political category of human (those who are adult, white, middle class, male, heterosexual and able-bodied) (Murris, 2016). This repeated engagement with my data during analysis helped to avoid simply reproducing dominant discourse in my field. It forced me to slow down and re-think the agential hierarches that exist between adult, child and non-human in contemporary Early Childhood discourse.

While there are conceptual alignments between multimodality and posthumanism they come from different ontological positions. In its current conceptualisation, multimodality studies pre-existing fixed entities that interact with one another. Within multimodality, the human researcher is positioned as distinct from the data which they collect and then analyse. In contrast with this, posthumanism derives from a relational ontology. Relational ontologies emphasise interdependence, claiming that individual entities do not pre-exist and that they materialise through intra-actions (Spyrou, 2018). Within posthumanism, the researcher is not seen as separate from the data, they are enmeshed, a part of the research assemblage (Mazzei, 2014). Thinking-with diffraction shifts the creation and reading of multimodal transcripts towards a relational ontology. My aim in diffracting multimodal transcripts was not to capture and represent pre-existing fixed entities interacting but to perform what is and what is not possible. I used diffraction to (re)consider the ways reality is materialised, and how boundaries are fixed through the process of transcription. Being responsive to how research gives the world material form is aligned with a relational ontology. The details that researchers choose to include and exclude from transcripts matter significantly for research participants and for how the world is (re)configured. This diffractive approach enables a researcher that uses multimodal transcription to acknowledge the interdependency of entities, take responsibility for their agential cuts and recognise they are part of the research assemblage. “That is, the diffractive methodology that I use in thinking insights from different disciplines (and interdisciplinary approaches) through one another is attentive to the relational ontology that is at the core of agential realism” (Barad, 2007: 93).

Conclusion

There are four key findings from this paper. Firstly, a multimodal approach can support the transcription and analysis of entanglements between humans and non-humans. Multimodal transcription provides researchers with a method to slow down their focus and transcribe the actions of non-humans. Secondly, diffraction and multimodality converge through (1) de-centring written and spoken language (2) recognising that the methodological choices a researcher makes enacts a partial representation of reality (3) repeated engagement with data. Thirdly, diffracting multimodal transcripts discloses hidden realities in the data. Through re-making transcripts and transducing modes, new meaning and understanding is produced. One transcript reduces reality, whereas reading multiple transcripts through one another and reflecting on differences can provide a much more nuanced and in-depth analysis. Finally, applying diffractive analysis to multimodal transcripts provides tools to work across (and re-conceptualise) the disciplines of multimodality and posthumanism, without watering down the specialised language or arguments from either field. Together, these findings demonstrate that a diffractive analysis of multimodal transcripts can provide researchers will a tool to de-centre humans and empirically ground posthuman research, which is this papers main contribution to knowledge.

A limitation of this article is that it presents only one case study out of eight. In my wider study, I noticed that alternative data sets produced different differences and thus made me reflect on different exclusions. For example, in the transcripts presented in this article there are very few sounds coming from the tablet application. This means that the visual had priority in the enactment of non-human participants, creating a potential imbalance with the modes recorded between human and non-human. Therefore, while these multimodal transcripts were suitable for presenting intra-actions in my specific data, I am in no means suggesting that these transcripts are immediately transferable, and it is important for researchers to “experience with” their own data and create their own transcripts. Indeed, it is through this process of creation that they can be affected by and “become with” their data.

As this was an exploration of what is possible, I did encounter a few challenges. Whilst completing a diffractive analysis. I found “reading through” transcription methods difficult. It was extremely challenging to not revert to “reading against”, critiquing and making comparisons. Certainly, there are gains and losses of each transcription method – some are more suited to re-configuring entanglements and thinking-through posthuman theories. However, my aim in applying a diffractive analysis to multimodal transcripts was not to find the most appropriate method but to see what difference each transcription method provides, what reading them through each other affords and how this process supports the empirical grounding of posthuman theories.

Another obstacle which I encountered when completing this process was in de-centring humans. I was cognisant of focusing my transcript on the humans and made an explicit effort not to do this. Nevertheless, my unconscious and automatic anthropocentric focus was a constant concern. This is an issue which other posthuman researchers have encountered when completing empirical posthuman research (Hackett and Somerville, 2017; Wargo, 2018). To address this, after completing each multimodal transcript, I reflected on the agential cuts I was making and how my choices were creating a partial, humanistic and linguistically centred representation of the data. I carefully questioned the variables of what each type of transcript might include or exclude and how this might support (or hinder) thinking-through a posthuman theoretical frame. This involved zooming in on the different modes of communication of both human and non-human.

Going forward, my recommendation is that researchers explore this diffractive process of analysis themselves, which can only be achieved by engaging with their own data set. Initially, video data collection needs to be precise in how it records all modes of intra-action between humans and non-humans, and a small excerpt should be chosen. Following this, a range of transcription methods can be explored. When exploring transcription methods, I recommend replicating Figure 1 as this will provide a starting point and de-centre an anthropocentric gaze. After completing this, I suggest reflecting on the gains and losses from this transcript. Then, a range of transcription methods can be created, using a combination of writing, layout, typography and image. As each new transcript is created, I advise reading these through each other, considering the differences they make, whether they materialise intra-actions and what they make invisible/visible. These transcripts can then be used to think-with posthuman theories and empirically ground posthumanism.