Abstract
Unveiling visual meaning-making progression of video ethnography
Systematically Working with Multimodal Data: Research Methods in Multimodal Discourse Analysis is a theory-grounded and research-oriented step-by-step guidebook for undergraduate and graduate students, as well as for emerging and established researchers who are interested in analysing multimodal (inter)actional data, particularly ethnographic video data. As Norris (2004) states, ‘all interactions are multimodal’ (p. 1) and multimodality includes linear and non-linear communicative data (Bateman et al., 2017) across disciplines and genres. To anatomize these data, scholars proposed many multimodal analytical approaches from different perspectives. The most influential approaches are Kress and Van Leeuwen’s social semiotic approach, systematic functional multimodal discourse analysis (SF-MDA) from O’Halloran, Bateman’s genre and modality framework (GeM) and Norris’ multimodal (inter)action analysis. Originating from mediated discourse analysis (Scollon, 1998), multimodal (inter)action analysis focuses on how different communicative modes are utilized by social actors in actual (inter)actions, rather than on functions and regularities of static modes. Being different from quantitative corpora data in social semiotic analysis, SF-MDA and GeM framework, multimodal (inter)actional data have no countable unit of analysis and are not suitable for quantitative scrutiny. Therefore, what are their units of analysis? How to distinguish one (inter)actional mode from another? How to investigate the functions of various modes playing together in interaction? These questions make multimodal video data processing overwhelming and remain the emergent concerns of the framework of multimodal (inter)action analysis. To facilitate the operability of this framework, in this volume, Norris demonstrates a five-phase approach with specific steps and examples for multimodal (inter)actional video data processing, including data collection, delineation, selection, transcription and analyses.
This volume consists of eight chapters. Chapter 1 provides an introduction to all the chapters and offers guide notes recommending certain sections to readers according to their academic levels. It also provides a literature review on the main existing multimodality theoretical frameworks, and then narrows it down to multimodal (inter)action analysis. Chapter 2 introduces the theoretical and philosophical underpinnings of the approach: ‘perception’ and ‘embodiment’. Chapters 3 through 7 provide a step-by-step guide from Data Phase I to Data Phase V with examples, showing the steps to process YouTube data, experimental data and ethnographic video data at each phase. Chapter 8 offers suggestions for instructors teaching bachelor, master and PhD levels.
The book is very well-structured. The beginning chapters present the theoretical and philosophical foundations that justify the necessity and validity of Norris’ framework. The middle sections illustrate a technical guide for data processing, which clarifies the applicability of the framework. The closing chapter concludes a classroom guide for instructors, endowing the framework with great pedagogical significance. To soothe the pressure on readers of overwhelming qualitative data processing, the author illustrates each technical phase in a gradual way, from the processing of micro YouTube data to the medium-sized experimental datasets and finally the sizeable ethnographic video data. Classroom tasks and guiding notes are included so that learners can quickly find the content they are interested in and stay fully engaged. In addition, a companion website offers technical and pedagogical support for learners and teachers with video samples and transcripts.
This volume further elaborates on Norris’ seminal monograph Analyzing Multimodal Interaction: A Methodological Framework (2004) and updates the operational mechanism of Norris’s framework. Norris had successfully shifted the attention of multimodality from static meaning representation and communication to real interaction (Jewitt, 2014: 37). In line with Norris (2004), Systematically Working with Multimodal Data: Research Methods in Multimodal Discourse Analysis still adopts the concepts of mediated actions (lower-level, higher-level and frozen mediated actions) as the units of analysis. Meanwhile, it further extends Norris (2004) by categorizing (inter)actional data analyses into micro- and macro-analyses. The main analytical tools from Norris (2004) – mediated actions, modal density, modal configuration (which was originally defined as the interconnection of modes), modal dense foreground–background continuum of attention/awareness and semantic/pragmatic means are categorized in micro-analysis. The redefined concept of ‘modal configuration’ examines the hierarchical importance of each lower-mediated action for the production of its higher-level mediated actions. In order to place the micro-analysis in a broader perspective, three additional tools are used to perform a mediated macro-analysis. These are: (1) scales of action, (2) the site of engagement, practices and discourses, and (3) time cycles and rhythms. Specifically, the tool ‘scales of action’ helps identify the embeddedness of mediated actions. The tool ‘site of engagement, practices and discourses’ measures how concrete mediated actions are produced together with practices and discourses at the same social time and place. The tool ‘time cycles and rhythms’ examines how concrete actions are rhythmically embedded in natural time cycles. To use these tools correctly, ethnographic field notes and interview data have to be collected, in addition to video data, to learn about the psychological perceptions of social actors. The philosophical concepts of perception and embodiment are added to analyse social actors’ perceptions of actions and interactions and help understand how their perceptions connect to their bodies and the surrounding environment.
As an essential reading for multimodality researchers, this volume provides an easily applicable, highly replicable and scientifically reliable methodological framework for multimodal (inter)action analysis. However, there are a few shortcomings: first, the book would be easier to find for the researchers to whom it is addressed if the title were narrowed down to ‘ethnographic video data’; second, the book would gain more clarity if the selection and application of analytical tools in micro- and macro-analyses in Chapter 7 were illustrated using an ethnographic video example. Some recommendations for the application of Norris’ approach are: (a) researchers need to adopt their own theoretical framework working for their research focus since Norris’ framework is more like a technical methodological guide instead of theoretical direction; (b) although Norris claims that her framework is holistic and comprehensive, researchers still need to situate their analysis in a broader cultural and contextual background to explore the impetus of (inter)actions; and (c) Norris is right when she says that no single method can analyse everything (p. 5). This approach is powerful in the analysis of ethnographic video data but may not be appropriate for analysing text–image or text–audio relations.
