Book review: Multimodal Analysis in Academic Settings: From Research to Teaching

Multimodal Analysis in Academic Settings: From Research to Teaching is a new edition to the series of Routledge Studies in Multimodality. Multimodality is described by Murray (2013) as communication practices using the textual, aural, linguistic, spatial and visual resources to compose messages. As the title of this volume indicates, teaching and research practices are closely connected with multimodal discourse in tertiary institutions. Meaning-making resources or modes are multimodal, going beyond mere linguistic artifacts. Academic communication occurs in a series of events such as conference presentations, plenary addresses, teachers’ lectures or student learning, all of which are multimodal. This book is intended to examine the discursive features (multimodality) and the interactions between various modes (verbal and non-verbal resources) in academic settings.

In addition to an introduction, Multimodal Analysis in Academic Settings consists of two parts. In Part I (Chapters 1–5), the contributors focus on research communications, exploring the generic features of conference presentations and plenary addresses aurally and visually. In Part II (Chapters 6–10), multimodal research in the classroom is carried out through the investigation of teacher–student or student–student interactions and teaching/learning practices combined with various meaning-making resources/modes (especially visual support). Overall, this volume is well organized. Indeed, it reflects a new trend of multimodal analysis in research, teaching and learning activities in the environment of higher education.

‘Disagreements in Plenary Addresses as Multimodal Action’ (Chapter 1) and ‘Contrastive Multimodal Analysis: Conference Plenary Lectures in English and Spanish’ (Chapter 2) investigate the ‘multimodality’ of plenary speeches at academic conferences. With the help of ELAN multimodal annotation software, the author, Zuocheng Zhang, examines how multimodal meaning-making resources (spoken language, hand movement, head movement, facial expression, posture and gaze) are utilized by academic presenters to express disagreements at conference presentations. Also, Zhang provides us with valuable insights into teaching English for Academic Purposes. The authors of Chapter 2, Ruiz-Madrid and Fortanet-Gómez, carry out a contrastive multimodal analysis of asides, including ‘digressions, anecdotes and humor’ (p. 57) that occur in plenary addresses, comparing two lectures: one delivered in English and the other in Spanish. During the process of analysis, the authors take into account linguistic features, visual communication and kinesics in order to offer a comprehensive understanding of plenary lectures, revealing that verbal and non-verbal resources should be jointly used at academic conferences.

‘Intensifying Adverbs in Academic Spoken Discourse: A Contrastive Study between English and Spanish’ (Chapter 3), ‘Visual Communication in Applied Linguistics Conference Presentations’ (Chapter 4) and ‘A Multimodal Approach to Persuasion in Conference Presentations’ (Chapter 5) address the issues of scientific research presentations delivered by academics in different ways. Miguel F Ruiz-Garrido (Chapter 3) studies the features of intensifying adverbs and those supporting non-linguistic devices (pitch, silence, syllable prolongation; head, hand and body movements). On the basis of a corpus called MASC, Ruiz-Garrido carries out a contrastive study between English and Spanish scientific presentations, showing that the intensifiers combined with relevant paralinguistic and kinesic devices, contribute to reinforcing the intended meanings (which are conveyed by a presenter). In Chapter 4, Giuliana Diani copes with the problems of visual communication in applied linguistics conference presentations. Specifically, Diani studies the designing of PowerPoint slides, including how to incorporate visual resources (scriptures, numbers, figures and graphs) into a single slide and how to organize the slide sequences logically. Based on a corpus of PowerPoint slides, this chapter has many useful benefits for presentation–skill training in the area of English for Academic Purposes. The author of Chapter 5, Julia Valeiras Jurado, explores how academics convey their persuasive information at conference presentations through a case study of English business students. Jurado performs an in-depth analysis of semiotic modes (including intonation, head movements and gestures) used in oral presentations. Thus, persuasion in research presentations is achieved in a more comprehensive way.

Chapter 6, with the title ‘There Is More to Multimodality than Discourse Features and Nonverbal Behaviors’, written by Christine Räisänen, is regarded as ‘a bridge between research and instruction’ (p. 10). Räisänen argues that multimodal discourse is a good combination of social and material elements. The author attaches equal importance to ‘textual and contextual characteristics’ and ‘the interactional features of speaker/audience, speech/visuals, and the verbal/nonverbal’ (p. 136). Räisänen raises our awareness of the importance of addressing the affordances and constraints of materials (e.g. spatial layout, technical props and visual illustrations) when delivering conference presentations. In this way, conference presentations are seen by Räisänen as academic genres that can be taught and learnt in higher education.

In ‘Elaborating Explanations during OpenCourseWare Humanities Lectures: The Interplay of Verbal and Nonverbal Strategies’ (Chapter 7) and ‘Multimodality in Business Communication: Body Language as a Visual Aid in Student Presentations’ (Chapter 8), multimodal analyses are conducted in the classroom. Belinda Crawford Camiciottoli (Chapter 7), seeks a better understanding of how verbal explanations work with the accompanying modes during humanities lectures. The dataset for this study consists of digitally recorded lectures on ‘Yale University’s Open Courses website’. The multimodal analysis proceeds in two steps: the first step is to decide the features of verbal explanations and the second is to concentrate on the accompanying non-verbal features (prosodic stress, gaze and gestures). Valuable implications can be drawn that non-verbal signals reinforce those verbal explanatory expressions and that this multimodal strategy is useful for non-native speakers for improving their listening comprehension in academic courses. Juan Carlos Palmer-Silveira, the author of Chapter 8, performs an analysis of business presentations delivered by master students in English Language for International Trade, describing and assessing these students’ use of non-verbal features, such as stage, hand movements, eye contacts and head movements. Palmer-Silveira focuses on these non-verbal features using three video-recorded samples of student business presentations. Drawing upon this study, we can see clearly that non-verbal devices enable those business presenters to engage their audience and reinforce their communicative abilities.

The last two contributions, ‘Assessing Multimodal Listening’, by Mari Carmen Campoy-Cubillo, and ‘Teaching Learners How to Use Pragmatic Routines through Audiovisual material’, by Silvia Bruti, situate multimodal applications in the context of language instruction. Campoy-Cubillo discusses how to assess listening comprehension skills from the perspective of multimodality. Listening and visualizing collaboratively become involved in multimodal listening activities and authentic life situations are created in the classroom. Different from the traditional audio input, multimodal input is closely related to verbal and non-verbal information. The aim of multimodal listening assessment is to see how students construct meanings through using various linguistic and non-linguistic resources (e.g. gestures, facial expressions, images, etc.). In short, non-linguistic components should not be neglected in designing the criteria for assessing multimodal listening. In the last chapter, Bruti exploits film clips as audiovisual materials, focusing on ‘the multimodal dimension of a small number of conversational routines’ (p. 218) in the language teaching classroom. During the process of multimodal analysis, the author selects two film clips consisting of interactional routines (greetings and leavings) from a film clip database at the Berkeley Language Center for multimodal transcription. Based upon this kind of transcription, multimodal resources are utilized to teach foreign language learners linguistically, pragmatically and culturally in a comparatively effective way. Finally, Bruti suggests the possibility of extending the model of ‘conversational routines’ to ‘face-to-face interactions’ in language teaching.

In conclusion, the authors of this book make valuable contributions to the emerging field of multimodal discourse analysis in academic settings. First, this volume provides us with a more comprehensive understanding of multimodal analysis in this area through bringing different approaches together under the same umbrella. According to Fortanet-Gómez and Camiciottoli in the introduction (p. 1), three approaches exert a profound influence upon the development of multimodal discourse analysis: ‘social semiotics’ (Kress and Van Leeuwen, 2001), ‘discourse analysis grounded in systemic functional theory’ (O’Halloran, 2004) and ‘multimodal interactional analysis’ (Scollon and Scollon, 2003). Actually, there are no clear-cut boundaries between the three approaches. But no matter what approaches to multimodality we take, the ultimate goal of this volume is to explore how discursive meanings are constructed by resorting to semiotic resources verbally/non-verbally and aurally/visually. Secondly, quantitative methods (e.g. corpus analysis) and qualitative methods (e.g. multimodal transcription) are applied to produce a range of different insights. Thirdly, it shows that adopted multimodal strategies have the potential to enhance the effectiveness of communication between academic presenters and their audience in academic settings. However, we should keep in mind that ‘conventionality’ has to be differentiated from ‘individuality’ in analysing non-verbal features, especially when the corpus is not large enough. Indeed, this volume is a valuable resource for those researchers, teachers and students (especially postgraduates) who are interested in communication studies, multimodal discourse analysis and foreign language teaching.