Abstract
In this issue, we celebrate the 20th anniversary of Visual Communication, launched in 2002. Over these 20 years, the journal has published 78 issues with over 10,000 pages, featuring research articles, visual essays, practitioner reflections, and book and exhibition reviews. At the time of the initial launch, the founding editors Carey Jewitt, Theo van Leeuwen, Ron Scollon and Teal Triggs wrote:
Visual Communication will address not a unified and well institutionalized academic discipline but a varied group of people from a wide range of fields who share, nevertheless, a common interest in visual communication and its role in society.
(Jewitt et al, 2002: 8)
It was an ambitious undertaking, to create a journal that was inherently multi-disciplinary, which brought the visual from the margins of academic study to the centre, which was brave enough to challenge conventional academic genres, which recognised the intersection of the visual with all modes, and which addressed the fundamental role the visual plays in social life. For this anniversary issue, and as the new editorial team of five years’ standing, we have selected papers which speak to each of these strengths. These papers demonstrate diverse methods from historical analysis to eye-tracking; draw on diverse fields from psychology to mathematics; address the visual as a two-dimensional image and as part of embodied experience; reveal its relevance to multiple fields from healthcare to education; and make their arguments in classic academic forms as well as through the visual itself.
The multi-disciplinarity of the journal remains one of its key features. As Thomson (2021) notes, the journal includes, inter alia, multimodal, critical, content, corpus, conversation, fractal and meta analyses, not to mention surveys, interviews, autoethnography, and photovoice. While interdisciplinarity is an oft-sought goal, contributions to the journal inevitably arise from strong disciplinary bases. Social semiotics is a frequently recurring framework found in the journal, drawing in particular on the seminal work of Kress and van Leeuwen 2021 [1996] in relation to visual analysis. But it is certainly not the sole framework which is made use of, and even social semiotics has its diverse manifestations. We do not promote one voice or one point of view, nor seek for competing voices to be resolved. In this issue, the perspectives and methods adopted include cognitive metaphor theory and conversation analysis, street photography and health design thinking, social semiotics, mediated discourse analysis, and eye-tracking. Our own editorial team favours different approaches and methods, from the strongly empirical, with scientific validity and the capacities of scale, to the highly qualitative, favouring the potential insights of close analysis. As Pflaeging et al (2021: 12-16) reveal, the empirical orientation included in the journal, i.e. the engagement with data and large-scale approaches, is becoming more common in our journal and in academia generally, and the proportion of large-scale studies has increased to a considerable degree. However, qualitative and participative studies continue to provide key insights to our understanding of the visual. One of the journal’s main challenges is to balance these complementary but also competing interests. Our solution is to come back to the issues of meaning-making and lived experience in every contribution: what does a particular paper add to our understanding of visual communication in relation to social life, past, present, or possibly future? The deployment of quantitative methods means little if it results only in a set of numbers, and the most insightful analysis of a single text is worthless if it has no broader implications.
With this, the journal continues to bring appreciation of the visual to the fore of academic study. Indeed, instead of being an outlier, study of the visual is now almost conventionalised – the ‘visual turn’ has hit many fields! Yet the original editors foreshadowed that the visual should always be addressed in relation to other modes, adopting a ‘broad view of the visual’ (Jewitt et al, 2002: 8) and resulting in ‘multimodality: the interaction between visual communication and other modes of communication such as language, music, sound and action’ (Jewitt et al, 2002: 9). The papers of this issue address intersections of the visual with aural environments, three-dimensional spatial models, and the ‘unseen’ algorithms of digital science. This ‘broad view’ is also reflected in Thomson’s (2021) word cloud of the topics covered in 17 years of the journal’s history (from 2002 to 2019, see Figure 1):
1 The top 100 words of article titles published in 17 years’ worth of VC studies (Thomson, 2021: 31).
The diverse approaches encompassed by the journal pose particular challenges for what counts as a ‘valid’ academic genre. Different approaches to knowledge have different ways of construing that knowledge – different ‘legitimation codes’ as Maton (2016) would say – and the genres of one approach do not necessarily make sense to the genres of another. There is a too-easy tendency for ‘the’ genre of the (scientific) research article to dominate. Part of the foresight of the original editors was their willingness to embrace new genres, including the visual essay and the practitioner piece, both of which continue to be a strength of the journal. These demonstrate that research includes insights that can’t necessarily be articulated in words, that creativity is as central to method as scientific inquiry, and that the domain of visual communication necessarily encompasses practice.
It goes without saying that much has changed in the visual domain since 2002, in how we make visible what we want to be seen, in how we share what we see, and what we can do with how we see.
Digital technologies and access to the data they generate have brought evident changes in great swathes of contemporary life: data visualisation, artificial intelligence, immersive media, new capacities for surveillance, and of course, new capacities for connecting (Figure 2). As Manovich asks (2012, 2018), how do we make sense of a world engaging with one billion images per second? The answer is: we need to come back to the fact that it is life which is changed by these developments, not least the agency afforded to individuals and groups through their capacity to engage directly in production, selection, and distribution processes (Kress and van Leeuwen, 2001). What is at issue is not only the ‘what’ and the ‘how’, but ‘for whom’ and ‘by whom’: ‘social actors’ (van Leeuwen, 1996, 2008) are intrinsic to all of these processes. Despite the centripetal forces of AI and digital technologies, which capture all in their wake, and which potentially impose commonalities of life experience on all, there are upswings in centrifugal forces, which see a drive to experimentation, individuation, innovation (in design, formality, and creativity), and the highlighting of concerns which previously were invisible. We trust that Visual Communication is and can be the dynamic equilibrium documenting and coordinating these forces: providing space for both traditional and innovative approaches to the visual; allowing for firmly established and maximally creative scientific practices, and witnessing the newest trends and evolutions.
2 Mosaic graphic. Image authors from left to right, top row to bottom row: Jana Pflaeging & Chiao-I Tseng, all other photos published on Unsplash: Luke Chesser, Peter Bond, Lucrezia Carnelos, British Library, Bruno Martins, Jason Blackeye, Mika Baumeister, Ryan Garry, William Hook, Manolo Chretien, Austin Distel, Johnny Briggs, Austin Distel, Kenny Eliason, Aleksandra Sapozhnikova, Ramon Kagie, Thomas Charters, Marvin Meyer, Markus Spiske, Tyler Nix, Obi Pixel6Propix, Ricardo Gomez Angel, Luke Chesser, Annie Spratt. Jana Pflaeging, 022.
The Focus of this Special Issue
The papers we have selected for this issue not only reflect this dynamic balance, but also speak to the challenges of a journal examining visual communication in the first two decades of the 21st century. The issue opens and closes with visual essays, respectively ‘Alien Domesticity: Representing home during a pandemic’ by Brent Luvaas, and ‘Applying health design thinking to uncover actors in the sustenance of health and wellbeing during hotel quarantine in Kuwait’ by Juhri Selamet. These respond to one of the greatest challenges faced by so many during the past few years: just how to survive, and get through, the pandemic. Luvaas focuses on the nature of home, and its transformation during this time to a place of ‘dozens of worlds competing within’. Using the methods of street photography, Luvaas presents his own home as if ‘an outside observer’, hoping to capture home as a process and as an affective landscape, and hoping to do justice ‘to what that process feels like during pandemic times’. In a complementary way, Selamit reflects on hotel quarantine, and based on his own experience of this, examines the interaction of human and non-human actors, and how this does – or does not – contribute to health and well-being during such a confinement. He applies health-design thinking to a photo journal of his own quarantine period, and develops journey maps of the interactions between himself and artefacts/services during this period. While this ‘uncovers a micro, individual perspective on living and going through this environment’, it also shows that ‘design practitioners, design researchers, and health workers must rethink and bring about change’ in this – and related – contexts.
A particular challenge, or opportunity, for the journal is its multidisciplinarity, and here the papers we have selected speak to the diversity of approaches encompassed. In their paper on ‘Michel Pastoureau and the history of visual communication’, Giorgia Aiello and Theo van Leeuwen take the field forward by taking us back, back to fundamental principles of social semiotic theory, to examine not only how semiotic resources and semiotic practices change, but also why they do so. They argue that historical research and contextualisation is a necessary foundation for this, and highlight the contribution of Pastoureau to social semiotics in this regard, even if Pastoureau did not identify himself in this way. The past, they argue, is necessary to illuminate the present, and this needs to be, or to become, a key tool in a social semiotician’s repertoire.
Jill Bennett’s article on ‘Visual communication and mental health’ draws on psychoanalysis and cognitive metaphor theory to demonstrate the importance of lived experience in creating novel visualisations of mental health conditions such as depression and anxiety. Through the analysis of key examples, including the use of virtual reality in the Anangu community of central Australia, Bennett demonstrates that such novel representations can both avoid and explicitly debunk the too-easily generated stereotypes of generic images and conventionalised tropes. Such re-imaginings prompt ‘a host of rich and contradictory associations rather than a singular classification’ and it is this multiplicity which can enable some to ‘move through’ their experience towards solutions. Bennett helps us see (pun intended?) that the visual is not, indeed, something just to be looked at: rather, it is part of lived experience. It arises from such experience and gives shape to it, with the capacity to either resonate, recast, or repel.
In Sara Merlino, Lorenza Mondada, and Ola Söderström’s article, ‘Walking through the city soundscape: An audio-visual analysis of sensory experience for people with psychosis’, the visual is addressed in two ways. Firstly, it is addressed as a method, using video to record patients with psychosis – for whom the experience of noise is often a challenge – walking through the city with their partner, and thus capturing both the noise of the urban environment and the participants’ bodily and verbal responses to it. Secondly, the visual is addressed as one component of a multimodal, multisensorial ensemble, in terms of how sound and responses to it may relate to what is seen and/or shown. The micro-observations of conversation analysis are augmented by this account of aural sensoriality, and as with Bennett’s work, it is lived experience that is key here: embodied, situated, and real. Also as with Bennett, generic assumptions are eschewed in favour of analyses which are deeply immersed within interactional and sequential contexts.
In contrast, Areej Albawardi and Rodney Jones’ article, ‘Saudi women driving: images, stereotyping and digital media’, examines the now-integral role of metadata attached to and circulated with images online, and reveals how this can reinforce already-problematic stereotypes. They examine the 2017 lifting of the ban on Saudi women driving, and the impact of international news sites, magazines and advertisements making use of stock images (Getty Images, Shutterstock and Google Images) to communicate about this event. Their methods combine content analysis, social semiotic analysis, and mediated discourse analysis, to not only examine how these images are composed, but also how they are ‘made available to users through search engines and how they are adapted to different rhetorical and cultural contexts’. Albawardi and Jones demonstrate that metatextual practices, including the metadata of semantic tags and the organization of search parameters via algorithms, introduce ‘new variables’ into issues of representation, particularly that of stereotyping, and how such homogenizing practices erase the complexities of actual experience and social realities, and as such, are ‘disconnected from the real world’.
Received assumptions are also questioned in Eva Brumberger’s article, ‘Generational differences in viewing behaviours: An eye-tracking study’. Eye-tracking as a method of study is enabled by the affordances of digital media, and was brought to the fore in Visual Communication as early as 2006, with Holsanova et al.’s study of reading paths in newspapers (2006, vol. 5, 1: pp. 65-93). Since then, it has been consolidated as a means of verifying what it is that we actually focus on when ‘seeing’, for how long our gaze dwells on particular areas of the page, and in what sequence. Brumberger uses eye-tracking to compare participants from two age groups, to test the assumption that ‘younger individuals are inherently more visually skilled than previous generations’. Here we see the important contribution of a rigorously empirical study, with somewhat counter-intuitive results. The study reveals that there is little difference between the two groups, except perhaps for a greater number of ‘secondary areas of attention’ in older participants. Contextualising the findings, Brumberger argues that the digital natives argument ‘rests on broad and largely untested claims that elide the complexities of visual literacy and learning more broadly’.
The intersection of visual communication with questions of literacy and learning is also at the heart of Christine Price and Arlene Archer’s article, ‘Resemiotisation: tracing the movement of resources in landscape architectural design trajectories’. They argue that the teaching of landscape architecture in South Africa is dominated by practices which derive from the ‘global north’, and which are unsuited to the contemporary South African context. As an alternative, Price and Archer propose a ‘multimodal pedagogy for diversity’: a pedagogy which valorises ‘the resources and resourcefulness of the meaning maker’, rather than pre-defined conceptions of what constitutes ‘good’ design. They trace the resemiotisation processes in the design trajectory of one student, and how she moves ‘between spatial, visual and verbal modes’ in the unfolding design process. The close attention to the prompts which initiate iterations in this process, and the deployment of social semiotic analyses of two-dimensional sketches and three-dimensional spatial models, reveal the ways in which meanings are mobilised, carried forward, and reshaped.
Finally, in ‘Matter, meaning, and semiotics’, Kay O’Halloran points us towards the future, interrogating the role of the visual in the context of a digital ecosystem, and how changes in technology impact upon semiosis itself, that is, how ‘the world of matter’ impacts upon ‘the world of meaning’. O’Halloran argues that it is the very nature of artificial intelligence in the digital environment which makes it ‘difficult to understand the legacies of digital media and their social, political, cultural and economic impacts’. O’Halloran draws upon her deep knowledge of both the history of mathematical symbolism and the functioning of software and algorithms to argue that contemporary data science is entrenching social inequality, and that semiotics – including studies of the visual – has a particularly important role to play in terms of interrogating the nature of these practices, as a means ‘to inform design, policymaking, and activism around future digital technologies’.
Collectively, these contributions demonstrate that both multidisciplinarity and interdisciplinarity are important features of Visual Communication. None of these could be written without deep engagement with insights and proposals from a broad range of fields: history, cultural studies, psychology, psychiatry, geography, mathematics, information science, architecture, pedagogy. Meaning-making is an entanglement of practices, and can only be illuminated with recourse to the expertise and insights of multiple contributing perspectives. All of them are part of and contribute to the dynamic equilibrium that Visual Communication maintains and that is open to many more perspectives and forces.
Challenges and Thanks
In his 2021 review, Thomson notes some of the challenges facing a journal such as ours, not least that of being more representative of world-wide scholarship on visual communication, and not just that of the ‘global north’ as dominated by the USA, UK, and western Europe. We could not agree more, and we seek to continually expand the diversity of our editorial and advisory boards, and to treat submissions from non-English scholars with the care and consideration they deserve. We also continue to seek diversity of submission types, not just research articles, but reflective practitioner pieces, and visual essays. The conventional research article remains dominant, but we see in an overview of genres the roles that other types play in the journal (Figure 3).
3 Genre profiles in Visual Communication, 2002–2022. Jana Pflaeging, 2022.
Another area we are continuously expanding – and experiencing, thanks to the diverse submissions we receive – is that of the issues with which the journal is engaged. If the study of visual communication is to have any impact on social life, then it must be appliable, in Halliday’s original sense of ‘a theory as a mode of action that is based on understanding’ with ‘cycles of mutual reinforcement between practice and theory’ (Halliday, 2015: 97). Our late colleague and co-editor, Martin Thomas, particularly valued this concept, and we hope this is evident throughout the journal, in all issues.
In this regard, we see much scholarship reflecting the need for greater diversity of disciplines and approaches, of topics and themes, and of visual and multimodal representations of factors such as ability, gender, and ethnicity (see for example, Caple and Tian, 2022; Zhongxuan and Lin, 2021). We see great potential for visual communication scholarship to impact positively on contexts and fields such as politics (Chu 2021; Varvantakis & Nolas 2021), healthcare, both mental and physical (as with Bennett, this issue; Forceville & Paling 2018; Mondada, this issue; Robson, 2022), and our understanding of critical social crises, such as the experience of refugees (Catalani 2019; De La Presa and Ruiz, 2021), environmental disaster (Björkvall and Archer, 2021; Kwesell, 2021) and of course, the pandemic (Luvaas, this issue; Selamet, this issue; Yu and Yan, 2021). We continue to seek examples of the best and most recent innovations in visual design, be it at the national scale of grand institutions and exhibitions (Jones, 2020), or individual experimentation with expression (Cooper 2022; Raappano-Luiro, 2020). Expanded theories and methodologies are needed to help make sense of our digital world, not only the semiotic technologies which enable and change this world, including digital artefacts and digitally-mediated experiences (Albawardi and Jones, this issue; Main and Yamada-Rice, 2022), but also artificial intelligence, deep learning and computational analysis (Butkowski et al. 2022; Pentzold & Rothe 2022).
More recently, and just as the first editors did, we are challenging the fact that the objects of analysis discussed in the journal are by no means purely visual and just as little, even audio-visual. Multimodal meaning-making also happens in haptic experiences and through embodiment, and the near future might bring us digital olfaction and taste as forms of new sensory perception. We are excited to drive the journal towards these innovations.
Having been Chief Editors for the last five of these 20 years, we feel as if we are just beginning to understand the processes, both scholarly and pragmatic, which enable such a journal to function. Despite the brilliant affordances of online production, not least the immediate availability of ‘online first’ publication, we are proud to continue with a print version of the journal also. Yet, a curious enigma of the production and distribution processes is that the relation between print and online versions of the journal remains fractured. Online enables speedy and widespread dissemination, but does not replicate the qualities, feel, or design of the print version. If you read the journal online (where you may already have seen some of the articles of this issue in their ‘online first’ format), make sure you access the pdf version of articles also, to compare the impact of production and distribution on the design of the finished product. This is particularly important for visual essays, which can be seen in their ‘true’ format only in print/pdf. We particularly thank the designers at Pony UK, Nigel Truswell and Niall Sweeney, for their creative and professional input on these.
Our work could not happen without the support of our editorial team. We have faced diverse challenges and changes, but our collaborations make it all worthwhile, through collegiality, insights, and hard work. As our administrative team of one, Anna Harold manages to keep processes flowing despite these unfolding across multiple time zones.
Thank you to her and the editors current and previous for all your efforts. This particular issue and the recent years would not have been possible without the support of Jana Pflaeging, Michele Zappavigna, Søren Vigild Poulsen, and Dušan Stamenković. We recently welcomed aboard Alexandra Crosby, and promised her much hard work to come! Jana Pflaeging has generously helped us with the cover design of this Special Issue, as well as the images for this editorial. Our thanks also go to the publishers Sage, who remain fiercely independent and supportive of endeavours such as ours and whose work makes sure the journal has a presence both physical and digital. We particularly thank James Skelding Tattle, James Leng, and Jane Price for their continuous work and collegiality.
The journal is of course deeply indebted to the vast retinue of scholars who contribute their work for consideration, engage in the exploration of visual communication with us, and upon whom we draw for their disciplinary expertise in evaluating submissions. The latter work is entirely voluntary and increasingly difficult for individuals to accommodate within the institutional processes of most contemporary universities, and we remain genuinely grateful to both authors and reviewers for their enthusiastic support of the journal.
The diverse contributions published by Visual Communication in the last 20 years reflect the complex entanglement of our agenda to embrace diverse theories and methods in the context of multiple areas of investigation and application. We hope that you – readers, writers, creative practitioners, reviewers – continue to engage with us in this exploration, and help us showcase the methods, tools, theories, and practices which shape visual communication, then, now, and in the years to come.
Thank you and enjoy.