Multimodal content analysis: expanding analytical approaches to content analysis

Abstract

Research methods and analytical approaches that support inquiry in the social sciences need to respond to continual changes in the theoretical frameworks, research methods, and technologies used to support data collection and analysis in contemporary research frameworks. This article describes a variation of qualitative content analysis, termed ‘multimodal content analysis’, that draws upon previous iterations of qualitative content analysis, interpretivist research designs, deductive and inductive reasoning, qualitative data collection and analysis methods, and theories of multimodality for conceptualizing and analyzing a selected corpus of multimodal phenomena. In addition, the analysis of selected commercial wine labels is presented to offer researchers an example of multimodal content analysis to guide future research and open up a dialogue focusing on the potential advantages and challenges to researching multimodal phenomena.

Keywords

content analysis multimodal content analysis multimodality qualitative content analysis social semiotics

Research methods and analytical approaches that support inquiry in the social sciences need to respond to continual changes in the theoretical frameworks, research methods, and technologies used to support data collection and analysis. More specifically, approaches to content analysis need to account for and more explicitly theorize the myriad ways that texts are defined, produced, disseminated, and experienced, in particular, aspects of multimodality and the evolution of technologies used in the processes of representation and communication (Jewitt, 2009). In addition, researchers working to expand their analytical frameworks and approaches to content analysis also need to expand their conceptualizations of the data being analyzed. In other words, in order to increase the rigor and potential quality and trustworthiness of various content analysis frameworks for analyzing multimodal phenomena, researchers must take into consideration the affordances, limitations, and relationships between the various media and modes used in contemporary forms of representation and communication.

This article describes an adaptation of content analysis we have termed ‘multimodal content analysis’ that draws upon previous iterations of content analysis (Krippendorf, 2004; Schreier, 2012), interpretivist research designs (Erickson, 1986; Lincoln and Guba, 1985), deductive and inductive reasoning (Graneheim et al., 2017), qualitative data collection and analysis methods (Strauss and Corbin, 1990), and theories of multimodality (Kress, 2010) for conceptualizing and analyzing a selected corpus of multimodal phenomena. Like other researchers (Schreier, 2012), we suggest the need for an approach to content analysis that moves beyond traditional analytical perspectives and procedures of quantitative content analysis to address the complexities inherent in the multimodal nature of contemporary modes of representation and communication.

In this article, we draw upon a research framework and data analysis procedures previously developed across a set of published research papers in order to explicate our analytical approaches, in particular studies focusing on professional development textbooks and metafictive picturebooks (Serafini and Reid, 2019; Serafini et al., 2015). By doing so, we will make transparent the theoretical underpinnings and analytical perspectives used across our studies in an effort to forecast the changes necessary to expand content analysis procedures to account for the multimodal nature of the phenomena under study. In addition, we feel it is important to reconsider the objectivist foundations that potentially limit traditional content analysis research designs (Elo and Kyngas, 2008; Schreier, 2012). Although the studies described here have focused on analogue, textually dominant data sets, the procedures referred to in this article suggest expanding the scope of qualitative content analysis to address the multimodal nature of contemporary representation and communication. Expanding the analytical frameworks to include multimodal phenomena is an important goal for social science researchers working across online platforms and other forms of digital phenomena as well.

Like earlier calls for more transparency and detailed descriptions of the data analysis procedures used in qualitative research studies (Elo et al., 2014; Smagorinsky, 2008), we agree that content analysts must make explicit the theoretical perspectives, research designs, criteria for data selection, and analytical procedures used in their research studies. In this article, the researchers have re-envisioned their data analysis techniques and procedures, moving beyond an objectivist orientation (Ryan and Bernard, 2000) to take up an interpretivist position addressing the representational and communicational meaning potential of multimodal phenomena. In addition to drawing upon previous published studies, we will provide an example of an ongoing research study focusing on the multimodal elements of commercial wine labels to more effectively illustrate the analytical approaches and procedures we are advocating in this article. This example may serve as a guide for other researchers to analyze the semiotic structures and resources used in various multimodal texts in both analogue and digital contexts.

The Evolution of Content Analysis: A Brief Review

Traditional, or quantitative, methods of content analysis have allowed scholars to conduct systematic analyses across a broad, yet manageable corpus of selected textual matter (Krippendorf, 2004). According to Ryan and Bernard (2000), quantitative content analysis includes techniques for reducing texts to a unit-by-variable matrix and analyzing that matrix to test hypotheses by applying a set of codes to a set of data, the assumption being that the codes of interest have already been discovered and described beforehand. In general, traditional content analysis has been a quantitative methodology with the central tool being its system of predetermined categories, statistical analyses of textual data, and a focus on reliability and generalizability of its findings (Mayring, 2000).

A shift towards qualitative content analysis has been seen as a response to the limitations of quantitative data analysis and the challenges with relying on predetermined categories and frequency counts as the primary focus of analysis (Kohlbacher, 2005). This shift can be seen as an evolution of both theoretical perspectives and methodological procedures. Qualitative researchers have suggested that quantitative orientations to content analysis neglected the particular qualities of the phenomena under investigation and that it was important to also reconstruct the contexts of production and avenues of dissemination of the phenomena under investigation. Moving from quantitative methods, featuring the analysis of identifiable aspects of textual content, to qualitative approaches, most frequently aligned with an interpretivist paradigm, has been seen as a response to the narrow parameters associated with traditional, quantitative approaches to content analysis (Elo and Kyngas, 2008; Hseih and Shannon, 2005).

Qualitative content analysis is a multifaceted or flexible methodology for analyzing linguistic and visual content across many social science research frameworks (Van Leeuwen and Jewitt, 2001; White and Marsh, 2006). Variations of qualitative content analysis have been utilized by diverse fields of inquiry, including literary studies (Serafini, 2015), advertising (Vedula et al., 2017), nursing (Elo and Kyngas, 2008), literacy education (Beach et al., 2009), and psychotherapy (Thyme et al., 2013). A wide range of social science researchers have drawn upon and created variations of qualitative content analysis to fit the needs and requirements of their disciplines, research studies, and publishing outlets.

Moving beyond frequency counts and predetermined templates, qualitative content analysis has been designed as a ‘research method for the subjective interpretation of the content of text data through the systematic classification process of coding and identifying themes or patterns’ (Hseih and Shannon, 2005: 1278). Unlike other literature reviews that separated variants of content analysis into quantitative and qualitative approaches, Hsieh and Shannon categorized qualitative content analysis into three families of analytical approaches: (1) conventional, (2) directed, and (3) summative. These three categories represent a range of approaches to qualitative content analysis that have addressed different research purposes, designs, and analytical procedures. However, the approaches outlined in many of the literature reviews focused on textual or linguistically-based data and suggested the foundations of qualitative content analysis are the coding processes and schemes used to organize large quantities of textual data into fewer content categories (Weber, 1990).

Epistemologically, multiple approaches to qualitative content analysis can be applied whether knowledge is believed to be either innate, acquired, or socially constructed (Lincoln and Guba, 1985). Working across epistemological paradigms, reviews of qualitative content analysis have posited both inductive and deductive approaches to qualitative content analysis (Graneheim et al., 2017). Inductive approaches are characterized by a systematic search for patterns in the manifest content of a text, and the interpretations of the underlying meanings constructed between the lines of a text (Graneheim and Lundman, 2004). In contrast, deductive approaches involve the testing of pre-existing theories or models against a selected data corpus. Inductive approaches move from particulars to generalities, while deductive approaches move from generalities to specifics. In actuality, a blend of both deductive and inductive reasoning has been used across many variations of qualitative content analysis.

Most applications of content analysis have been used as a form of analysis for investigating textual or linguistically-based data in qualitative social science research studies (Kohlbacher, 2005). Abrahamson (1983) has asserted that qualitative content analysis can be fruitfully used to examine virtually any kind of communication materials, including narrative responses, open-end survey questions, interviews, and printed media such as articles, books, or manuals. Although the data cited referenced a wide range of text types, the focus of these studies has remained the linguistic elements manifest in the content of written texts, whereas the multimodal nature of contemporary forms of representation and communication have not been explicitly addressed across the theoretical frameworks or research designs mentioned in these reviews.

A Social Semiotic and Multimodal Theory of Representation and Communication

Jewitt (2009) has asserted that multimodality can be conceptualized as an approach to understanding the semiotic resources and contexts associated with various forms of representation and communication. Multimodal approaches to understanding representation and communication draw upon social semiotics as a foundational theory that provides hypotheses, descriptions, frames, and principles upon which these approaches are supported and framed (Adami and Kress, 2014). Social semiotic theories move beyond the logocentric focus of traditional theories of communication to acknowledge the repertoire of modes that comprise contemporary forms of representation and communication (Van Leeuwen, 2005).

Rejecting the notion that purely monomodal communicational systems exist, theorists addressing issues across social semiotics and approaches to multimodality have recognized the thoroughly multimodal nature of representational and communicative phenomena (Mitchell, 2005). Contemporary researchers working within these frameworks view multimodal texts as print-based and digital ensembles that utilize more than one mode or semiotic resource to represent meaning potentials, where mode is defined as a socioculturally shaped resource for meaning making (Kress, 2010). Multimodal ensembles appear in a variety of contexts, utilizing different cultural and semiotic resources to articulate, render, represent, and communicate an array of concepts and information (Serafini, 2014).

Representation through any single mode, for example written language or visual images, is always plural and partial, with any particular mode offering meaning potential, given the affordances and limitations of its material and social dimensions. This plurality and partiality of representation of meanings challenges scholars and researchers to theorize the interpretive space and processes necessary to make sense of these ensembles in particular social contexts. It is of vital importance that researchers address the multimodal aspects of any phenomena under investigation, focusing on the social construction of meaning, the theories used to support one’s research framework, and the analytical procedures used in data collection and analysis.

Approaches for analyzing singular or relatively small numbers of multimodal ensembles have been offered across analytical frameworks. Detailed procedures for multimodal transcription (Baldry and Thibault, 2006), semiotic, anthropological, cultural studies, and ethnomethodological approaches to visual analysis (Van Leeuwen and Jewitt, 2001), and multimodal discourse analysis (O’Halloran, 2004) offer extensive analytical approaches for investigating individual ensembles and texts in both analogue and digital formats. The intention of this article is to provide a detailed analytical procedure that blends the procedures used for analyzing multiple texts, without resorting to simple frequency counts, and adapting the approaches for detailed analyses already available.

Acknowledging the ‘Multimodal’ in Qualitative Content Analysis

Researchers and theorists have drawn upon approaches for analyzing linguistic, semiotic, and visual data to investigate multimodal phenomena across a wide range of disciplines. For example, researchers have adapted linguistically-focused discourse analysis theories and methods ( Fairclough, 1995; Gee, 1999) into forms of multimodal analysis (Machin, 2007), critical discourse analysis (Machin and Mayr, 2012), and visual discourse analysis (Albers, 2007). In addition, researchers have adapted systemic functional linguistic approaches for analyzing multimodal texts, in particular advertisements (Aiello, 2006), picturebooks (Painter et al., 2013), and mathematical formulae (O’Halloran, 2004).

Jewitt (2009) asserted that the theoretical perspectives and systematic approaches for understanding multimodal phenomena can be organized under the following three categories: (1) social semiotic approaches, usually based on Halliday’s (1978) concepts associated with systemic functional linguistics; (2) multimodal discourse analysis, usually associated with the work of O’Halloran (2004), based on the metafunctional systems and semiotic theories of O’Toole (1994); and (3) multimodal interactional analysis, derived from the sociolinguistic interactional theories of Scollon and Scollon (2003), and expanded and revised by Norris (2004). Various theoretical perspectives and analytical frameworks that address the multimodal aspects of contemporary forms of representation and communication may be drawn upon to expand the quality and potential of qualitative content analysis (Bateman, 2008).

In addition to expanding the theories underlying the research design and analytical procedures associated with qualitative content analysis, it is important for researchers to also expand their conceptualizations of the nature of the data being investigated. For example, by considering the texts under investigation as multimodal entities that may include visual, textual, haptic, aural, and digitally-enhanced modes of representation and communication, researchers are able to move beyond a focus on written language as the primary, if not sole, mode addressed during qualitative approaches to content analyses. In conjunction, the traditional act of transcribing the multimodal aspects of face-to-face interviews or classroom interactions into written scripts neglects other procedures and analytical approaches for analyzing data represented across other modalities, for example data constructed as visual images, video segments, or web-based content (Adami, 2014).

Although some researchers are developing innovative approaches for multimodal transcriptions (Cowan, 2014; Flewitt, 2011), numerous studies in education, nursing, and other social science-based research areas rely primarily on text-based transcripts as ways of capturing and representing interactions, interviews, or observations. Focusing on the multimodal aspects of the data being analyzed, including intermodal relationships, is an important distinction between traditional quantitative content analysis and early variants of qualitative content analysis methods (Schreier, 2012). This distinction sets our approaches apart from earlier work in content analysis.

Although specific researchers have used the term ‘multimodal content analysis’, we assert that, in the studies cited that have adopted this term, researchers under-theorized the properties of the data being studied and simply applied quantitative, a priori (deductive) coding procedures across modalities without fully theorizing the multimodal aspects of modern forms of representation and communication (Vedula et al., 2017). The frameworks, procedures, and analytical techniques proposed here are designed to demonstrate the need to more fully theorize the multimodal aspects of the data being analyzed as researchers move from qualitative analyses of linguistically-based content, to qualitative analyses of multimodal content.

The aim of our adaptation of qualitative content analysis, referred to here as ‘multimodal content analysis’ (MMCA) is to expand the focus of qualitative content analysis frameworks to consider the ways multiple modes – for example, illustrations, photography, written language, and design elements – add to or expand the meaning potential of texts beyond the meaning potentials of individual modes. In other words, we needed to consider the complex relationships among modes present in analogue and digital multimodal ensembles, in addition to the affordances and limitations of individual modes.

Although some scholars postulate inductive and deductive approaches as mutually exclusive methodologies, analytical approaches that blend previously determined or identified categories with themes and assertions constructed through interpretive or grounded theory designs do not have to be dichotomized in this way. Whereas quantitative content analysis methods have long been associated with deductive approaches, and qualitative content analysis primarily associated with inductive approaches, this distinction may be misleading, or worse, counterproductive to the quality and application of qualitative methods in content analysis (Armat et al., 2018).

Drawing upon existing theories and previously conducted research studies and predetermined categories for initial data analysis does not reduce the need, nor make unproductive, the application of inductive reasoning or approaches to data analysis. In fact, we, like Armat et al. (2018), argue that both types of reasoning are usually drawn upon in most forms of qualitative content analysis and strengthen, rather than diminish, the rigor and quality of these approaches to content analysis. Although qualitative research is less likely to impose restrictive a priori classifications on the collection of data, and is thus less driven by specific hypotheses and categorical frameworks (Cassell and Symon, 1994), drawing upon existing theoretical perspectives and research findings does not have to be counterintuitive to the approaches associated with qualitative content analysis.

Rose (2016) asserted that researchers utilizing visual methodologies recognize different sites of analysis, including the sites of production, dissemination, the phenomenon itself, and reception. Although the procedures of MMCA explicated here are primarily concerned with the site of the multimodal phenomena itself, additional analyses of multimodal phenomena have provided the foundation for expanding the research to understand the production and dissemination of multimodal texts, as well as how audiences respond to these phenomena (Jewitt and Kress, 2003; Sunderland and McGlashan, 2013). Once a more extensive understanding of the multimodal phenomena under consideration is accomplished, researchers are better positioned to understand how particular multimodal ensembles work at the site of audience or reception.

Why Study Wine Labels?

Peirce (1960) asserted that inquiry begins with a nagging sensation of doubt, and that doubt is a sense of disequilibrium that humans attempt to eliminate through systematic inquiry. In a sense, research and inquiry begins with doubt or uncertainty that leads to developing and asking questions. In addition, Dewey (1910) suggested humans approach problems or doubt with acquired habitual states of mind and an accumulation of previously stored meanings and experiences. Both of these aspects of inquiry affected the researchers’ interests in studying commercial wine labels; a sense of doubt about how these labels represent and communicate meanings through various modes, and the influences that past experiences and stored meanings associated with multimodal texts have on our understandings, in particular advertisements.

The design and production of wine labels has become a burgeoning industry with increased interest and expansion in the consumption of wine in the United States and around the world (Bosker, 2017). This topic has recently been explored from a variety of perspectives, including consumer research (Celhay and Remaud, 2018), graphic design (Celhay et al., 2017), and semiotic analysis (Konig and Lick, 2014). Our research project was designed to expand the analytical procedures for understanding the multimodal aspects of wine labels, and provide researchers with an all too infrequent look into the reflective processes needed to adapt the research design and procedures utilized in an actual study.

Although a plethora of mobile apps that allow wine drinkers to access wine reviews and information about wine production and vintages are currently available, wine labels are often the only information provided about a particular wine or winery that customers have access to when making purchases. Wine labels have aesthetic qualities that include visual images and textual information, as well as commercial and advertising functions (Reynolds, 2011).

In addition to the researchers’ interest in wine label design, one of the authors had recently studied for and was awarded a Certified Specialist of Wine degree by the Society of Wine Educators in 2017 and passed the classes and exams to be recognized as a Certified Sommelier by the Court of Master Sommeliers. As part of the preparation for these certifications, one of the authors was attracted to the various visual and design elements featured on the wine labels encountered as part of the training and examinations. In addition, as part of a doctoral seminar on multimodal theories and research designs, the author conducted investigations with his students into self-selected wine labels from the Wine Spectator List of Top 100 Wines of 2016. The discussions and analyses that were part of this course became the impetus for this study and article.

From a multimodal perspective, qualitative forms of research have focused on advertisements (Carvalho, 2013), websites (Adami, 2014), children’s picturebooks (Guijarro, 2014; Painter et al., 2013), textbook covers (Serafini et al., 2015), and other multimodal ensembles. The researchers became interested in the variety of semiotic resources used on contemporary wine labels to represent and communicate particular messages or information, primarily intended for consumers. In addition, the various modes of representations included on the wine label were used to convey some aspects of a particular winery, bottler, or distributor of the products. How various meaning potentials and information are represented and communicated across modalities is of interest to us as scholars and researchers investigating multimodal phenomena. We felt the collection of wine labels represented an enjoyable way to blend our interests in wine and our research on multimodal phenomena.

Multimodal Content Analysis Procedures

Generally speaking, the procedures of analysis that make up the analytical framework for conducting MMCA are organized and described as follows: (1) Recognizing an Area of Interest; (2) Developing Initial Research Questions; (3) Constructing the Data Corpus; (4) Defining the Object of Study; (5) Developing Initial Categories; (6) Developing the Analytical Template; (7) Testing the Analytical Template; (8) Applying the Analytical Template to the Data Corpus; (9) Constructing Potential Themes; (10) implications of the analysis; and (11) Dissemination of Findings. Although these procedures are presented in linear fashion, they actually overlap at times, and should be viewed as recursive processes that inform procedures before and after the order presented.

The final three steps in the procedures listed here feature the process of constructing themes, considering implications of the analyses, and the presentation of the findings. It is in these final stages that the data is organized in ways that allow the researcher to draw conclusions and make implications visible to wider audiences. In the following section, we will use an ongoing study, focusing on the multimodal aspects of commercial wine labels, to exemplify the procedures outlined above in order to better explain each element in our analytical framework.

Developing initial research questions

We decided to focus on the modes of representation included on the labels themselves and not on the production of the labels or the process of making specific choices by label designers, nor the site of reception, by interviewing consumers about their wine label preferences. We initiated our inquiry by asking, ‘How do commercial wine labels represent and communicate ideas about the wine, the vineyard, the winery, or the vintners or wine owners associated with the label through a combination of visual images, textual and typographic elements, and particular design features?’

Approaches to qualitative research allow for the continuing revision and expansion of research questions (Merriam, 1998). Throughout the study, the focus of our inquiry remained the labels located on the front of selected wine bottles, deciding that the additional information provided on the backs of bottles would expand our study into different areas than intended. Additional sub-questions were subsequently developed that focused on particular modalities, for example the visual elements, fonts used, and size and position of the labels.

Constructing the data corpus

Once the initial research questions were determined, the researchers had to decide on which set of wine labels would be analyzed. In other words, they had to define the corpus of multimodal phenomenon to include in the study. It is as important to thoroughly define what to include as it is to construct reasons and justifications for what not to include. For some studies, we have used chronological parameters to determine the data corpus, for example, all the professional literacy textbooks by seven educational publishers published in a particular year. In other studies, we searched databases, in one case Amazon book catalogues and library inventory software (e.g. Follett) to locate as many examples of a particular type of picturebook as possible.

For this study, we decided to use a predetermined set of wine labels that was put together for commercial and institutional purposes, specifically advertising and sales promotion. The data corpus was determined by an ‘official’ designation, namely, the Top 100 Wines of 2016 published in Wine Spectator magazine. Once this decision had been made, the researchers used online resources to locate representative images of each label from each wine on the list, and downloaded and organized these images as JPEG files including all the labels of the Top 100 wines of 2016. Although this corpus was not created by a random sampling of wine labels, for example from a particular wine store or by gathering wine magazines and studying the advertisements and articles, we felt that the Wine Spectator Top 100 list was an adequate data set to generate assertions about the modes of communication used on the commercial wine labels.

Defining the object of analysis

Defining the object of analysis is different from selecting the images or multimodal texts that comprise the data corpus. In actuality, these two procedures overlap and affect one another. For instance, as the ways we define an object of analysis evolves, the images or texts we included in a particular study will change as well. In order to analyze the labels we have selected as part of the data corpus, we must define what we mean by a ‘wine label’, what to include as part of the analysis, and what aspects may not be analyzed during a particular study.

In past studies of textbook covers and metafictive devices in contemporary picturebooks, the researchers had to explicitly define and describe what the object of analysis would be. In one study, the front covers of the selected textbooks were analyzed, paying attention to the title and any subtitles, and the visual images or design elements, but not the spine or back covers. The objective of this part of the MMCA procedures is to ‘bound the case’ (Stake, 1995) to be studied. In other words, we needed to explicitly define the focus of our analysis to distinguish which elements would be part of the data corpus and which would not.

Both inductive and deductive forms of reasoning were utilized in defining the object of analysis for the commercial wine label study. We drew upon previous studies of commercial products and advertising to see how other researchers defined their objects of analysis (Messaris, 1997; Scott, 1994). In each of the studies in our review, the researchers provided explicit definitions of how that data corpus was created and what was actually analyzed in the study. In addition to our review of these studies, we also organized the 100 labels in different ways and created an inventory of the types of things that were included on the bottles.

After an initial viewing and reading of the full set of wine labels, we expanded and clarified our definition of the object of analysis even further. Some bottles had embellishments beyond the central wine label, such as wax seals and promotional ribbons. Other bottles had two labels: one label on the front and a second label on the back. The label on the back tended to elaborate on the production of the wine, the history of the wine and winery, or pairings of particular types of food with which it might be served. We restricted our study to the front label of the wine bottle (for example, see Figures 1a and 1b). In general, the front label included a visual image or textual component, the name of the wine, or the producer or winery. There were no wine bottles that did not have some form of front label. Any embellishment or visual and textual component directly incorporated onto the front label was considered part of the object of analysis. However, objects contained along the neck of the bottle or back label were excluded.

Figure 1(a).

Bottle label from Beaux Frères Vineyard (2014). Reproduced with permission.

Figure 1(b).

‘Carnival of Love’ bottle label from Mollydookers Wine Company (2019). Reproduced with permission.

These decisions enabled us to finalize our data corpus and the objects of analysis. We bound our corpus to the front labels of the 100 wine bottles featured on the Wine Spectator List of Top 100 Wines of 2016. We used various photography software applications (Lightroom, Photoshop) to download and organize the images by their Wine Spectator ranking of numbers 1 through 100 as a means of keeping the wine labels in order for sharing our notes and observations.

Developing initial categories

Before we conducted an initial coding of the full data set, we reviewed all 100 wine labels to generate theoretical or analytical memos (Erickson, 1986) and observational field notes to develop potential categories for coding. We began by constructing two overarching categories to account for the full corpus of wine labels: labels with visual images and labels with text only. This would seem a rather straightforward distinction; however, some labels included visual design elements that were a blend of textual and spatial forms.

We then decided to expand the initial two categories to three general categories to account for some of this overlap and identified variations: (1) visual images, (2) design elements, and (3) textual elements. Within these three general categories, we started to construct more specific subcategories to account for variations within each category. For example, within the category of textual elements, we noted differences in font selections, font sizes, colors, textures, and placement of textual elements. Within the category of visual images, we noted that some images featured human characters, others depicted objects, places, or locations, while others simply displayed the name of the wine or winery. We decided to use Kress and Van Leeuwen’s (1996) distinction between narrative images, images that contain people or objects in actions and particular settings, and classification images, images without narrative vectors, as a way to organize and distinguish the visual image components of the wine labels. Elements that did not fit into either image or textual categories were reviewed as design elements – for example borders, lines, frames, and other adornments.

Developing the analytical template

One of the most successful analytical techniques or procedures that has emerged from the development of our MMCA framework has been the creation of analytical templates to focus observations and support our analysis of the data corpus. The primary goal of developing and applying the analytical templates was to focus our attention on particular visual, textual, design, and in particular the intermodal associations featured on the wine labels. The construction of these templates has been used to make our observations and analysis more systematic, to guide our perceptions, and to generate additional sub-categories for further analysis. The templates we have used in past studies have all gone through numerous iterations as our observations and analysis were hampered, as well as supported, by the various categories we proposed, generated, and included across these templates. It is important to understand that any analytical template or procedure will bring into focus particular aspects of multimodal phenomena, while diminishing or making invisible other aspects of the same phenomena.

When organizing the categories for our initial analytical template, we looked to several theoretical frameworks that have been used in prior studies. For example, frameworks have been associated with multimodal analysis (Machin, 2007), multimodal analysis based on Halliday’s systemic functional linguistics (Kress and Van Leeuwen, 1996), and a tripartite analytical framework associated with social semiotics (Serafini, 2010). As a framework for our initial template, we utilized the three metafunctions proposed in Halliday’s (1978) work on systemic functional linguistics and further developed for visual images (Kress and Van Leeuwen, 1996), namely: (1) ideational, (2) interpersonal, and (3) textual metafunctions. Using Halliday’s metafunctions as overarching categories for our analysis, we developed an initial analytical template and applied it to several wine labels.

After using this template to guide our trial analysis of several wine labels, we found this template to be cumbersome and that it included aspects of multimodal phenomena that were irrelevant for our study. Although the initial template framework covered many aspects of the labels, it did not allow us to include many of the intermodal and intramodal relations (Painter et al., 2013; Unsworth, 2006), or connections within and across various modalities that we wanted to be sure to include. We needed a template that focused our observations without being overly detailed but that included enough details to make the analysis thorough and rigorous, and focused our analysis on the intramodal aspects of the wine labels.

A second analytical template (see Table 1) was developed that focused on the textual elements, visual images, and design features as over-arching sections of the template. In addition, a fourth category, labeled intermodal relations, was added to help analyze the ways in which various modalities interacted with one another. The impetus for revising the template was due to the fact we would need to apply it to our analysis of 100 labels and found the original template unsuitable for that task. The second template was tested with a couple of labels before we applied it to a larger set of labels to test the efficacy of the template.

Table 1.

Blank template for multimodal content analysis.

Analytical category	Features	Analytical notes
Textual elements: Content	Wine name
	Winery
	Location/AVA (American Viticultural Area)
	Varietal/blend
	Miscellaneous
Textual elements: Expression	Font
	Palette
	Position/size
	Salience
	Style – era
	Miscellaneous
Visual images: Narrative	Setting
	Actors
	Actions – poses
	Vectors
	Theme
	Miscellaneous
Visual images: Classificatory	Logo–seal
	Classification scheme
	Information provided
	Hierarchy
	Miscellaneous
Design elements	Compositional elements
	Position/size
	Materiality
	Palette
	Salience
	Miscellaneous
Intermodal relations	Composition
	Complementarity
	Dissonance
	Framing
	Genre

Testing the analytical template

To increase the validity and reliability of the second template, the researchers selected four wine labels and used the revised analytical template to analyze each image individually. After the researchers completed their individual analyses of the wine labels, they compared their results. These comparisons led to further revisions of the template and enabled us to generate a final template that addressed the important multimodal features of the data under investigation. In addition, these comparisons helped establish reliability and trustworthiness in our analytic procedures in much the same fashion as inter-rater reliability procedures are used across multiple researchers and larger data sets. The final template featured in Table 1 draws upon the three general categories of textual, visual, and design elements but adds several features that were missing from the original template and incorporates a miscellaneous category to allow researchers to include elements that didn’t neatly fit into one of the predetermined categories. The final template was used as an observation guide to provide researchers with a framework for systematically approaching each data point in a similar manner.

Applying the analytical template to the data corpus

At this point, the final analytical template was used to begin our analysis of the 100 labels included in the Wine Spectator List of Top 100 Wines of 2016. All the wine labels in the data corpus were to be examined through the lens of the analytical template, and researchers completed a template for every label. During the analysis, the researchers recorded theoretical memos as they applied the template to each label to keep track of potential patterns or categories as they progressed through the data. In addition to the theoretical memos focusing on the wine labels themselves, the researchers also kept methodological memos to keep track of the effectiveness of the analytical template and to recognize any aspects of the labels that were not fully addressed in the template. These memos could be used for further revisions to the template for future studies, or to simply include any extraneous details that were constructed during the analysis.

At the time of the writing of this article, the researchers have analyzed the 100 templates and are working across the data points to construct potential categories that will lead conclusions about the data corpus. To illuminate what this analysis looked like, we include in Table 2 a completed analysis of the wine label Carnival of Love produced by the Molly Dooker winery (see Figure 1b). Utilizing our final template, we focused our attention on both the individual modes included in the label as well as the intramodal aspects of the label. We highlighted the ways in which the visual images, textual elements, and design features complemented and contradicted one another, drawing upon the work of Painter et al. (2013). In their work, they described intramodal relationships as being complementary or dissonant. Although designers may work to create harmonious labels and graphic elements to attract consumers’ attention, our analysis revealed both complementary as well as dissonant elements and relationships across many of the labels. It was of vital importance that our final analytical template should address both the intramodal elements as well as the intermodal relationships of the semiotic resources being used to represent and communicate concepts and information on various wine labels.

Table 2.

Completed template for multimodal content analysis of Carnival of Love wine label.

Category	Features	Analytical notes
Textual elements: Content	Wine name	Carnival of Love
	Winery	Mollydooker
	Location/AVA (American Viticultural Area)	McLaren Vale, Australia
	Varietal/blend	Shiraz
	Miscellaneous	Sarah & Sparky Marquis (vineyard owners)
Textual elements: Expression	Font	Winery & Seller details: Narrow, capitals, sans serif – straightforward, fancy-free. Carnival of Love (label within the label): Circus font – makes the label within label resemble a Carnival/Circus poster. The words ‘Carnival of Love’ are spelled out on the blue/red banner behind the jester. Script. The angle of the letters is reminiscent of medieval or Renaissance lettering. Mollydooker logo: font contains flourishes as if individually hand-drawn. Suggests quirky individuality.
	Palette	Red dominates the label within label/Carnival poster. All details are in same shade of red. Red carries the symbolism of love and desire. Logo and banner: Lettering here is yellow – picks up the background color of the label/Carnival poster. The yellow tends towards earthy, orange color. Warm color – matches the almost autumnal tones in the image. The yellow looks like faded paper – harkens to an older time.
	Position/size	Carnival (banner): Top left.Love (banner): Top rightMollydooker (label): Just above center – left justifiedCarnival (label): Center – left justifiedLove (label): Just below center – left justifiedWinery owners: Bottom left cornerLogo: Bottom right
	Salience	Love (label): large, stretched font. Carnival (label): large, stretched font.Mollydooker: Logo.
	Style – era	The Circus lettering (and their backing onto what appears to be a faded poster) speaks to the Renaissance era or to the modern renditions of the Renaissance Fayre.
	Miscellaneous	The style of the lettering is congruent with the human figures that comprise the images that comprise the rest of the lettering. The jester, the ruffled collars, the young lady in the Renaissance-style dress. It is also important to note that the lettering forms part of the scene – on a poster or a banner. A jester sits atop the banner, his dangling legs hanging over the Carnival posters. With the exception of the Mollydooker logo, which overlays the square label, the text is incorporated into the Carnival scene.
Visual images: Narrative	Setting	A Carnival – there are jesters, a strongman, a female violinist, and masked dancers. The strongman holds the banner stating Carnival of Love, and the backdrop looks to be the draped black fabric of a tent. Note: place is signaled by people and their costume and place are anchored by the wording – both what they say (Carnival of Love) and how they are presented (Circus font). Image and words together communicate the Carnival.
	Actors	From top left clockwise to bottom right. Their clothing and props determine whom we perceive the actors to be. Jester – costume signals jester. Strongman – wrestling singlet, muscled shoulders. Jester – ruffled collar, cap, harlequin pattern. Violinist – female, crown of white flowers, white dress. Male masked dancer – ruffled collar, shirt, waistcoat, cap – from another era. Female masked dancer – red-haired, low-cut Renaissance maiden’s dress, white mask.
	Actions – poses	Jester – seated atop the label/poster. Holding a heart on a stick in one hand, an owl with an eye patch in the other. The fingers on his left hand are curled towards him, as if he is beckoning the viewer closer. Strongman – faces left, holding the banner, carrying a pair of boxing gloves. Jester – facing left, juggling three red/white balls. Violinist – female, crown of white flowers, white dress. Her dress seems in motion and she appears to lean back. Male masked dancer – holding out a white handkerchief to the female masked dancer. Female masked dancer – holds a rose in her right hand.
	Vectors	The jester atop the poster looks directly out at the viewer – visual demand. He invites the viewer to participate in the party scene. The violinist looks to her left, which draws our eyes to the male masked dancer. His eyes are focused right – on the face of the female dancer in front of him. Her eyes look off towards the right. Her gaze takes us beyond the edges of the label to where, we presume, the Carnival parade or party continues. There is a clear connection between these three and an especially intense connection between the male and female dancer.
	Theme	Carnivalesque romp
	Miscellaneous	Although the image is by nature a still with the actors frozen in time, the vectors and the range of poses give the impression of movement. The violinist shows there is music and sound. The vectors suggest the parade passes right in front of us.
Visual images: Classificatory	Logo – seal	Interrupts the wine label image – placed over the bottom right-hand corner. The signature/logo label has an irregular shape with curves and edges.
	Classification scheme	Not applicable
	Information provided	Not applicable
	Hierarchy	Not applicable
	Miscellaneous	Not applicable
Design elements	Compositional elements	The linguistic wine details are contained in a rectangle frame on the left/bottom side. This frame is 1/2 the width and 2/3 the height of the entire label. The image takes up the rest of the label.
	Position/size	Ideal: The jester with the eye-patched owl and the heart-staff sits at the top near the center. Real: At the bottom are the dancer, musician, pickpocket(?). Left: Known – the name of the wine and the verbal information. Right: Possible – the party, the beauty, the Carnival, the music, the love.
	Materiality	Paper label. The edges of the label look textured – as if the paper has been torn by hand, not sliced by machine. In contrast, the Mollydooker signature/logo label is smooth-edged, seems separate from the torn label.
	Palette	Red is the predominant – which makes sense: Carnival of Love is the theme. Red = desire, love, appeal, passion. Yellow background to red text also draws attention.
	Salience	Size: the two largest characters are the jester and dancing girl. Color: Red occurs for each character – the color of love linking them all. Foregrounding: the jester and dancing girl are foregrounded through their overlaid position over the frame. The image is busy and eye-catching – the ‘bustle’ or Carnival is memorable if not individual characters within the image. BUT so is the phrase ‘Carnival of LOVE’ – in uppercase, red. I remembered this wine’s name above all else.
Intermodal relations	Composition	The linguistic wine details are contained in a rectangle frame on the left/bottom side. This frame is 1/2 the width and 2/3 the height of the entire label. Although there are a lot of words on this label, and although the words initially seem cordoned off by their containment within the poster, the fact that the jester sits atop the poster dangling his feet over its edge subsumes the verbal aspect into the image.
		The image takes up the rest of the label. The font chosen makes the verbal text akin to a Carnival or Circus poster. The red of the font is the same red used throughout the image to color characters’ hair, clothing or props. The heart-stamped juggling balls and the rose held by the female dancer are examples of red items. The color connects the words and the images, too – as well as the red wine product the label is selling.
	Complementarity	Words: ‘Love would be that place where you desire no more/Everyone wants to come, and no one ever wants to leave/Every heart is full and the joy of love permeates all things/This is as life should be: A Carnival of Love.’ The permeation talked about in the words is shown in the image as the red of love permeates the figures in the pictures and even the written text. Other symbols of love – hearts and roses also appear. The notion of desire is shown in the intense connection between the male and female masked dancers. The female dancer’s gaze takes us beyond the label, as if she is looking at the rest of the parade or dance that continues beyond the label. No one is exiting the dance – there is only invitation to join (courtesy of the seated jester).
	Dissonance	There is one element of the language that is not shown in the image. The words speak of the ‘joy of love’ but joy is not visible in the actors on the label. There is intensity and desire and the sense that the actors are caught up in a Carnivalesque dance or parade – but there is no visible facial expression of joy on any of the characters’ faces. The seated jester smiles at us, but there is almost a menace to his smile. The second jester and the female dancer are half smiling – but neither are beaming as if joyful.
		Joy is not expressed in the color palette – the red is not a bright red, the yellow is not a bright red.
	Framing	Several frames: the curved edge of the banner, the straight edges of the verbal frame, a patterned frame along the bottom. No frame: The image on the right side goes to the edge – there is no line marking the realm of the frame. This is true for the top-left of the image, too. Party goes on. The dancing girl’s arm interrupts the frame of the verbal text, as do the feet of the jester sitting atop the verbal frame. A sense of rule breaking? The Carnival running amok?
	Genre	This is neither abstract or photography. The image suggests a fictional world – but we can recognize the characters and scene. It’s designed to be fantasy.

Constructing potential themes

The data analysis procedures used in this study were consistent with an interpretivist approach to qualitative research. According to Erickson (1986: 146), ‘the basic task of data analysis is to generate assertions that vary in scope and level of inference, largely through induction, and to establish an evidentiary warrant for the assertions one wishes to make.’ Theoretical memos were generated during the application of the final template to the individual wine labels to organize potential categories, assertions, and themes during our analysis.

The construction and revision of categories and themes across multimodal texts is an important focus of data analysis frameworks and procedures. Constructing categories is done before themes are constructed from the data set. It is important to distinguish the differences between categories and themes in order to make the analytical process more visible and transparent. Erickson (1986) asserted the importance of linking data points together and looking for key linkages during data analysis. Our analysis of the data gathered using the analytical templates will proceed in much the same manner as grounded theory, consisting of phases of coding and analysis (Strauss and Corbin, 1990).

The difference between a category and a theme is:

A category is a collection of similar data sorted into the same place, and this arrangement enables the researchers to identify and describe the characteristics of the category, whereas a theme is a constructed interpretation of the characteristics of a category, or a meaningful essence that is believed to run through the data. (Morse, 2008: 727)

Themes or essences do not just emerge; rather, they are constructed by the researchers during data analysis by close readings and interpretation of the data corpus. It is important to continually recognize the role of the researchers in the interpretive and analytical processes, and to once again assert that themes are constructed and do not simply emerge without researcher intervention.

Although this study is being used to illustrate the procedures of MMCA, and we are challenged to speak authoritatively about our findings, we feel it is important to share some preliminary ideas and assertions in order to demonstrate the final steps of our analytical procedures. However, these assertions are tentative and will surely be expanded and revised before being submitted as part of a completed study. During our preliminary analysis of the wine labels, we constructed several themes that will certainly be revised and expanded to illuminate several assertions from the data analysis.

First, many of the wine labels focus on particular geographical features, either through visual images or as part of the textual information provided about the winery location or production zone. For example, valleys, creeks, hills, ridges, rivers, and mountains were used as geographical indicators to draw a relationship between the place where the grapes were grown or the wine was produced and the suggested quality of the wine in the bottle. The concept of terroir, or the connection between the environment, climate and types of earth featured in high-quality vineyards, has become an important marketing device, and many sommeliers use this concept to discuss and sell wine. By featuring these geographical characteristics, wine labels help customers to connect the wine in the bottle to specific, highly sought after vineyard locations.

Second, in addition to the geographical indicators, words like estate, chateau, reserve, limited selection, classical, and premier allude to the elite quality of the wine or the grapes from which the wine is made. Many of these terms have legal designations depending on the political boundaries, or viticultural institutions regulating wine production; however, many of these terms are simply marketing devices and can be used on labels without any reference to actual wine regulations. These two preliminary findings serve as examples from our ongoing analysis and will doubtless be refined further before we submit the study for publication.

Concluding Remarks

The shift from the traditional analogue modes of printed text and page-based visual images to contemporary digital resources and multimodal platforms has forced both quantitative and qualitative researchers to rethink how they analyze various modes of representation and communication in modern times. As researchers begin to consider the multimodal nature of the modes of representation and communication, content analysis procedures need to be re-envisioned to include a variety of constructs for reconceptualizing the textual, visual, and multimodal nature of the data under investigation.

Research in multimodality has been referred to as the ‘late discovery of the obvious’ (Stockl, 2007), suggesting the ubiquity of these ensembles in contemporary visual culture. The analysis of selected commercial wine labels presented here is intended to offer the reader an example of content analysis to guide future research and open up a dialogue focusing on the potential advantages and challenges to researching multimodal phenomena. Drawing assertions from sets of multimodal data requires both inductive and deductive logic that allows researchers a flexible, yet systematic, set of analytical procedures. It is our hope that, by delineating the procedures used throughout our analyses, and by providing an example of an ongoing study, our framework for multimodal content analysis will inspire quantitative and qualitative researchers to rethink their analytical approaches to address the complexities of multimodal phenomena.

Footnotes

Funding

The authors received no financial support for the research, authorship and publication of this article and there is no conflict of interest.

ORCID iD

Frank Serafini

Biographical Notes

FRANK SERAFINI is a Professor of Literacy Education and Children’s Literature in the Mary Lou Fulton Teachers College at Arizona State University.

Address: Mary Lou Fulton Teachers College, Arizona State University, Tempe AZ, USA. [email: serafini@asu.edu]

STEPHANIE F REID is a doctoral student in the Mary Lou Fulton Teachers College at Arizona State University.

Address: as Frank Serafini. [email: sfreid@asu.edu]

References

Abrahamson

(1983) Social Research Methods. Englewood Cliffs, NJ: Prentice Hall.

Adami

(2014) What’s in a click? A social semiotic framework for the multimodal analysis of website interactivity. Visual Communication 14: 133–153.

Adami

Kress

(2014) Introduction: Multimodality, meaning making, and the issue of ‘text’. Text & Talk 34: 231–237.

Aiello

(2006) Theoretical advances in critical visual analysis: Perception, ideology, mythologies, and social semiotics. Journal of Visual Literacy 26: 89–102.

Albers

(2007) Visual discourse analysis: An introduction to the analysis of school-generated visual texts. In: Rowe

, et al. (eds) 56th Yearbook of the National Reading Conference. Oak Creek, WI: NRC.

Armat

, et al. (2018) Inductive and deductive: Ambiguous labels in qualitative content analysis. The Qualitative Report 23: 219–221.

Baldry

Thibault

(2006) Multimodal Transcription and Analysis: A Multimedia Toolkit and Coursebook. London: Equinox.

Bateman

(2008) Multimodality and Genre: A Foundation for the Systematic Analysis of Multimodal Documents. New York: Palgrave Macmillan.

Beach

, et al. (2009) Exploring the ‘critical’ in critical content analysis of children’s literature. In: Leander

, et al. (eds) 58th Yearbook of the National Reading Conference. Oak Creek, WI: NRC.

10.

Bosker

(2017) Cork Dork. New York: Penguin.

11.

Carvalho

(2013) Social semiotics and literacy: A case study about the social meanings constructed by ads of a children’s magazine. Australian Journal of Language and Literacy 36: 169–180.

12.

Cassell

Symon

(1994) Qualitative Methods in Organizational Research: A Practical Guide. London: Sage.

13.

Celhay

Remaud

(2018) What does your wine label mean to consumers? A semiotic investigation of Bordeaux wine visual codes. Food Quality and Preference 65: 129–145.

14.

Celhay

, et al. (2017) Package graphic design and innovation: A comparative study of Bordeaux and Barossa wine visual codes. Research and Applications in Marketing 32: 46–70.

15.

Cowan

(2014) Multimodal transcription of video: Examining interaction is early years classrooms. Classroom Discourse 5: 6–21.

16.

Dewey

(1910) How We Think. Boston, MA: DC Heath and Co.

17.

Elo

Kyngas

(2008) The qualitative content analysis process. Journal of Advanced Nursing 62: 107–115.

18.

Elo

, et al. (2014) Qualitative content analysis: A focus on trustworthiness. SAGE Open 4.

19.

Erickson

(1986) Qualitative methods in research on teaching. In: Wittrock

(ed.) Handbook of Research on Teaching, 3rd edn. New York: Macmillan, 119–161.

20.

Fairclough

(1995) Critical Discourse Analysis: The Critical Study of Language. London: Longman.

21.

Flewitt

(2011) Bringing ethnography to a multimodal investigation of early literacy in a digital age. Qualitative Research 11: 293–310.

22.

Gee

(1999) An Introduction to Discourse Analysis: Theory and Method. London: Routledge.

23.

Graneheim

Lundman

(2004) Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Education Today 24.

24.

Graneheim

Lindgren

B-M

Lundman

(2017) Methdological challenges in qualitative content analysis: A discussion paper. Nurse Education Today 56: 29–34.

25.

Guijarro

AJM

(2014) A Multimodal Analysis of Picture Books for Children: A Systemic Functional approach. Sheffield: Equinox.

26.

Halliday

MAK

(1978) Language as Social Semiotic: The Social Interpretation of Language and Meaning. London: Edward Arnold.

27.

Hseih

H-F

Shannon

(2005) Three approaches to qualitative content analysis. Qualitative Health Research 15: 1277–1288.

28.

Jewitt

(2009) The Routledge Handbook of Multimodal Analysis. London: Routledge.

29.

Jewitt

Kress

(2003) Multimodal Literacy. New York: Peter Lang.

30.

Kohlbacher

(2005) The use of qualitative content analysis in case study research. Forum: Qualitative Social Research 7: Art 21.

31.

Konig

Lick

(2014) Wine labels in Austrian food retail stores: A semiotic analysis of multimodal red wine labels. Semiotica 200.

32.

Kress

(2010) Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.

33.

Kress

Van Leeuwen

(1996) Reading Images: The Grammar of Visual Design. London: Routledge Falmer.

34.

Krippendorf

(2004) Content Analysis: An Introduction to Its Methodology. Thousand Oaks, CA: Sage.

35.

Lincoln

Guba

(1985) Naturalistic Inquiry, Thousand Oaks, CA: Sage.

36.

Machin

(2007) Introduction to Multimodal Analysis. London: Hodder Arnold.

37.

Machin

Mayr

(2012) How to Do Critical Discourse Analysis. Los Angeles: Sage.

38.

Mayring

(2000) Qualitative content analysis. Forum: Qualitative Social Research 1.

39.

Merriam

(1998) Qualitative Research and Case Study Applications in Education. San Francisco: Jossey-Bass.

40.

Messaris

(1997) Visual Persuasion: The Role of Images in Advertising. Thousand Oaks, CA: Sage.

41.

Mitchell

WJT

(2005) There are no visual media. Journal of Visual Culture 4: 257–266.

42.

Morse

(2008) Confusing categories and themes. Qualitative Health Research 18: 727–728.

43.

Norris

(2004) Analyzing Multimodal Interaction: A Methodological Framework. New York: Routledge.

44.

O’Halloran

(2004) Multimodal Discourse Analysis: Systemic Functional Perspectives. London: Continuum.

45.

O’Toole

(1994) The Language of Displayed Art. Leicester: Leicester University Press.

46.

Painter

Martin

Unsworth

(2013) Reading Visual Narratives: Image Analysis of Children’s Picture Books. Bristol, CT: Equinox Publishing.

47.

Peirce

(1960) Collected Papers, Vols I & II. Cambridge, MA: Harvard University Press.

48.

Reynolds

(2011) The Inside Story of a Wine Label. Napa, CA: Wine Compliance Alliance.

49.

Rose

(2016) Visual Methodologies: An Introduction to Researching with Visual Materials. Los Angeles: Sage.

50.

Ryan

Bernard

(2000) Data management and analysis methods. In: Denzin

Lincoln

(eds) Handbook of Qualitative Research. Thousand Oaks, CA: Sage.

51.

Schreier

(2012) Qualitative Content Analysis. Thousand Oaks, CA: Sage.

52.

Scollon

(2003) Discourses in Place: Language in the Material World. London: Routledge.

53.

Scott

(1994) Images in advertising: The need for a theory of visual rhetoric. Journal of Consumer Research 21: 252–273.

54.

Serafini

(2010) Reading multimodal texts: Perceptual, structural and ideological perspectives. Children’s Literature in Education 41: 85–104.

55.

Serafini

(2014) Reading the Visual: An Introduction to Teaching Multimodal Literacy. New York: Teachers College Press.

56.

Serafini

(2015) The appropriation of fine art in contemporary narrative picturebooks. Children’s Literature in Education 46(4): 438–453.

57.

Serafini

Reid

(2019) Crossing boundaries: Exploring metaleptic transgressions in contemporary picturebooks. Children’s Literature in Education. Available at: https://doi.org/10.1007/s10583–019–09382–9 (accessed 22 July 2019).

58.

Serafini

Kachorsky

Goff

(2015) Representing reading: An analysis of professional development book covers. Journal of Language and Literacy Education 11(2): 94–112.

59.

Smagorinsky

(2008) The methods section as conceptual epicenter in constructing social science research reports. Written Communication 25: 389–411.

60.

Stake

(1995) The Art of Case Study Research. Thousand Oaks, CA: Sage.

61.

Stockl

(2007) In between modes: Language and image in printed media. In: Ventola

, et al. (eds) Perspectives on Multimodality. Amsterdam: John Benjamins, 9–30.

62.

Strauss

Corbin

(1990) Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, CA: Sage.

63.

Sunderland

McGlashan

(2013) Looking at picturebook covers multimodally: The case of two-Mum and two-Dad picturebooks. Visual Communication 12: 473–496.

64.

Thyme

, et al. (2013) Qualitative content analysis in art psychotherapy research: Concepts, procedures, and measures to reveal latent meaning in pictures and the words attached to pictures. The Arts in Psychotherapy 40.

65.

Unsworth

(2006) Towards a metalanguage for multiliteracies education: Describing the meaning-making resources of language-image interaction. English Teaching: Practice and Critique 5: 55–76.

66.

Van Leeuwen

(2005) Introducing Social Semiotics. London: Routledge.

67.

Van Leeuwen

Jewitt

(2001) Handbook of Visual Analysis. London: Sage.

68.

Vedula

, et al. (2017) Multimodal content analysis for effective advertisements on YouTube. arXiv.

69.

Weber

(1990) Basic Content Analysis. Beverly Hills, CA: Sage.

70.

White

Marsh

(2006) Content analysis: A flexible methodology. Library Trends 55: 22–45.