Abstract
The multimodal and visual nature of children’s picturebooks has been documented in research emanating from multiple fields of inquiry. In this article, the authors present three types of analytical frameworks that are useful for conducting research on contemporary picturebooks as multimodal entities. Each framework draws upon different aspects of visual images, design features, and written language, and uses different theoretical lenses to call forth particular aspects of contemporary picturebooks. The three analytical frameworks are: (1) social semiotic frameworks, (2) literary frameworks, and (3) artistic frameworks. This article suggests that only through an orchestration of a range of analytical frameworks can scholars and educators begin to understand the complexity of contemporary picturebooks and their role in educational settings.
The evolution of various analytical frameworks for understanding how images and text work across a range of children’s literature has been predicated on attention to the increasingly multimodal landscape of contemporary forms of representation and communication (Duncum, 2004; Kress, 2010). Analytical frameworks that focus on contemporary picturebooks are based on the assertion that texts children experience in and out of school settings include a range of modalities, including visual, textual and graphic design elements, and are becoming increasingly more complex, necessitating a rethinking of how multimodal ensembles function across a range of educational and social contexts (Author, 2010; Beauvais, 2015).
Since the landmark publication of Reading Images: The Grammar of Visual Design (Kress and Van Leeuwen, 1996), appeals for incorporating theories of multimodality, social semiotics, and visual literacy into research on picturebooks have increased dramatically (Lewis, 2001; Pantaleo, 2004; Author, 2014). Drawing upon a variety of theoretical orientations, picturebook researchers have begun to explore new analytical frameworks for understanding the multimodal nature of picturebooks and the intersemiotic complexity inherent in these ensembles (Albers, 2007; Guijarro, 2014; Painter et al., 2013). It is our assertion that picturebook scholars and researchers need to move past the dominance of linguistically-based theories and analytical approaches, and take up a variety of analytical frameworks that conceptualize picturebooks as artistic, literary, semiotic, and multimodal entities.
Different analytical frameworks conceptualize picturebooks in different ways, and approach analysis from a variety of theoretical and methodological perspectives (Author, in press). Semiotic and social semiotic perspectives conceptualize picturebooks as multimodal objects and focus on the inclusion and orchestration of different modalities and the social contexts of their meaning potential, often drawing on Halliday’s (1978) ideational, interpersonal, and textual metafunctions (Painter et al., 2013; Sipe, 1998). Artistic perspectives conceptualize picturebooks as artistic objects, focusing primarily on the illustrations and graphic design elements, and the interplay of visual and textual modalities (Schwarcz and Schwarcz, 1990). Literary perspectives, including much scholarship on picturebooks as a literary genre or format, have conceptualized picturebooks as literary objects and have focused on narratological structures and how the relationship or interplay between visual, textual, and design components are presented (Nikolajeva, 2010; Pantaleo and Sipe, 2012).
In recent publications, researchers have proposed an array of analytical frameworks for examining the ways written text, visual images, graphic design features, and other modalities are used to construct narrative picturebooks (Kummerling-Meibauer, 2018; Author, in press). In addition to semiotics and social semiotics, researchers investigating multimodal phenomena have drawn from analytical frameworks associated with art criticism (Valleau, 2006), narratology (Genette, 1980), metafiction and postmodernism (Sipe and Pantaleo, 2008), cultural studies (Sturken and Cartwright, 2001), and critical approaches to multimodal discourse (Machin, 2007) to expand their understandings of various forms of representation and communication across particular social contexts. Many of these same analytical frameworks have been incorporated into studies designed to understand the visual and multimodal aspects of contemporary picturebooks (Albers, 2008; Kummerling-Meibauer, 2018).
Focusing on picturebooks as multimodal entities, researchers have conducted studies on the meanings constructed during children’s interactions with contemporary picturebooks in classroom contexts (Arizpe and Styles, 2003; Pantaleo, 2004; Sipe, 1999). Researchers have also focused on the ways young children respond to the visual, textual, and design features of contemporary and classic picturebooks, and have drawn on reader response theories (Beach, 1993; Iser, 1978; Tompkins, 1980), semiotics (Sipe, 1998), and critical theories (Lewis, 2000) to understand the variety of social and cognitive experiences readers have with picturebooks across a range of educational and non-educational contexts. For the purposes of this article, the researchers have selected to focus on research and scholarship that attends to picturebooks as objects themselves, as visual, textual, cultural, and multimodal phenomena, omitting scholarship on children’s responses to picturebooks, the pedagogical roles picturebooks play in educational settings, or research on the design or production of picturebooks.
Understanding Picturebooks
The compound word picturebook has been used by various researchers and theorists to connote the unified nature of the written language and visual images of this literary form (Author, 2014; Lewis, 2001). For these researchers, picturebooks are considered a multimodal, literary experience, where meaning is generated simultaneously from written text, visual images, and the overall layout or design (Nodelman, 1988). Based on this premise, picturebook scholars have posited a range of metaphors to describe the interactions among the visual images and written language of picturebooks, including plate tectonics (Moebuis, 1986), interanimation (Lewis, 2001), ironic (Nodelman, 1984), and congruency (Schwarcz and Schwarcz, 1990). Sipe (1998) has described the relationship between written text and visual images in the picturebook as synergistic, suggesting the combination of the two sign systems is greater than the potential meanings offered by considering either written text or visual images in isolation.
From a social semiotic perspective, contemporary picturebooks are multimodal ensembles drawing upon various modes of representation, namely written language, visual images, and layout and design elements to render a narrative or provide information (Author, 2014). Although the interplay between visual image and written language may vary across picturebooks and other multimodal texts (Nikolajeva and Scott, 2006), in order to construct meaning in transaction with picturebooks, readers need to attend to all semiotic systems of meaning to fully experience these multimodal ensembles.
There is no single theoretical orientation or all-encompassing analytical framework used for understanding contemporary picturebooks that would reveal all there is to consider about these complex, multimodal phenomena. Each of the analytical frameworks presented here focuses on different aspects of picturebooks and conceptualizes the relationship among text, image, layout, and graphic design in different ways. This diversity of perspectives allows researchers to build a more comprehensive view of children’s literature, in particular picturebooks, by bringing new forms of analysis that highlight different aspects of the production, design, distribution, and reception of picturebooks, as well as the structures and systems of meaning inherent in picturebooks themselves. In the same way that affordances and limitation have been considered across different modalities (Kress, 2010), each analytical framework brings possibilities and challenges to the ways researchers approach and analyze contemporary picturebooks.
Conceptualizing and Organizing Analytical Frameworks
An analytical framework is a set of assumptions, procedures, and techniques purposefully organized and designed to investigate textual, visual, and multimodal phenomena across a variety of sociocultural contexts and spaces (Author, in press). In general, the development of analytical frameworks has been affected by the visual, social, multimodal, and cultural turns across the social sciences, and has evolved to consider the social, cultural, historical, and material aspects of various multimodal phenomena across the contexts of production, dissemination, and production. The range of analytical frameworks available to researchers for investigating various aspects of picturebooks should address the expanding forms of representation and communication beyond language and linguistic elements to include additional modalities, including visual images, graphic design elements, and potentially music, gesture, and sound effects across digital platforms.
Based on a review of the literature on picturebook research and scholarship presented in subsequent sections of this article, and our previous experiences of working with a variety of analytical frameworks for understanding multimodal phenomena (Reid and Author, 2018; Author, 2011; Author et al., 2018), we have organized the approaches for understanding contemporary picturebooks into three general categories: (1) semiotic frameworks, (2) literary frameworks, and (3) artistic frameworks (see Table 1). Although there is some overlap and blurring across the boundaries associated with these categories, and not all frameworks included within a single category are grounded in identical theoretical orientations, we believe this way of organizing the existing scholarship offers picturebook researchers a map of the terrain that may support future research endeavors. Each analytical framework draws upon different ways of approaching and analyzing visual images, graphic design features, and written language, and uses different theoretical orientations to call forth various semiotic, literary, artistic, and multimodal aspects of contemporary picturebooks.
Overview of the analytical frameworks.
In addition to any theoretical differences among semiotic, literary, and artistic perspectives across the social sciences, and the academic domains from which these frameworks are derived, there are also differences in the data collection and analytical procedures associated with each framework. For example, literary perspectives tend to focus on the narrative or metafictive structures of picturebooks, whereas artistic perspectives tend to focus on picturebooks as works of visual art with written language viewed as a supplementary modality (Stevenson, 1998). Semiotic perspectives focus on the semiotic systems of meaning used in picturebooks and the interrelationships among these different elements, often drawing on the structures, grammars, and metafunctions of language to organize their analytical approaches (Halliday, 1975). Rather than seeing the differences across analytical frameworks as challenges to overcome, we view the diversity of analytical perspectives included here adding to the potential for new insights and understandings moving forward in the areas of picturebook research. Although there are some researchers and picturebook scholars whose work seems to transcend the boundaries of the three categories presented in this article, we feel the distinctions made among these frameworks can be used as an heuristic to illuminate important considerations for contemporary picturebook scholarship.
Semiotic and Social Semiotic Frameworks
Research that falls into this category views picturebooks from a semiotic perspective as complex signs that work across modalities and metafunctions, and can be investigated by attending to the systems of signs and choices made across visual, textual and graphic design elements in their production. Additionally, semiotic and social semiotic frameworks explore the multimodal nature of picturebooks and how meaning is constructed by readers as they transact with and oscillate between the textual and visual sign systems and other design elements (Sipe, 1998). Readers read the words through the pictures and the pictures through the words (Lewis, 1990). Scholars approaching picturebooks and other multimodal texts from semiotic and social semiotic perspectives must consider the semiotic work done by the various verbal, visual, and design elements, and the social, cultural, and educational practices and contexts in which they are embedded.
Researchers viewing picturebooks from a semiotic or social semiotic perspective ground their analysis in the central belief that various components of a picturebook are signs that stand for something, in some capacity, for particular readers (Peirce, 1960). From this perspective, picturebooks should be analyzed by considering the visual, textual, and design elements separately and how they interact with one another. The discussion here does not allow for an extensive review of all forms of semiotic and social semiotic approaches but will focus on the work of literary scholars that have used these theories and perspectives to analyze and conceptualize picturebooks.
In earlier scholarship, Golden (1988) described how meaning is generated simultaneously from visual and verbal cues, and stated that the verbal and visual systems contained in picturebooks offer a unified meaning. Golden’s work asserted that semiotic systems are responsible for representing different aspects of the narrative discourse or storyworld, where the linguistic text unfolds temporally, while the image is constructed of components that function spatially or simultaneously.
Sipe (1998) presented a theory of text–picture relations based on the semiotic concept of transmediation, and applied his analytical framework to the classic picturebook Where the Wild Things Are (Sendak, 1963). Transmediation occurs as readers work across sign systems, in this case visual and textual signs, during the process of comprehending picturebooks, where readers consider the text in terms of the images and images in conjunction with the texts. Sipe (1998) contested the assertion that understanding how images and text work in a picturebook is simply a matter of balancing the semiotic loads of individual modalities; rather, Sipe suggested it involves understanding how these various systems of meaning work in concert to extend, enhance or contradict potential meanings. Additionally, words and images act upon one another in shifting relationships to enhance, as well as destabilize, the potential meanings of a picturebook. Readers are invited to oscillate back and forth among text and image as they draw upon particular semiotic resources to make sense of the multimodal nature of contemporary picturebooks.
Drawing on Barthes’ (1977) classifications of text–image relations, including: (1) relay, where text and image contribute equally to meaning potentials; (2) anchorage, where the text anchors the potential meanings of the image; and (3) illustrating, where the image supports the text and adds detail to the meanings offered in the verbal text, Sipe (2012) expanded his original typology of image–text relations. In his later work, Sipe (2012) suggested congruency, deviation, augmentation, and contradiction as more nuanced explanations for how images and texts work in concert in contemporary picturebooks. Each sign system, or semiotic resource, in a picturebook complements, enhances, or contradicts the other systems of meaning to produce multiple, yet anchored, meanings.
Nodelman (1988: ix) posited the analysis of picturebooks should be grounded in some traditional form of semiotic theory which suggested focusing on ‘the codes and contexts on which the communication of meaning depends’. Drawing on the work of Barthes (1977), Quinn (2009: 144) stated that: semiotic analysis is an examination of the complex interplay of the literal elements of a text (termed denotations), and how these work through shared (or cultural) understanding to produce connotations, which are the second level of a reader’s understanding of the meaning of a (linguistic, pictorial, or textual) sign. (emphases in original).
Lewis (1990: 141) asserted that ‘an adequate theory of the picture book must directly address the bifurcated nature of the form (words and pictures) and must account for the whole range of types and kinds, including the metafictive’ (emphasis in original). Nodelman (1984) stated that an understanding of the whole picturebook involves integration of these temporal and spatial components.
Drawing upon semiotic theories and analytical frameworks associated with multimodality, Fahmi (2015) presented an analysis of The Tale of Peter Rabbit (Potter, 1902) and The Rabbits (Marsden, 1998). In this analysis, Fahmi selected these two picturebooks to show the differences between realistic and surrealistic imagery, and to develop a metalanguage for describing the different signifying systems present in these texts. Kabuto (2014) drew upon Peirce’s (1960) semiotic theories to outline the various semiotic relationships (iconic, indexical, and symbolic) that are present in visual narratives, in particular Alexander and the Wind-Up Mouse (Lionni, 1969). This analysis provided a semiotic profile that was the basis for analyzing how readers employed signs in the readings, asserting that the triadic, semiotic model of Peirce (1960) was superior to the dyadic model (De Saussure, 1910) and highlighted the complexity of readers’ transactions with the visual and verbal signs in picturebooks (Kabuto, 2014).
Working across the fields of children’s literature and semiotics, Trifonas (1998) offers some interesting work involving picturebooks. Because Trifonas is not associated with traditional children’s literature or educational contexts, his work is often overlooked by educational and literary scholars. However, his semiotic analysis suggested, ‘in order to understand the semiotic intraspecificity of how meaning-making is produced through the cross-medial dynamics of picture-book construction, it is essential to identify the formal elements of its lexical and visual textual components’ (p. 3). In his research, Trifonas (2002) outlined a method of analysis designed to account for the intramodal and intermodal aspects and elements of a picturebook, one incorporating traditional semiotic techniques that have been utilized for the examination of lexical and visual texts. He concluded, ‘The picture-book itself demonstrates that there is a definite self-supportive framework of cross-medial engagement between the lexical and visual components of the text on all levels that functions to develop the linear narrative manifestations of the plot in each codic milieu’ (p. 20).
Quinn (2009) conducted a hybrid semiotic analysis of the depictions of fathers and children in best-selling picturebooks, where she blended aspects of qualitative content analysis (Krippendorf, 2004) to give power to an in-depth semiotic analysis in what she termed a hybrid semiotic analysis. Her analysis revealed ‘connotations related to the roles of fathers as masculine, protective, nurturing, and playful, and the conceptualization of children as naïve, vulnerable, and playful’ (Quinn, 2009: 140).
In an investigation of the visual and verbal representations of same-sex parents included on picturebook covers, Sunderland and McGlashan (2013) drew upon theories of social semiotics and multimodality to better understand the various ways gay families were depicted in children’s texts. Sunderland asserted aspects of gay identity were represented in different ways through text and image that relate to the cultural context of gay identity in contemporary society. In the picturebooks included in the investigation, the authors asserted gay identities and practices can be, and need to be, read through an appreciation of mutual enhancement, rather than through an analysis of the image or text alone. Additionally, Labitsi (2009) constructed a visual narrative analysis model, informed by various semiotic theories that treated visual images as a specific form of communication. Labitsi (2009: 63) proposed an analytical model that ‘focuses on how a visual narrative interacts with and affects the meaning communicated in the written story’.
Crawford and Hade (2000) undertook research to better understand the semiotic, sense-making processes that occur when readers transact with wordless picturebooks. They asserted that wordless picturebooks ‘invite more divergent types of readings and may be more open to a range of semiotic interpretations than books that are accompanied by text’ (p. 69). Finally, Dowhower (1997: 57) emphasized the semiotic process of transacting with images in picturebooks and asserted that ‘visual images have an almost infinite capacity for verbal extension, because viewers must become their own narrators, changing the images into some form of internalized verbal expression’.
Closely associated, yet distinct from a traditional semiotic orientation, scholars have drawn upon theories of social semiotics (Hodge and Kress, 1988) and adapted the systemic functional theories developed by Halliday (1975, 1978) as a framework for analyzing the visual images and textual elements contained in picturebooks and other multimodal ensembles. Social semiotic theorists, in particular the work of Kress (2010) and Van Leeuwen (2005), have adapted the structures originally associated with analyzing language to analyze visual images and multimodal entities. The primary difference between semiotic and social semiotic approaches for understanding picturebooks is whether the primary focus is on Halliday’s systemic functional theories, and how the ideational, interpersonal, and textual metafunctions can be used as a model for understanding visual and multimodal phenomena, specifically picturebooks.
Sometimes referred to as multimodal discourse analysis, approaches aligned with Halliday’s (1978) systemic functional theories (SF-MDA) have been used for analyzing a range of multimodal texts, for example, advertisements (Machin, 2007), mathematical formulae (O’Halloran, 2004), print-based texts (Royce and Bowcher, 2007), and posters (Jewitt and Oyama, 2001). From a multimodal discourse perspective, a picturebook is conceptualized as a visual and verbal narrative composed of linguistic and visual grammars, where individual elements work in isolation (intramodally) and in concert with other elements (intermodally) to present a multimodal narrative (Unsworth, 2006; Unsworth et al., 2014).
Painter et al. (2013) and Guijarro (2014) have offered extensive analyses of picturebooks from a multimodal discourse perspective based on Halliday’s systemic functional theories. Painter et al. (2013) believed the analytical categories in the visual grammar put forth by Kress and Van Leeuwen (1996), while immensely valuable, are limited in their ability to analyze sequential and multimodal aspects of picturebooks. For example, regarding the interpersonal metafunction, they added five additional systems – (1) focalization, (2) affect, (3) pathos, (4) ambience, and (5) graduation – to Kress and Van Leeuwen’s original categories of contact and modality. They question Kress and Van Leeuwen’s assertion that vectors are critical to depict physical, mental, or verbal illustrations, and focus instead on understanding the roles of participants, processes, and circumstances by exploring these in the context of narrative sequence. In addition, Painter et al. (2010) analyzed the ways the concepts of framing, balance, and intermodal integration in multimodal texts affect the meaning potential when considering systems of visual and textual modalities. They asserted the semiotic choices made by authors, illustrators, and picturebook designers affect the various ways narratives are presented and the ways readers transact with these multimodal texts.
Also working from a social semiotic perspective, Guijarro (2014: 1–2) asserted the superiority of a linguistically-based analytical approach to analyzing picturebooks, and suggested that other frameworks, ‘although revealing, unfortunately disregard the meaningful exegetical possibilities afforded by a purely linguistic analysis’. He also asserted that considering the age of readers is an important variable to include in any analytical approach. Guijarro explored the choices available to writers and illustrators across representational, interpersonal, and textual meanings in verbal and visual aspects of picturebooks. Adopting systemic functionalist approaches over formalist linguistic approaches, Guijarro claimed that systemic functional theories are preferable to formalist approaches because they focus on grammar and meaning in context. Guijarro extensively analyzed nine picturebooks using the metafunctions of SFL and concepts associated with multimodal discourse analysis before proceeding to discuss the interdependent and synergistic interplay of visual and verbal grammars. In addition, Guijarro (2016) used linguistically-based analytical frameworks to reveal how images and text are combined to create meaning in multimodal artifacts, positing the concept of semiotic metaphor to the understanding of text–image relations.
Also working from a social semiotic perspective, Wu (2014: 1416) suggested visual and verbal relations in multimodal texts should be investigated from two perspectives: (1) inter-semiotic relations, examining how the pictorial and verbal elements complement each other to form a single and cohesive text; and (2) logico-semantic relations, derived from the inter-clause relations to interpret the logical relation between image and text across structural syntagmatic units. These analyses were founded upon the belief that multimodal texts, in particular picturebooks, involved linguistic, visual, and intersemiotic forms of semiosis, including concepts based on Halliday’s (1978) original work in systemic functional grammar, for example elaboration, extension, enhancement, and projection.
Scholars working from a semiotic and social semiotic orientation take the view that language, layout, image, and typography are all important aspects of contemporary forms of representation and communication. Analyzing picturebooks across visual and verbal modalities, scholars have begun to understand how various semiotic resources contribute to the representations and meanings associated with contemporary picturebooks.
Literary Frameworks
Scholars approaching picturebooks from a literary framework are interested in understanding the complexity of picturebooks as literary forms that include visual images, written language, design features, and the intermodal connections among these elements. What distinguishes the literary frameworks from the others is the focus on the picturebook as a whole, as a complex literary entity that must be considered holistically, not as independently operating components when being analyzed. The basis of these frameworks hinges on the assumption that visual images, written language and the oft-neglected elements of visual and graphic design (Dondis, 1973), including typography, borders, orientation, speech bubbles, and endpapers, work in different ways, serve different roles, and interact with one another to render a more complex narrative than is possible within a single modality.
The oft-quoted definition of a picturebook offered by Bader (1976: 1) included the following statement germane to this perspective, ‘As an art form it [the picturebook genre] hinges on the interdependence of pictures and words, written text, on the simultaneous display of two facing pages, and on the drama of the turning of the page.’ A basic analogy has been offered suggesting that ‘words tell, images show’ (Lewis, 1990). Nodelman (1984) suggested that pictures or illustrations limit text, and text limits the meaning potential of pictures or visual images. This way of conceiving picturebooks as an interdependent, multimodal entity relying on various literary and visual structures in their presentation is the bedrock of the analytical perspectives included in the literary framework.
Various scholars have taken a more text-analytic approach, investigating certain linguistic and visual characteristics, and the affordances, complementarities, and divergences of the two modalities. McCloud (1994) offered a leveled taxonomy of seven stages of image–text relationships based on their balance of how individual modes contributed to meaning potentials. Nikolajeva and Scott (2006) constructed an elaborate taxonomy of the relationship between text and images in picturebooks that range from symmetrical relationships, where the images and text work together to present similar meanings, as opposed to counterpoint relationships, where the images and text offer ironic or contradictory meanings.
However, the interrelationship among various modalities does not fall into as neat a taxonomy as some scholars have suggested. Although words and images may make a seemingly autonomous contribution to the overall meaning of the multimodal narrative, Lewis (1990) has problematized these various taxonomies by asserting that words and images can never offer the same meanings because they work in different ways across different temporal and spatial structures, rendering a truly symmetrical relationship theoretically impossible. While texts and images in picturebooks are not mutually exclusive, they are not congruent either. These constructed taxonomies too often diminish attention to the various ways that images and text work together to offer meaning potentials not depicted in any individual mode.
Agosto (1999) described the connections among words and images as interdependent storytelling, positioning the ways words and images work together in terms of augmentation and contradiction. These categories are reminiscent of Barthes’ (1977) concepts of relaying and anchoring, where texts worked to anchor images or to enhance the potential meanings through extension and derivation. In a picturebook, the written narrative propels the reader forward in the narrative as the visual images serve to slow the reader down to linger with the details included in the visual images and graphic design elements on each page.
Drawing on the work of Iser (1978), Beauvais (2015: 1) posited the gap as a central concept of picturebook theory, describing picturebooks as ‘pieces of a jigsaw waiting a reader’. Bodmer (1992: 72) suggested illustrations serve to ‘expand, explain, interpret, or decorate a written text’. From these perspectives, illustrations or visual images are included to serve the needs of the written narrative and may, at times, only work as a means of illustrating or decorating the textual narrative. This minimizing of the role of visual images does not conceptualize the picturebook as a coherent whole posited by other scholars working within this framework. Bodmer (1992) set the foundation upon which many picturebook scholars have extended the ways images and texts work together across a continuum ranging from decorative, as in minimally illustrated texts, to contradictory, as in many postmodern picturebooks (Sipe and Pantaleo, 2008).
Moebius (1986) offered a set of codes that are useful for understanding how illustrations in picturebooks work, calling our attention to the design and communicative aspects of this modern art form. The plain or literal sense of an image or picturebook illustration provides a starting point for one’s experience, offering a way into the illustrations and a foundation for further analysis and interpretation. Unlike famous works of art hanging in a gallery, Moebius posited the visual images in a picturebook cannot hang by themselves, and do not fare well when they are extracted from the context of a picturebook. All aspects of a picturebook should be analyzed as carefully chosen parts of a whole, with each element of design adding to the picturebooks’ cohesive nature.
Through an exploration of the various roles visual images play in picturebooks, Fang (1996) suggested illustrations work to establish setting, determine mood, define and develop character, extend or develop plot, provide a different viewpoint, reinforce text, and contribute to textual coherence. Additional research from a literary perspective has investigated the concepts of frame-making and frame breaking (Scott, 2010), verbal and visual page-turners (Gressnich, 2012), interpretive codes (Nikolajeva, 2010), and twist-endings (Bellorin and Silva-Diaz, 2010).
Often omitted from discussions aligned with literary perspectives are studies that focus on elements of visual design that do not fall neatly into either category of visual or textual narrative elements. Typographical elements, including font, weight, and color, graphic design features, including panels, gutters, and borders, and visual conventions including upfixes, reduplication, movement lines, and speech bubbles, are all conventions traditionally associated with graphic novels or comics, and are often overlooked in picturebook analyses. When discussing the importance of design and the key role played by the designer in picturebook production, Scieszcka (1998) suggested paying closer attention to the missing vocabulary and analytical perspectives that a study of the basics of graphic art and visual design might provide. For Scieszcka, good design sets the tone for everything and tells as much of the story as the words and images. In addition, Bandre and Button (2010) used the work of noted picturebook illustrator David Wiesner to demonstrate the potential for design elements, including typography, and suggested that nothing should be left to chance, and all design features contribute to the overall narrative syntax and coherence.
Author and Clausen (2012) focused on the typographical elements in contemporary picturebooks and offered a typology for analyzing these visual elements as part of the literary and graphic design of picturebooks. They asserted, ‘rather than acting as a naturalized conduit for the communication of a verbal narrative, typographical elements have become an integral part of the narrative itself, a semiotic resource that adds to the potential meanings of a picturebook’ (p. 5). Typographical elements need to be conceptualized as resources for authors, illustrators, publishers, book designers and readers to draw upon to realize textual or expressive meanings.
While semiotic approaches to analyzing picturebooks may offer more detailed analytical frames of analysis, literary frameworks approach picturebooks as multimodal entities where the synergistic relations between text, image, and design are interconnected in ways that trying to analyze separate modalities diminishes the nature of the phenomena under investigation. To understand picturebooks, researchers must not only consider the separate modalities from a variety of perspectives, but they must consider the ways modalities intersect, overlap, and contradict one another.
Artistic Frameworks
The artistic framework and associated analytical approaches conceptualize the picturebook as a piece of visual art, and the focuses on the various roles that visual images and illustrations play in the process of meaning-making (Dondis, 1973; Gombrich, 1972). What distinguishes this framework from the others is the primary focus on the visual images contained in picturebooks rather than the textual narrative, and the employment of perspectives from art history and criticism. Working from an artistic perspective, the picturebook is conceptualized as a work of art, and scholars have focused their analyses on the visual images in picturebooks from the traditions of art history and art criticism, where picturebooks have been described, evaluated, and interpreted using the language and processes belonging to art history and criticism (Eubanks, 1999).
From an artistic perspective, there are different ways visual images can be analyzed or considered. Picturebook scholars may focus on the visual or compositional elements traditionally associated with visual literacy and art criticism (Dondis, 1973), by analyzing the various art elements such as line, shape, color, position and other elements of visual design, including tone, textures, dimension, scale, movement, and composition. These elements are considered the basic ingredients of any visual image, and careful analysis of these components provides deep insight into the potential meanings of a particular visual image. While Eubanks (1999) referenced the positioning of images within groups, series, and sequences, the artistic analytical framework focuses primarily on individual images. By doing so, the focus on the interplay of word and image is diminished, and each visual image is viewed as a holistic entity that can be interpreted separately from the verbal narrative sequence.
Another artistically-based analytical approach focused on the intertextual–intervisual associations between picturebook art and works of fine art displayed in museums and other contexts (Beckett, 2010; Author, 2015). Picturebook artists often appropriate works of fine art by reproducing, transforming and transfiguring original art into a new context that challenges the reader to consider the work of art both within and outside the context of the picturebook. In order to understand the picturebook as a separate art object, readers must consider the contexts and nature of the original work of art and the recontextualization of that work in its new context.
It is important to note that artistic appropriation always results in some form of transformation; due to the different contextual factors of the original art and the picturebook image, an exact replication can never occur (Author, 2015). Reproduction allows the reader to consider how faithfully the original work of art was represented in its new context. Transfiguration permits exploration of how picturebook art was transformed from the original to fit the specific context and purpose of a picturebook and its narrative. Stylization requires the reader to consider picturebook art from an historical perspective by recognizing picturebook art as an homage to certain artists or art movements.
The intervisual relationships among picturebook art and works of fine art has also been considered a form of imitation (Mitchell, 1990) or adaptation (Hutcheon, 2006). Mitchell (1990) noted references to fine art in biographical and informational picturebooks on art and artists, as well as in picturebooks that contain fictional accounts of children or animals interacting with famous works of fine art. Valleau (2006) observed how some picturebook artists cut and pasted fine artwork into a picturebook, imitated other artists while adding their own personal flair, or translated visual elements from classical works of art into their images. Parody and pastiche are examples of cross-genre play that reference known and culturally important works of art or iconic conventions and depend upon the viewer sharing the artist’s or illustrator’s cultural tradition (Beckett, 2010).
Albers (2007) posited a visual discourse analysis by considering the presence of representational or schematic codes in picturebook artworks, codes that are present in images from across time and cultures. Albers (2008) analyzed children’s literature from the perspective of art theory and considered how and why artists render visual representations as they do, then used this approach to analyze images in Caldecott award-winning picturebooks. Three findings were presented: (1) image types cut across time, culture, and artists’ rendering; (2) images embody stable representations of culture; and (3) images tend to render visual binaries and invite oppositional readings (p. 163). Observing the codes present in picturebook art and noticing intertextual and intervisual relationships to works of art outside picturebooks enabled the reader to engage in the continual construction of knowledge about the world.
While many of these scholars have focused primarily on the picturebook itself, others have asserted the emphasis should be placed on the art history traditions from which individual illustrations and illustrators have evolved (Mitchell, 1990). Picturebooks may be used to direct students’ attention towards artistic traditions and art styles, thus building students’ conceptual knowledge of art in general. Eubanks (1999) asserted critically informed judgement rests on the reader’s ability to compare the picturebook against a relevant class of art objects that the critic has assembled.
The Whole Book Approach is a critical framework implemented in elementary schools to encourage students to see, hear, and talk about picturebooks (Lambert, 2015). Drawing on a Visual Thinking Strategies curriculum (Yenawine, 2013), the Whole Book Approach engages students in facilitated discussions surrounding their visual reading and meaning-making. These approaches have drawn upon strategies for discussing art in museums and other educational settings.
In addition, pop culture has been viewed as a starting point for children’s construction of picturebook meanings, and has served as a bridge between picturebook viewers and picturebooks (Eckhoff and Guberman, 2006). Yohlin (2012) has offered the concept of frame-breaking and has suggested that viewers approach and consider picturebooks by personally entering into the works of art and becoming involved in ways that make new, personal meanings possible. The artistic frameworks presented here have made reference to various art movements and traditions associated with picturebook art, and extended meaning-making processes beyond the pages of picturebooks to the world of art to help support readers’ exploration of both art history and picturebooks.
Concluding Remarks
If one were to define semiotics as the study of signs and forms of representation and communication generally, then it could be asserted that all the analytical frameworks might fall under this umbrella category. However, social semiotics, literary and artistic perspectives and analytical frameworks have evolved different assumptions, theories, and traditions of analysis that render them different from one another, if not mutually exclusive or unique. We also recognize the overlap of some of these approaches and acknowledge the highly connected nature of their research designs.
Although the focus of this article is on the array of analytical frameworks for investigating the nature of picturebooks as artistic, semiotic, and literary entities, it must be noted that research must consider picturebooks as socio-culturally situated artifacts, as well as multimodal objects, contextualizing these objects in the social contexts of their production, distribution, and reception. Although analyses from the site of the picturebook itself are an important first step in understanding how the multimodal ensembles work, understanding the contexts of their creation and consumption is an equally valid line of research.
The analytical frameworks presented here are grounded in specific literary, artistic, and semiotic theories that offer affordances, as well as limitations. Each of the categories draws upon different analytical lenses to call forth different aspects of picturebooks. Scholars working to understand the interdependent meaning-making systems of picturebooks should recognize the wide range of analytical frameworks available for conducting their analyses. It is only through an orchestration of each of these analytical frames that we begin to understand the complexity, meaning potentials, and underlying structures of contemporary picturebooks.
Footnotes
Funding
The authors received no financial support for the research, authorship, and publication of this article, and there is no conflict of interest.
Biographical Notes
FRANK SERAFINI is a Professor of Literacy Education and Children’s Literature in the Mary Lou Fulton Teachers College at Arizona State University. His research continues to focus on the multimodal nature of contemporary representation and communication and how critical, multimodal, and social theories impact literacy education.
Franks new book, Beyond the Visual: An Introduction to Researching Multimodal Phenomena will be available in 2022 from Teachers College Press.
Address: Arizona State University, 5038 E Weldon Ave, Phoenix, AZ 85018, USA. [ email:
STEPHANIE F REID is an Assistant Professor of Literacy Education in the Phyllis J. Washington College of Education at the University of Montana. Her scholarship focuses on multimodal literacy pedagogies in classroom contexts. Stephanie’s work appears in journals such as the Journal of Language and Literacy Education, AERA Open, the Journal of Children’s Literature and Visual Communication amongst others.
Address: Phyllis J. Washington College of Education, University of Montana Missoula, 32 Campus Drive, Missoula, MT 59812, USA. [ email:
