Understanding retrospection: Blended joint attention and the multimodal construction of film flashbacks

2.1. The Help (Taylor, 2011)

The Help (Taylor, 2011) tells the story of young journalist Skeeter Phelan and a group of African American maids in the 1960s, during the civil rights movement. By writing a book with the maids’ testimonies about the hardships in their daily work, Skeeter condemns publicly the way white families treat them, and helps them fight for their rights.

One of the flashbacks in the film is introduced right after the scene of Skeeter’s argument with her mother (00:23:00–00:25:50). She has just found out that, while she was away in college, her mother dismissed their lifelong maid, Constantine. Visibly upset, Skeeter leaves the house and crosses the backyard, and that is when the sight of an empty bench makes her remember a conversation she had with Constantine right there, a few years ago. As Figure 1 shows, ¹ we first see Skeeter in a medium shot, from the side: she has stopped and she is looking somewhere off-screen (a). The following shot shows an empty bench in the garden (b), and then a third shot goes back to Skeeter again, although this time it is a front-view close-up (c). That is, first we see the protagonist herself, but afterwards we observe the bench with her, adopting her point of view. Finally we see her again but from a different optical perspective. Thus, the combination of shots a, b and c builds an ‘eyeline match’ structure, a classic film technique which contributes to the ‘continuity system’, ² and which consists of the combination of at least one shot of a character looking at a certain point off-screen and a shot of an object (or another character) towards which the first person looks. This schema is equivalent to the point-of-view (POV) shot as defined by Branigan (1984): a combination of shot A (point/glance), showing the character’s face and gaze direction, and shot B (point/object), which shows the object the character is looking at. Thus, in shot B the viewer adopts the character’s perspective (although he or she does not necessarily take the character’s precise optical vantage point).

OPEN IN VIEWER

Figure 1.

Skeeter’s flashback.

The eyeline match, then, is the basis upon which the flashback is built, as the next shot shows that what Skeeter is looking at is already an image of the past, and that it is Skeeter herself who is sitting on the bench, though the scene took place some years earlier (d).

Furthermore, what lies beneath this conventional editing structure, and therefore beneath the construction of the flashback analysed, is a scene of classic joint attention: a situation in which two individuals sharing spatiotemporal coordinates attend simultaneously to a third object, and also communicate about it. That is, they ‘know together’ that they are focused on the same element (Carpenter and Call, 2013: 50) (see section 2.3). Oakley and Tobin (2012: 73) state that ‘all the standard devices of continuity editing are endowed with the logic of joint attention’, and this also seems to be the case of the eyeline match technique, for the viewer follows the character’s gaze, and both subjects end up taking notice of the same object. However, this interpretation poses a problem if we take a closer look at the flashback scene and at the concept of joint attention. It is true that both Skeeter and the viewer pay attention to the same element, but one of the main features of joint attention is not present: the character is not aware of that shared attention, and she does not intend to share anything with the viewer (Tobin, 2008: 24–28). Therefore, the attentional triangle in this case cannot be formed by the viewer, the character and the object. But, since it is clear that the flashback is based on an eyeline match structure, and that a joint-attention logic underlies this kind of construction, the necessary attentional triangle must consist of other components, namely the camera, the viewer and the object observed (Skeeter first, and then the bench). It is the camera eye and the viewer who pay attention to the same thing; the camera guides (and controls significantly) the viewer’s gaze, and thus, by paying attention to a certain object, they are both engaged in a goal-directed activity (and they are conscious of it) (Oakley and Tobin, 2012).

Even though the character (Skeeter) is not one of the subjects of the joint-attentional triangle, her gaze being directed somewhere off-screen also plays an important part in the construction of the flashback. Human beings naturally tend to follow one another’s gaze in order to see what somebody else is looking at (see Tomasello, 1999). Thus, this seemingly universal feature constitutes the perfect basis for the eyeline match/point-of-view structure. This is precisely what Carroll (1996: 128) is pointing at when he affirms that ‘we might think of point-of-view editing as an automatization, via editing, of our own natural perceptual reaction to track a glance to its target’. The viewer, always guided by the camera, follows the character’s gaze and reaches the object being looked at (which is in fact ‘given’ to him or her by the camera, although the viewer may feel that they are completing by themselves a totally natural process). But the film technique goes even further, for, as Carroll states, in point-of-view editing our gaze-following tendency goes beyond its information-gathering purpose and it is used as a communicative device (1996: 129). Although the character in question (Skeeter, in this case) does not take part in a joint-attentional triangle, her gaze is intentionally communicative because the camera (the filmmaker and their crew) has decided to use it for that aim, and it directs the viewer’s attention to an object of importance in the story (the bench), which is in fact the object that will introduce the flashback.

Another factor that underlies the use of an eyeline match in the construction of certain flashbacks could be the relation of the flashback’s content to the character’s memories. By employing an eyeline match schema right before the flashback, the filmmaker clearly shows that the particular object the character looks at is prompting her to recall the past. It is as if Skeeter’s eyes were directly related to memories: they once witnessed certain events, and now they work as ‘projectors’ of those facts (memories), which she keeps in her mind. This idea is somewhat connected to the spatial-temporal metaphor ‘past is in front of ego’ that Coëgnarts and Kravanja (2015: 237) prove to be underneath the construction of several film flashbacks. In short, their claim is that, in flashbacks like the one in The Help where an eyeline match is established, what the character sees corresponds to what she knows, and that reality is located in front of her. As we can only know the past, and not the future, what the character sees in front of her must correlate with the past (2015: 227–228). Thus, ‘character perception is mapped onto the conceptual domain of time’ (2015: 237).

2.2. Big Fish (Burton, 2003)

Looking now at one of the flashbacks from Big Fish (Burton, 2003), we can see that the mechanism used in it is very similar to that from The Help. The film narrates Ed Bloom’s life, which he tells as being full of adventures and uncommon events. But his son, Will, does not really believe all the stories his father has been telling him about his childhood and youth. When Ed gets seriously ill and is close to death, Will seizes his final opportunity to discover the truth about his father’s life.

While flying with his wife from Paris to his hometown in Alabama, Will is very quiet and pensive (00:08:20–00:08:55). From what we already know about the story, we can infer that he is rather concerned about his relationship with his father. He looks at the hand shadows that a child nearby is making, and this moves him to recall his own father projecting hand shadows for him, many years ago, while telling him bedtime stories. As Figure 2 shows, we first have a long shot with Will in the background and, in the foreground, the child he is looking at (a). There is an eyeline match within this shot already. We then see a medium shot of the kid making hand shadows, but the perspective is different: it is as if we were seeing him from Will’s position (b). The following shot is a close-up of Will, his gaze directed off-screen (c); the camera zooms in slightly. Shot d is a big close-up of the kid’s hands, also with a zoom-in, and shot e is another close-up of Will’s face and look (zooming in), thus closing the eyeline match structure. Then, the next shot following this last close-up of Will takes us directly to the past (f): although it also shows a shadow projected with hands, it is a different one and it belongs to a previous moment in the story.

OPEN IN VIEWER

Figure 2.

Will’s flashback.

While this scene and the one from The Help share essentially the same structure, there are some meaningful variations in Big Fish that should be considered. First, the eyeline match structure is repeated: instead of using the basic schema of a ‘character shot’ plus an ‘object shot’ (as is the case with The Help), the scene starts with a long shot (a) which already displays an eyeline match, and which functions as the ‘character shot’ for the object in shot b. Then there is another eyeline match between shots c and d, and finally the last one between shots e and f; with this last shot the leap to the past is introduced.

The question is: why does Big Fish repeat the eyeline match structure, if a single pairing of the character and the object is enough to understand the sequence (as The Help shows)? What is it trying to reinforce? A possible explanation comes by also taking into account that the camera keeps zooming in, both when it shows Will’s face and the hand shadows (shots c, d and e). Will’s close-ups in shots c and e are progressively narrowed down to his face, thus bringing the viewer closer to the character and his experience. On the other hand, the zoom upon the kid’s hands (d) should be interpreted as being shown from Will’s perspective. Obviously, Will is not getting physically closer to the kid, and he cannot zoom with his eyes, but what this effect (alongside the repetition of the eyeline match) is trying to represent is Will’s mental state: concentrated and focused on a very particular aspect of his surroundings, he is progressively ‘taken’ by his memories, which are prompted by that external object calling his attention. By employing both a repetition of the eyeline match and a zoom-in effect, the scene underlines the character’s experience and state of mind. The subjectivity of the scene is enhanced by using those particular poetic resources. ³

Also, as it is the case with the flashback from The Help, a joint-attention structure lies behind this flashback scene, again with the camera, the viewer and the object observed (first Will, then the hand shadows) setting up the joint-attentional triangle. Will’s gaze directed off-screen is important as well, just as Skeeter’s is, for it guides the viewer’s attention towards the kid’s hands and, ultimately, towards an episode in the past.

2.3. Blended joint attention: the key to retrospect

Nonetheless, the construction and understanding of the flashback scenes in both films involves much more than just joint attention. The meaning of each retrospective scene is clear and obvious to the viewer, but the mental operations behind it are many and complex.

On the surface of the scenes from The Help and Big Fish, we see a leap to the past within the story, and a further shot-by-shot analysis of each sequence reveals the joint-attention schema as the pivotal structure for the flashback. Yet this is only the tip of the iceberg. The whole idea of retrospection does not come up solely by using a joint-attention structure. The leaps to Skeeter’s and to Will’s childhoods can only be understood as such when a variety of ideas are combined in our minds. There are several packets of meaning which must be brought together in order to produce the idea of retrospection, and just one of those ideas is that of jointly attending to something. In what follows, in order to explain the basic functioning of blended joint attention, the main concepts of blending theory will be outlined first. Secondly, it will be shown how that theoretical approach works in the three flashbacks analysed.

The mental process of combining different meanings in order to come up with a new one is also known as conceptual integration. It is a mental activity that human beings carry out unconsciously and is a fundamental part of almost everything they think, say and do. It is, blending theory argues, the way to explain how human beings think (Fauconnier and Turner, 2002). Blending theory proposes a hypothetical explanation of how human beings cognitively deal with a variety of phenomena. Many situations in everyday life call for the activation of several mental spaces, ‘small conceptual packets constructed as we think and talk, for purposes of local understanding and action’ (Fauconnier and Turner, 2002: 40). These packets of meaning are then projected into a single new mental space (a blended space) that integrates ideas from all the input mental spaces of meaning. Those projections are selective, meaning that not all the content from every mental space is transferred to the blend. Also, there are meanings that emerge in the resulting blended space that were not present in any of the input mental spaces. A convenient example is found in counterfactual statements of the ‘If I were…’ kind. Let us consider, for instance, the following sentence, said by a professor to one of their younger colleagues: ‘If I were you, I’d be working on finishing my book’. There are at least two main inputs, one for each of the professors, and they are projected into the blended space. In it, some selected features of the two persons are integrated: there is the situation of the younger professor, who needs to publish a book in order to attain a position, and there is also the attitude and experience of the senior professor (Grady et al., 1999: 119). Features of ‘I’ and features of ‘you’ are projected, and it is only inside the new blended space that the sentence can be spoken and understood, for only in it are the two viewpoints blended and do they result in a new perspective that combines both professors. Thus, only in the blend is the meaning of the counterfactual statement made available to our minds: it integrates all the relevant information needed to comprehend the statement, but it also presents it in a way simple enough for our minds to grasp its content.

It must be made clear that these kinds of mental operations are not extraordinary processes which our minds need to use on certain occasions that involve particularly complex meanings. In fact, our thinking is always complex, but since we only see the surface of it (its products and not its working mechanisms) we have the impression of simplicity. The truth is that we are constantly integrating different concepts and different mental spaces of meaning in order to come up with new ideas that shape new spaces. There are many examples of everyday activities which are only possible thanks to this capacity for conceptual integration, and some of those daily activities involve both joint attention and conceptual integration, just like the flashback scenes seen above. Take, for instance, the nowadays common activity of having a conversation through text messages. At least two people are talking to each other, paying attention to the same thing (the written messages shown on the cell phone screen, and their content) and communicating about certain matters. But, as obvious as this classical joint-attention scene may seem, its simplicity is only apparent. These two people are not speaking face-to-face, sharing time and space coordinates. They may even be miles apart, and one may read the other’s message several hours after it was written. Yet they are capable of engaging in an activity that requires jointly attending to an object, even if it is a non-physical one, like their topic of conversation. How does this work, cognitively speaking?

There is, to start with, the idea of a local classic scene of joint attention (taking place here and now), which serves as the ground for this other communicative situation that takes place beyond here and now (Thomas and Turner, 2011: 189–200). Classic joint attention is also distinguished by one of the participants drawing attention towards a near object, or pointing out something about it, and the result is the ‘common and interactive attention’ of all the participants (Thomas and Turner, 2011: 190). There is communication about that specific object of attention (see section 2.1), although the process may be nonverbal (see Tomasello and Farrar, 1986). It is because we have the experience of local and immediate joint attention that we can ‘project’ it to other experiences that are not local, not so familiar. In the same way, our experience of face-to-face conversation serves as the ground for the text-message conversation, which doesn’t occur face-to-face. What mental spaces of meaning are brought together, then, in this common texting activity? There are basic, human-scale packets of meaning: the idea of here and now, the idea of face-to-face conversation, the idea of classic joint attention, and so on. And there are also complex sets of meaning that, although easy for us to understand, surpass the here and now coordinates: two people in two different places and moments in time, two different points of view, the ability to acknowledge and adopt someone else’s viewpoint, and so on. All those mental spaces or packets of meaning are integrated, brought together, and the result is a new packet of meaning, a new mental space, which is anchored in a local scene of joint attention, but which goes beyond that. It is a space of blended joint attention, and only in it and from it does the text-message conversation make sense to us. A blended joint attention space is also the key to our understanding of the flashback scenes in The Help and Big Fish.

Going back to the flashback examples, at first glance it seems that they can be easily explained as simple joint-attention scenes. However, if we look closely at them we find that they don’t meet the conditions of direct joint attention. Classic joint attention takes place ‘here’ and ‘now’, whereas the components of the joint-attention triangles in the flashback scenes (the viewer, the camera and the character or object in the movie) are not present in the same place and time. That is why these film flashbacks should be analysed as cases of blended joint attention.

Oakley and Tobin (2012) identify four defined mental spaces that configure the conceptual integration network that is activated in the mind every time a movie is watched: the presentation space, which involves all those elements related to film production and to the way movies tell stories, including the camera’s gaze guiding the viewer (66); the reference space, which refers to the diegesis, the story world shown on the screen (66); and the virtual space, which issues from the integration of the presentation and the reference spaces (67). Those three spaces make up a conceptual integration network that emerges from and in turn affects the ground, defined as ‘the ontological status of the relevant actors in a film-viewing scenario’ (65). The joint-attention triangles that constitute the basis of the two flashback scenes are composed by elements from these four mental spaces: the viewer belongs to the ground; the camera guiding the viewer by using different techniques (close-ups, zooms, POV shots and so on) is part of the presentation space; and, finally, the characters and the objects towards which the viewer’s attention is directed belong to the reference space. The integration of the story told with the cinematic techniques (e.g. Skeeter stopping and remembering in front of the bench, and this being told using eyeline matches and POV editing) results in the virtual space.

All those spaces contain such an enormous amount of information that it is not possible to process all of it at once. Also, not all that information is necessary for us to understand the flashback scenes in terms of joint attention. Only a selection of relevant inputs from each space needs to be projected into a single, blended space, in which the meaning of the flashback emerges and is made accessible to our minds. Basically, the viewer’s attention and act of watching is projected from the ground; the camera’s gaze, directing the viewer, is added from the presentation space; the fictional world displayed on the screen (with the characters and their actions, emotions, etc.) is projected from the reference space; and finally, and most importantly, a classic joint-attention schema in its simplest, human-scale form is needed to understand this vast mental network. All these inputs are blended into a mental space of blended joint attention. Without this cognitive operation, it would be impossible to merge such diverse mental spaces and to make sense of them, and so the flashback scenes would be unintelligible. Blended joint attention operates in such a way as to create a human-scale joint-attention scene whose components don’t share time and space coordinates, and don’t even have the same ontological nature.

Some of the inputs projected to the blended mental space are products of a blend in themselves. That is the case of the viewer, a figure who brings together all the possible actual viewers a film can have. The common characteristics of all those spectators are projected to the blend (those features relevant in a film-viewing scenario), and as a result a general idea of a viewer emerges, one that can be in turn projected as an input in a new blend. In these terms, it is possible to speak of the viewer’s role in the joint-attention triangle without the need to test how a group of particular viewers watch a flashback.

The camera, as one of the components of the joint-attention triangle, is also the product of a blend. We think of it as a conscious agent that controls what the viewer sees, but actually such an intelligence only exists inside a blend. The idea of the camera as an independent subjectivity is the result of combining a variety of participants and their decisions in the movie-production field regarding camera operations. All those individuals’ decisions are blended into one single abstract agent that we call ‘the camera’ (Oakley and Tobin, 2012: 62). The camera as the product of a blend serves in turn as one of the inputs for the blended joint attention space.

Furthermore, as the viewer, the camera and the story world are projected into the blended space, three different perspectives are projected and merged in it as well. The viewer’s point of view is always forced to align itself with the camera’s perspective (otherwise, if he/she refuses to do so, he/she loses their as spectator), and so these two input elements are integrated into one in the blended space. But this doesn’t mean that in the blend the viewer’s POV disappears, subsumed by the camera’s. Both perspectives are merged in the blend, but at the same time they are always distinguishable, and they need each other. The camera’s reason for being lies in guiding someone’s attention, while the viewer cannot be so without a camera showing him the way along the story.

An additional perspective, that of the character, is introduced in the POV shots when the eyeline match structure is closed with the image of an object. In the example from The Help, as Figure 1 shows, the first shot of every eyeline match shows Skeeter from an ‘external’ position (shots a and c), and then the camera and the viewer join her optical perspective in shots b and d. The same technique is used in Big Fish (Figure 2): first we see Will (shots a, c and e), and immediately after each one of those shots the camera and the viewer adopt Will’s point of view, when he looks at the hand shadows (shots b, d and f). These object shots bring together the three perspectives aforementioned: that of the camera, that of the viewer, and the one related to the character in question.

Moreover, both in The Help and Big Fish the retrospection scene combines two different viewpoints from the main character: her perspective from the present moment, while recalling the past, and her past experience of the event which she remembers. In The Help, the flashback shows Skeeter’s relationship with Constantine when she was a teenager, and, while Skeeter’s experience of those moments belongs to the past, at the same time the flashback never stops being linked to Skeeter’s perspective from the present. In the same way, the flashback in Big Fish shows Will’s memories of his good relationship with his dad when he was a kid, which we experience with him. But, simultaneously, we are somehow linked to Will’s point of view from the present throughout the flashback, while he looks back at the events in the past.

Again, in order to make this combination of points of view available to our understanding, they need to be projected into the blend and compressed to a human-scale size. There is no other way to make sense of four perspectives that belong to different mental spaces, and that are not rooted in the same time and space coordinates. They are projected alongside the idea of classic joint attention, and also with the idea of putting oneself in someone else’s shoes (which is also the result of a blend, because we need to project the idea of ourselves, of our own mindedness, and the idea of someone else at the same time, and integrate both) (Turner, 2014: 93).

Finally, there is one last idea closely connected to that of viewpoint compression, and particularly to that of having two different points of view from the same character at once. It is the question of the past: where does it come from?

In The Help and Big Fish, the main character’s attention is suddenly absorbed by a specific object in their surroundings. This object is, eventually, the one that serves as a transition to the flashback scene, that is, to the narration of some past events in the story. But then, how does the viewer know that what he/she is watching is, in fact, a flashback? The introduction of a retrospection within the narrative requires a new narrative space to be set up, and that space needs to be activated by a ‘space builder’ (see Fauconnier, 1994, 1997). That is the function of the bench and the hand shadows in The Help and Big Fish, respectively: each of these objects appears first in the narrative space of the present, and then they reappear in a different narrative space, the past space, thus activating it and linking it to the previous one by means of an analogy connection (see Dancygier, 2008, 2012).

This new space is further developed and establishes identity connections between the characters in it and the ones in the narrative present: Skeeter, a young adult in the present, is now seen as a teenager; and Will, already an adult in the present, is just a kid in the past. There is also a relation of change that links these two narrative spaces, for the characters in them have changed as they got older (‘identity’ and ‘change’ are two of the ‘vital relations’ that, according to Fauconnier and Turner (2002: 91–96), conceptually link elements from different input spaces). In addition to this, a process of time compression is necessary to understand the flashback as such. The retrospection scenes consist of a narrative leap from the narrative present to a particular previous moment in the story (i.e. ‘the past’). But neither the character nor the viewer has to pass through all the events and the time elapsed between the present and the past moments in the story (Fauconnier and Turner, 2002: 317). The flashback transition (that is, the space builder, which sets up the narrative past space and gives us access to it) takes us directly back to a particular moment in the story. However, that new narrative space doesn’t come out of nowhere: it only exists in a blended space, it only emerges as a result of integrating in the blend this new narrative space with the present narrative space and, most importantly, the relation between them, for the past can only be so in relation to the present. For this purpose, the story as a whole needs to be projected as well.

Finally, returning to the blended joint attention schema discussed above, one could ask what the role of the past space in the network is. Considering that the character doesn’t take part in the activity of jointly attending, it can be said that it is again the viewer and the camera who, together, pay attention to the events in the past. The character, however, is added to the triangle by means of parallel attention (see Carpenter and Call, 2013), and so the past becomes the object of attention, either joint or parallel, of the camera, the viewer and the main character.

The analysis of the two scenes from The Help and Big Fish shows that, even though it is easy for us as spectators to understand a basic movie flashback, the device itself is not a simple one in cognitive terms: there are many different mental spaces which need to be activated and blended in our minds in order to understand a flashback. Blending processes are thus essential for the comprehension of this narrative tool. And more specifically, as we have argued, these particular flashbacks need to be understood as products of blended joint attention. But this cognitive operation doesn’t just function as a ‘final touch’ that comes in at the end of the scene and gives meaning to it. Rather, it is a constituent element that lies at the very heart of each flashback, and without it this device could not exist at all.