Embodied Music Cognition and Mediation Technology : A critical review

Embodied dualism

From the outset of the book, Leman identifies one of the crucial problems of music research as explaining how to fill the gap between music seen as encoded physical energy (the way modern digital media considers music) and music as a matter of beliefs, intentions, interpretations, evaluations and significations (the human way of dealing with music) (2007, p. xiii). In his words, the book is based on a hypothesis about musical communication, which is supposed to be rooted in a particular relationship between musical experience (mind) and sound energy (matter).

What appears problematic, however, seems to be the way Leman takes for granted the ontological separation between mind and matter, considering in particular that the author himself states that “[n]euroscience has provided compelling arguments that the Cartesian division between mind and matter can no longer be maintained and that a disembodied mind as such does not exist” (p. 13).

We can notice a strong contradiction between this point and the problem the book wants to solve. Indeed, how is the body supposed to fill the gap between mind and matter if they cannot be considered anymore two different categories? After discussing the problems of a subjectivist approach (pp. 12–13), Leman states that “[a] way to proceed is based on the idea that action may play a key role in mediation processes between the mental and the physical world” (p. 14). In assuming this ontological separation, the author actually follows Descartes (1641/1984), who considered immaterial mind and the material body as two distinct substances (Rozemond, 2002; see also Hart, 1996, for an overview of modern discussions on dualism). Among the main theoretical models proposed to overcome the difficulties raised by the Cartesian dichotomy (how are mental events and physical events linked? How can an immaterial mind interact with a material body?), the classic cognitive perspective (Boden, 2006) has emerged as one of the most discussed and controversial approaches during the past century. It has been developed leaving the dualistic presuppositions unaltered, as in the bottom up / top down model for information processing (see Dennett, 1994).

Leman’s account for “an action-oriented approach, based on the notion of corporeality” (2007, p. 26) is, in fact, supposed to overcome the problem of dualism, but its aim to provide an “epistemological foundation for bridging the gap between musical mind and matter” (2007, p. 26) falls inevitably short due to the intrinsic contradiction in its assumptions. As Hutto (2013) shows:

This intellectualist way of understanding the basic nature of minds taps into a long tradition stretching back at least as far as Plato; it was revived by Descartes in the modern era, and regained ascendency, most recently, through the work of Chomsky during the most recent cognitive revolution. As Noë (2009) observes: “What these views have in common – and what they have bequeathed to cognitive science – is the idea that we are, in our truest nature, thinkers. It is this intellectualist background that shapes the way cognitive scientists think about human beings” (p. 98). (Hutto, 2013, pp. 252)

This dualistic theoretical background, moreover, will affect most of the topics discussed in the book. As Keller and Janata pointed out in their book review, Leman’s analyses focus on the levels of descriptions used in musical research:

[p]revious approaches to understanding music (e.g., traditional musicology) have fixated upon these two levels of description without giving adequate treatment to the “rules” that govern the mapping between objective representations and subjective interpretations. Such rules are needed to achieve his scientific goal of developing a complete theory of music, as well as his practical goal of developing a successful mediation technology. The key to Leman’s solution is the proposal that an understanding of musical intentions requires third-person and first-person descriptions to be linked via second-person descriptions, which are corporeal in nature. At this intermediate level, expressive bodily gestures from an individual’s repertoire of actions are used to describe moving sonic forms in a manner that the individual can interpret based on his or her personal experience of interacting with others in the world. (Keller & Janata, 2009, pp. 289–290)

Leman’s approach aims to replace an objectivistic third-person description and a standpoint linked to subjective intentions’ interpretations (first-person description) with a second-person description based on the notion of corporeal intentionality. In the next section, we will first discuss Leman’s understanding of the first-person standpoint in relationship to the theoretical background of the book, before analysing the second person descriptive level and its influence for the role of intentionality.

Phenomenological points of view and musical intentionality

If we look at the classic phenomenological account for the first-person standpoint, we could notice that it is supposed to ensure that the respective perceived object (in our case, music) is described exactly as is experienced, or intended, by the subject:

Perception is not a simple empty “having” of perceived thing, but rather a flowing lived experience of subjective appearances synthetically uniting themselves in a consciousness of the self-same entity existing in this way or that. (Husserl 1925/1977, quoted in Smith, 2003, p. 67)

Husserl (1912–1929/1988, first book) considered the epistemic infallibility of the first person perspective as the core of transcendental phenomenology, and Leman refers to it as drawn upon “interpretations of intentions attributed to music, such as attribution of “inner victory of the spirit” to a passage in Beethoven’s Piano Sonata in A” (2007, p. 79). But the phenomenology of embodiment claims something different from Leman’s standpoint: in Merleau-Ponty’s masterpiece Phenomenology of perception (1945/1962), the French philosopher argues that the real nature of perception is not consciousness, but the body, intended not as the simple piece existing into the physical world, rather considered a lived and living body. “Merleau-Ponty bases his entire phenomenological project on an account of bodily intentionality and the challenge it poses to any adequate concept of mind. Embodiment thus has a philosophical significance for Merleau-Ponty that it could not have for Husserl” (Carman 1999, p. 206). According to this view, the perceiver’s ability to perceive is mainly constituted by sensorimotor knowledge and, consequently, the role of representations in any perceptual theory should be reconsidered (Noë, 2001; O’Regan & Noë, 2001).

No doubt perception depends on what takes place in the brain, and very likely there are internal representations in the brain (e.g. content-bearing internal states). What perception is, however, is not a process in the brain, but a kind of skinfull activity on the part of the animal as a whole. (Noë, 2004, p. 2)

However, Leman, conceiving this standpoint as generated completely in a disembodied mind, prefers to use another level of description. But also the third-person standpoint will not be used: it

is based on repeatable measurements of the given phenomena that can be obtained by a person, a machine or a person with the help of a machine. Putting knowledge of human information processing and the user’s convention into a machine, it is possible to achieve the measurement of the high – level structural and semantic properties (pitch, loudness, tempo or particular affects such as happy or sad) directly from the physical properties of the musical piece. The agent’s subjective involvement with music can be observed from the third – person point of view. For instance, the brain activity causing the movement of arms and legs in response to music can be recorded and then analysed from the third – person perspective. (Matyja, 2010, p. 9)

Leman basically agrees with the refutation of the third person perspective provided by the Husserlian phenomenology: it refers to the scientific and naturalistic claim that reality is objective, and that sense data correspond with it, excluding from the research the first-person viewpoint. So, Leman, in league with Husserl, denies this scientific perspective but, instead of using a pure egocentric standpoint, advocates for a second, intermediate level of description. To explain this second person point of view, the author uses the following metaphor:

It is like a doctor who asks a patient to describe what he or she feels. The nonverbal and verbal articulations of the patient express what is perceived and experienced. Based on that information, the doctor should be able to make a correct diagnosis . . . Second-person descriptions reflect involvement with physical energy in a context of intersubjective communication (Leman, 2007, p. 82).

As mentioned above, Leman’s solution is to investigate musical experience with a level of description of the motor repertoire of the subjects, trying to put scientific and subjective approaches on the same level. As Keller and Janata explain:

For example, describing a piece of music as “driving” connotes structural properties at the sensory level (fast tempo, repetitiveness of pitch, strong accentuation), at the gestural level (e.g., vigorous pounding of the air with a fist), and at the semantic/affective level by implying a high level of arousal. Leman assumes that corporeal articulations arise in response to music (though actual movement can be inhibited) because the human motor system resonates with patterns of physical sonic energy. Consistent with the view that perception involves covert action simulation, Leman claims that this behavioural resonance occurs due to the tight coupling of action and perception. As he points out, the most compelling evidence for action/perception coupling comes from behavioural studies of imitation and neuroscientific studies of the so-called “mirror system” in the brain. (Keller & Janata, 2009, pp. 289–290)

The discovery of mirror (Di Pellegrino, Fadiga, Fogassi, Gallese, & Rizzolatti, 1992; Gallese, Fadiga, Fogassi, & Rizzolatti, 1996; Kohler et al., 2002; Rizzolatti & Sinigaglia, 2008; Sakata, Taira, Murata, & Mine, 1995) and canonical neurons (Rizzolatti et al., 1988; Rizzolatti & Sinigaglia, 2008) contributed to the gradual renewal process of many ideas at the basis of the classic cognitivistic position (Gallese & Sinigaglia, 2011), and, in particular, the idea of a disembodied mind appears far from the new theoretical background provided by the discovered functional role of the motor system.

In virtue of the translation of others’ bodily movements into something that the observer is able to grasp as being part of a given motor act accomplished with a given motor intention, the observer is immediately tuned in with the witnessed motor behaviour of others. This enables the observer to understand others’ motor goals and motor intentions in terms of her/his own motor goals and motor intentions. (Gallese, Gernsbacher, Heyes, Hickok, & Iacoboni, 2011, p. 370)

As Sinigaglia (2008a) noticed, these findings suggest that the basic aspects of intentional understanding can only be fully appreciated using a motor approach to intentionality; indeed, at an sub-cognitive and pre-linguistic level, the understanding of the others’ actions, intentions and emotions does not require a cognitive subordination because our brain has a mechanism for feeling them as if (see Damasio, 2003) we feel our own intentions, actions and emotions. It is probably to this kind of intentionality that Leman is referring when he uses the expression “corporeal intentionality” (p. 84). But, again, the author refers to another kind of intentional understanding, a “cerebral intentionality “which – in music – explores the speculative pursuit of potential interpretations. The essence of cerebral intentionality is interpreting the source of intentions attributed to music. The essence of corporeal intentionality is the articulation of moving sonic forms . . .” (p. 84). This theoretical paradigm, despite its noble intentions, seems counterproductive in nature: if an embodied approach to music perception should consider the sensorimotor integration which link the subject and the object of musical experience (Pelinski, 2005; Schiavio, 2012), there is no need to postulate an additional high-level form of intentional understanding. (Musical) intentionality, in its new determination provided by the discovery of mirror neurons, does not deal with mental states but, on the contrary, should be studied in its enactive, pre-attentive aspects (e.g., Koelsch, Schroger, & Gunter, 2002).

Simulating and understanding

Let us now look at the central issue concerning how musical understanding actually takes place in the subject/object relationship defining musical experience. In Leman’s work, as previously seen, the concept of embodiment has a pivotal role in the meaning-giving processes. This notion, however, is strictly connected with a radical simulationist claim. It basically states that music understanding consists in a mental simulation of acts appropriate to make sense of musical surface. As simulation is in fact a very extensive issue, we will need to explore the conceptual topography of Leman’s peculiar approach, starting from its basic formulation. The first question to be answered is about the characterisation of the subject of musical experience, and the identification of other agents s/he would understands via simulation.

The above findings agree with the notion of self as a mental model having access to a proprioceptive and kinesthetic representation of one’s own body in combination with a representation of intended relationships with the other (Gallagher, 2000; Metzinger, 2003). Both aspects of the self, its inward and outward directedness, can be considered aspects of the action-oriented ontology. In the present context, the other is of course the music. The above findings are consistent with the idea that empathy involves regulatory mechanisms by which the subject keeps track of the self in relation to music (Leman, 2007, p. 126).

According to Leman’s account, therefore, the subject of musical experience should be identified with a mental model, whereas “the other,” the agent whose behaviour is understood in intentional terms, should be the musical stimulus itself. Of course Leman does not mean musical surface as some sort of subject to whom we can ascribe beliefs, values and meanings. His claim should be rather understood in these terms: what matters for musical, as well as different kinds of understanding to take place is the attribution of intentionality the subject makes on the basis of some kind of mirroring, that is, “on the basis of a simulation of the perceived action in the subject’s own action” (Leman, 2007, p. 92). Here we have to notice the radical incompatibility of such a position with the embodied approach according to which motor understanding takes place without and before any kind of inferential mediation. Another crucial issue in defining a simulationist approach concerns the way we mean simulation itself. As Rockwell (2008) pointed out, indeed, a naïve approach to this topic is no longer sustainable, in light of the long-term controversy between Theory Theory and Simulation Theory. According to Gordon (1995) and Rockwell (2008), there are two main ways simulation has been intended in literature:

Both these approaches have been widely questioned regarding their theoretical presuppositions and empirical evidences, yet, what we want to point out here is the theoretical impossibility to embrace both of them at the same time. According to the first view, indeed, there is no need to postulate any kind of conceptual understanding preceding simulation, whereas the second view is known to collapse into a hybrid TT/ST theory. The fact that Leman does not address explicitly the issue of this distinction raises several coherence problems for his account questioning whether he always means “simulation” in the same terms or not. The following excerpt (in which Leman quotes his colleague Godøy) highlights a view of “simulation” as pictorial representation, where the subjects would create an image in the mind from which to extract information for action planning.

When a trajectory in the inner space is simulated – for example, a walk from the front door of the university building to my office – I can associate this motor image with a trajectory in the outer space. I can explore the visual-audio-tactile features of the objects I encounter along my imaginary walk. In that sense, the sensorimotor couplings allow the transition from imagined movement to predicted sensory qualities. If the moving sonic forms can engage humans in body movements, then it is straightforward to assume that this movement will engender sensory qualities which can be attributed to music as well. If corporeal imitation of movement in sound is possible, then the association with sensory qualities is straightforward (Godøy, 2003). (Leman, 2007, p. 88)

A second quotation, however, will show a different – and, we claim, incompatible – way Leman means “simulation,” here understood in terms of imitation.

Mimesis theory assumes a transitivity relationship: (1) music imitates something, (2) the subject imitates the music, and hence (3) the subject imitates that same something . . . A related version of this transitivity relationship is based on the notion of expressiveness: (1) music expresses something, (2) the subject captures that expression, and (3) the subject captures the source of that expression. (Leman, 2007, p. 128)

This view, exemplified in several excerpts, outlines a circular scenario in which simulation (or mimesis) cannot account by itself for an understanding of actions, concepts or musical objects, requiring therefore, as the ST/TT dispute showed, a conceptual knowledge prior to simulation itself, as we can argue from this well-known argument of Dennett:

If I make believe I am a suspension bridge and wonder what I will do when the wind blows, what “comes to me” in my make-believe state depends on how sophisticated my knowledge is of the physics and engineering of suspension bridges. Why should my making believe I have your beliefs be any different? In both cases, knowledge of the imitated object is needed to drive the make-believe “simulation”, and knowledge must be organized into something rather like a theory. (Dennett, 1987, pp. 100–101)

Furthermore, it seems that the emphasis on attribution of intentionality reflects a position similar to Dennett’s, where the understanding is a matter of concepts, not percepts:

This attribution of intentionality can also be extended to material things that move, such as cars. In traffic, another car is not just a moving object. It is an object with particular intentions which I can understand by using the experience of my being a driver. Responsible drivers aim at understanding the intended movements of other cars, and on that basis predict their future behaviour. It is likely that for a dog, a car is just another moving object. It is not an intentional object because the dog is not involved with driving a car. The moving car is not something that the dog can relate to its own actions. Hence, it may not have an understanding of it in terms of a mental simulation. Like cars, music can be understood as an intentional object. (Leman, 2007, p. 78)

This view seems to compromise Leman’s understanding of mirror neurons issue, despite Keller’s and Janata’s optimism about this point. The main claim related to the so-called “mirror neuron system” is, as we saw in the last paragraph, about its crucial role in action understanding, so, if we assume with Leman that intentionality is mentally attributed to agents, or even material things that move, we will lose the chance to gain insight into motor understanding which occurs “prior to and apart from any forms of deliberate mentalizing” (Sinigaglia, 2008b, p. 70). According to these considerations, Leman’s way of dealing with intentionality should be considered as irremediably compromised with the same disembodied approach he claims to overcome. In summary, we found that Leman’s way to tackle the simulation problem not only highlights remarkable internal incoherencies, but also points to a substantially disembodied approach to musical understanding, according to which the subject, meant as a mental model, attributes intentional states to musical surface simulating different actions. These actions, furthermore, have nothing to do with goal-directed motor actions, but are conceived from an a-posteriori point of view, consistent with the intentional (or teleological) stance paradigm, adopted by Dennett and other scholars.

Mediation, body and instruments

According to Keller and Janata (2009), mediation refers to the mappings between the intentions and desires on the part of active musical participants and the technology that renders the music. This first meaning, related to the dualistic presuppositions previously examined, is not, however, the only one adopted by Leman: he also refers to mediation in respect of the role of technology itself as mediating between the performer and actual sonic outcomes. We have already discussed about the first kind of mediation, concerning the link between the allegedly separate domains of mind and matter. We will now turn to the second one, wondering what kind of mediation Leman poses between a musician and the music he plays. We could also look at the problem from this side: what is the role of musical instruments, whether they are classical acoustic ones or electronic devices, in musical practice and experience? We have at first to notice that Leman is facing two different issues:

Though Leman seems not to distinguish between these two sub-problems, he made clear his persuasion that an understanding of the more theoretical one would be useful in order to implement more effective music technologies (2007, p. 2). In order to delineate Leman’s approach to this crucial issue, we should look back to his position about the body–mind problem. Once recognized the dualistic bias which informs his whole theoretical paradigm, we’ll be able to fully understand the operating concept of “mediation” adopted, though not explicitly discussed. Indeed, his view about musical technologies should be understood as based upon the previously examined dualistic account: musical instruments work as mediation technologies in a similar way to how the body functions “as a biologically designed mediator that transfers physical energy up to a level of action-oriented meanings, to a mental level in which experiences, values, and intentions form the basic components of music signification” (Leman, 2007, p. xiii). This is, apparently, a classical embodied claim: instruments, as tools can be seen as extensions of the human body or, more precisely, of the peripersonal space defined by the subject’s action possibilities (Costantini, Ambrosini, Scorolli, & Borghi, 2011). Nevertheless, a closer analysis will show that Leman reduces embodied cognition to a kind of mediation compatible with classical, disembodied, paradigms about tool use, once more misunderstanding the deeper meaning of embodiment. The reason for this misunderstanding is to be found precisely in the dualistic presuppositions examined previously: if mind and body are conceived as distinct substances the main theoretical problem becomes, as for post-Cartesian thinkers, to justify the mediation between these separate dimensions. This is not, however, the way an embodied approach would address the questions concerning mediation and tool use, given the non-dualistic stance implicit in its core claims: conversely, the notion of body schema, widely discussed in the last decades, provides a non-dualistic way to conceive tool use as well as the role of “proper body” in human action, perception and cognition (see Maravita & Iriki, 2004).

Conclusion

In our opinion, Leman’s attempt to develop a new approach defining musical experience is not sufficient in providing a coherent theoretical paradigm, and most of his intentions fall short according to the implicit contradictory background underlining his arguments. The explicit use of an ontological distinction between mind and matter is indeed incompatible not only with an embodied approach to musical understanding, but also with a plausible notion of musical intentionality based on subject’s motor repertoire. At the basis of Leman’s perspective, indeed, there is, we claim, a misunderstanding of the role of the sensorimotor integration provided by the mirror neuron system: while the mirror mechanism “maps the sensory representation of the action, emotion or sensation of another onto the perceiver’s own motor, viscero-motor or somatosensory representation of that action, emotion or sensation” (Gallese & Sinigaglia, 2011) without any cognitive subordination, the author, conceiving the subject as a disembodied mind, prefers to investigate other (high-level) forms of sense-giving abilities, shaped upon an rough notion of simulation. On the contrary, in order to provide a persuasive theory of musical understanding based on embodied cognition, the author should have considered the goal-directedness of the motor acts grasped and mirrored by the subjects (for example by studying their involuntary resonance during passive listening tasks, or measuring their pre-attentive neural response) focusing on the basic level of the intentional relationship that links the subject and the object in any musical experience. A musical-directed repertoire of acts is indeed the theoretical consequence of the background provided by the SAME model ³ proposed by Katie Overy and Istvan Molnar Szacaks (Molnar-Szakacs & Overy, 2006; Overy & Molnar-Szakacs, 2009) and by many behavioural and neuroscientific studies (e.g., Bangert et al., 2006; D’Ausilio, Altenmuller, Olivetti-Belardinelli, & Lotze, 2006; Lahav, Salztman, & Schlaug, 2007). Indeed, we consider the study of such a repertoire the key for a deeper understanding of the basis of musicality and a gateway for any embodied approach to music perception. In particular, the understanding of the development of musical expertise in musicians, non musicians and children can be improved considering the subjects’ sense-giving ability as resulting directly from the motor knowledge of our body, taken as a permanent condition of experience (Merleau-Ponty, 1945/1962) and not a mere mediator between a disembodied, subjective mind and the objectiveness of the musical material.