Music interaction with others as conveyor of relational intent: A response to Cross (2014)

Music performance as communication

Music is communication in any performance, communication that is governed by principles that are actively manipulated or unconsciously invoked by skilled musicians and less experienced performers alike. Almost all of the music that we hear is performed music, music that has been produced by a human player (or singer) to be received by a human listener. Performing music serves as one of the most visceral and direct forms of transmitting information (knowledge, perceptions, needs and desires) and eliciting emotion responses, unencumbered by the physical and cognitive barriers of spoken or written language.

Ian Cross argues, in his article (2014), for the need for research that treats music as communication, to fill the gap in prevalent approaches which focus almost exclusively on music as a ‘sonic commodity for hedonic consumption,’ and as ‘aesthetic object for presentation or consumption.’ He makes the case for ‘an account of music as interactive process that we can operationalize,’ on which we can build and validate hypotheses, and thus understand and test the impact of music communication in music therapy.

As a starting point, Cross examines the social functions of music that cut across cultures. Noting that music is a common feature in ‘moments of potential social transformation or situations of social uncertainty,’ he suggests that music ‘may be an optimal means of managing situations of social uncertainty.’ Furthermore, music achieves this through two properties: that of entrainment, which enhances a ‘sense of mutual affiliation’ amongst the interactants; and that of floating intentionality, which allows participants to experience an event as ‘both deeply personal yet shared.’

Comparing music to speech, he points out that music privileges the relational, while speech the transactional, dimension of communication. The transactional dimension of communication enables interactants to ‘exchange information and coordinate goal-directed behaviour;’ the relational dimension of communication brings participants’ ‘affective and motivational states into alignment.’ While this focus on social function provides valuable insights into why music is an interactive process, an account of music as communication would not be complete without also addressing the process (the how) of this interaction. To examine the process of musical communication, I propose an alternate starting point, that of the performer (the musician)’s perspective.

From the musician’s perspective, which is operational in the most practical sense of the word, I posit that the communication, and some might say the manipulation, of the emotional content of music derives from the active choreography of expectation.

In Leonard Meyer’s seminal volume on music and emotion (Meyer, 1956), he draws connections between formalist (intellectual) and expressionist (emotional) forms of musical meaning, thereby ‘establish[ing] an intellectual meeting place where “music lovers” (with their penchant for emotional conceptualizing) can meet music theorists (with their penchant for formalist conceptualizing)’ (Huron, 2001, Critical reflections on Meyer, para. 2). In Meyer’s work, much attention is afforded the composer’s role in choreographing expectation: by thwarting expectations, delaying expected outcomes, and simply providing the expected in note sequence of the music (see Huron, 2008).

There exists little research, both described in Meyer’s treatise and in the literature in general, focussing on the performer’s use of these very same devices in their choreography of expectation. The performer’s choreography of expectation through the expression of music forms a means of eliciting emotion responses as effective as, and sometimes more so than, the composer’s choreography of expectation.

Like Cross, Meyer eschews the notion that the aesthetic experience of music boils down to sensuous enjoyment (hedonism). Instead, he relates the form and syntax of music to its meaning and expression. In an alternative critique of Meyer’s thesis, Charles Keil (1966, p. 338) points out that there exists a part of expression that cannot be explained by syntactic analysis alone, and that ‘every piece of teleological music involves both syntax and an elusive quality designated here as “process.”’ His argument stems from his experience with non-Western musical traditions, which are almost exclusively performance-based, and which he feels Meyer’s theory is less successful at describing.

Keil discusses at length a straightforward classification of selected types of jazz bassists (chunky or stringy) and drummers (on-top or lay-back), and bassist-drummer combinations. In Keil’s (1966, p. 344) discussion, drummers’ strategies are described as serving the function of ‘generating a vital drive around a common pulse.’ For example, ‘“lay-back” drummers take more drastic liberties with the pulse than their “on-top” compatriots.’ It is worth pointing out that this description can be readily framed in the context of plays on expectation.

Taking Keil’s argument further, the process of timing in music performance may be only partially explained by Meyer’s syntactic approach, but it also extends beyond the partitioning into the common pulse and timing deviations from the pulse. The next example describes a kind of extreme elastic pulse stretching to the extent of impacting, even suspending, the pulse flow, which I call a ‘tipping point’ (Chew, 2013).

As described in Chew (2013), consider Puccini’s O mio babbino caro from the opera Gianni Schicchi as performed by Kathleen Battle (Battle & Williams, 1989), Maria Callas (Callas & Serafin, 1954), and Kiri te Kanawa (Kanawa & Pritchard, 2000). (Some of the more extreme cases of time stretching (rubato) in performance can be found in opera.) Eighth note lengths from an excerpt of the piece corresponding to the text:

Mi struggo e mi tormento!

O Dio, vorrei morir!

Babbo, pietà, pietà!

Babbo, pietà, pietà!

are plotted and shown in Figure 1. Refer to the video posted at https://vimeo.com/70618222 or Supplemental File A, available in the Supplementary Material, for an animation showing the eighth note lengths corresponding to Maria Callas’ performance in real time.

OPEN IN VIEWER

Figure 1.

Eighth note lengths in performances of Puccini’s O mio babbino caro.

The hyper-extension of the eighth notes corresponding to the onsets and ends of the words ‘Dio’ and ‘pietà’ (highlighted in bold in the text above and labelled in Figure 1) are points at which the singers take special liberties with the note lengths, deviating significantly from the durations indicated in the score. Observe, for example, in the final ‘pietà,’ that Kiri te Kanawa extends the penultimate syllable to more than seven times the normative length of an eighth note near the beginning of the excerpt.

In each case, the prolongation of the note length delays the expected outcome so as to communicate to, and perhaps induce in, the listener feelings of longing and poignancy. Not only is the listener expecting the second syllable to complete the word, the expectation is further piqued, in many cases, by the promise of a return following a melodic leap up or down. This is reminiscent of Meyer’s Inhibition Thesis, which states that emotion response is intensified when a tendency or response is inhibited or arrested.

In the case of the word ‘Dio,’ the composer has written into the score an elongation from what should have been a quarter note (crotchet) to a dotted minim tied to a crotchet note, setting it apart from the melodic pattern established with the falling figure for {‘struggo e mi,’ ‘tormento O.’} Despite this written-in extension, each of the three sopranos linger well beyond the already long prescribed note length, with Maria Callas topping the chart by extending the ‘o’ in ‘Di-o’ to about four times the eighth note (quaver) length at the beginning of the excerpt. This musical hyperbole introduced by the performers further lengthens the delay built into the score by the composer, just before what listeners would understand to be the inevitable outcome, the completion of the falling figure with ‘vorrei morir!’ to end the phrase on the tonic.

Not only are these timing anomalies moments of individual expression, they are also critical junctures of communication amongst the musical interactants. In the singer’s interaction with the accompanist or with an orchestra led by a conductor, this choreography of expectation must be led and shaped in a way that is natural and improvisatory, yet controlled and predictable. How does the singer design these timing hyperboles in such a way as to make the return of the pulse seem sure-footed and inevitable; indeed, how do the accompanying musicians successfully join in at the return of the train of pulses? These considerations form a significant part of the nature of interaction in ensemble performance, and are essential to the establishing of trust amongst players.

Temporal communication in music performance

Cross credits music’s relational focus for allowing a sense to emerge that ‘attitudes and motivations of each interactant are being honestly communicated to each other and are in alignment.’ Here, I take honest communication to refer to the relaying of true states of being, which can be validated by the meeting of expectations set. When expectations are consistently met, interactants can with confidence predict, adapt, and align their sound-producing behaviours, thus enhancing the sense of trust and mutual affiliation. The most obvious manifestation of this setting of expectations and coordination of behaviours can be found in temporal communication in music performance.

How does a musician communicate timing in music performance? The common pulse in ensemble music provides the framework for this communication. The push and pull of the timing of events that supposedly reside on the pulse, but fall before or after it generates what Keil calls a vital drive around the common pulse in jazz. This division of timing into the underlying pulse and deviations from the pulse works well for music with an approximately stable pulse. In some music traditions, such as music of the romantic era, the common pulse itself can exhibit extreme forms of elasticity. In such cases, smoothly varying temporal structures allow a listener to follow how the player is shaping time (a communication to the listener). They also provide a means for an interactant to conceptualize evolving time structures and predict the onset of the next event. This allows the interactants to join in at the right time, and to coordinate actions toward joint goals in ensemble playing (a communication amongst players).

Models and metaphors for performance timing form essential tools for this joint behaviour. The evolution of beat durations, or inversely the changes in the rates of the beat onsets (the tempo), is most often likened to transformations in velocity in physical motion (see Shove & Repp, 1995). If beat locations form the markers for distance in time, then tempo corresponds to velocity. The same term for velocity increase (acceleration) or decrease (deceleration) is also applied to tempo. The connection between music and motion can be traced back to the ancient Greeks; Todd (1995) has more recently formalized this relationship using mathematical equations for acceleration and deceleration. On a related front, researchers have also noted and modelled behaviours of phrase final lengthening in music (see Honing, 2003; Sundberg & Verrillo, 2003), similar to that observed in spoken language.

When tempi of phrases are mapped out over time, they form the outlines of arcs that are often modelled using simple quadratic functions. Following their earlier work that showed that runners slowing down to a stop best modelled the final ritardando (Friberg & Sundberg, 1999), in ways better than a simple quadratic function of the score position, Friberg, Sundberg and Frydén (2000) continued the locomotive analogy for music performance to map vertical force patterns from different walking and dancing footsteps to sound envelopes. These efforts represent attempts to create models that reflect the process of music communication or that mimic the human ability to predict and align to sound producing gestures.

Later, Chew, François, Liu, and Yang (2005), and Chew, Liu, and François (2006) took the locomotive analogy for music performance literally to create an automotive interface for controlling the musical parameters of tempo, loudness (linked to tempo), and articulation (operationalized as note lengths). The goal of creating the interface was to simulate an environment for active decision-making in music performance so that novices and experts alike can experience the process of music communication. The driving interface, named ESP for the Expression Synthesis Project, is shown in Figure 2. It allowed the user to accelerate and decelerate using pedals while the approaching white segments of the road’s centreline served as a visual indicator of the vehicle’s speed. Road bends corresponded to suggestions to slow down – such as where a ritardando was written into the score, at phrase endings, and at tonally complex passages – and straight sections allowed the driver to speed ahead. Please see Supplemental File B in Supplementary Material, for a video demonstration of ESP.

OPEN IN VIEWER

Figure 2.

The ESP driving interface for synthesizing natural-sounding expressive performances.

As a step towards understanding the nature of temporal communication in music performance, Cambouropoulos, Dixon, Goebl, and Widmer’s (2001) study affirmed that humans prefer tempi that vary smoothly. This is likely because smooth varying information is more predictable; however, the ability to vary tempo smoothly necessitates a degree of control over the instrument. Regardless of driving skill, the ESP system invokes a virtual radius mapping strategy (Liu, Chew, & François, 2006) to ensure tempo smoothness, a hallmark of expert performance. In this case, a motion metaphor for music performance was used to assist a human player in creating natural sounding expressive renderings of expressionless Music Instrument Digital Interface (MIDI) files. By removing the barrier of physical command over an instrument, and by overlaying the effect of smooth tempo variation, even novices can experience the fruits of expressive control in music communication.

Returning to the Puccini example of Figure 1, I have recently proposed (Chew, 2013) that the kind of extreme elongation of (score) time as exemplified by the sopranos’ eighth note plots can be modelled as a ‘tipping point,’ in a literal and physics sense. Like other motion models of music, this invokes mathematical equations of classical mechanics; unlike previous kinematics models, the tipping point analogy focuses on a transition between a moment of stasis and one of movement. Tipping points thus represent a form of temporal communication not yet captured by Meyer’s syntactically oriented nor Kiel’s vital-drive-around-a-common-pulse models.

The focus of the discussion here has been on the shaping of beat rates and beat durations (periodic events), which are critical to the process of entrainment. It is important to note that temporal structures can also be generated or reinforced by organising patterns in pitch and timbre, at varying time scales, and by coordinating behaviours, such as cueing between interactants and entrainment amongst participants, observed and studied through these features.

With an eye (or ear) on the process, the goal of creating models for temporal communication is to understand how a musician might shape a performance, and how a human interactant (whether a participating musician or an active listener) might be able to predict and align with the musician’s sound producing behaviours. These performer-centred models describe the process of music communication and serve as valuable tools that can lend rigor to the validation of hypotheses on the nature and impact of music communication in music therapy.

Revisiting Ian Cross’ music as communication

Music as conveyor of relational intent

In Cross’ characterization of the dichotomy between music and speech, speech possesses the ‘capacity to be unambiguously referential,’ and is the preferred medium for information exchange and for coordinating goal-directed behaviour (having a transactional function), while music is seen as ‘[privileging] the relational dimension to the apparent exclusion of the transactional.’

While I am in general agreement with this concept, the casual reader may misconstrue music communication as being deliberately vague and ill defined. Thus, I find it worthwhile to point out that the entrainment requirements of musical interaction impose stringent requirements that a player’s relational intent be communicated just as, if not more, directly and unambiguously in music interaction even though no words may be exchanged.

Consider the moment when the orchestra joins the soloist after a stormy cadenza in Martha Argerich’s performance of Strauss’ Burleske with the Berlin Philharmonic and Claudio Abbado conducting (1992) – see Chew (2014) for an analysis of the performance. As the soloist approaches the end of the cadenza, an operatic prolongation of the final note in the descending melodic line sequence draws out the suspense before Argerich turns to the conductor and smiles to signal for the orchestra to join in the return of the lyrical theme. Similar examples of the orchestra joining after an extensively elongated note can be found in the Puccini aria example as shown in Figure 3.

OPEN IN VIEWER

Figure 3.

Points of elongation (marked by a hump) followed by points requiring alignment (marked by arrows).

This musical handoff or joining in forms an essential part of every music communication. Its importance is most obvious in the case of communication between performers. As in Olympic athletes passing the baton in a relay race, there are ways to coordinate it successfully, gracefully, and there are many ways to err, through miscalculation or deliberately. In both the Strauss and the Puccini examples, the soloist acts as leader, and the orchestra, led by a conductor, is the follower. The leader is expected to set the pulse in a follow-able way, and the follower is expected to join in at the agreed moment.

The more clear and understandable the gesture, the easier it is for the orchestra to join in at the right time. Successful alignment requires the cooperation of all parties involved – the sender must be absolutely clear in her intentions and follow through, and the receiver must accurately decode the intentions and move to join in at the appropriate time.

Just as one can assess the skill of a composer by the quality of her transitions, one can determine if communication in music is healthy by successful musical handoffs and joining in, which are emblems of working communication amongst musical interactants. Through music interaction, navigating moments of social uncertainty is substituted by the common goal of managing musical transitions.

A stream of successfully navigated musical alignments and re-alignments establishes the sense of honest communication and a joint commitment to shared sound-producing goals. Conversely, failed handoffs can disrupt the flow of ideas, and be a cause of embarrassment. Just as entrained behaviours foster a sense of mutual affiliation and positive attitudes one interactant to another, derailment of this entrainment can lead to uncertainty and distrust.

Thus, in an ensemble setting, in order to satisfy the requirement for entrainment, interactants must convey relational intention clearly and unambiguously. Jointly navigating moments of musical transitions, where coordination is critical, not only nurtures a sense of mutual affiliation, it also exposes behavioural tendencies through the interactants’ willingness for, and nature of, cooperation. While active engagement with others through music may not reveal what one thinks (thus the floating intentionality feature of music as communication), it can demonstrate how one thinks (relational intention), exposing attitudes and motivations.

Clarity of intention also plays an important role in communication between performers and active listeners. Here, a missed handoff or joining in may not have audible consequences, such as a disruption in the flow of the music, and thus the consequences of miscommunication may appear less dire than in ensemble interaction. However, unambiguous indications of intentions and clear follow through help manage listeners’ expectations and convey a sense of authority and trustworthiness. When the intentions are clear, listeners can be more readily led to parse the structure and affect of the moulded sound as interpreted by the player. The audience’s response, whether it be rapt attention, a release of bated breath, or synchronized movement, guides the performer’s ensuing choices.

The richness of music coordination behaviours

In summary, I have argued that the entrainment required in playing together results in conveyance of relational intent deliberately and unambiguously, and that music interaction in both scripted and unscripted music provides an honest signal revealing attitudes and motivations.

As Cross noted, there is a dearth of research on music as communication, research that could potentially benefit music therapy. A modest exception to the paucity of ensemble interaction research can be found in work addressing musical interaction over distance (see for example Bartlette, Headlam, Bocko, & Velikic, 2006; Chafe, Caceres, & Gurevich, 2010; Chafe & Gurevich, 2004). In a set of networked music performance experiments that tested the impact of auditory delay on duet piano (keyboard) performance, Chew, Sawchuk, Tanoue, and Zimmermann (2005), Chew et al. (2004) and Chew, Zimmermann, et al. (2005) discovered that delaying the players’ auditory feedback from their own instruments so that it lined up with that from their partner (as shown in Figure 4) improved their capacity to set common goals, and to tolerate network latency. The added delay, while seemingly unintuitive, put the players on a common clock, allowing them to create a sense of togetherness.

OPEN IN VIEWER

Figure 4.

The Tosheff piano duo in distributed performance experiments where audio feedback from their own instruments were delayed to line up with their partner’s signal.

In more recent experiments, designed with the end goal of improving ensemble interaction over distance, Vera, Chew and Healey (2013) tested the impact of visual feedback on ensemble coordination between a violinist and a cellist. Quantitative analysis of note synchronization showed the open line of sight to be similar to the partial line of sight (silhouettes) scenario, both being significantly better than no line of sight when playing a duet comprising of long notes interspersed with long pauses. Little difference was found for all three scenarios when the test piece consisted of only long notes, where the string players had the capability to shape notes to encode metric cues – see examples from Vera and Chew (in press) reproduced in Figure 5. Each plot overlays numerous instances of intensity shapes of long notes played by the cellist and violinist, respectively. The cello expression curves plot shows a preponderance for the intensity to peak at 0.25 and 0.75 of the way through the note duration, while the violin expression curves tend to peak at 0.25, 0.5, and 0.75. The peaks occur at natural subdivisions of the note duration, and a peak at 0.75 acts as an upbeat into the next note. The interactants also used breath cues to coordinate joint actions. The metrical encoding in both intensity envelopes and the breath cues can be extracted from the sound signal, thus providing the opportunity to analyse more rigorously the coordinating behaviours between the musicians.

OPEN IN VIEWER

Figure 5.

Intensity shapes over long notes encoding metrical cues (reproduced from Vera and Chew, in press).

While it is true the research on musical interaction has focused primarily on symmetric dyadic processes, some research on asymmetric processes can be found in studies on music tutoring, in particular, remote music tuition, whether the tutor and student interact over the internet (see Duffy & Healey, 2012, 2013).

Finally, Cross mentions interesting research in neuroscience that showed common brain patterns when two people coordinate their behaviours, and that even joint listening can align individuals’ brain networks. Given the richness of coordinating behaviours inherent in musical interaction, it promises to be an interesting avenue to explore for eliciting and influencing attitudes and behaviours.