Abstract
That music elicits motor signals has been suggested as the key to understanding how emotion may be experienced from it. While previous studies have demonstrated that music both induces movement and affects ongoing motor activities, little is known about the effect of its emotional content on neural-motor representation and how such an effect may be modified by concurrent movement. To fill this gap, we asked participants to tap their right index finger following flashes of a dot (movement) or just to count the flashes silently (no-movement) while listening to happy versus emotionally neutral music, when EEG was recorded. Results showed that happy music induced greater mu (8–13 Hz) suppression than neutral music in the no-movement but not the movement condition. For beta oscillations (16–24 Hz), happy music induced greater suppression than neutral music irrespective of concurrent movement. These findings suggest a close association between music emotion and motor representation at a neural level, supporting theories stipulating that emotion in music is experienced through motor representation, such as embodied music cognition and the Shared Affective Motion Experience model.
Music moves us literally. While listening to it, we tap our fingers, nod our heads, and dance. The tendency to display spontaneous movement upon hearing music appears to be common across musical styles and cultures, suggestive of a shared, general cognitive-motoric structure underlying music and movement for emotional expression (Sievers et al., 2013). To understand the close relationship between music and movement, previous studies have focused on how musical features such as tempo, beat, and rhythm, and some individual-level factors such as musical training may moderate the effect of music on motor responses. When listening to music, people not only move along with the tempo and beat (Janata et al., 2012) but also show brain-level neural responses representative of motor activities even when they are asked to refrain from moving (Zatorre et al., 2007). These effects are stronger for musically trained than nonmusically trained individuals (Haslinger et al., 2005).
More often, in everyday life, music provides a background for different ongoing activities such as shopping and exercising. How may music affect the performance of such activities? In an early study, Milliman (1982) showed that the tempo of background music played in a store affected in-store traffic flow speed. Kampfe et al. (2010) concluded in their meta-analysis that background music could exert both facilitatory and inhibitory effects depending on the specific ongoing activities in question. For instance, background music might disturb reading and memory processes but facilitate performance in sports. At a neural-motor level, a fast tempo and high intensity of pretask music are associated with neural activation integral to visual perception, allocation of attention, and motor control during later reactive performance (Bishop et al., 2014). Applying transcranial magnetic stimulation (TMS) to the primary motor cortex to artificially induce motor responses while music was playing, Stupacher et al. (2013) showed that high-groove music strengthened the motor responses more than low-groove music and noise, especially when the TMS pulses were applied on the beat. Hence, music not only induces movement giving rise to motor responses (Janata et al., 2012; Zatorre et al., 2007) but also moderates on-going motor activities both at a behavioral (Milliman, 1982) and neural level (Stupacher et al., 2013).
But what does the close link between music and movement tell us about the nature of music? What functional role does movement play in the representation of music emotion? The embodied music cognition theory and Shared Affective Motion Experience (SAME) model may provide answers to these questions. Embodied music cognition stipulates that music is perceived through the gesturing and movement implied in or suggested by the flow of the music toward the derivation of emotion from music (Leman & Maes, 2014; Siu & Cheung, 2016, 2017). In other words, the emotional content of music is extracted through perceiving the movement or motion implied in the music. The SAME model suggests that music evokes motor responses through the mirror neuron system (MNS), which is associated with the limbic system and thus the whole circuit may be implicated in emotion activation (Molnar-Szakacs & Overy, 2006). The MNS supports the listener’s spontaneous imitation of the movement implied in the music, which gives rise to emotion activation because of the MNS’s connection with the limbic system. For instance, McGarry et al. (2015) reported that emotional compared with nonemotional judgment of sung melodic intervals elicited enhanced neural responses from the MNS corresponding to both self-executed action and observation of others’ action. This finding supports the hypothetical link between music-induced emotional and movement responses assumed in embodied music cognition and the SAME model.
Embodied music cognition and the SAME model both suggest that music evokes emotion via its implied movement and the listener’s spontaneous imitation or reproduction of such movement at a behavioral or neural level. In this connection, McGarry et al. (2015) provided evidence for an association between emotional judgment of musical intervals and its impact on motor responses. In the present research, we ask a further question: What effect does musical-emotional valence, for example, happy versus neutral music, have on neural-motor responses? If we feel the emotion behind music through the development of a neural-motor representation as assumed in embodied music cognition and the SAME model, then this representation should be closely associated with what we are supposed to feel in the music. For instance, a piece of music that is supposed to give rise to a happy feeling should produce neural-motor responses that are quite different from those produced by a piece of sad music. To date, most studies have examined how music-level characteristics such as rhythm and tempo, and musical expertise may modulate the music effect on motor representation without looking at music emotion as a unique factor. The current study fills this gap by examining how emotionally up-lifting (i.e., happy) and neutral music may be differentially related to neural responses that are associated with motor representation. According to embodied music cognition and the SAME model, music emotion originates from motor activation, and therefore, we hypothesize a music emotion effect on neural-motor responses:
Hypothesis 1: Emotionally “strong” happy music is more likely to produce neural responses that are associated with motor representation than emotionally “flat” neutral music.
The second question asked in the present research is: How may the hypothesized music emotion effect be modified by requiring the participants to make explicit movements when listening to the music? Past studies have shown that in addition to inducing motor responses, music also interacts with ongoing activities and neural-motor responses elicited by other stimuli (Milliman, 1982; Stupacher et al., 2013). Although these acitvities and responses are independent of, that is, not induced by, the music in question, they obviously share the same neural basis with the music-induced responses because they all belong to the same broader motor representation, and are therefore expected to interfere with each other in some way. Hence, the effect of music emotion as predicted in the first hypothesis may be made less observable by concurrent movement because of interference: Concurrent movement may give rise to additional, extraneous neural-motor activation that overshadows the music emotion effect or decreases the observable difference in neural-motor activation between emotionally strong and neutral music. This hypothesized interaction between concurrent movement and music emotion rests on the assumed link between music emotion and motor representation, and thus provides further evidence for the SAME model and embodied music cognition.
Hypothesis 2: The difference in neural-motor responses between happy and neutral music is smaller with than without concurrent movement.
We used mu (~10 Hz) and beta (~20 Hz) brain oscillations as the neural-motor responses to test the two hypotheses. Brain activities are dominated by oscillations between populations of neurons; oscillations at different frequencies reflect the activities of different neuronal populations and serve different psychological functions. Both mu and beta oscillations originate from motor areas, showing reduced power relative to rest when executing and observing movements (Babiloni et al., 2002; Orgs et al., 2008; Pfurtscheller et al., 2006; Pfurtscheller & Neuper, 1994; Quandt et al., 2012). Mu rhythms are desynchronized and their powers are attenuated when the individual is engaged in movements (Gastaut, 1952) and when observing others’ movements (e.g., Muthukumaraswamy & Johnson, 2004). Similar mu suppression or desynchronization is observed even when the individual is only asked to imagine movements (Pfurtscheller et al., 2008). Mu suppression in electroencephalogram (EEG) has also been found to covariate with BOLD activities in typical MNS regions responsible for action observation and execution (Arnstein et al., 2011). Hence, mu suppression, which indicates neuronal desynchronization, has been considered a reliable index of neural-motor activation. Beta oscillations are similarly related to movements, observed movements, and imagined movements. Some studies have demonstrated that beta suppression reflects motor activation in a wider range of situations than mu suppression. For instance, while mu suppression is observable only for consciously perceived movement, beta suppression is associated with both consciously and unconsciously perceived movement (Simon & Mukamel, 2016). Another study has demonstrated that individuals with or without rich motor experience show similar beta suppression in response to observed movement, whereas mu suppression is more dependent on the individual’s motor experience (Cannon et al., 2014). On the basis of these findings, we considered the possibility that mu and beta suppressions may capture the hypothesized effects to different degrees.
Method
Participants
Eighteen college students (11 females) aged 18–26 years (mean age = 20.6 years) participated in this study after giving written informed consent. Number of participants was determined in accordance with a power analysis using the G*Power software (Erdfelder et al., 1996), indicating that a sample of 18 participants was needed to detect differences with a repeated-measures ANOVA with power (1 − β) set at .80, alpha at .05, and an estimated of .06. The study was approved by the ethics panel of the Faculty of Social Science, The Chinese University of Hong Kong. All participants were right-handed with normal or corrected-to-normal vision and no neurological or psychiatric disorders. Participants were asked to perform some simple cognitive-motoric tasks while EEG was recorded. They were also asked to report their music experience in terms of number of years learning music and/or learning musical instruments. Each participant received US$19.20 as an incentive.
Design, material, and procedure
Each trial lasted 15 s; this 15-s duration was divided into ten 1.5-s segments. A dot lasting 833 ms was flashed at the screen center at the beginnings of eight or nine randomly chosen segments, leaving two or one blank segment in which no flash of dot occurred. In other words, ISIs were 1.5 s most of the time and the stimulus (dot) was on for 833 ms and off for 677 ms in each of the eight or nine randomly chosen time segments out of the 10 segments in a trial (see Figure 1(a)). Participants were asked to either tap their right index finger to each flash in the movement condition or count the flashes silently without any body movement in the no-movement condition, while listening to a 15-s excerpt of emotionally neutral or happy music through a pair of headphones. These 15-s excerpts were selected from a pilot study involving 60 college students who were asked to listen to and rate the emotions of a large pool of excerpts extracted from western classical music pieces from the Baroque period (e.g., Tomaso Albinoni’s oboe concertos) up to the late Romantic period around the turn of the 20th century (e.g., Gustav Mahler’s Fifth Symphony). One excerpt was extracted at a random point in each piece. Emotion rating was done on a 15-point Likert-type scale, with −7 = very sad, 0 = neutral, and +7 = very happy. The 40 excerpts with averaged ratings closest to 0 were selected for use in the present study as neutral music, and the 40 with ratings closest to +7 were used as happy music. Under the no-movement condition, participants were asked to decide if there were eight or nine flashes by pressing the “J” and “K” key, respectively, on a keyboard right after each trial.
(a) The Experimental Paradigm and (b) Mood Evaluations.
The experimental design included two within-subject factors: emotion (happy vs. neutral music) and movement (movement vs. no-movement). Each participant received 80 trials blocked by condition (happy-movement, happy-no-movement, neutral-movement, neutral-no-movement) and thus each block contained 20 trials. The presentation of blocks was randomized. After each block, an evaluation bar was shown and the participants were asked to report their mood on a 7-point Likert-type scale from 1 (neutral mood) to 7 (happy mood). The music excerpts were never repeated for any participants.
EEG/ECG recording and preprocessing
EEG was recorded with Ag/AgCl electrodes at 64 scalp locations using the 10–20 system. Two pairs of bipolar electrodes were placed at the outer canthi of both eyes and above and below one eye for horizontal and vertical eye movements, respectively. The online reference and ground electrodes were placed at the left mastoid and AFz, respectively. Electrode impedance was kept below 5,000 Ohms. EEGs were sampled at 500 Hz using a Neuroscan Synamps 2 amplifier (Compumedics USA, EL Paso, TX, USA). The digitized EEG data were re-referenced to the average of all electrodes and bandpass filtered at 0.5–70 Hz. The filtered data were segmented into epochs of 18 s including the 3 s prior to the onset of music. All segments of EEG data exceeding 100 μV were rejected as artifact and ocular artifacts were corrected using ICA. On average, 7.72 (SD = 8.38, ranging from 1 to 35) trials were excluded due to artifact rejection. At least 10 trials were included in the analysis for each condition.
Time-frequency power spectra were computed by Fourier transforms on a moving window of 250 ms (i.e., 125 data points) for every 10 ms with Hanning taper in each artifact-free EEG epochs by using the Fieldtrip software (Oostenveld et al., 2011). The time-frequency spectra were normalized by the powers in the baseline period (–2 to 0 s) using the formula 10 × log10(Signal/Baseline), then averaged for each condition, channel, and participant. Averaged mu and beta powers were calculated from 8 to 13 Hz in the 250–550 ms time window and from 16 to 24 Hz in the 150–500 ms time window, respectively, at locations C3, Cz, and C4 after the onset of each dot flash. This strategy of analysis was adopted from previous studies (e.g., Hao et al., 2020; Moreno et al., 2013) and was also based on visual inspection of the timings and frequency bands of the motor signals.
Results
Mood
A 2 × 2 repeated-measures analysis of covariance (ANCOVA) was performed on the mood ratings collected after each block of trials with the independent factors emotion (happy vs. neutral music) and movement (movement vs. no-movement), and music experience (the number of years learning music and/or musical instruments) as the covariate. The emotion main effect was significant, F(1, 16) = 5.504, p = .032,
Mu and beta powers
A 2 × 2 × 2 × 3 repeated-measures ANCOVA with the independent factors emotion (happy vs. neutral music), movement (movement vs. no-movement), frequency (mu vs. beta), and electrodes (C3, Cz, and C4) was performed on the oscillation power data. The covariate was music experience. A significant 4-way interaction, F(2, 32) = 4.954, p = .013,
A 2 × 2 repeated-measures ANCOVA with emotion and movement as independent factors and music experience as a covariate was performed on C3 mu power (Figure 2(b)). The interaction was significant, F(1, 16) = 5.338, p = .035,
Time-Frequency Results: (a) Time-Frequency Spectra at C3 Electrode and (b) Mean Mu and Beta Powers.
For C3 beta power, a 2 × 2 repeated-measures ANCOVA with music experience as a covariate showed a significant main effect of emotion, F(1, 16) = 4.628, p = .047,
To examine how much the oscillational response pattern might be due to acoustic differences between the happy and neutral music, information on the loudness, tempo, and pitch of the 80 music pieces was extracted using the mirrms, mirtempo, and mircentroid functions, respectively, from MIRToolbox and compared between the two emotion conditions. These three acoustic dimensions were examined because previous studies had shown that they were among the features that were the most associated with music emotions (e.g., Juslin & Laukka, 2003). On average, the happy pieces, M = 0.866, 1 SD = 0.008, were louder than the neutral pieces, M = 0.082, SD = 0.006, t(78) = 2.588, p = .012. Average tempo, M = 112.513 beats per minute (BPM), SD = 31.059, and pitch of the happy pieces, M = 470.038 Hz, 2 SD = 319.802, however, did not differ from those of the neutral pieces, tempo: M = 114.616 BPM, SD = 31.217, t(78) = 0.302, p = .763; pitch: M = 433.152 Hz, SD = 357.943, t(78) = −0.486, p = .628.
To further examine the contribution of acoustic features to the main pattern of results, loudness, tempo, and pitch were used as covariates in a 2 × 2 × 2 by-item ANCOVA using the 80 music excerpts as cases for analysis, frequency and movement as the within-subject factors, emotion as the between-subject factor, and oscillational responses at C3 as the dependent measure. The three-way interaction remained significant after the effects of these acoustic features were removed, F(1, 75) = 4.629, p = .035,
To examine how much the present results might reflect the peripheral process of visual attention involved in an observation task (e.g., Debnath et al., 2019; Pineda et al., 2013), we analyzed oscillational power data from the Oz electrode (occipital alpha) using a 2 × 2 × 2 repeated-measures ANCOVA with emotion, movement, and frequency as independent factors and music experience as a covariate. Only the main effect of frequency was significant F(1, 16) = 4.973, p = .041,
Discussion
In this study, we examined the effect of emotional valence in music on neural-motor responses as reflected in mu and beta powers. We also compared the neural-motor responses elicited against a no-movement background to those accompanied by concurrent movement, that is, finger tapping. EEG results revealed that mu suppression was modulated by music emotion under the no-movement condition but not under the movement condition. On the other hand, greater beta suppression was found during happy than neutral music listening both with and without concurrent movement.
At a general level, the music emotion effect hypothesized in the present study manifested itself in both mu and beta suppression, providing support for embodied music cognition and the SAME model. Both accounts assume a close link between music emotion and movement because the emotional content of music is derived through the listener’s perception of movement implied in the music or reproduction of such movement at a behavioral or neural level. Our finding that listening to happy music generally induced greater beta suppression than listening to neutral music, as predicted in Hypothesis 1, suggests a close link between motor representation and perceived music emotion. We take this as direct evidence for the main tenet of embodied music cognition and the SAME model, that emotion in music is experienced through motor representation. Some may argue that the different patterns of motor responses while listening to happy versus neutral music could be driven by acoustic differences between the two types of music, which may have nothing to do with the emotional content of the music. However, we factored in differences in loudness, tempo, and pitch between the happy and neutral excerpts in a follow-up analysis and found the same pattern of results. The happy and neutral excerpts we used differed only in loudness. These suggest that acoustic contrasts between musical pieces may not fully explain the music effect on motor responses, and we think the emotional valence of the music does play a role in it.
The other finding, that happy music elicited greater mu suppression than neutral music only under the no-movement condition, confirms hypothesis 2 and suggests that concurrent movement interferes with and overshadows the music emotion effect. Such interference is attributed to the fact that music emotion and concurrent movement contribute to the same broader neural-motor representation when the individual is listening to the music. This result provides additional evidence for an association between motor representation and perceived music emotion, supporting embodied music cognition and the SAME model. Mu suppression is more sensitive to changes under the test condition than beta suppression in that it reflects motor activation only in specific situations (e.g., Cannon et al., 2014; Simon & Mukamel, 2016). Hence, the hypothesized music emotion effect was picked up by mu suppression only when concurrent movement, or interference, was absent. Nevertheless, note that the emotion × movement interaction was at p = .035 for mu suppression, which was not entirely incomparable with the corresponding marginal interaction at p = .064 for beta suppression. The difference between the two powers in indicating the hypothesized music emotion effect, therefore, may not be overly interpreted.
We use mu and beta powers to index neural-motor activation because suppressions in these signals have been reported during self-executed action and observation of actions produced by others (Hari, 2006; Kühn et al., 2006; Moreno et al., 2013; Muthukumaraswamy & Johnson, 2004). Mu suppression has been observed in the auditory domain as well, in that it is elicited by both action execution and action-associated sounds produced by others (Pineda et al., 2013). The signal has also been found to correlate with BOLD activities in MNS regions typically responsible for action observation and execution (Arnstein et al., 2011). These findings suggest that mu suppression may be understood as an index of mirror neuron responses, coding self- and others-generated actions in both the visual and auditory modalities. Beta oscillations are generally observed in different cognitive-motoric tasks in a variety of situations and are thus considered to reflect the general maintenance of the overall sensorimotor state (Engel & Fries, 2010). Nevertheless, one should be cautious about interpreting mu and beta suppressions exclusively as strengths of motor signals. This is because attenuated mu and beta powers capture neuronal desynchronization which can be a result of different processes. For instance, desynchronization could occur because of reduced entrainment to the beat of the music when the individual is asked to perform concurrent movements that are out of phase with the music. This alternative interpretation is consistent with the insignificant trend that concurrent movement seemed to increase power attenuation generally, ps = .204 and .605 for mu and beta, respectively; see Figure 2(b), although it may not explain the obtained music emotion effect that happy music was associated with greater power suppression than neutral music (hypothesis 1). Hence, the possibility that mu and beta suppressions may indicate reduced entrainment makes hypothesis 2 speculative: Concurrent movement may reduce the difference between happy and neutral music via interference in entrainment, not interference in motoric activation as hypothesized.
Several issues may be addressed in future research given the present findings. First, as discussed, mu and beta suppressions capture neuronal desynchronization which may indicate processes other than motoric activation. It is therefore important for future research on neural-motor activation in music to complement EEG recordings with other types of neuroimaging data, such as BOLD activities in functional magnetic resonance imaging (fMRI), to increase the internal validity of the design and interpretability of the findings. This applies especially to future investigations of the interaction between concurrent movement and music emotion (Hypothesis 2) (Milliman, 1982; Stupacher et al., 2013; see also Kampfe et al., 2010). Second, in addition to emotional valence, what roles do some other music dimensions, including tempo, rhythm, and emotional intensity, play in the motor representation behind music? Third, future research may examine the effects of different neural signals on emotional experiences from listening to music in the most direct way, by using brain stimulation techniques. This method provides the most stringent test of causality.
To conclude, the present study showed the main effect of music emotion and an interaction between music emotion and concurrent movement on beta and mu oscillations, respectively. These findings provide evidence for a close relation between music emotion and motor representation at a brain level which may be taken as empirical support for the embodied music cognition view and the SAME model, both stipulating that the emotional content of music is interpreted via motor representation.
Footnotes
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study is funded by a CRAC Grant (no. 04A16), The Education University of Hong Kong, awarded to Him Cheung.
Open practices statement
Data and materials for the experiments reported here are available on request; none of the experiments was preregistered.
