Abstract
The present study used the affective priming paradigm to understand interference and facilitation effects of cross-modal emotional interactions; specifically, the ability for five-chord progressions to affect processing efficiency of visual targets. Twenty-five-chord progressions were selected based on the degree that they fulfilled participants’ automatically formulated expectations of how each musical sequence should sound. The current study is an extension of previous research that revealed the influence of music-like stimuli on the identification of valence in emotional words. The potential of music-like stimuli to affect emotional processing, as measured by the efficiency of valence categorization, was assessed across two experiments. Experiment 1 presented word-targets, whereas Experiment 2 presented facial expressions. The processing of words and faces primed with affectively congruent chord-progressions was facilitated, whereas the processing of words primed with affectively incongruent chord-progressions was not. Incongruent pairings with faces engendered interference effects and the second experiment revealed a predictive relationship between behavioral processing speed and self-ratings of anxiety. The processing of word-targets was compared to facial expressions in the presence and absence of music. The results suggest that short musical sequences influence individuals’ emotional processing, which could inform intervention research into how to attenuate potential attention biases.
Keywords
In many domains, it is beneficial to direct research efforts toward understanding the factors that allow individuals to perform optimally. For example, facilitating the speed and efficacy of emotional processing could be beneficial in situations where a perceiver must make split-second decisions about his or her behavior. One method of studying the influence of emotion is the affective priming paradigm (APP), which presents a priming stimulus of a specific valence immediately before an emotionally valenced-target, then participants must categorize their valence as either negative or positive (Muller, Klein, & Jacobsen, 2011). According to the spreading activation theory, closely related concepts are stored in the mind as cognitive units and priming stimulates these conceptual nodes in a way that subconsciously activates an entire network of similar ideas (Collins & Loftus, 1975). The present study aims to support previous research that has proposed the ability of music to influence the emotional processing of other non-musical stimuli, such as words (Ignacio, 2016; Muller et al., 2011) and facial expressions (Kamiyama, Abla, Iwanaga, & Okanoya, 2013). Moreover, emotional processing in silence will be assessed as a basis to suggest using music as a non-invasive adjunct to existing clinical practices.
Interference and facilitation
Successfully interacting with the surrounding environment requires selective attention to focus on task-relevant stimuli, amid non-relevant stimuli. Emotional processing can be affected by cognitive interference, which occurs when noticing task-irrelevant information (Algom, Zakay, Monar, & Chajut, 2009)—this is also a process involved in the classic Stroop effect (Stroop, 1935). This type of interference has been extensively studied using semantic and affective variations of the Stroop paradigm presented in the visual and auditory modalities. In one study, reaction times (RTs) were slower when the word “high” was spoken in a low pitch versus a high pitch (Cohen & Martin, 1975). Similarly, judgments of a speaker’s sex were quicker when the speaker’s sex (e.g., female) was congruent with the word spoken (e.g., girl) when compared to incongruent pairings (Green & Barber, 1983). In a more currently relevant variation, researchers of the emotional Stroop effect observed that participants identified the colored-ink of printed emotional words significantly slower than neutral words (Dresler, Meriau, Heekeren, & Van der Meer, 2009). These Stroop-like sensory variations discussed above inspired the current investigation to assess the potential of emotion elicited by music-like stimuli to affect the processing of valenced words and facial expressions.
Cognitive penetration states that previously received information influences the perception of currently available information (Marchi & Newen, 2015). Cognitive efficiency is impacted when knowledge from past experiences anticipate events in the future. In one study on interference effects, the classification of positive facial expressions was significantly slower when a negative word was previously presented (Cothran, Larsen, Zelenski, & Prizmic, 2012). Similarly, in another study on affective priming, the identification of valence in negative or positive words was differentially affected by priming with negative or positive auditory stimuli (single-chords) that were congruent or incongruent (Sollberger, Reber, & Eckstein, 2003). Internal states, such as mood or affect, can also be considered to be another example of prior information influencing the processing of incoming information (Lovibond & Lovibond, 1995). For example, patients diagnosed with anxiety may have increased attention toward negativity and decreased attention toward positivity in their daily environment (Cooke, Chaboyer, & Hiratos, 2005). Overall, previously received affective information appears to have a considerable influence on the processing of subsequently presented stimuli.
Affective communication
Emotion is communicated through verbal and nonverbal behaviors, such as gestures, postures, and facial expressions (Posamentier & Abdi, 2003; Radice-Neumann, Zupan, Babbage, & Willer, 2007). Perception, interpretation, recall, and management of affect are required for successfully integrating various inputs of information to facilitate problem-solving proficiency, which are skills relevant to emotional processing (Mayer, Salovery, Caruso, & Sitarenios, 2003). Many have experienced situations where integrating information is difficult. For example, having a conversation with someone who is saying one thing, but whose facial expression seems to be signaling something very different can be viewed as a naturalistic instance of the cognitive interference seen in Stroop paradigms. Facial features, as well as the semantic meaning of words, assist perceivers in recognizing valence; a process that may be influenced.
Music may also communicate emotion through expectations that listeners generate about how musical sequences will sound; this indicates valence to a listener (Muller et al., 2011). Pitch expectations are examples of expectations about what is coming next in a listener’s auditory environment. Listeners experience pleasure when these pitch expectations are satisfied and disappointment when they are violated (Pearce & Wiggins, 2012). If valence in music is communicated through automatically evaluated pitch expectations, then completely removing pitch expectations would presumably remove available affective information (Ignacio, 2016); therefore, it was expected that silence would not engender differences in the processing of valence in the experimental targets.
Integrating cognition and emotion
Linear sequences, like music or speech, naturally facilitate the cognitive ability of abstraction and pattern recognition so that the mind can begin to formulate systematic hypotheses about their relationships. The degree of regularity in structured musical sequences may allow for predictions or hypotheses to be generated in the minds of listeners. In accordance with the spreading activation theory, primes that satisfy listeners’ pitch expectations, for example, would activate closely related conceptual nodes and, therefore, engender a response tendency to categorize the immediately following word-target as positive; resulting in quicker processing for congruent pairs. Likewise, expectations–unfulfilled primes would facilitate a response “for targets with the same valence as the prime” (Goerlich et al., 2012, p. 1725). It was suggested that the RT needed to accurately categorize the valence of word-targets depended on the automatic tendency to respond behaviorally in a way that is congruent to the preceding prime. Although not a direct measure, the number of incorrect categorizations made (errors) could be an index of an automatic tendency that could provide support for response competition.
One alternative to the response competition hypothesis, which suggests rivalry at the categorization level, is the automatic vigilance hypothesis (Estes & Adelman, 2008). According to the automatic vigilance hypothesis, negative words hold more attention than positive words, thereby longer RT is expected for negative words. Cognitive hierarchies have evolved to decide which information, among the mass amount of available data, to process first (Beall & Herbert, 2008). Therefore, the strongest stimulus would be most salient and attract initial attention, and result in quicker processing. The following experiments will present evidence to support the potential of music to facilitate the emotional processing of words and facial expressions.
The present study
In one variant of APP experiments, music has been validated for use as an emotional auditory stimulus (Kamiyama, Abla, Iwanaga, & Okanoya, 2013). Muller and colleagues (2011) used five-chord progressions, as opposed to single-chords (Sollberger et al., 2003), as primes to illustrate the capability of short musical excerpts to communicate affective information. Facilitated processing speed, as a result of priming, has been suggested to be due to the affective context established by an entire progression, rather than by the qualities of any single chord (Muller et al., 2011). Moreover, previous research using music in the APP revealed that neither single-chords nor single-chords repeated five times (to emulate the duration of a five-chord progression) differentially affected the processing of emotional non-musical stimuli (Ignacio, 2016). Taken together, the influence of music could be the result of emotional consequences to automatically evaluated appraisals that form in response to a rule-based structure, like linguistic syntax; therefore, five-chord progressions were used in the present study.
We hypothesize that music-like stimuli (five-chord progressions) will transmit enough affective information to participants through automatic appraisals of pitch expectations to reliably communicate valence. Specifically, when a musical sequence follows an implicitly expected pattern (i.e., pitch expectations are satisfied), a pleasurable emotional response will be engendered, and, when a musical sequence does not follow (i.e., pitch expectations are unfulfilled), a disappointed emotional response will be produced. Moreover, we hypothesized that the RT required to categorize the valence of the targets would be facilitated if their valence was congruent with musical primes, and interfered if incongruent, after controlling for the number of errors made. Any differences observed between targets (i.e., negative and positive) were expected to be absent when targets were primed with silence; consistent with the spreading activation theory of priming (Steinbeis & Koelsch, 2010).
Method
Population and recruitment
Experiment 1 gathered data from undergraduate students; a population that typically consumes mass amounts of music, as evidenced by streaming demographics in the United States (Lamere, 2014). Participants were recruited using the undergraduate psychology research participant pool, which comprised students enrolled in psychology courses at California State University, Fullerton. The experiment required participants to be of normal hearing, normal vision, and have sufficient motor function to engage in voluntary key presses; any participant who did not meet these criteria was excluded. Participation in the present experiment involved the use of in-ear headphones and typical listening conditions.
Participants
Previous research revealed small (r = –.043) to moderate (r = .575) correlations across conditions between the repeated RT measurements (Ignacio, 2016). When positive word-targets were congruently primed by chord-progressions that satisfied listeners’ pitch expectations, the RT to identify valence was facilitated with a medium effect size (
Materials
Chord-progressions
There were 20 experimental five-chord progressions selected for the present study: 10 chord-progressions that denied participants’ pitch expectations and 10 chord-progressions that satisfied their pitch expectations. The first four chords for the experimental sequences were all diatonic chord-progressions within the key of C major. The final chord was always a dominant seventh. Each chord consisted of four tones played simultaneously. Progressions shown in Figure 1 (negative) and Figure 2 (positive) from the same diatonic step were created to vary on one chord, specifically, the fifth chord. For example, the first four chords in the progressions I-3 (denied) and I-6 (denied), as well as I-2 (satisfied) and I-7 (satisfied), followed the same structure: I (tonic), IV (subdominant), V (dominant), and I (tonic). The structure of the first four chords differed across progression types (e.g., I, II, III, IV). The chord indices used in the present study (e.g., I-3, I-2, II-3, II-0; shown in Figures 1 and 2) represent the diatonic step to the fourth chord from the third chord and the chromatic step from the fourth chord to the fifth chord. There were seven different diatonic steps represented in the fourth chord and 12 chromatic steps of the fifth and final chord for a total of 84 possible combinations. For example, I-3 is a chord-progression that includes the tonic C step in the fourth chord before the final C dominant seventh chord (C, E, G, Bb), which was raised to the third chromatic step (Eb, G, Bb, Db). The first four chords, for both negative and positive progressions, were the same and followed one of the following musical structures depending on the fourth chord: tonic, subdominant, dominant, tonic (I-IV-V-I); tonic, tonic, subdominant, supertonic (I-I-IV-II); tonic, dominant, submediant, mediant (I-V-VI-III); tonic, submediant, mediant, subdominant (I-VI-III-IV); and tonic, subdominant, dominant, submediant (I-IV-V-VI). The musical stimuli were short musical sequences (two measures long) comprised of five chords played over 3 s (to listen to the stimuli, visit: dandreignacio.wix.com/musicalbeauty).
The musical notation for the experimental chord-progressions that denied participants’ pitch expectations is shown above. The first four chords are all diatonic chord-progressions within the C major key signature. All progressions within the same diatonic step followed the same structure (listed below each chord): I (tonic), II (supertonic), III (mediant), IV (subdominant), V (dominant), and VI (submediant). The last chord was always a seventh and differed for each progression depending on which chromatic step was represented (12 possible steps).
The musical notation for the experimental chord-progressions that satisfied participants’ pitch expectations is shown above. The first four chords are all diatonic chord-progressions within the C major key signature. All progressions within the same diatonic step followed the same structure (listed below each chord): I (tonic), II (supertonic), III (mediant), IV (subdominant), V (dominant), and VI (submediant). The last chord was always a seventh and differed for each progression depending on which chromatic step was represented (12 possible steps).
Participants in the pre-study (N = 30) were asked to rate their perceived beauty on a 5-point scale (1 = not beautiful; 5 = beautiful) for each of the 84 sequences (Table 1 for ratings). Participants were explicitly instructed to base their ratings on how well each progression fit their expectations of how each should sound, so participants presumably rated expected sequences as beautiful. Muller and colleagues (2011) suggested that beauty is innately positive; therefore, beautiful progressions were considered positive. Since the first four chords in the progressions were the same across negative and positive sequences, participants’ ratings of beauty were presumably due to the manipulations in the fifth and final chord. All primes were played on a piano and presented at the same standardized volume (60 dB).
Central tendency measures for the 10 negative and 10 positive five-chord progression primes.
From pre-study survey (N = 30).
Word-targets
Experiment 1 of the present study primed negative and positive word-targets with the experimental chord-progressions that either denied or satisfied participants’ pitch expectations. The word-targets used were selected from previous research using the APP (Steinbeis & Koelsch, 2010); the frequency and familiarity of the words were not controlled, but they were “not found to differ in terms of the abstractness or the concreteness of their content” (p. 607). Sixty English words of a negative valence (e.g., hate, disgust, grieve) and 60 English words of a positive valence (e.g., love, peace, beauty) were presented in the singular, noun-form (Table 2). Verbs, adjectives, and proper nouns were excluded. In addition, words longer than two syllables were also excluded.
List of the 120 word-targets used in Experiment 1, separated by valence.
Procedure
Both experiments of the present study were computer automated using PsychoPy, a stimulus presentation software package, to ensure standardized administration (Peirce, 2009). The experiment was installed on 12 Windows desktop computers, which were randomized to present the three blocks in different orders. Participants were randomly seated at one of the computers, approximately 50 cm from 22″ monitors, with an experimental packet in front of each station. The first page of the experimental packet was an informed consent sheet, which was signed and collected before beginning. The task of the participants was to categorize the valence of the presented word-targets using the keyboard’s arrow keys: the “left” arrow key indicated negative valence and the “right” arrow key indicated positive valence.
The experiment was computer-based, so the timing of each aspect was specifically programmed. The duration of each five-chord progression was 3,000 ms: the first four chords played for 500 ms each, and the final chord played for 1,000 ms. At the beginning of each trial, a 1 × 1″ white-colored fixation point was presented at the center of the computer screen during the first three chords (1,500 ms). The fixation point changed to a red color during the fourth chord (500 ms) to indicate that the final chord and valenced-target were impending. The word-targets were presented shortly after the onset of the fifth and final chord. Word-targets were 4 × 2″ white-colored text in Times New Roman font on a gray background at the center of the screen. The optimal stimulus onset asynchrony, or the time in between the onset of the final musical chord-prime and the onset of the target, is 200 ms for the APP (Steinbeis & Koelsch, 2010). The final chord and target overlapped for 800 ms before simultaneously disappearing. A blank screen and a period of silence followed for 2,000 ms, such that each trial was exactly 5,000 ms. Participant responses were accepted from the onset of the target until the end of the trial (response acceptance time of 2,800 ms), and the response did not end the trial earlier.
Each prime (i.e., unfulfilled, satisfied, and silence) and target (i.e., negative and positive) pairing constituted a single trial. The experimental condition included trials primed with either an unfulfilled or satisfied chord-progression, and the control condition consisted of trials completed in silence. There were a total of 120 randomized trials divided evenly across three counter-balanced blocks. Twenty negative targets and twenty positive targets were pseudorandomized with one of three different auditory primes (i.e., unfulfilled, satisfied, and silence) such that each participant experienced exactly 20 trials of each condition (120 trials).
Data cleaning and statistical assumptions
Experiment 1 utilized emotional words as targets. A 3 × 2 (Prime [Silence, Unfulfilled, Satisfied] by Target [Negative, Positive]) within-subjects design was used to assess participants’ emotional processing speed (RT scores) to categorize the valence of targets. First, the averages of all four conditions were subtracted from the grand mean to obtain an adjustment factor, then the adjustment factor was added to the original scores for both RT and errors; this procedure eliminated the between-subjects variance from the error estimates of the within-subjects measurements illustrated in Figures 1 and 2 (Field, 2009). Change scores from silence were calculated to correct the significant negative skew in the RT scores (Zskewness < –3.29); the assumption of normality was met after this adjustment. There was no significant difference between negative (M = 934 ms, SD = 56 ms) and positive words (M = 922 ms, SD = 53 ms) in silence, t(75) = 1.38, p = .171 (Mdifference = 0.01, 95% confidence interval [CI] [–0.01, 0.03]).
Statistically speaking, there was no difference in the silence condition, so scores from control trials of negative and positive targets were subtracted from corresponding experimental trials, such that the resulting change scores would reflect the influence of the experimental primes (i.e., unfulfilled and satisfied) on valence categorization. Negative change scores reflected facilitation effects; while positive change scores reflected interference effects relative to silence (Beall & Herbert, 2008). One-sample t-tests were conducted to compare the change scores to zero (no change). There was significant facilitation for congruent music pairings of negative words, t(75) = –4.06, p < .001, and positive words, t(75) = –3.65, p < .001, relative to silence. There were no significant effects for incongruent pairings (ps > .05).
The accuracy of participants’ valence judgments (incorrect categorizations) was controlled in the present analysis to further assess the emotional processing efficiency of targets using an analysis of covariance (ANCOVA). The assumption of independence between the covariate and the treatment effects was assessed with a 2 × 2 (Prime [Unfulfilled, Satisfied] by Word-Target [Negative, Positive]) within-subjects ANOVA. The prime main effect, F(1, 75) = 3.90, p = .052, and the word-target main effect, F(1, 75) = 0.28, p = .600, were non-significant influences on the covariate, suggesting that the covariate main effects are independent from the within-subjects factors. The assumption of sphericity (Greenhouse–Geisser ε = 1.00) was within acceptable ranges.
Results
A Bonferroni alpha adjustment was applied to account for the two hypotheses in Experiment 1 (α Bonferroni = .025): there would be no processing differences between negative and positive word-targets in silence, and there would be a reliable prime and word interaction effect on processing of word-targets after controlling for errors in the experimental conditions. To assess the influence of five-chord progressions relative to silence on the categorization RT of negative and positive emotional word-targets, a 2 × 2 (Prime [Unfulfilled, Satisfied] by Word-Target [Negative, Positive]) within-subjects ANCOVA was conducted while controlling for the influence of total errors made. There was a non-significant main effect of Prime before, F(1, 74) = 0.31, p = .580, and after, F(1, 74) = 0.57, p = .451, controlling for the covariate. The main effect of Word was also non-significant, F(1, 74) = 0.84, p = .364, but the two-way interaction with the covariate was significant, F(1, 74) = 6.64, p = .012. The RT change scores of negative words (M = –22.10 ms, SE = 8.02) were statistically longer than positive words (M = –24.39 ms, SE = 6.97) after consideration of the covariate. The two-way Prime by Word interaction effect was significant on the RT change scores, F(1, 74) = 13.71, p < .001 (Figure 3). However, this interaction was rendered non-significant after consideration of the covariate in the three-way effect, F(1, 74) = 0.34, p = .564.
The Prime × Word interaction effect for RT change scores in the first experiment is shown above. The error bars reflect values that have been adjusted to eliminate between-subjects variance (negative words = square end-caps on the left; positive words = round end-caps to the right). The data above reflect change scores relative to silence (see y-axis). Negative change scores reflect facilitation effects.
Discussion
The type of prime and the type of word-target main effects were not significant, individually. Corroborating previous research, congruent word–music pairings were categorized faster than incongruent pairings in the current results (Goerlich et al., 2012; Muller et al., 2011). The automatic vigilance hypothesis was supported as the RT scores were longer for negative words relative to positive words (Estes & Adelman, 2008), although this pattern was only revealed after controlling for incorrect response tendencies (errors), supporting the response competition hypothesis (Goerlich et al., 2012). Our hypothesis was partially supported as the interaction between the two factors was significant but not after consideration of the covariate. In other words, the affective priming effect was not evident after the number of errors were controlled, suggesting that the differences observed in the RT scores were dependent on the pattern of errors made. The mean patterns suggest that as one condition slows down, more errors are made relative to faster conditions (see Table 3 for untransformed scores). The assumption analysis ruled out a mediation effect (Hayes et al., 2016); however, the errors made (covariate) might still be a moderator on the relationship between the within-subjects factors and RT scores. For example, if the experimental musical sequences improve RT only for participants with fewer errors made, relative to participants with more errors, and that silence has no effect on RT scores, then moderation may still be a possibility to investigate in future research.
Below are descriptive statistics for the average RT (ms) and error scores for the six conditions in both experiments of the main study.
Numbers reflect values before being converted to change scores and the numbers have been adjusted to eliminate between-subjects variances for all of the participants in the main study (n for Experiment 1 = 76; n for Experiment 2 = 99). RT: reaction time; CI: confidence interval.
Experiment 2
People perceive and process many affective facial expressions on any given day, and accurately interpreting affect through these nonverbal cues must occur rapidly to navigate social situations successfully. In Experiment 2, we were interested in investigating if the categorization of facial expressions would be differentially influenced by pitch expectations evaluated to either be satisfied or unfulfilled. It was hypothesized that valence derived from short musical sequences would be able to modify attention; therefore, an affective measurement was included in Experiment 2. All four of the conditions in Experiment 1 produced negative change scores, so the present discussion is limited only to the comparison of facilitation effects in response to manipulated music-like stimuli, and not interference effects, relative to silence. Since facial expressions are processed more automatically than word-targets (Beall & Herbert, 2008), they may constitute a more salient stimulus that might be strong enough of an attentional influence to cross the threshold between facilitation and interference.
Participants
Experiment 2 assessed the potential influence of pitch expectations on the emotional processing of facial expressions, as opposed to the word-targets used in Experiment 1. If a medium effect size was revealed for the influence of pitch expectations on the emotional processing of word-targets, it is reasonable to expect a similarly sized effect (small to moderate) for the processing of facial expressions (
Materials
Chord-progressions
The five-chord experimental progressions used in Experiment 2 were the same progressions used in Experiment 1, which were selected based on the results of a pre-study survey.
Facial expressions
In Experiment 2, experimental chord-progressions primed the appearance of negative and positive facial expressions. Positive faces have very distinct and specific features (e.g., smiling), whereas negative facial expressions have many different overlapping aspects that increase the difficulty in discriminating subtle negative emotions (Ribeiro & Fearon, 2010). Therefore, to prepare equivalent facial expressions in respect to emotional recognition, a survey like the one distributed to prepare the experimental chord-progressions was also used to select the facial stimuli. There are 43 models (18 females and 25 males) each posing in negative, positive, and neutral facial expressions in the NimStim facial stimulus set (Tottenham, Tanaka, & Leon, 2009). Any models who did not have a closed-mouth negative and positive expression were excluded (n = 2). This online survey asked participants to rate a total of 82 facial expressions on a 5-point Likert-type-like scale from 1 (negative) to 5 (positive).
Measures
Since internal states may affect the processing of subsequently presented stimuli, the Depression Anxiety and Stress Scale 21 (DASS-21), which is a more parsimonious version of the original 42-item measure, was used to measure affect in Experiment 2 (Lovibond & Lovibond, 1995). The DASS-21 is proposed to measure three dimensions: depression (low positive affect), anxiety (physiological hyperarousal), and stress (negative affect; Brown, Chorpita, Korotitsch, & Barlow, 1997). Test takers provide responses on a 4-point Likert-type scale about the severity that they experienced an emotional symptom over the past week from did not apply (0) to applied very much or most of the time (3; Lovibond & Lovibond, 1995). Higher scores indicated greater symptom severity. Considering that the DASS-21 is a brief measure of emotional states, and not emotional traits, the internal reliability has been revealed to be acceptable for the Depression (α = .88), Anxiety (α = .82), and Stress (α = .90) subscales (Henry & Crawford, 2005).
Procedure
Similar to the previous experiment, participants in Experiment 2 saw two types of faces (i.e., negative, positive) paired with three different auditory conditions (i.e., expectations–unfulfilled, expectations–satisfied, and silence) for a total of six repeated conditions (e.g., expectations–unfulfilled primes paired with negative faces, expectations–satisfied primes paired with positive faces). The rapid presentation of the stimuli would also approximate daily-life conditions, especially with faces, as spontaneous affective facial expressions must be processed quickly (Maurage, Campanella, Philippot, Martin, & de Timary, 2008). Before the final demographic questionnaire in Experiment 2, participants were asked to self-report their level of agreement over the past week on the items of the DASS-21 (e.g., “I was aware of dryness of my mouth”). Standardized mechanical pencils, in-ear headphones, and experimental packets were provided to the participants.
Data cleaning and statistical assumptions
The experiment was a 3 × 2 (Prime [Silence, Unfulfilled, Satisfied] by Target [Negative, Positive]) within-subjects design with 120 trials per participant. The same procedure in Experiment 1 was used to eliminate between-subjects variance in Experiment 2 (Field, 2009). Also similar to Experiment 1, change scores were calculated to remove the significant negative skew in the RT scores (Zskewness < –3.29). After the adjustment, the assumptions of normality and sphericity (Greenhouse–Geisser ε = 1.00) were met. There was a non-significant difference between the RT scores of negative faces (M = 831 ms, SD = 76 ms) and positive faces (M = 839 ms, SD = 86 ms), t(98) = 0.67, p = .502 (Mdifference = –0.01 [–0.03, 0.02]). Therefore, RT scores in the control condition (silence) were subtracted from the experimental conditions (unfulfilled, satisfied) to create change scores for further analysis. One-sample t-tests revealed no significant facilitation or interference effects for either the congruent, t(98) = 1.17, p = .246, or incongruent, t(98) = –0.26, p = .793, face–music pairings relative to silence. A 2 × 2 (Prime [Unfulfilled, Satisfied] by Face [Negative, Positive]) within-subjects ANOVA revealed no significant main effects of Prime, F(1, 98) = 3.26, p = .074, and of Face, F(1, 98) = 0.57, p = .528, on errors, suggesting the independence between the treatment effect (within-subjects factors) and covariate assumption is met.
Results
The alpha criterion was adjusted to account for the three hypotheses in Experiment 2 (Bonferroni α = .017): there would be no processing differences between negative and positive facial expressions in silence; there would be a reliable prime and face interaction effect on processing of facial expressions after controlling for errors in the experimental conditions; and that anxiety scores would significantly predict RT scores relative to depression and stress as measured by the DASS-21. A Prime (two levels: Unfulfilled, Satisfied) × Face (two levels: Negative, Positive) within-measures ANCOVA assessed the impact of five-chord progressions on RT change scores while controlling for error change scores. The main effects of prime, F(1, 97) = 0.28, p = .600, and face, F(1, 97) = 0.20, p = .888, were non-significant before and after consideration of the covariate. The two-way Prime by Face interaction effect, F(1, 97) = 4.86, p = .030, was not significant (α = .017), but the three-way (Prime × Face × Errors) interaction effect on RT, F(1, 97) = 7.72, p = .010 (Figure 4), was significant. Unfulfilled primes paired with negative faces (M = –3.53 ms, SE = 12.21, 95% CI [–27.76, 20.71]) and the satisfied primes paired with positive faces (M = –3.23, SE = 12.01 [–27.06, 20.61]), both engendered negative change scores and illustrated significant facilitation effects for congruent pairings.
The Prime × Face interaction effect for the average RT change scores for the experimental conditions in the second experiment are shown above. The error bars reflect values that have been adjusted to eliminate between-subjects variance (negative faces = square end-caps on the left; positive faces = round end-caps to the right). The data above reflect change scores relative to silence (see y-axis). Negative change scores reflect facilitation effects; while positive change scores reflect interference effects.
The attention bias to threat theory of anxiety disorders was assessed through two multiple linear regressions conducted on the three DASS-21 subscales. The criterion was anxiety scores, as measured by the DASS-21, and the predictor variables were the RT scores from the three conditions of negative (threat) faces. RT scores before being transformed to change scores were used so that the predictive value of silence could be compared to the experimental primes. The first model regressed the anxiety subscale of the DASS-21 on the RT scores from the three conditions of negative faces (e.g., negative faces primed with silence, negative faces primed with unfulfilled chord-progressions, negative faces primed with satisfied chord-progressions). A significant regression equation was found, F(3, 92) = 4.95, p = .003, with an R2 = .139. The analysis revealed that only negative facial expressions primed with unfulfilled chord-progressions significantly predicted participants’ anxiety scores, β = .399, t(95) = 3.85, p < .001 (B = .043 [.021, .064]). Negative faces primed with silence, β = .149, t(95) = 1.43, p = .156 (B = .016 [–.006, .038]), or when primed with satisfied chord-progressions, β = .056, t(95) = 0.57, p = .572 (B = .006 [–.014, .025]), did not significantly predict anxiety scores. To further support the concurrent validity of the current findings, the other subscales of the DASS-21 were analyzed. The second multiple regression regressed the RT scores from the negative faces primed with unfulfilled chord-progressions on the depression, anxiety, and stress subscales of the DASS-21. Neither the depression, β = –.044, t(94) = –0.33, p = .745 (B = –.405 [–2.87, 2.06]), or stress, β = –.081, t(94) = –0.53, p = .596 (B = –.700 [–3.3, 1.91]), subscales were significant.
Discussion
There was a non-significant main effect of faces on RT change scores in the current study, which does not corroborate Nelson, Jackson, Amir, and Hajcak (2015); however, this study did not include any individuals with affective disorder at the clinical level. The results of Experiment 2 support the claim that short musical sequences can significantly facilitate or interfere with the emotional processing of facial expressions, depending on whether the pairings are congruent or incongruent on the valence dimension. Emotionally congruent face–music pairs were categorized faster than incongruent pairs, which corroborates previous research (Kamiyama et al., 2013). There were negative change scores (facilitation effects) observed for congruent pairings of Prime and Face, and positive change scores (interference effects) for incongruent pairings, although none of the effects were statistically different from zero.
The musical primes appear to induce acute emotional states, possibly through automatically evaluated pitch expectations that influence emotional processing. Behavioral RT indicators significantly predicted anxiety ratings. Specifically, there was a direct relationship between the RT scores required to categorize threatening (negative) faces primed with unfulfilled chord-progressions and self-reported anxiety ratings. This significant relationship was attenuated when threatening faces were primed with chord-progressions that satisfied participants’ pitch expectations. There were dissociations with the depression and stress subscales of the DASS-21; these were not significantly predicted by RT scores of congruently negative trials, which support divergent validity for these behavioral scores as indicators for anxiety in an attention bias framework. However, it is important to note that the direction of this relationship could not be specified. The first regression predicted anxiety scores on the DASS-21 using RT scores from the congruent-negative condition among the other conditions. Then, the second regression predicted RT scores from this congruent-negative condition using anxiety scores on the DASS-21 among the two other subscales (i.e., depression and stress). In other words, participants who categorized threatening faces quicker may have been more prone to self-reporting higher ratings of anxiety, or participants who self-reported higher ratings of anxiety might have been more sensitive to processing threatening faces.
General discussion
The current study contributes to previous research that experimentally manipulated auditory stimuli and measured congruence effects on the processing of other stimuli. Facial expressions were processed faster than words with a large effect size (d = 2.69). Corroborating previous research on pitch expectations, we found no processing differences between negative and positive targets in silence since there were no pitch expectations to communicate influencing affective information (Ignacio, 2016; Pearce & Wiggins, 2012). Incongruent pairings of Prime and Target (both words and faces) resulted in longer RTs to categorize the target’s valence, which may support the automatic vigilance hypothesis. Incongruence may signal negativity or threat, hence resulting in more attention held by the experimental stimuli. The affective priming effect was revealed in Experiment 1 and Experiment 2; however, there was a dissociation between the two. The affective priming effect was not present in Experiment 1 after controlling for errors, while it was only present after controlling for errors in Experiment 2 (α Bonferroni = .017). Since word-targets have a more specific linguistic value than facial expressions, this may explain the differential impact of errors observed in Experiment 1.
In terms of facilitation and interference, there were differential results observed across Experiment 1 and Experiment 2. The results of Experiment 1 revealed negative change scores for all conditions, although only congruent pairings produced significant facilitation relative to silence (no change). In Experiment 2, the interference effects were larger for the incongruent pairings when visually compared to the facilitation effects of the congruent pairings; however, none of the effects were statistically different from silence. When the musical stimuli that presumably prompted congruent response tendencies were removed (silence) in both experiments, there was no difference in the processing of negative and positive targets. The current results support the affective influence of pitch expectations.
The automatic vigilance hypothesis corroborates the response competition hypothesis since controlling for conflict at the categorization level attenuated the significant difference between prime and word-targets in Experiment 1. Goerlich and colleagues (2012) propose longer RTs for words of a negative valence, and longer RTs were observed for incongruent pairings of prime and target. Incongruence may also represent negativity and could therefore potentially influence the behavioral findings by holding attention more, thus resulting in longer RTs. The findings support both the response competition and automatic vigilance hypotheses; therefore, future research is warranted to disentangle the effects.
Music could potentially be useful in treating the attention biases or emotional processing deficits in clinical populations (Chafin, Roy, Gerin, & Christenfeld, 2004; Neumann, Malec, & Hammond, 2017; Wong, Dahm, & Ponsford, 2013). According to Beck and Haigh (2014), simple behavioral manipulations can successfully modify attention; for example, treating social anxiety by looking up when entering a crowded room. Changing attention repeatedly over time may adjust an individual’s focus in the long term. As proposed by Hitchcock, Golden, Werner-Seidler, Kuyken, and Dalgleish (2018), cognitive training programs may result in improvements to clinical disturbances. For example, the dot-probe task is used to treat attention biases to threatening faces (Nelson et al., 2015; Shechner et al., 2014). Music could prime participants during the identification of the dot in the task of the paradigm to facilitate the categorization of non-threatening stimuli.
The current findings suggest that behavioral paradigms currently used to treat affective complications could be enhanced with the use of specifically selected musical stimuli designed to satisfy patients’ pitch expectations. The methods of the current study may be a non-intrusive, behavioral indicator for the attentional factors involved in the assignment of affective salience. Moreover, modified attention and facilitated emotional processing can be used to enhance learning outcomes and performance in daily activities. For example, quickly and accurately perceiving emotion in the environment better informs behavioral responses, and better-informed reactions result in more thoughtful interpersonal exchanges (Kornreich et al., 2012).
Emotions are communicated to perceivers through charged-words, affective facial expressions, and musical sequences via automatically evaluated pitch expectations regarding how they should sound. The current results support the proposition that the perception of music is integrated into the emotional processing and judgment of visual stimuli such as words and human faces (Ignacio, 2016; Kamiyama et al., 2013). The affective information derived from one stimulus is also able to interact with the affective information derived from another stimulus, even when the stimuli are from different sensory modalities; like a cross-modal Stroop-like effect. Emotional processing can be influenced within seconds through modulating attentional resources that perceivers append to their environments.
