Abstract
Western music improvisers learn to realize chord symbols in multiple ways according to functional classifications, and practice making substitutions of these realizations accordingly. In contrast, Western classical musicians read music that specifies particular realizations so that they rarely make such functional substitutions. We advance a theory that experienced improvisers more readily perceive musical structures with similar functions as sounding similar by virtue of this categorization, and that this categorization partly enables the ability to improvise by allowing performers to make substitutions. We tested this with an oddball task while recording electroencephalography. In the task, a repeating standard chord progression was randomly interspersed with two kinds of deviants: one in which one of the chords was substituted with a chord from the same functional class (“exemplar deviant”), and one in which the substitution was outside the functional class (“function deviant”). For function compared to exemplar deviants, participants with more improvisation experience responded more quickly and accurately and had more discriminable N2c and P3b ERP components. Further, N2c and P3b signal discriminability predicted participants’ behavioral ability to discriminate the stimuli. Our research contributes to the cognitive science of creativity through identifying differences in knowledge organization as a trait that facilitates creative ability.
Many human behaviors can be characterized as improvisatory: past knowledge allows us, in real time, to understand and respond to a wide range of possible situations that we may never have exactly encountered before. Improvisation may seem esoteric—the term most often being applied to musicians—but the concept has been used to describe a wide variety of human activities (Lewis & Piekut, 2016). For the purposes of cognitive-scientific research, a better definition of the construct and according methods to measure and explain its mechanisms would contribute to a better understanding of the creative mind.
Western musical practices, more so than other improvisatory behaviors like conversing and navigating, allow improvisatory ability to be isolated. People who know a language are almost always able to generate novel utterances in that language. However, there are many musicians with considerable expertise and deep understanding of musical styles who cannot generate new “utterances,” while others with similar levels of training (albeit different kinds of training) are able to improvise. To understand this difference is to understand what ways of learning and ways of knowing facilitate the ability to use that knowledge creatively.
Past neuroscientific studies on musical improvisation (Bengtsson, Csíkszentmihályi, & Ullén, 2007; Berkowitz & Ansari, 2008, 2010; de Manzano & Ullén, 2012; Donnay, Rankin, Lopez-Gonzalez, Jiradejvong, & Limb, 2014; Limb & Braun, 2008; Liu et al., 2012) have been concerned with what brain states underlie the spontaneous generation of novel motor behavior. Other studies have taken a formalistic perspective (i.e., examine the structural characteristics of the sounds produced by performers) and make inferences about the generative algorithms they use (Johnson-Laird, 1991, 2002), or psychological and cognitive processes they use (Frieler, Pfleiderer, Zaddach, & Abeßer, 2016; Goldman, 2013; Hargreaves, 2012; Norgaard, 2011, 2014; Norgaard, Spencer, & Montiel, 2013; Pfleiderer & Frieler, 2010; Pressing, 1988, 1998). Essentially, these studies isolate and describe improvisatory states and processes and describe the different kinds of knowledge improvisers use.
Another approach is to examine traits of experienced versus inexperienced improvisers that arise through differences in training and experience. Differences in sensitivity to changes in various musical features (Tervaniemi, Janhunen, Kruck, Putkinen, & Huotilainen, 2015; Vuust et al., 2005; Vuust, Brattico, Seppänen, Näätänen, & Tervaniemi, 2012), spatial reasoning (Harris & de Jong, 2015), and functional connectivity in the brain (Pinho, de Manzano, Fransson, Eriksson, & Ullén, 2014) have been observed between these groups. The present study builds on these between-group comparison approaches by providing another hypothesis as to how improvisers’ and non-improvisers’ cognitive processes may differ, and, importantly, the theoretical framework to help explain why such a difference would help enable the ability to improvise.
Differences in the use of musical structures
Western improvising musicians in tonal contexts often use notation that does not fully specify which notes to play (e.g., figured bass and lead sheets); rather, symbols specify classes of harmonies and melodies. The performer chooses between a set of possible realizations within the functional category specified by the symbol. By function, we mean that different exemplars can be used similarly according to both explicit music theoretical rules or through implicit knowledge of the regularities of a style. For an improviser, such an ability to switch fluently between exemplars in performance would enable, for instance, harmonizing a melody in a new way, or soloing over a familiar harmonic progression with a performance partner who is using different exemplars of that progression. By contrast, practitioners of Western classical art music—who rarely improvise harmonic and melodic aspects of music—typically use notation that fully specifies which notes to play. In a given piece, a classical musician could, in principle, substitute certain musical structures for other similarly functioning ones, but they rarely do this as it contradicts the practice and tradition of classical music, which treats music as fixed works (Goehr, 1992).
Hypothetically, for improvisers, two exemplars within the same functional category would be perceived as more similar than a chord outside the functional category. For less experienced improvisers, this difference should be less pronounced since these kinds of exemplar substitutions are rarely made in practice, thus not reinforcing musical function as a categorization criterion. This research question is formulated with tonal improvisation in mind, although similar ideas may apply to other traditions of improvisation (see Discussion).
The current study
We tested these differences in categorization using an oddball paradigm while recording electroencephalography (EEG). Our musical stimuli allowed us to compare the relative salience of different kinds of harmonic changes we termed “exemplar deviants” (changes within a functional category) and “function deviants” (changes between functional categories) in a sample of musicians with varying levels of improvisatory experience. For example, if the standard chord progression was ii–V–I, an exemplar deviant would be ii–V6–I and a function deviant would be ii–IV6–I. In addition to harmonic differences, such changes also necessarily change the intervals between specific notes, which could also change the relative salience of each kind of deviant; we controlled for these changes (see Methods). Improvisation experience should lead to categorizing exemplar deviants as being more similar to standards than function deviants are; behaviorally, function deviants should thus be identified more quickly and accurately than exemplar deviants, dependent on the amount of improvisation experience.
EEG data allowed us to assess the neurophysiological activity that discriminated each kind of deviant stimulus from the standard. Specifically, in an oddball task, if detection of a deviant requires classification (e.g., detecting deviant male names among standard female names) rather than detecting physical differences in stimuli only, an N2c component is evoked (Ritter, Simson, & Vaughan, 1983). This requires a controlled process rather than a purely stimulus driven one. Although other N2 subcomponents—namely the mismatch negativity (MMN) and the N2b—have similar latencies and topographies (for auditory stimuli), we interpret our results in terms of the N2c due to the nature of the task. Thus, for those with more improvisation experience, function deviants should constitute a greater violation of expectation, and this should be predicted by greater discriminability of the N2c signal.
N2 components often occur in conjunction with P3 components. The P3 is elicited when participants detect relatively rare deviant stimuli in oddball tasks. It is also divided into subcomponents, namely the P3a and P3b (Polich, 2007). The P3a is elicited by the detection of deviants in oddball paradigms when a response is not required while the P3b, which is later in latency and more posterior topographically, occurs when a response is required by deviant detection. We thus expected our task to evoke the P3b. Stimulus probability modulates P3b activity, with less probable stimuli evoking stronger signals. However, it is not the objective probability of the stimuli; rather, how a participant categorizes stimuli can change the perceived subjective probability and thus also modulate the P3b (Johnson & Donchin, 1980). In our task, the exemplar and function deviants are close to equiprobable in occurrence, so the objective stimulus probability alone should not modulate the P3b activity, but those with more improvisation experience are hypothesized to categorize exemplars as similar to standards, making the function deviants appear relatively less probable, and relatively easier to detect. Because the subjective probability is lower, we expect those with more improvisation experience to show more discriminable P3b activity for function relative to exemplar deviants.
Overall, we predicted that behavioral differences in discriminability would be predicted by the discriminability of N2c and P3b activity. These components are thus analyzed as neurophysiological indexes of the information processing leading to participants’ behavioral performance.
Methods
Participants
To take part in the study, participants had to have formal training (or significant professional experience) and play regularly, and those who had improvisation experience had to have formal training in improvisation. This allowed us to selectively include participants with a wide range of improvisatory ability. All participants reported having normal hearing, no history of neurological disorders, and proficiency on a pitched instrument. Refer to Table 1 for additional participant demographic information. The study took approximately 2 hrs and participants were remunerated for participating. Our university’s Institutional Review Board approved all procedures and informed consent was obtained before the start of each experiment.
Participant demographic information.
Experimental design
Forty-one participants qualified and took part in the study. We excluded one participant’s data for leaving the experiment before completion, leaving 40 participants (M = 25.28 years old, SD = 5.52; 24 male).
Stimuli
Participants filled in a questionnaire with standard demographic information on age, gender, handedness—as assessed by the Edinburgh Handedness Inventory (EHI; Oldfield, 1971)—absolute pitch (self-reported yes or no), years of formal study on each instrument they played, years of those that included formal improvisation training, and self-identification as an improviser (on a scale from 1–7) and following Pinho et al. (2014), we also collected information on the number of hours per week participants played their primary instruments in the age ranges of under 11, between 12 and 17, and 18 and older, as well as the number of those hours that were specifically spent improvising. We instructed participants that this includes hours spent both practicing and performing and that they should be as accurate as possible. Few participants asked for clarification about what counts as improvisation experience, but in those cases we encouraged participants to use their own judgment as to what constituted improvisation. For our final analysis, we decided only to examine how many hours per week participants played since the age of 18 because many participants started their training after the younger age range cutoffs and some could not accurately recall how much they played when they were younger. Further, we decided that this measure was a more accurate index of how active the musicians were at the time of the study. We thus obtained an index of improvisatory activity and experience (impHours), and an index of non-improvisatory activity and experience (nonImpHours), which was calculated by subtracting the impHours from the total hours indicated on our questionnaire.
Because participants listed all instruments on which they have formal training, we were able to assess how many years of piano training each participant had. We included this exploratory variable, pianoYears, because we suspected that training specifically on the piano might affect participants’ judgments on the auditory task owing to the piano timbre used for our stimuli.
Participants also briefly described their improvisation activities on the questionnaire. All participants were skilled in a Western tonal improvisatory style. Jazz was by far the most represented type of improvisatory music, although some participants also indicated experience with pop, classical, and folk improvisatory music in addition to jazz. We did not analyze our data with respect to this information, but further studies could examine differences between types of improvisatory experience (see Discussion).
We initially considered grouping participants into “improviser” and “non-improviser” categories on the basis of their self-identification as an improviser (4 or greater) and years of formal training (1 or more). However, we ultimately decided to use the impHours variable as an index of improvisatory experience because impHours provides a continuous measure, which allows us to show a continuous effect of the role of improvisation experience. The partial regression plots below retain the improviser/non-improviser grouping for visualization purposes.
The task required listening to different chord progressions of three chords each. The chord progressions were all either ii–IV–I, ii–V–I, ii–IV6–I, or ii–V6–I. A given chord progression could function as a standard or either type of deviant depending on the block type (see Figure 1). Each chord lasted 400 ms, with 400 ms of silence after the last chord (1600 ms total per trial). These 1600 ms stimuli were presented continuously, one after the other. There were 180 chord progressions per block, and eight blocks total in the experiment (two of each block type). There was also one practice block. Stimuli were created with a custom Python script using the pyMidi module to generate MIDI files, which were then converted to audio using a grand piano sound in GarageBand (Apple Inc., Cupertino, CA). The same stimuli were used for every participant, but the block order was randomized. A short break was taken after each block.
Stimuli used in the experiment. Each block type has three kinds of progressions. The different block types control for differences in contour (e.g., intervals a–c). However, note “e” is a fourth away from the expected standard note “d,” while note “f” is a third away from what would be expected in the standard. A larger deviation here could also potentially affect salience. According to this, stimuli can also be categorized into three groups (deviations of thirds, fourths, and fifths). We test the effect of this with a separate analysis that uses this grouping (see main text).
Each block’s standard chord progression had a corresponding exemplar deviant and function deviant, each interspersed pseudorandomly with 7.5% probability (15% total deviants) such that a deviant was always followed by at least three standards, and each block always started with at least eight standard stimuli. Because deviants were dispersed probabilistically, the blocks did not have identical numbers of deviants (the combined set of stimuli from the four block types totaled 111 function deviants and 109 exemplar deviants). Deviant stimuli differed from the standard stimuli in the middle chord of the progression. In the exemplar deviant, the middle chord was substituted with a chord of the same musical function but a different inversion (e.g., a IV chord vs. a IV6 chord in which the same notes are “stacked” in a different order). In the function deviant, the middle chord was substituted with a chord that was built on a different scale degree (e.g., a V chord instead of a IV chord), constituting a different functional class. Note that the middle chords of each kind of deviant have the same inversion. Further, each chord progression was presented in a random key with the constraint that the same key was never presented twice in a row. The middle chord of each kind of deviant would thus be indistinguishable from each other in isolation; the type of deviant can only be determined by assessing the middle chord’s relationship to the first and third chords of the progression, i.e., by assessing its function. In improvisational practice, exemplar deviants would be much more likely to be substituted for the standard than the function deviants.
In a given block type, the three types of chord progressions also differ in terms of the intervals between the chords and the contour of the top notes. Contour could potentially change the relative salience of the two kinds of deviants (Fujioka, Trainor, Ross, Kakigi, & Pantev, 2004). We thus included four block types such that each chord progression could function as each condition (standard, exemplar deviant, and function deviant), allowing us to control for this variable and the number of common tones between chords (refer to Figure 1).
Finally, in any given block, the deviant conditions also differ from the standard in terms of the intervals of the top notes of the middle chord relative to the standard; this is similar to the previous point about intervals, but not the same. For example, if a participant listens to the top notes of the chords and expects a descending second but hears an ascending third, the deviation in the expectation of the interval (in this case, a fourth) might also affect the chord progression’s salience (Saarinen, Paavilainen, Schöger, Tervaniemi, & Näätänen, 1992). A larger interval difference could potentially be more salient in itself, regardless of functional categorization. Figure 1 illustrates this alternative categorization. We account for this with a separate analysis that compares the salience of each type of deviant progression based on these intervals rather than their grouping as function or exemplar deviants.
Procedure
After completing the questionnaire, participants completed the oddball task while their EEG was recorded. Participants listened to the chord progressions over Bose in-ear noise cancelling headphones (Model QC-20) and were instructed to press the spacebar on a computer keyboard as quickly as possible if they heard a chord progression that differed “in any way” from what they perceived to be the most common chord progression within the context of that block. They were told that the block would begin with several of the standard stimuli to help them judge what was the most common one local to that particular block (since the standard stimulus and corresponding types of deviants differed between blocks). They were also told to be accurate, i.e., to only press the spacebar if they are confident there is a difference. We collected reaction time and accuracy data in addition to EEG data. Importantly, participants were not told that there would be different types of deviant progressions, and were thus not encouraged to classify the stimuli in any particular way.
During the oddball task, EEG signal was acquired through 64 active scalp electrodes arranged according to the 10-20 system, sampled at 2048 Hz using a BioSemi Active Two AD Box ADC-12 amplifier (BioSemi, The Netherlands) in an electrostatically shielded room (ETS-Lindgren, Glendale Heights, IL, USA). The stimuli audio signal was also recorded on a separate channel to ensure proper alignment of the event markers, which were added offline after data collection. Separate computers were used to collect the EEG data and present the audio stimuli.
Behavioral data analysis
For each participant, we first computed the median reaction time of correct trials regardless of deviant type as well as the overall accuracy (total number correctly identified out of total number of deviants) of each participant. There was a statistically significant Pearson correlation between reaction time and accuracy values of the 40 participants, rs(38) = –.39, p = .012. Faster participants were also more accurate. Both of these values assess a participant’s behavioral performance, so we captured information from both metrics by combining them into the Inverse Efficiency Score (IES), which is simply the reaction time divided by the accuracy (Townsend and Ashby, 1978, 1983). We then calculated the median RTs and accuracies for the individual deviant conditions separately and calculated an IES value for each that represented their behavioral performance for each deviant condition. Because the IES is a ratio variable, relatively small changes in low accuracy values can disproportionately affect the overall metric. To attenuate this effect, we log-transformed the IES values. Finally, for each participant, we subtracted the log-transformed IES value for function deviants from the value for exemplar deviants, yielding a single value, termed logIES, that we used as a dependent variable in statistical models described below. A higher logIES value means that the participant performed faster and more accurately for the function deviants relative to the exemplar deviants.
EEG preprocessing
EEG data were preprocessed using EEGLAB (Delorme & Makeig, 2004). Event markers for each chord were added offline using the audio channel data. Bad channels were interpolated using EEGLAB’s inbuilt function, resulting in an average of 3.16 ± 0.25 channels interpolated per block. Data were down-sampled from 2048 Hz to 64 Hz, band-pass filtered between 0.50 Hz and 30 Hz and re-referenced to the average activity of all electrodes. Eye movement artifacts were identified and rejected by eye using independent components analysis (ICA). Data from standard trials (not including false alarms) and deviant trials (only including hits) were epoched into 2000 ms epochs, ranging from 400 ms before the second chord of the progression to 1600 ms after it. In these epochs, 0 ms corresponds to the onset of the middle chord because that is the first point in time when a participant can possibly identify the deviants. The baseline was removed using the latency range from 100 ms before the onset of the middle chord. We excluded deviant epochs that had reaction times less than 200 ms from the onset of chord 2 as we considered these to be spurious. We used EEGLAB’s functions to reject epochs based on extreme values in the EEG signal (outside of −250 to 250 µV) and based on probability (outside of 5 SD within channel, and across all channels). This left us with a set of exemplar deviant epochs (68.35 ± 3.95 per participant), function deviant epochs (78.25 ± 3.06 per participant) and standard deviant epochs (994.70 ± 23.51 per participant).
Linear discriminant analysis of EEG
We used linear discriminant analysis (LDA) to determine the EEG activity that best discriminated each participant’s set of exemplar deviants and function deviants from the standard epochs (correct trials only). For full details on how this is calculated and the advantages of this method over traditional ERP analysis, please see the Supplementary Materials online.
In short, the analysis provides a measure (the AUC value) of how discriminable each set of deviant epochs is from the set of standard epochs. AUC values are calculated over the course of the epoch showing the degree of discriminability at different latencies. Further, the analysis provides a forward model, which shows what combination of electrode activity best discriminates the sets of epochs at different times. This pattern of activity coupled with the latency information can be interpreted in similar terms as ERP analyses (Parra, Spence, Gerson, & Sajda, 2005).
Results
Identifying regions of single-trial discriminability
We conducted a two-tailed paired sample t-test (df = 39) between the exemplar and function conditions for each point of the area under the curve (AUC) curves (see Figure 2). We used a conservative alpha value of .01 for this test only (the others in our study use the standard .05 value). The t-tests revealed two fully contiguous regions where the curves differed significantly: an early range from 214 ms to 293 ms, t(39) = 3.08, p = .004, on average in this region, and a later range from 419 ms to 576 ms, t(39) = 4.48, p < .001, on average in this region. We considered this contiguity and occurrence in time windows of hypothetical interest (corresponding to N2 and P3 latencies) to be evidence against type I errors and thus did not further correct for multiple comparisons. We then computed the average forward model over the time points in these two time ranges (see Figure 3), which corresponds to the electrode activity that discriminates each type of deviant epoch from the standard epochs at those time ranges. The forward model corresponding to the earlier region has a frontocentral negativity that matches the latency and topography parameters of an auditory N2c ERP component, whereas the forward model for the later region has a posterior positivity that matches the latency and topography parameters of a P3b component. The topographies and latencies were very similar for function and exemplar deviants.
AUC values calculated at sliding time windows over the epochs. A high AUC value means the sets of that type of deviant were more discriminable from standard epochs at that latency. The shaded area represents one standard error across participants in each direction. Green lines indicate points at which a paired t-test shows a significant difference between the curves (alpha = 0.01). Black lines show where the sound of each chord in the progression occurred. We did not consider the single window outside the two contiguous regions because it was temporally isolated and we thus assumed it reached significance by chance.
Forward models corresponding to the two significant time regions identified in Figure 2. This can be interpreted as the electrode activity that discriminates each type of deviant from the standard chord progression activity. The exemplar and function conditions were not appreciably different in their topographies: in both, the early region corresponds to the latency and topography of the N2c ERP component and the later component corresponds to the latency and topography of the P3b component.
This difference can be interpreted as a main effect of condition—the function deviant has more discriminable N2c and P3b activity than the exemplar deviant—but to assess the relationship to the other variables of interest, for each participant, we averaged the AUC values in these two regions for each type of deviant and subtracted the exemplar value from the function value, yielding two values per participant that we termed EarlyAUC and LateAUC. A higher value for these variables means that the function deviants are more discriminable than the exemplar deviants on the basis of the corresponding forward model activity (N2c and P3b component activity respectively). We used these two values as dependent variables in our statistical modeling.
Behavioral and neural discriminability of deviant types
We assessed the relationship between the three dependent variables logIES, EarlyAUC, and LateAUC, and the predictor variables impHours, nonImpHours, age, EHI score (from the handedness inventory), pianoYears, and the grouping variable absolute pitch (yes or no). We first conducted ANCOVAs to check for an effect of the grouping variable, absolute pitch, using type III sum of squares (because only eight participants had perfect pitch, leading to unequal group sizes). As this factor was not of experimental interest for our particular study, we wanted to ensure it was not interacting with the other continuous predictor variables in our models.
For the logIES and EarlyAUC variables, the ANCOVA showed no significant main effects or interactions with the absolute pitch categorical variable. However, in the LateAUC model, absolute pitch showed a significant interaction with nonImpHours, F(1, 28) = 4.45, p = .044, and pianoYears, F(1, 28) = 6.03, p < .020. This means that our covariates of interest affect people with and without absolute pitch differently. Absolute pitch has various effects on perceptual processing, and in particular has been linked to differences in P3 activity (Hantz, Crummer, Wayman, Walton, & Frisina, 1992), and, since we found that it was violating an assumption of ANCOVA modeling and was not a variable of hypothetical interest for the purposes of our study, we excluded perfect pitch participants (n = 8, 5 of whom were clustered into the non-improvisers group) from the LateAUC multiple regression models described below. For the sake of completion, we report the results of every multiple regression model both with and without the absolute pitch participants, although it is important to note that, strictly speaking, it is only necessary, statistically, to exclude these participants for the model in which there was a significant interaction in the ANCOVA (the LateAUC model).
We next performed a series of multiple regression models with the same independent and dependent variables as for the ANCOVAs, but without the grouping variable of absolute pitch. The overall statistics for the models are shown in Table 2, and the statistics for the individual regressors are shown in Table 3. The logIES model was significant overall, as was the LateAUC model. The impHours variable significantly predicted the logIES variable, β = 0.68, p < .001. Those with more impHours performed relatively faster and more accurately for function deviants than for exemplar deviants. For the LateAUC model, after excluding participants with absolute pitch, impHours is a significant predictor, β = 0.50, p = 0.008, showing more discriminability for function compared to exemplar deviants for those with more improvisation experience. Interestingly, pianoYears also significantly predicted the LateAUC variable, β = –0.33, p = .041. Those with more piano training showed less discriminability between function and exemplar deviants in the LateAUC measure. The EarlyAUC model was not significant overall, although the effect of impHours itself was marginally significant, β = 0.36, p = .064. The results of the multiple linear regressions are visualized in partial regression plots shown in Figure 4.
Overall statistics for our multiple regression models.
p < .05; **p < .01; ***p < .001.
Complete statistics with individual regressors for each our multiple regression models, both with and without the participants with absolute pitch included.
p < .05; **p < .01; ***p < .001.
Partial regression plots for the main analysis. Note that the logIES and EarlyAUC plots include participants with absolute pitch while the LateAUC plots do not due to the statistical violation detected by the ANCOVA (see main text). These plots show the partial contribution of each predictor variable to the overall linear models. Plots for EHI Score and age are not shown because neither of them showed significant effects and are not hypothetical variables of interest. All variables were standardized. Improviser and non-improviser group identity is distinguished for visualization only and was not included as a factor in our models. EHI: Edinburgh Handedness Inventory.
Finally, we tested the Pearson correlations between each of the dependent variables. logIES correlated with EarlyAUC, r(38) = .47, p = .002, and with LateAUC, r(38) = .41, p = .008, showing that behavioral performance correlates with N2c and P3b activity. EarlyAUC and LateAUC also significantly correlate with each other, r(38) = .36, p = .021, indicating that greater N2c discriminability corresponds to greater P3b discriminability.
Interval analysis
As mentioned above, stimuli can also be categorized according to the intervallic distance from the top note of the middle chord in each kind of deviant to the top note of the middle chord in the standard. We wanted to ensure the effects we found based on the function and exemplar stimuli categorizations were not being driven by this acoustic feature of the stimuli. We performed a similar analysis as with the main categorization of the stimuli. These models showed that larger intervals were not predicting greater discriminability in either the behavioral or neural data, preserving the validity of the findings from our main analysis. Please refer to the supplementary materials for more details of this alternative analysis.
Discussion
In this study, we examined the effect of improvisational experience on the categorization of musical structures by testing how salient different kinds of deviants were in an oddball task. We hypothesized that those with more improvisational training would perceive exemplar deviants (chord progressions within a functional category) to be less salient relative to function deviants (chord progressions outside of a functional category) because of our theory that improvisers categorize musical structures based on function. This hypothesis was supported by both our behavioral and neural data. Those with more improvisation experience were better able to identify function deviants than exemplar deviants. Importantly, nonImpHours was not a predictive variable: it is specifically improvisation experience that predicts the differences.
The correlation between the behavioral measures to each of the neural measures lends consistency to our argument and allows us to tie in our explanation of this difference between these groups to the broader neuroscientific literature on categorization. Previous studies have found MMN and P3a differences between improvisers and classical musicians (Tervaniemi, Janhunen, Kruck, Putkinen & Huotilainen, 2015; Vuust et al., 2005, 2012). The N2c activity involved in other classification tasks and the way subjective probability modulates the P3b also evidently applies to our musical stimuli and can be leveraged to explain differences in how more experienced improvisers perceive and categorize stimuli: classification and categorization are indeed the relevant cognitive processes at play here.
More importantly, our study helps describe how improvisers perceive differently with specific regard to harmonic function. Thinking in terms of function may arise from improvisatory pedagogies and may aid the ability to improvise. Being able to “reach for” several possible realizations of functionally appropriate musical structures would aid the ability to improvise (cf. Sudnow, 1978). More experienced improvisers perceive different exemplars of a functional class as being similar, which provides evidence that their knowledge of structures is organized in this kind of way, and further research could explore how this is manifested in the neural instantiations of the representations of these structures.
Scientific observations of improvisers do not necessitate having participants actually improvise as they would outside the laboratory. Improvisation experience, as we have shown here, is associated with particular characteristics of how people perceive and categorize sounds. Studying this perception and categorization is an indirect way to study improvisation abilities, but nevertheless advances an understanding of how people improvise, as is framed by our theoretical context outlined above. This link back to ecological practice should not be assumed, but we argue here that the research question is grounded well enough in an understanding of musical practice that in our case the link is strong.
Extrapolating our theories to other genres and traditions of improvisation is not trivial, but we believe it is possible. Improvisers in other traditions may also categorize different musical structures according to function (be it music-theoretical function or another kind of function), generalizing characteristics of specific exemplars within that category. Of course, to show how viable this is would require further theoretical inquiry and elaboration, and any experiments in other styles would rely on that foundation.
Explicit instructions can modulate deviant detection (Sussman, Winkler, Huotilainen, Ritter, & Näätänen, 2002), even in early perceptual processes indexed by the MMN. Our data suggest that participants did indeed categorize differently, but how explicit that categorization effort was remains unclear. Two interpretations with respect to our data are possible, and perhaps interrelated. First, those with more improvisation training may consciously choose to classify by function, which could worsen their detection of exemplar deviants. In debriefing interviews, few participants reported this, nor did they report noticing the different types of deviants. Further, we did not instruct participants to listen to any particular features of the stimuli to accomplish the task. Nevertheless, this would still need to be more conclusively ruled out. Another interpretation is that there is something different about improvisers’ representations of musical structures that would predispose them to categorize this way, such that even if they tried to actively attend to other features, functional classes would still influence their perception. Future studies could specifically manipulate this variable by giving different categorization instructions to see if improvisers are still predisposed to hear function despite trying to attend to other characteristics of the stimuli.
A representational similarity analysis (RSA) study (Kriegeskorte, Mur, & Bandettini, 2008) could also build on our findings. This study assessed categorization on the basis of perceptual similarity. Given that our definition of musical function is tied to how a particular structure is used, the large and growing body of work on perception-action coupling in music (Maes, Leman, Palmer, & Wanderley, 2014; Novembre & Keller, 2014) and new promising work describing improvisation in terms of Gibsonian affordances (Love, 2017), it could be that motor areas are involved in representing such structures. A functional magnetic resonance imaging (fMRI) RSA analysis could help show what representational brain activity corresponds to the perceptual similarity. Functional classes of movements from other (non-musical) creative and everyday domains could also be assessed with a similar approach, similar to work on the neuroscience of goal representation (Hamilton & Grafton, 2006), and could be correlated with measures of creativity and improvisation ability.
Our findings show how certain organizations of knowledge may facilitate the ability to employ that knowledge in improvisatory behaviors. To be able to shift fluently between exemplars of a given functional class, as a similarity in representation would facilitate, could allow improvisers to have many different task-appropriate ways to behave in response to a particular stimulus. Note that this study situates knowledge organization as a relevant feature of improvisers’ minds which bears on their ability to improvise; in this way, creative music performance is not addressed as a state of mind alone, but is predicated on particular ways of learning and knowing. Such a theory is cross-culturally compatible (cf. Nooshin, 2003) and compatible with theories of non-musical creative behaviors such as a chef substituting ingredients with a similar function or a martial artist responding to an attack with a variety of functionally similar blocks or parries. Tasks from the creativity literature like the Alternate Uses Test (cf. Arden, Chavez, Grazioplene, & Jung, 2010) could also be understood in light of this theory. Speculatively, the ability to think of new uses of an object could be aided by organizing one’s knowledge of objects by their function.
Differences in representational organizations could in principle be based on a kind of self-selection: those who have a predisposition to categorize by function for some other reason (be it inherent or learned in some other behavioral domain) may be more likely to become improvisers. Because our findings were based on a continuous predictor variable, the evidence from our study shows that the amount of experience at least contributes to the difference, highlighting the importance of training.
One alternative explanation for why knowledge organization differs comes from how improvising musicians are taught to value music structures. Students of jazz learn to make value judgments through a number of techniques including: reharmonizing pre-existing harmonic structures to actualize harmonic and melodic interaction, transcribing other musicians’ solos or accompaniments to decode their “vocabulary” over harmonic changes, and comparing two or more pieces of music to find and utilize structural commonalities in their improvisational development (Berliner, 1994). These practices suggest, at least in part, that there is value placed on the utility of these music-structural elements, that is, how they assist the improviser in achieving the aural aesthetic that they desire. Taken this way, the functional deviant in our experiment could have presented as a structure with more utility for those with greater improviation experience, thus pushing it to the front of the participants purview.
In contrast, Western non-improvising musicians may place value in completing challenging passages in a score at the desired tempo, or finding the right dynamics to accentuate a certain section of a compositon. As such, educational practices in this environment emphasize playing sections of a compositon and focusing on the dynamics, articulation, tone, and, expressivity (Zhukov, 2015), but not usually learning to manipulate functional-harmonic aspects of musical structures. As it relates to our experiment, a possible explanation for the non-improvisers’ performance is that the non-improvisers placed value on more specific, local changes to how each chord in the 3-chord structure changed, such that exemplar and functional deviants were perceived as more equally different from the standard.
Finally, this work was motivated by the more general theory that the concept of improvisation can be understood as a particular way of knowing (Goldman, 2016). The structural differences in the knowledge of improvisers (e.g., differences in representational organization) are inextricable from the processes that they underlie in performance (i.e., the way they are able to employ them in action); because of this, understanding differences in the processes and the structures both contribute to an understanding of what it means to improvise. Difference in the categorization of structures on the basis of their function is one way improvisers have now been shown to differ (in addition to other ways they differ shown by the previous between-group comparison studies cited above). There may be many other ways that can meaningfully distinguish improvisatory ways of knowing.
Supplemental Material
Final_Supplementary_Materials – Supplemental material for Improvisation experience predicts how musicians categorize musical structures
Supplemental material, Final_Supplementary_Materials for Improvisation experience predicts how musicians categorize musical structures by Andrew Goldman, Tyreek Jackson and Paul Sajda in Psychology of Music
Footnotes
Acknowledgements
Sincere thanks to Josef Faller, Raphael Gerraty, Daphna Shohamy, George Lewis, Peter Gordon, Lori Custodero, the LIINC Lab, and the Learning Lab for supporting this project. Thanks also to the Presidential Scholars in Society and Neuroscience program.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclose receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by the Presidential Scholars in Society and Neuroscience program at Columbia University, and was partially supported by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-15-2-0074.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
