Dynamics of Musical Success: A Machine Learning Approach for Multimedia Data Fusion

Abstract

The success of creative products depends on the felt experience of consumers. Capturing such consumer reactions requires the fusing of different types of experiential covariates and perceptual data in an integrated modeling framework. In this article, the authors develop a novel multimodal machine learning framework that combines multimedia data (e.g., metadata, acoustic features, user-generated textual data) in creative product settings and apply it to predict the success of musical albums and playlists. The authors estimate the proposed model on a unique data set collected using different online sources. The model integrates different types of nonparametrics to flexibly accommodate diverse types of effects. It uses penalized splines to capture the nonlinear impact of acoustic features and a supervised hierarchical Dirichlet process to represent crowd sourced textual tags, and it captures dynamics via a state-space specification. The authors show the predictive superiority of the model with respect to several benchmarks. The results illuminate the dynamics of musical success over the past five decades. The authors then use the components of the model for marketing decisions such as forecasting the success of new albums, conducting album tuning and diagnostics, constructing playlists for different generations of music listeners, and providing contextual recommendations.

Keywords

Bayesian nonparametrics big data data fusion multimedia music industry product recommendations supervised hierarchical Dirichlet process

Creative industries such as music, movies, podcasts, videos, and art are unique in terms of the factors that determine product success. While the demand for utilitarian goods can be studied using easily measured and structured data (e.g., price, promotion, advertising), quantifying the success of intangible and expressive products (e.g., albums, movies) is more challenging. Product success in creative industries can be only partly explained using readily available metadata such as the product genre and the expertise, resources, and past successes of the artists and production houses, as these variables alone are not sufficient to fully capture the felt experience of the consumer.

Recently, data on experiential and aesthetic features of creative products have become increasingly available, offering a unique opportunity to quantify success in terms of the experiential aspects of consumption. Such features include the acoustical fingerprints of songs (e.g., physical features such as the pitch, loudness, or tempo; more subjective features such as energy, danceability, or valence); the stylistic, visual, and audio features of movies (e.g., mise-en-scène features; harmonic, rhythmic, or tonal features); and the “Art Genome” for describing the experiential aspects of art. In addition, thematic features that represent the emotions and perceptions that such creative products evoke are becoming available through user-generated textual content such as product tags and reviews. Table 1 shows the types of data that are relevant for modeling in different experiential contexts. Simultaneous quantification of the effects of these features on success is challenging. These multimedia features are multimodal (structured, unstructured, discrete, continuous, and textual) and high dimensional. Predicting success in such settings therefore requires a modeling framework that can flexibly fuse data from multiple modalities to capture the experiential aspects of consumption.

Table 1.

Some Relevant Domains of Application of the Proposed Model.

Context	Metadata	Experiential/Physical Features	Textual Features	Outcome
Music	Genre, artists, time periods, labels, etc.	Acoustical fingerprints	Tags, lyrics, reviews, etc.	Rankings
Movies	Genre, cast, time periods, studios, etc.	Visual features (mise-en-scène) and audio features (spectral features related to harmony, rhythm, and tone), suspense, pace, and narrative	Tags, script, reviews, etc.	Box office
Plays	Genre, cast, etc.	Mise-en-scène features, narrative, etc.	Dialogue, reviews, etc.	Attendance
Art	Genre, artists, time periods	Artsy’s Art Genome (1,000 features)	Tags, textual descriptions, etc.	Price

In this article, we develop a novel machine learning framework that integrates such multimedia data using a Bayesian semiparametric approach and focus on the music industry to showcase the utility of our framework. Despite the importance of the music industry, relatively little work has been done in marketing to model and predict musical success. We therefore assemble a comprehensive data set that includes Billboard 200 album ranking data for the last 50 years, acoustical fingerprints of the songs within each album, user-generated textual tags that capture people’s perceptions, and traditional album metadata. We then use these data to quantify the experiential determinants of musical success and to design and recommend creative products, such as albums and playlists. Specifically, we make three contributions. First, we develop a methodological framework that flexibly fuses multimedia data for prediction and managerial action. Second, on the substantive side, we examine how experiential features and textual collections correlate with success in the music domain. Third, we illustrate various managerial uses of our framework in forecasting success and in recommending and designing albums and playlists.

Relevant to our modeling is the fact that music is often consumed in listening sessions involving assortments of songs played in a sequence.¹ For much of musical history, these sessions have been fueled by the availability of albums and playlists. Consumer reactions to such assortments depend on the balance (Bradlow and Rao 2000) and distribution of acoustic characteristics across the songs within the bundle. From a production perspective, both albums and playlists are designed to be more than just collections of songs. They are designed around specific themes, balance various acoustic features, and contain songs that evoke specific moods and emotions. Predicting the success of such assortments requires that we not only capture listeners’ subjective experience evoked by the acoustic features but also accommodate how these features are distributed across the songs within the collection.

To date, albums continue to be the primary format for launching popular music. According to Nielsen, more than 70,000 albums were released by mid-2018 (RIAA 2018), and Vogel (2014) reports that major labels alone introduce 11,000 albums each year. Less than 10% of these are profitable, however, and fewer than 100 sell more than half a million units. Moreover, new music is costly to produce, as an album can cost between $250,000 to $400,000. It is therefore important to diagnose albums (assessing which acoustic features need to be changed for higher success) and evaluate their potential success before their release. Moreover, musical consumption has recently been driven by popular streaming providers such as Spotify and Pandora as well as more specialized ones, such as Qobuz, which construct and curate a variety of playlists to suit different tastes, generations, and contexts.

We focus on albums because, as stated previously, most new music has been and continues to be launched via albums. Analyzing playlists is more difficult, as these are relatively recent, and information on unsuccessful playlists is hard to get. In contrast, album data date to at least the 1960s and are available for both successful and unsuccessful albums.² Even if we were only interested in playlist design, we could still learn a great deal by studying the musical features that distinguish successful from unsuccessful albums. Like albums, playlists are curated and designed, and the good ones are balanced in their musical content and consistent in their themes.

We use data on albums that were released from 1963 to 2016. While a majority of our albums are from the Billboard 200 charts, we supplemented these with additional noncharting albums that were recorded by the artists that are represented in the Billboard data. The dependent variable in our model is a (censored) score reflecting the success of musical albums. The score combines data on sales of albums, track-equivalent albums, and streaming-equivalent albums (Pietroluongo 2012). Our data contain information on all major musical artists, genres, and historically significant albums produced during this period. We collected the acoustic features using the Spotify web application programming interface (API) and the textual tags using the Last.fm web API.

Given the extensive time span of our data, a static specification is unlikely to capture the relationship between musical success and the album features. Our model therefore accommodates dynamics in the coefficients of the genre and acoustical variables to capture their heterogeneous impact over time. It flexibly captures the possible nonlinear impact of the acoustical variables using nonparametric basis splines and uses a supervised hierarchical Dirichlet process (SHDP) to infer latent album themes or topics that summarize the semantic content within the textual tags. This results in a Bayesian semiparametric machine learning model, which we then use for several marketing tasks involving album recommendation and playlist construction.

Methodologically, our model is novel in many respects. It integrates multiple nonparametric components that flexibly fuse multimodal features. The use of nonparametrics is important, as we do not have existing knowledge of the functional forms of these features in most of the experiential settings in Table 1. The SHDP infers themes that yield a mixed membership representation of the albums in topic space and are useful for predicting album success. In addition, the SHDP allows the albums to share a common set of themes and automatically infers the number of those themes. While state-space dynamics (Naik 2015) and Bayesian nonparametrics (Ansari and Iyengar 2006; Ansari and Mela 2003; Bruce 2019; Dew and Ansari 2018; Kim, Menzefricke, and Feinberg 2007; Li and Ansari 2013; Rossi 2014; Shively, Allenby, and, Kohn 2000; Wedel and Zhang 2004) have been previously used in marketing, we combine these tools in several novel ways to yield a new modeling approach. Apart from the fact that we are the first to use an SHDP in marketing, our model extends the machine learning and statistical literature via its integration of state-space dynamics and different types of Bayesian nonparametric approaches to simultaneously handle different data modalities.

Substantively, our application builds on the marketing literature on musical success and consumption. Bradlow and Fader (2001) model the movement of songs up and down the Billboard 200 charts, and Lee, Boatwright, and Kamakura (2003) forecast sales of new albums prior to their release. Elberse (2010) shows that unbundling songs and selling them individually has a smaller effect on album sales when the songs were produced by a musician who had a relatively strong reputation and/or the songs’ features were similarly appealing. Ocasio, Mauskapf, and Steele (2016) examine the effects of technology on changes in the music industry, and Papies and Van Heerde (2017) study the dynamic interplay between recorded music and live concerts as well as the effects of piracy, unbundling, and artist characteristics on demand elasticities. Datta, Knox, and Bronnenberg (2017) examine how adoption of online streaming services affected listening behavior, and Chung, Rust, and Wedel (2009) propose an adaptive system for music personalization. More broadly, our research is related to work in other experiential contexts such as movies (Eliashberg et al. 2000).

Several behavioral studies are also relevant to our application. Holbrook and Hirschman (1982) and Schmitt (2010) broadly examine experiential consumption, while Bruner (1990) review how music impacts mood and emotions. Juslin and Laukka (2004) and Juslin and Västfjäll (2008) address mechanisms underlying emotional responses to music; Nave et al. (2018) relate musical preferences to listeners’ personalities; and Holbrook and Anand (1990) consider the effect of musical tempo on perceptions of activity, affective responses, and situational arousal. Most important for our research is Holbrook and Schindler (1989), who show that development of tastes for popular music follows an inverted U-shaped pattern that peaks at about 24 years of age.

Our results showcase the use of our model for several managerial tasks. We compare the predictive accuracy of our model with several benchmark models and show its superiority for both in-sample and out-of-sample predictions. More importantly, we use several applications to show how our model can be used to (1) forecast the performance of new albums before they are launched, (2) recommend albums and playlists that are consistent with the musical styles associated with specific eras, (3) recommend contextual playlists based on user queries, and (4) provide diagnostics for albums that are predicted to be unsuccessful. The model also generates substantive insights relating album success with acoustics and themes and shows how the role of acoustics in shaping popular success has changed over time. We also discover several themes from the user-generated tags that summarize how the albums are perceived. Among other things, these themes can represent variables for which we do not have explicit data (e.g., subgenres, emotions, situations, artist characteristics).

The article is organized as follows. The next section describes our data, its sources, and its collection methods. This is followed by a section in which we develop the proposed model. Then, we present the results, discuss qualitative insights, and compare the predictions of our model with those obtained from several benchmark models. The penultimate section uses the model for album and playlist scoring, recommendation, and design. We conclude by acknowledging the limitations of our research and identifying areas for future research.

Data Description

We used different online sources to assemble a multimedia data set of American popular music spanning the 54 years from 1963 to 2016. Our data set includes not only metadata such as the genres within the album but also acoustical measures that characterize the listening experience and textual tags that summarize listeners’ perceptions. Next, we describe the data and its sources, starting with our dependent variable.

Album Success: The Billboard 200

The Billboard 200 is a ranking, released by Billboard magazine, of the 200 best-performing albums in a week. Since 1991, the ranking has been based on sales data obtained from Nielsen SoundScan (now MRC Data), and these data have been augmented to include the sales of digital albums/tracks and revenue from online streaming since 2014. We scraped Billboard’s website to obtain the weekly Billboard 200 rankings for the 1963–2016 period.

Album sales would have been ideal as our dependent variable, but as these are unavailable, we used the Billboard’s year-end score, which is obtained by summing the weekly inverse rankings of an album across all the weeks of a year in which it was on the charts (e.g., the top-selling album on Billboard 200 is assigned an inverse rank of 200, and the worst-performing album an inverse rank of 1). Billboard exclusively used this score before switching to Nielsen SoundScan in 1991. The Billboard rankings and the year-end score are industry standards that have also been extensively used for research on popular music (see Alexander 1996; Anand and Peterson 2000; Bradlow and Fader 2001; Dowd 2004; Lena 2006; Lena and Pachucki 2013; Peterson and Berger 1975). A limitation of this score is that it aggregates weekly rankings. Its strength, however, is that it considers both peak success and longevity of an album on the charts.

Focusing only on albums that make the charts can provide a biased picture of the determinants of musical success. We handled this selection issue by augmenting the Billboard 200 data with albums that did not appear on the charts. To this end, we collected the full discography (a catalog of all the recordings of an artist) of all our artists and identified their albums that did not appear on the Billboard 200. From this set of noncharting albums, for each year, we randomly selected 200 albums that were released in the preceding three years.³

The original number of unique albums on the Billboard 200 over the 54 years is 25,750. However, complete data were available only for 15,233 charting albums, which we include in our sample. The original number of unique groups/artists on the Billboard 200 is 8,039, out of which full data were available for 5,852. We collected a total of 12,653 noncharting albums with full data, and we sample 200 every year with replacement (because an album does not necessarily chart on the year of its release but within a window of three years after release), resulting in 7,273 unique noncharting albums in our data. Altogether, we obtained 34,214 observations across 22,506 unique albums that were created by 5,852 unique artists or groups. We compared the distribution of the year-end scores for the included and missing albums and found these to be almost identical. Thus, the missing albums can be considered missing at random. Moreover, because the dependent variable is available but the acoustical covariates or the textual tags are missing, it is not possible to impute the missing values.

Table 2 shows the ten albums with the all-time highest year-end Billboard 200 scores. The top album is Michael Jackson’s Thriller, which sold an estimated 66 million copies and is the best-selling album of all time. The next two are Alanis Morissette’s Jagged Little Pill, which sold 33 million copies, and Taylor Swift’s 1989, which has sold 10 million copies since its release in 2014.

Table 2.

Ten Albums with the All-Time Highest Year-End Scores on Billboard 200.

Album	Artist	Label	Year	Year-End Score
Thriller	Michael Jackson	Epic	1983	10,526
Jagged Little Pill	Alanis Morissette	Maverick	1996	10,233
1989	Taylor Swift	Big Machine Records	2015	10,181
Cracked Rear View	Hootie & The Blowfish	Atlantic Records	1995	10,111
Fearless	Taylor Swift	Big Machine Records	2009	10,080
Breakaway	Kelly Clarkson	RCA Records Label	2005	10,044
Human Clay	Creed	The Bicycle Music Company	2000	10,044
21	Adele	XL Recordings/Columbia	2012	10,036
Backstreet Boys	Backstreet Boys	Jive	1998	10,021
Crazysexycool	TLC	Arista/LaFace Records	1995	10,016

Figure 1 shows the histograms of the observed year-end scores and their log-transformed values. The year-end scores have an average value of 169 and exhibit a long-tail distribution. Only 10% of the albums have scores higher than 2,887. The log-transformed scores are less skewed and have a distribution that resembles the normal. For this reason, we use the log-transformed score as the dependent variable in the following analysis. Next, we describe the different sets of variables we used to model success.

Figure 1.

Distributions of the year-end scores and their logarithmic transformations.

Genres

Musical styles are conventionally described by their genre. We identified the genres of each album using the API for Discogs, which is a comprehensive crowd sourced database of audio recordings. An album can contain multiple genres if it fuses different types of music (e.g., electronic-rock combines electronic and rock music), if it results from a collaboration of different types of artists, or if its tracks have different genres. Figure 2 shows that rock is the most common genre in our data.

Figure 2.

Distribution of genres across all observations in all years.

The production and popularity of genres has changed over time. Jazz and pop were dominant until rock took over in 1965. Since then, rock has been prevalent in at least a third of all released albums. Hip hop emerged in the early 1970s and peaked in the 1990s. Funk/soul peaked in the 1970s and held its own until the early 1980s. The “folk, world, and country” music genre has peaked twice, in the 1960s and 1990s, and electronic music in the mid-1970s. The other genres have appeared only infrequently on the charts.

Acoustic Fingerprints

It is difficult to measure how a song impacts the listening experience. Acoustic fingerprints, which are digital summaries of a song’s phonic features, are the best available measures for capturing a song’s effect on a listener. Acoustic features encapsulate the creative experience on multiple dimensions, capture the underlying artistic style, and relate to the instruments and technologies used for producing music. Some acoustical fingerprints capture the physical aspects of music: key, loudness, mode, tempo, and time signature. Others describe the listening experience: acousticness, danceability, energy, instrumentalness, liveness, speechiness, and valence. We also consider track duration and the explicitness of lyrics as acoustic features. We used the 14 acoustic features described in the Web Appendix. We used the Spotify API to collect the acoustic fingerprints of the songs in each album. The fingerprints are produced using machine learning techniques by The Echo Nest (Spotify’s music intelligence and data platform). An album is a bundle of songs that can differ in their acoustical profiles. As in the balance model, we use the means and standard deviations of the different acoustic measures to capture the average acoustical level and the variability of the acoustics across the songs in an album (Bradlow and Rao 2000; Farquhar and Rao 1976).^4,5

User-Generated Tags

User-generated tags reflect how listeners categorize and perceive different albums. Last.fm is an online music platform that started collecting member-generated tags describing music in 2002. Even though this effort started in 2002, the community of users have tagged current albums as well as albums released prior to 2002. These tags contain a mix of factual and perceptual information about the albums. We collected tags for the albums in our data set using the Last.fm API.

Bag-of-tags representation

Last.fm data reports only the relative frequencies of the tags associated with an album. For example, it might report that 95% of the listeners who tagged an album used the word “romantic.” We used this information to construct a bag-of-words representation of size 100 for each album.⁶ To prune the vocabulary and retain important tags that distinguish among albums, we used the term frequency-inverse document frequency (tf-idf) (Rajaraman and Ullman 2011) to choose the top V tags across all albums. We deleted the tags with the .1% lowest tf-idf scores and retained the remaining 15,059 tags. This procedure resulted in a reasonable number of tags per album—a minimum of 9 and an average of 95 tag applications. Less than 1% of the albums had fewer than 63 tag applications. Altogether, there were 2,145,139 tag applications across all albums and years. Figure 3 shows a word cloud of all unique tags in our data. Tags displayed with larger fonts appear more frequently.

Figure 3.

Word cloud of all unique tags.

Figure 4 shows the tags associated with the albums Thriller by Michael Jackson, 25 by Adele, and 1989 by Taylor Swift. The unique tags are “male vocalists,” “Halloween,” and “classic” for Thriller; “British,” “blue,” and “epic” for 25; and “love at first listen,” “electropop,” and “synthpop” for 1989. The tags “pop” and “albums I own” are common to the three albums.

Figure 4.

Word clouds of tags for Michael Jackson’s Thriller, Adele’s 25, and Taylor Swift’s 1989.

Other Covariates

We also considered artist popularity, the collaboration of multiple artists in an album, and major labels.

Superstardom

Artists differ in their popularity, with some being superstars. Moreover, their popularity varies over time. We measure artists’ superstardom at any point in time using the number of previous albums they had on the Billboard 200. Artists who have more previous albums on Billboard 200 have more fans, are more visible (e.g., they appear on television shows, movies, and advertisements), and are better supported by record houses. As Krueger (2005) and Giles (2007) observe, superstardom can have a spillover effect on the success of new albums.

Figure 5 shows the histogram of the number of previous albums an artist had on the Billboard 200 at any point in time. About 90% of the observations correspond to artists with fewer than seven previous charting albums, with a median of one album. Barbra Streisand is the biggest superstar in our data, being the only artist whose every album has appeared on Billboard 200. The latest, her 35th entry on the charts, was Encore: Movie Partners Sing Broadway. Five of her albums have achieved the top rank.

Figure 5.

Histogram of previous charting albums.

Number of artists

We used the Spotify API to obtain data on the number of artists featured in each album. Most of the albums in our data set feature only one artist or group. The Boston Symphony Orchestra’s 1981 recording of Mahler’s Symphony no. 8 is the album with the most featured artists (12).

Major and minor labels

Albums launched by major labels usually have several advantages over those from independent (indie) producers and artists. Major labels have larger, more skilled production teams; more innovative technologies; larger budgets for marketing, recruiting popular artists, and training new talent; and better connections with media outlets (Rossman 2012). We used Spotify’s web API to identify each album’s production house, resulting in 2,959 unique labels. We then manually compiled an exhaustive list of 1,746 labels that are associated with major music groups (e.g., EMI, Sony Music, Universal Music Group) from different online sources⁷ and combined this with a list of major record labels (historical and current) from the U.S. Library of Congress website.⁸ Out of the 22,506 recordings in our data set, 14,218 albums were released by major labels. While 66% of the charting albums were released by major labels, only 58% of the noncharting albums are from major labels. Having described our data, we next present our modeling framework.

Modeling Framework

We develop a novel Bayesian framework that flexibly and semiparametrically integrates the different data modalities (genres, acoustic features, and textual tags) to model album success. Let $i = 1$ to I denote albums and $t = 1$ to T the calendar years (i.e., 1963 to 2016) in our data. An album can appear on the charts for multiple years and therefore can be present several times in our data. Let album i appear in $J_{i}$ years. Let $t (ij) \in {1, \dots T}$ denote the calendar year associated with the $j$ th observation for album i. Let $s (i) \in {1, \dots N_{s}}$ denote the artist/group associated with album i, where $N_{s}$ is the total number of artists/groups in our data. The following variables are used in our model.

$y_{ij}$ is the success score of album i on its $j$ th observation, where $j \in {1, \dots J_{i}}$ . This variable is not observed for albums that do not make it to the charts.

$x_{i} = [x_{i 1}, \dots, x_{iM}]$ is an $M$ -vector containing the binary genre and major label indicator variables for album i.

$a_{ij}$ is an $L_{1}$ -vector of continuous variables containing the means of the different acoustical measures across all the songs in an album as well as the superstardom and number of artists covariates for album i. Of these, only the superstardom variable changes across the different observations of the album.

$a_{i}^{sd}$ is an $L_{2}$ -vector of continuous variables containing the standard deviations of the different acoustical measures for album i.

$w_{i} = {w_{in}}_{n = 1}^{N_{i}}$ is a vector of $N_{i}$ textual tags associated with album i.

We normalized all continuous variables (e.g., acoustic features) to a scale between 0 and 1 to allow for meaningful comparisons across variables. Next, we describe how all the previous variables are used in different components of our model. We begin with the dependent variable.

Censored Success Score

The Billboard year-end score is a yearly measure of an album’s success. It is available only if an album appears on the charts. For other albums, the score is unobserved. This score summarizes both the appearance and persistence of an album on the charts for a given year. As these two outcomes are manifestations of a single underlying process—popular success—which is based on album sales, we use a Type 1 Tobit model (Tobin 1958) to handle the censoring. A Type 2 Tobit would be more appropriate if one process placed the album on the charts and a different one determined its position on the charts.

We model this censored variable by assuming that the observed value of $y_{ij}$ stems from a partially latent score $y_{ij}^{*}$ , such that

y_{ij} = {\begin{matrix} y_{ij}^{*}, & {if y}_{ij}^{*} \geq 0, \\ unobserved, & {if y}_{ij}^{*} < 0. \end{matrix}

(1)

The latent score

y_{ij}^{*}

for the noncharting albums then has the interpretation of the album’s score, had Billboard included more albums annually. We use a flexible link function to relate this partially latent variable to the different sets of covariates. This relationship can be written as

y_{ij}^{*} = F_{t (ij)} (x_{i}, a_{ij}, a_{i}^{sd}, w_{i}) + ε_{ij},

(2)

where

F_{t (ij)}

is a time-varying function and

ε_{ij} \sim N (0, σ_{ε}^{2})

is an idiosyncratic, normally distributed error term. We next describe the link function in greater detail.

Link Function

The link function $F_{t (ij)}$ flexibly captures the impact of the different types of album features and latent variables on success. For simplicity, we use an additive specification,

F_{t (ij)} (x_{i}, a_{ij}, a_{i}^{sd}, w_{i}) = η_{t (ij)} + η_{s (i)} + a_{i}^{sd T} ϑ + x_{i}^{T} β_{t (ij)} + f_{t (ij)} (a_{ij}) + g_{i} (w_{i}),

(3)

where

η_{t (ij)}

are year-specific intercepts;

η_{s (i)}

are artist/group random effects;

ϑ

is a vector of static coefficients that captures the effect of the acoustic varieties;

β_{t (ij)}

is a vector of time-varying coefficients that captures the effects of the binary genre variables;

f_{t (ij)} (.)

is an unknown, time-varying, multivariate function that captures the effects of the continuous variables (including the acoustic means); and

g_{i} (.)

is a function that quantifies the relationship between the tags and the success of an album. We separately discuss each component next.

Artist Random Effects

We control for individual artist/group success by estimating a random effect $η_{s}$ for each artist/group s. We assume a normal population distribution $N (0, τ_{η}^{2})$ for the random effects. As we also have a time-varying intercept, the population mean is set to 0, and we estimate the population variance $τ_{η}^{2}$ , which has an inverse-gamma prior.

Static Linear Effects of Acoustic Standard Deviations

For simplicity, we assume⁹ that the standard deviations of the acoustics are linearly and statically related to album success via the parameters $ϑ$ , which have a normal prior $N (0, C_{ϑ})$ , where $C_{ϑ}$ is a diagonal matrix.

Linear Effects of Discrete Variables Including Genres

We use a linear specification, $x_{i}^{T} β_{t (ij)}$ , to model the effects of the binary genres and the major label indicator. It is unlikely that the popularity of different binary genre variables remained unchanged over time. We therefore use a state-space approach to allow for dynamics in the popularity of genres over the 54 years for which we analyze data. Note that the coefficients in $β_{t (ij)}$ are indexed by $t (ij)$ , which gives the calendar year for the $j$ th observation of album i. All albums within a year t share the same coefficients, $β_{t}$ . We assume that $β_{t}$ varies across the calendar years $t = 1$ to T according to a transition equation that we describe in detail in our “State-Space Dynamics” subsection.¹⁰

Nonlinear Nonparametric Effects of Continuous Variables Including Acoustics

We use the functions $f_{t (ij)} (a_{ij})$ , which are specific to calendar time, to model the dynamic effects of the acoustic fingerprints. Given the large number of variables, for simplicity, we use an additive specification:

f_{t (ij)} (a_{ij}) = \sum_{ℓ = 1}^{L_{1}} h_{t (ij)}^{ℓ} (a_{ij ℓ}),

where

h_{t (ij)}^{ℓ} (.)

is a smooth function for the (continuous) variable

ℓ

. Given the lack of theory, it is not possible to hypothesize a priori how these covariates influence album success. We therefore consider a nonparametric specification to model these unknown functions, using penalized splines (Wand and Ormerod 2008). We assume that for a given calendar year

t (ij)

, each component

h_{t (ij)}^{ℓ} (.)

is a piecewise, second-order polynomial:

\begin{aligned} h_{t (ij)}^{ℓ} (a_{it ℓ}) = & ψ_{t (ij) 0}^{ℓ} + ψ_{t (ij) 1}^{ℓ} a_{ij ℓ} + ψ_{t (ij) 2}^{ℓ} a_{ij ℓ}^{2} \\ + \sum_{q = 1}^{Q} ψ_{κ q}^{ℓ} (a_{ij ℓ} - κ_{q}^{ℓ})_{+}^{2}, \end{aligned}

(4)

where

{ψ_{t 0}, ψ_{t 1}^{ℓ}, ψ_{t 2}^{ℓ}, ψ_{κ}^{ℓ} = [ψ_{κ q}^{ℓ}]_{q = 1}^{Q}}

are parameters and

κ_{1}^{ℓ} < \dots < κ_{Q}^{ℓ}

are fixed knots. We use quadratic basis splines involving the second-degree truncated basis function where

(a_{ij ℓ} - κ)_{+}^{2} = (a_{ij ℓ} - κ)^{2}

a_{it ℓ} > κ

and 0 otherwise;

κ

is a fixed knot in the compact support of

a_{ij ℓ}

. We used second-order polynomials because these offer the simplest departure from linearity. The use of penalized splines enables us to seamlessly trade off the flexibility and smoothness of the inferred functions. Usually, there is a trade-off between using a larger number of knots (which increases the fidelity to the data) and the smoothness of the resulting function. The trade-off is resolved using a roughness penalty. We consider a fixed number of knots with known positions to avoid the complexities arising from the need to infer the number of knots and their locations. We constrain their influence by regularizing their basis coefficients using shrinkage priors, thus ensuring smoothness and avoiding overfitting. Because the continuous variables are scaled between [0,1], we assume five equidistant fixed knots for each variable.

Additive model

Summing over all $L_{1}$ additive nonlinear effects yields the following form:

f_{t (ij)} (a_{ij}) = ψ_{t (ij) 0} + \sum_{ℓ = 1}^{L_{1}} [ψ_{t (ij) 1}^{ℓ} a_{ij ℓ} + ψ_{t (ij) 2}^{ℓ} a_{ij ℓ}^{2} + \sum_{q = 1}^{Q} ψ_{κ q}^{ℓ} (a_{ij ℓ} - κ_{q}^{ℓ})_{+}^{2}],

(5)

where

ψ_{t (ij) 0} = \sum_{ℓ = 1}^{L_{1}} ψ_{t (ij) 0}^{ℓ}

is a nonidentified intercept, which we fix to 0. Sun, Tsutakawa, and Speckman (1999) showed that Equation 5 can be represented as a mixed model with additional covariate-specific hierarchical variance components

σ_{ψ_{κ} ℓ}^{2}

, which regularize the basis coefficients and therefore control the extent of the smoothing by assuming that

ψ_{κ q}^{ℓ} \sim N (0, σ_{ψ_{κ} ℓ}^{2})

The polynomial coefficients $ψ_{t (ij) 1}^{ℓ}$ and $ψ_{t (ij) 2}^{ℓ}$ , the basis coefficients $ψ_{κ q}^{ℓ}$ , and the prior variances $σ_{ψ_{κ} ℓ}^{2}$ are inferred from the data for each continuous variable $ℓ$ . Because $σ_{ψ_{κ} ℓ}^{2}$ is estimated, the implicit regularization it induces ensures that the parameters of only the important knots are given significant weights. Small values of $σ_{ψ_{κ} ℓ}^{2}$ diminish the effect of the truncated basis; larger values allow the basis coefficients to significantly deviate from 0 and allow a greater effect of the truncated basis. This type of regularization is similar to what is done in ridge regression.

State-Space Dynamics

Note that the time-varying intercepts $η_{t (ij)}$ , the coefficients of the genre variables in $β_{t (ij)}$ , and the coefficients $ψ_{t (ij) 1}^{ℓ}$ and $ψ_{t (ij) 2}^{ℓ}$ in Equation 4 for each variable $ℓ$ vary across calendar years, $t = 1$ to T. All albums within a given calendar year t (i.e., for albums with $t (ij) = t$ ) have the same coefficients, $η_{t}$ , $β_{t}$ , $ψ_{t 1}^{ℓ}$ , and $ψ_{t 2}^{ℓ}$ . In modeling the evolution of these coefficients, it is important to capture the fact that they are likely to evolve smoothly, as dramatic changes from one year to the next are unlikely. We use a linear state-space specification to model such evolution.

Collecting all the time-varying coefficients into a single vector $γ_{t} = {η_{t}, β_{t}, {ψ_{t 1}^{ℓ}, ψ_{t 2}^{ℓ}}_{ℓ = 1}^{L_{1}}}$ , we model their evolution via a second-order autoregressive (AR[2]) process (for a variant of this, see Biyalogorsky and Naik [2003]) as follows:

γ_{t} = φ_{1} + φ_{2} \circ γ_{t - 1} + φ_{3} \circ γ_{t - 2} + ω_{t} .

(6)

In Equation 6,

x \circ y

denotes the element-wise product of two vectors

x

and

y

. The residuals in

ω_{t}

are distributed normally,

N (0, Σ_{γ})

, with a diagonal covariance matrix

Σ_{γ}

. Using an AR(2) transition equation allows us to capture different patterns of evolution. Depending on the values of the

φ

coefficients, cyclical patterns of evolution are also possible.¹¹ The variance components

σ_{2 γ, k}

Σ_{γ}

capture the volatility in the values of the coefficient over time. It is possible to include additional terms in Equation 6 to accommodate factors that drive the evolution in the coefficients (e.g., technological drivers of music). Because we do not have access to these drivers in our data, we do not include these in the article.

Having discussed the first components of Equation 3, we focus on the last component that incorporates the perceptual tags.

Nonparametric Themes

Textual tags can also be useful for explaining album success. Given the large number of tags in our data set, we cannot directly include them in the model as covariates. Instead, we incorporate their effect via topics or themes that predict album success. We use an SHDP to model the themes. This is a generalization of the commonly used Latent Dirichlet Allocation (LDA) topic model (Blei, Ng, and Jordan 2003; Tirunillai and Tellis 2014) that summarizes documents into topics; however, it differs from LDA on two aspects. First, the inferred themes are obtained in a supervised fashion, as they are informed not only by the tag collections across the albums but also by the variability in the success scores of the albums. Thus, the themes are predictive of success. Unsupervised approaches like the LDA can yield topics that have no relationship with musical success. Thus, supervision here is akin to “variable” selection. Second, unlike in an LDA, the hierarchical Dirichlet process nonparametrically assumes a countably infinite number of themes and ensures that only a finite number of these themes have significant weights in any given application. In summary, the SHDP allows us to automatically infer the number of themes that correlate with album success.

Themes

A theme/topic is a discrete probability distribution over the vocabulary of V tags. Formally, the $k$ th theme is described by $ϕ_{k}^{w} = [ϕ_{kv}^{w}]_{v = 1}^{V}$ , where $\sum_{v = 1}^{V} ϕ_{kv}^{w} = 1$ and $ϕ_{kv}^{w}$ is the probability that tag $v \in {1, \dots, V}$ appears in that theme. Thus themes reside in a $V - 1$ dimensional simplex. All the tags appear in all the themes but with different (possibly near-zero) probabilities. That is, the themes differ in the weights they place on each of the V tags. Because the number of themes is a priori unknown, we consider a Bayesian nonparametric framework where we assume that there is a countably infinite number of themes $[ϕ_{k}^{w}]_{k = 1}^{\infty}$ that are shared across all the albums. For each theme $ϕ_{k}^{w}$ , we assign a scalar $ϕ_{k}^{y}$ that indicates the contribution of the theme to the success of an album. Each “augmented” theme is then fully defined by the tuple $ϕ_{k} = (ϕ_{k}^{w}, ϕ_{k}^{y})$ .

Mother distribution of themes

The themes need to be shared across all the albums. We therefore need a discrete distribution $G_{0}$ that generates the countably infinite number of themes. Because this distribution is unknown, we assume a Dirichlet process (DP) prior for it, that is,

G_{0} \sim DP (ζ, H),

(7)

where the baseline distribution

H = H^{w} \times H^{y}

is a joint discrete-continuous probability measure on the themes and their success coefficients, and

ζ > 0

is a precision parameter. H represents the mean of the process and sets its location, and

ζ

controls the dispersion of the DP realizations around H. We use a symmetric Dirichlet distribution

H^{w} = Dir (1 / V)

for the themes and a diffuse normal distribution

H^{y}

for generating the success coefficients for the themes.

Because the DP realizations are discrete distributions, $G_{0}$ places its mass over a countably infinite collection of augmented themes (i.e., probability vectors and their success coefficients). Sethuraman (1994) showed that a DP can be alternatively construed via a stick-breaking construction. According to this representation, $G_{0} \sim DP (ζ, H)$ can be written as

G_{0} = \sum_{k = 1}^{\infty} ς_{k} δ_{ϕ_{k}}, ϕ_{k} \sim H,

(8)

ς_{k} = ς_{k}^{'} \prod_{l = 1}^{k - 1} (1 - ς_{l}^{'}), ς_{k}^{'} \sim Beta (1, ζ),

(9)

where

[ϕ_{k}]_{k = 1}^{\infty}

is an infinite sequence of augmented themes drawn from the base distribution H,

δ_{ϕ_{k}}

is a discrete random measure concentrated at atom

ϕ_{k}

, and

[ς_{k}]_{k = 1}^{\infty}

is an infinite sequence of weights. The stick-breaking construction in Equation 9 ensures that the weights sum to 1. In this way,

G_{0}

initializes the infinite collection of themes and their success coefficients that are then shared across the albums.

The clustering properties of the DP depend on its precision parameter. If $ζ$ is large, the DP realizations mimic the baseline distribution H. As $ζ$ approaches zero, the DP generates distributions that are concentrated on a few mass points (i.e., augmented themes). We infer $ζ$ from the data and therefore automatically determine the number of themes present in a given data set.

Albums as mixtures of themes

The set of tags for an album can be considered as a mixture of the themes $[ϕ_{k}]_{k = 1}^{\infty}$ . The albums differ in the proportions with which the themes are present in them and therefore place different weights on the themes. Let $G_{i}$ be the unknown discrete distribution, defined over the themes $[ϕ_{k}]_{k = 1}^{\infty}$ , from which album i draws its themes. To ensure that the album-specific distributions $G_{i}$ place weights only on the augmented themes in $G_{0}$ , (i.e., on the atoms of $G_{0}$ ), we assume a common DP population distribution for all $G_{i}$ , that is,

G_{i} \sim DP (α_{0}, G_{0}),

where

G_{0}

is the baseline distribution and

α_{0} > 0

is a precision parameter.

G_{i}

can be represented as an infinite mixture,

G_{i} = \sum_{k = 1}^{\infty} π_{ik} δ_{ϕ_{k}},

(10)

where the atoms

[ϕ_{k}]_{k = 1}^{\infty}

are drawn from

G_{0}

, and

π_{ik}

is the probability weight for atom

ϕ_{k}

. The

G_{i}

s are independent across albums; each

G_{i}

selects a specific subset of themes for album i.

Each tag in album i comes from its associated theme. Let $θ_{in} = (θ_{in}^{w}, θ_{in}^{y})$ denote the augmented theme for the $n$ th tag, $w_{in}$ , of album i, where $θ_{in}^{w}$ is equal to one of the probability vectors in $[ϕ_{k}^{w}]_{k = 1}^{\infty}$ and $θ_{in}^{y}$ is its corresponding success score. Then

θ_{in} \sim G_{i} .

Given the theme

θ_{in}

, the tag

w_{in}

comes from the categorical distribution

w_{in} \sim Categorical (θ_{in}^{w}) .

(11)

The set

w_{i}

of the

N_{i}

tokens within the album can contain many replications of the same tag that were assigned by different users, and two replicates of the same tags can come from different themes. Let

C_{i}

be the set of themes present in the album across all of its tokens. The

g_{i} (w_{i})

component in Equation 3 links the themes to overall album success as in

g_{i} (w_{i}) = \sum_{k = 1}^{\infty} ϕ_{k}^{y} 1_{C_{i}} (ϕ_{k}^{w}),

(12)

where

1_{C_{i}} (x)

is an indicator function that takes a value of 1 if

x \in C_{i}

, and 0 otherwise. Thus, the

g_{i} (w_{i})

just sums the coefficients of all the themes that are present in album i. This is different from what is done in traditional supervised topic models, where the theme proportions within an album are used, instead of the binary variables that we employ. Our approach has the significant advantage of yielding coefficients that are readily and cleanly interpretable. Moreover, it allows us to also identify the intercept in the regression. Figure 6 gives a schematic representation of the SHDP (see Dai and Storkey 2015) for a simplified vocabulary of three tags. The figure shows both the distribution hierarchy and the data-generative process for the tags.

Figure 6.

Illustrative representation of the SHDP with a vocabulary of three tags.

In summary, our model flexibly integrates data from multiple media and modalities, using a combination of static and dynamic components. The dynamics in our model are parametric and linear in scope, whereas the nonlinear and nonparametric components (i.e., the basis terms and the supervised HDP components) are static in nature. This results in a partially linear semiparametric model. Next, we further summarize the entire modeling framework via its generative process.

Generative Process

Our model can be specified using the following generative process.

Draw the artist random effects $η_{s} \sim N (0, τ_{η}^{2})$ for each artist s.

Draw the linear static effects of the standard deviations of the acoustic features $ϑ \sim N (0, C_{ϑ})$ .

Draw the hyperdistribution of themes and their success coefficients, $G_{0} \sim DP (ζ, H)$ .

For each album i,

Draw the thematic mixture $G_{i} \sim DP (α_{0}, G_{0})$ .

For each tag application n,

Draw theme $θ_{in} = (θ_{in}^{w}, θ_{in}^{y}) \sim G_{i}$ .

Draw tag $w_{in} \sim Categorical (θ_{in}^{w})$ .

Draw for all $ℓ$ the variance components, $σ_{ψ_{κ} ℓ}^{2}$ , from an inverse-gamma $(a_{ψ, ℓ}, b_{ψ, ℓ})$ .

Draw for all $ℓ$ the truncated basis coefficients of the nonlinear effects, $ψ_{κ}^{ℓ} \sim N (0, σ_{ψ_{κ} ℓ}^{2})$ .

For each year t, draw the time-varying coefficients for the genre, means of the acoustic features, and other covariates, $γ_{t} \sim N (φ_{1} + φ_{2} \circ γ_{t - 1} + φ_{3} \circ γ_{t - 2}, Σ_{γ})$ .

Draw the error variance $σ_{ε}^{2}$ from an inverse-gamma $(a_{σ_{ε}}, b_{σ_{ε}})$ .

For each observation j of album i, in year $t (ij)$ , draw score

y_{ij} \sim N [F_{t (ij)} (x_{i}, a_{ij}, a_{i}^{sd}, w_{i}), σ_{ε}^{2}] .

Posterior Inference via Markov Chain Monte Carlo methods

We use a fully Bayesian approach to infer the unknowns in our model. Let the collection of all unknowns be given by $Δ = {{y_{ij}^{*}}, σ_{ε}^{2}, {η_{s}}, τ_{η}^{2}, ϑ, {γ_{t}}, φ, Σ_{γ}, ψ_{κ}, {σ_{ψ_{κ} ℓ}^{2}}, {θ_{in}}, {G_{i}}, G_{0}, α_{0}, ζ}$ . Then, the joint distribution of the data and unknowns can be written as

\begin{aligned} p ({y_{ij}}, {w_{ij}}, Δ) \\ = [\prod_{i = 1}^{I} \prod_{j \in J_{i}} p (y_{ij} | y_{ij}^{*}) p (y_{ij}^{*} | η_{s (i)}, ϑ, γ_{t (ij)}, ψ_{κ}, {θ_{in}}, σ_{ε}^{2})] \\ \times p (σ_{ε}^{2}) [p ({η_{s}} | τ_{η}^{2}) p (τ_{η}^{2})] p (ϑ) [p (ψ_{κ} | {σ_{ψ_{κ} ℓ}^{2}}) p ({σ_{ψ_{κ} ℓ}^{2}})] \\ \times [p (γ_{- 1}) p (γ_{0}) p (φ) p (Σ_{γ}) \prod_{t = 1}^{T} p (γ_{t} | γ_{t - 1}, γ_{t - 2}, φ, Σ_{γ})] \\ \times [p (ζ) p (α_{0}) p (G_{0} | ζ, H) \prod_{i = 1}^{I} p (G_{i} | α_{0}, G_{0}) \prod_{n = 1}^{N_{i}} p (w_{in} | θ_{in}) p (θ_{in} | G_{i})] . \end{aligned}

Because the full posterior distribution

p (Δ | {y_{ij}}, {w_{ij}})

is not available in closed form, we use Markov chain Monte Carlo (MCMC) methods to summarize the posterior. We use data augmentation to infer the censored score for the noncharting albums in each MCMC iteration. This and the use of conjugate priors yields closed-form full conditionals resulting in a Gibbs sampling algorithm.¹² In other nonlinear contexts where such data augmentation is not possible, advanced methods such as those described in Rubel and Naik (2017) and Bruce (2019) are needed. Integrating out the SHDP priors results in a collapsed Gibbs sampling scheme for inferring the themes. Here, we rely on the algorithm based on direct assignment of the tags as in Teh et al. (2005), Heinrich (2011), and Dai and Storkey (2015). We programmed the inference algorithm in Python and used Numba to compile our code into optimized machine language for efficient computation in our big-data setting.

Results

We first compare the predictive performance of our model with that of several benchmark models. We then highlight the main qualitative results from our model.

Model Comparison

Benchmark models

We carefully selected a set of benchmark models to show how the various data types (e.g., metadata, artist effects, acoustic features, tags) and the different components of our model contribute to the predictions of musical success. Table 3 shows the different models and the associated predictive performance. As the table shows, we use three sets of benchmarks. The first set of three models includes only the metadata and control variables, and the random effects, in a linear and static manner. The models are of increasing complexity. Model LS1 includes only the control variables, LS2 adds artists’ random effects, and Model LS3 adds genres to LS2.

Table 3.

Comparison of the Fit and Predictive Performances of the Different Models.

Model	CV	RE	G	AF	# Themes	In Sample					Out of Sample
Model	CV	RE	G	AF	# Themes	LL	PSIS	AUC	RMSE	p(nc)	AUC	RMSE	p(nc)
LS1	x				0	−70,594.43	−70,598.13	.57	3.19	.23	.57	3.22	.23
LS2	x	x			0	−61,939.90	−63,613.27	.91	2.45	.48	.85	2.76	.44
LS3	x	x	x		0	−61,889.91	−63,466.39	.90	2.46	.48	.85	2.76	.44
NLS	x	x	x	x	0	−61,198.40	−62,764.76	.91	2.45	.49	.87	2.72	.45
NLD	x	x	x	x	0	−60,944.48	−63,801.91	.92	2.32	.50	.86	2.67	.46
NLD-LDA	x	x	x	x	20	−60,931.07	−63,849.04	.92	2.32	.50	.87	2.66	.45
NLD-LDA	x	x	x	x	40	−60,752.24	−63,544.41	.92	2.31	.50	.86	2.71	.44
NLD-LDA	x	x	x	x	60	−60,842.36	−63,731.39	.92	2.31	.50	.87	2.74	.47
NLD-SHDP	x	x	x	x	—	−60,346.97	−62,617.85	.92	2.32	.51	.87	2.67	.47

Notes: CV = control variables; RE = random effects; G = genres; AF = acoustic features; LL = log-likelihood; PSIS = Pareto smoothed importance sampling; AUC = area under the curve; average estimated probability of failure (i.e., not appearing on the charts) for the noncharting albums; LS = linear and static.

The next set of two models adds the acoustic features as a model component. Model NLS (nonlinear and static) incorporates the mean acoustics and the other continuous variables nonlinearly via the penalized splines, but in a static fashion, whereas Model NLD (nonlinear and dynamic) accounts for both nonlinearity and dynamics via the AR(2) specification. The third set of NLD-LDA models incorporates the textual tags in a two-staged approach. First, the themes are estimated using an ordinary LDA model with a fixed number of themes. The variants in this set differ in the number of themes $K_{ϕ} = {20, 40, 60}$ . The LDA yields theme membership probabilities $ρ_{i}$ for each album i. These are used in the second stage as covariates in the parametric models. The genres, acoustic fingerprints, theme memberships, and other control variables are included linearly in the link function.¹³ Because the $ρ_{i}$ vectors sum to 1 (i.e., $\sum_{k = 1}^{K_{ϕ}} ρ_{ik} = 1$ ), an intercept is not identifiable. We therefore fixed it to 0.

The final model, NLD-SHDP, is the proposed model. It uses the SHDP to automatically infer the number of themes, all in one step.

Estimation details

We used MCMC methods to estimate these models. For the benchmark models, we ran the Markov chain for 10,000 iterations. The last 5,000 post-burn-in draws are used for reporting the results. For Model NLD-SHDP, we ran the chain for 100,000 iterations and retained the last 10,000. The estimation time for this model depends on the vocabulary size. Given the large size of our vocabulary, this model takes about two weeks to sample 100,000 MCMC draws. The most time-consuming part of the computation stems from SHDP. Next, we focus on the predictive performance of these models.

Predictive performance

We evaluated the predictive performance of the models using the log-likelihood (LL) and the Pareto smoothed importance sampling leave-one-out statistic (PSIS-LOO), which is asymptotically equivalent to the Watanabe–Akaike information criterion but is more robust in finite sample settings (Vehtari, Gelman, and Gabry 2017). The PSIS-LOO accounts for both model fit and complexity via an implicit penalty for complex models. It uses PSIS to implement LOO cross validation by reweighting observations. It is a measure of out-of-sample predictive accuracy, as it relies on LOO procedure for cross-validation (across many calibration/training and holdout/test data splits).The model with the highest PSIS-LOO is considered the one with the best predictive ability when comparing a set of models.

We also used a traditional calibration and holdout data splitting strategy to further assess predictive performance. For this, we randomly split the data set into a calibration sample with 30,828 observations and a holdout data set of 3,386 observations. The holdout sample contains 10% of the albums in each year. We estimated all models on the calibration data and used the estimated parameters for holdout predictions. The theme membership indicators for the holdout albums are sampled with the themes fixed to those from the calibration sample and by ignoring the information in the success scores. We then used the area under the curve (AUC) of a model’s binary receiver operating characteristic curve (Swets 2014) to assess the closeness of the predicted and actual probabilities of an album’s appearance on the charts. We also used the root mean square error (RMSE) of the observed and estimated success scores for albums that made it to the charts. Finally, we used the average estimated probability of failure (i.e., not appearing on the charts) for the noncharting albums, p(nc). This statistic assesses the capability of detecting noncharting albums. Table 3 reports these statistics both for the calibration and holdout data.

The PSIS-LOO statistics in Table 3 show that the proposed model, NLD-SHDP, has the best predictive performance across the multiple LOO cross-validation data sets. This is confirmed from the last row of the table, which indicates that the model has the best in-sample and out-of-sample performance (for the one validation data set) across all metrics. Looking across the benchmark models, we note that the addition of artist random effects and acoustic features results in considerable improvements in predictive performance. The addition of dynamics improves the AUC and RMSE measures. Finally, we see that adding the nonparametric SHDP component improves the PSIS-LOO statistics and the in-sample as well as out-of-sample AUC and RMSE measures. The table shows that the addition of the random effects has the greatest impact. However, this could also reflect the order in which the different components are added across the rows in the table. Having compared our model with other benchmarks, we next briefly discuss our results regarding the random effects, genres, acoustics, and themes.

Random Effects

As discussed previously, artist random effects contribute significantly to the models’ predictive power. Figure 7 shows the distribution of the posterior means of the estimated random effects, which has a long left-tail. This mirrors an album’s difficulty in making it to the charts, as 38.07% of the artists have negative estimated random effects. Consistent with our expectations, popular artists such as George Winston ( $η_{s} =$ 7.38), Josh Groban ( $η_{s} =$ 5.25), Michael Bublé ( $η_{s} =$ 5.06), and Norah Jones ( $η_{s} =$ 5.05) have large random effects. Artists with smaller random effects are less successful, such as Boney M. ( $η_{s} =$ −7.83), Heaven 17 ( $η_{s} =$ −7.52), and Sarah Vaughan ( $η_{s} =$ −7.39).

Figure 7.

Distribution of estimated random effects.

Dynamics of Genre Popularity

We briefly illustrate how our model captures the dynamics of genre popularity. Figure 8 shows how the appeal of rock and pop genres has changed over time. The vertical axis in each plot indicates the estimated popularity of each genre.

Figure 8.

Patterns in the popularity of rock and pop over time.

We see that the model captures the multiple peaks associated with rock music, the most produced genre in our data set. We observe the increasing trend after the “British invasion” of American popular music in the 1960s, with the Beatles’ albums such as Revolver, The Beatles (“The White Album”), and Sgt. Pepper’s Lonely Hearts Club Band. Albums such as Pink Floyd’s The Dark Side of the Moon drove rock music to a peak in the 1970s. Another peak is observed in the 1990s, corresponding to the rise of alternative rock music and the release of Nirvana’s Nevermind, which topped the charts in 1991. Other hard rock and heavy metal bands such as Aerosmith and Metallica also topped the charts. Since the 1990s, the popularity of rock has waned and fragmented into specialized forms, while pop and hip hop have risen in popularity.

Pop has had significant periods of popularity. The 1960s peak in Figure 8 corresponds with the release of More of the Monkees by the Monkees and Herb Alpert and the Tijuana Brass’s Whipped Cream & Other Delights. New styles of pop emerged in the 1980s, the most notable being Michael Jackson’s Thriller and Prince’s 1999. Recently, pop has again shown a positive trend in popularity with the launch of albums such as Taylor Swift’s 1989, Adele’s 21, and Ed Sheeran’s ×. The model also recovers interesting patterns for the other genres, which are presented in the Web Appendix.

Acoustic Balance and Album Success

We next illustrate how the success of an album is related to the average levels of the acoustic features and their standard deviations across its songs. As we have noted, our acoustic fingerprints contain “technical” and “experiential” features. Technical features refer to physical characteristics of the music itself, and experiential features relate to how the listener experiences the music. We start with how the mean acoustics measures of an album are related to success.

Mean acoustics

Recall that we modeled album success to be nonlinearly related to the acoustic means and allowed the nature of this nonlinear relationship to vary across the years. Our results indicate that such flexibility is needed, as many of the acoustics indeed exhibit dynamic nonlinear effects, as shown in Figure 9. The estimated effect of each acoustic feature is represented in the subfigures. For a given acoustic feature, the right axis refers to the mean of that acoustic across all the songs in the album and the front axis represents the calendar years. The vertical axis gives the value of the estimated effect of the acoustic fingerprint. The shading in these figures reflects the magnitude of the estimated effects, with red corresponding to a higher value.

Figure 9.

Estimated nonlinear effects for the means of the acoustic fingerprints.

Figure 9 shows that certain acoustics exhibit a consistent pattern of impact over the years. These include acousticness, duration, instrumentalness, loudness, speechiness, and valence. We begin with acousticness, which represents the extent to which a song contains natural sounds such as a guitar or harmonica, rather than electronic sounds. Our results indicate that lower levels of acousticness are associated with success across the years. For duration, we see that albums containing long songs are, on average, less successful, and this effect becomes accentuated beyond a threshold. We also see that albums with low average instrumentalness (i.e., albums that contain more vocals) were relatively more successful. Interestingly, loudness has a pronounced nonlinear pattern and has been positively associated with success over the years. This is consistent with Serrà et al. (2012), who also found that popular songs have become louder over the years. Speechiness, which captures the presence of spoken (rather than sung) words that can evoke emotions, tell stories, and convey messages, has been positively related with success. Another interesting pattern pertains to valence, which shows that albums with negative valences (sad or angry) have been more successful. This is consistent with findings by Koelsch et al. (2006) that sad music can be more enjoyable because it has a greater aesthetic appeal and activates other positive emotions (Scherer 2004; Zentner, Grandjean, and Scherer 2008).

Other acoustic features show interesting patterns that exhibit both nonlinearity and time variation. For example, regarding explicitness, we find that albums with a greater level of explicit language have been more appealing across the years. We also see that danceability, which captures whether the album evokes a desire to dance, has been positively related to success throughout the study period. There was several peaks in its impact prior to the 1990s as the popularity of dance clubs expanded. This trend began with the twist dance craze of the late 1960s and continued during the disco craze of the 1970s. Musical energy represents the experienced energetic strength of a song. For example, a rock-metal track has more energy than a smooth jazz track. Our results suggest that albums that had high energy were, on average, more successful than albums with low energy. Rock and other types of dance-oriented music, which were popular in the 1960s, were highly energetic. The peak prior to the 1970s reflects the popularity of disco and rock (with the rise of Led Zeppelin and Stevie Wonder). Similarly, liveness shows an interesting pattern of impact. Live recordings convey the energy and the experience of attending a concert or live performance by an artist or band. Albums focused on songs that were recorded live were most successful in rock music, particularly albums produced during the 1960s and 1970s. Since then, the trend has reversed, and albums featuring live recordings have been less successful.

The technical acoustics also exhibit interesting dynamic patterns. Keys map songs to corresponding pitches, and some keys are more convenient than others for composing music on certain instruments (e.g., keys of E, C, and G for piano and guitar). We see that successful albums generally were composed in a low key, with the exception of albums in the 1960s. Music composers use minor and major modes to create certain moods in their songs. Major modes convey feelings of stability and happiness (e.g., Verdi’s “Grand March” from Aida). Minor modes often sound sad and evoke bittersweet feelings (e.g., Chopin’s Prelude in E-Minor). Our results show that albums containing, on average, more songs recorded in a major mode were more successful before 1970. Since then, albums with minor modes have been more successful, with an exception during the 1990s. Time signature is a measure of the number of beats in a bar and can reflect the complexity of the music. The time-signature number associated with an album is the average of the time signatures present across the songs in the album. For example, jazz music has higher time signature values than other genres. We see that successful albums have had predominantly low levels of average time signature, with the exception of albums produced in the 1960s and 1970s, which inherited aspects of 1950s jazz music. Successful albums in the subsequent years are characterized with lower average time signatures and are therefore musically less complex. Finally, tempo characterizes the speed of music. A slow tempo tends to evoke sadness, while a fast tempo tends to induce feelings of happiness and excitement. We find that, on average, albums with faster tracks generally have been more successful in most years mainly during the 1960s (for rock albums), early 1980s (emergence of new rock genres such as punk and heavy metal), and the recent 2000s.

Acoustic standard deviations

The acoustic standard deviations capture the extent to which the albums are balanced on the acoustic fingerprints. Recall that we assumed that the effects of the standard deviations of the acoustic features are linear and static. Our model yields interesting relationships between these variables and albums' success. Table 4 shows the estimated success coefficients of these standard deviations.

Table 4.

Estimated Linear Effects for the Standard Deviations of the Acoustic Fingerprints.

Acoustic Feature	Posterior Estimate	Acoustic Feature	Posterior Estimate
Acousticness	.22 (−.21, .66)	Liveness	.39 (−.15, .89)
Danceability	.79 (.24, 1.33)	Loudness	3.71 (2.56, 4.85)
Duration	.22 (−1.40, 2.10)	Mode	.13 (−.45, .90)
Energy	.55 (−.06, 1.17)	Speechiness	−.42 (−1.38, .28)
Explicitness	.95 (.60, 1.35)	Tempo	1.04 (.59, 1.51)
Instrumentalness	−1.23 (−1.56, –.91)	Time signature	−.50 (−.98, −.01)
Key	.12 (−.35, .61)	Valence	−.06 (−.52, .40)

Notes: Parentheses indicate the 95% posterior interval.

We observe that six acoustic standard deviations exhibit significant effects on album success. Some fingerprints have positive coefficients suggesting a preference for variety across songs within an album. These include danceability, energy, explicitness, loudness, and tempo. These fingerprints are associated with the intensity of the music and suggest that successful albums alternate between high- and low-intensity songs, which could give the listeners a moment to rest between intense experiences. Fast tempos can also become tiresome, while slow tempos are more relaxing. Slow beats also feel more natural because they are closer to the natural rhythm of the human heart (Iwanaga 1995). Successful albums also contain songs that are similar in terms of instrumentalness and time signature. These fingerprints are technical musical features. Recall that instrumentalness indicates if a particular song contains any vocals. These results suggest that diversifying musical complexity or frequently alternating the performance styles (i.e., instrumental/vocal) of the songs within an album can result in less successful outcomes.

Thematic Insights

The SHDP component yielded 75 themes. These are the atoms of the $G_{0}$ distribution with substantial weights. After 40,000 iterations, the number of themes stabilized between 70 and 78 and the discovery of themes was frozen for the last 20,000 iterations. We report results based on the last 10,000 iterations, for which the number of themes was fixed to the modal value of 75.

Theme distribution and prevalence

Table 5 shows the top tags (i.e., those with the highest probability of occurring within the theme) associated with the most prevalent themes ( $p (ϕ_{k}) \geq .01$ ). The second column of the table shows the success coefficient associated with the theme $ϕ^{y}$ , whereas the third column shows the probability with which the theme appears across all the tag applications. These 42 themes account for 90% of all themes represented within the albums. The themes yield meaningful categories: specific types of music, artist gender, musical eras, and different groupings related to music consumption. For example, themes 14, 17, 23, and 35 refer to subgenres (e.g., country pop, metal rock, jazz, and Christian). Theme 54 is about female vocalists singing love songs. Themes 19, 36, 37, 58, and 65 refer to the music from particular decades (1960s, 1970s, 1980s, 1990s, and 2000s, respectively). Theme 37 is also about the British influence, theme 62 is about Christmas and holidays, and theme 23 is about smooth jazz. The themes are ordered in descending order of their success coefficient, $ϕ^{y}$ in Table 5. For example, the presence of theme 14, which is about modern country music, improves by .25 the success score of an album. However, theme 64, which is about indie rock, negatively impacts the success score by −.12.

Table 5.

Description of Top Themes with $p (ϕ_{k}^{w}) \geq .01$ .

Theme k	$ϕ_{k}^{y}$	$p (ϕ_{k}^{w})$	Top Tags
13	.26	.011	2015, best of 2015, bbk, best albums of the 2010s, t tucker, wants, 10s
14	.25	.028	country, library, entire, my whole damn collection, modern country, country pop, honky tonk, contemporary country, c m rae, country music
17	.21	.025	heavy metal, metal, thrash metal, progressive metal, death metal, groove metal, speed metal, melodic death metal, power metal, mam
19	.13	.02	60s, southern rock, psychedelic, 1969, 1968, psychedelic rock, 1967, 1966, male vocalist, debut
20	.11	.013	alternative metal, nu metal, piano, basically bass, metal, ska, 2017, rapcore, ska punk, lopez 88
23	.07	.04	jazz, smooth jazz, jazz fusion, fusion, saxophone, jazz piano, jazz guitar, trumpet, modernjazz, jazz rock
26	.06	.016	easy listening, jazz, swing, jazz vocal, vocal jazz, vocalistas femeninas, mastersinger, lounge, 2014 albums, nat king cole
27	.04	.013	disco, industrial, country rock, industrial metal, covers, cover, industrial rock, gothic rock, stoner rock, gothic
28	.03	.013	oldies, flashback alternatives, 50s, kids, male vocalists, close harmony, e harris, everything, rotfl, blue-eyed soul
29	.03	.033	soul, funk, motown, rhythm and blues, jazz-funk, funky, california, aleister crowley, aleyster crowley, argeu california seixas
30	.02	.01	instrumental, bossa nova, awesome, 1978, surf, schlageroldies, jam, 60’s, a gilberto, favorites
31	.02	.03	pop, electronic, dance, male vocalists, synthpop, electronica, house, electropop, british, dance-pop
32	.01	.068	albums i own, favorite albums, favourite albums, 00s, 2006, rock, 2005, 2003, 2002, 2000s
33	.01	.012	2013, bernstein, best of 2013, my country, property, mmfwcl, 10s, detroit, psychopathic, liblistened
34	.00	.038	alternative rock, rock, alternative, pop rock, indie, grunge, post-grunge, power pop, emo, britpop
35	.00	.014	christian, christian rock, contemporary christian, new age, guitar virtuoso, l pavarotti, sweet, religious, gospel, instrumental rock
36	.00	.02	90s, 1993, 1990, 1998, 1996, 1999, 1991, 1994, favorites, philly soul
37	.00	.033	70s, classic rock, rock, 1973, 1974, 1979, british, 1970s, 1972, 1975
38	.00	.022	laptop, i have this album, usa, male singer songwriter, american musician, female singer songwriter
39	−.02	.017	experimental, electronic, ambient, chillout, post-rock, trip-hop, experimental rock, downtempo, alternative, beautiful
40	−.02	.034	rap, hip-hop, hip hop, my collection great 150 albumz of rap, gangsta rap, east coast rap, best album, hiphop, west coast rap, real life rhymes
41	−.02	.041	rock, hard rock, classic rock, heavy metal, canadian, glam metal, album rock, hair metal, rock n roll, arena rock
43	−.03	.016	progressive rock, classic country, 1970, progressive, space rock, art rock, symphonic rock, pickett, psychedelic rock, rock progressif
44	−.03	.017	blues rock, blues, guitar, musicalmessradio, sympathy68, guitar virtuoso, blues-rock, raga music, song noir, stonedbeat
45	−.04	.013	folk, 1001 albums you must hear before you die, britten, irish, celtic, leadbelly and co, m haggard, not streamable, country albums to check out
46	−.06	.034	80s, new wave, post-punk, happy music for happy people, 1988, 1985, 1986, 1987, 1984, 1980s
47	−.06	.026	rnb, soul, r&b, neo-soul, smooth, sexy, neo soul, new jack swing, classic soul, urban
50	−.07	.017	punk, pop punk, punk rock, grammy nominated, 1983, 1980, bs orchestra, melodic hardcore, regine-disc, emo
51	−.07	.011	country christmas, own on vinyl, nelson, 1972, playlist, allboutguitar, 1981, fully streamable albums, brc blues band karlsruhe
52	−.08	.015	blues, soundtrack, killforpeace, bluezzz, frank sinatra, etta james, music i tried but didnt like, vandellas, best albums of the 1960s, star wars
54	−.09	.026	female vocalists, pop, female vocalist, soft rock, love songs, american, adult contemporary, do nagrania, london symph, female
56	−.09	.023	singer-songwriter, folk, folk rock, acoustic, soft-rock, outlaw country, 1982, deshannon, folk-rock, mellow
58	−.10	.011	2012, best of 2012, 10s, donald byrd, 2010s, 1963, rich, modern country, johnny cash, stuff i need
59	−.10	.014	latin grammy nominated, latin, latin jazz, got it on 8 track, latin pop, reggaeton, 1970’s, salsa, m callas, 1960’s
61	−.10	.011	americana, alt-country, bluegrass, rockabilly, rock and roll, 1989, my gang 10, electro, bubblegum, gl4 m
62	−.11	.016	christmas, 2010, xmas, best of 2010, holiday, christmas easy listening, christmas music, weihnachten, 1965, christmas songs
63	−.11	.018	metalcore, post-hardcore, hardcore, deathcore, metal, emo, screamo, emocore, melodic metalcore, mathcore
64	−.12	.025	2011, indie, indie rock, 2008, indie pop, 10s, best of 2011, best of 2008, dream pop, weallgetold
65	−.15	.017	2014, best of 2014, 10s, in my collection, 2014
66	−.17	.012	comedy, gospel, spoken word, stand-up, love, 1975, rb, freestyle, contemporary gospel, elvis presley
68	−.22	.01	reggae, roots reggae, dub, gangsta rap, chill, bay area, rap-a-lot records, dancehall, 2001, w judd
69	−.26	.012	vinyl i own, classical, opera, piano, contemporary classical, babyfacepatsy, west coast rap, wb recording, glass, orchestral

Notes: These 42 themes account for 90% of all the themes of the entire tags.

Figure 10 shows the average of the theme proportions of all the albums within each decade. A darker cell means that the corresponding theme is present in higher proportions among the albums in a decade. This can be verified by looking at the themes in Table 5 that focus on particular decades. These themes have a higher density in the corresponding row in Figure 10. For example, theme 19, which has “60 s,” “1966,” and “1969” as the top tags in Table 5, peaks exactly during the 1960s, and this is true for each of the decade-related themes—theme 32 (2000s), theme 36 (1990s), theme 46 (1980s), theme 37 (1970s), and theme 19 (1960s).

Figure 10.

Theme proportions across decades.

The themes that are associated with different styles and listening experiences in Table 5 also appear in the relevant years in Figure 10. For example, rap and hip hop (Theme 40) was more prevalent in the 1990s and 2000s. Classic rock, hard rock, and garage rock (Theme 41) appeared more in the 1970s and 1980s. This was driven by the British invasion (the Beatles, the Rolling Stones) in the 1960s. Hard rock became mainstream with the success of bands such as Iron Maiden, Saxon, and Def Leppard. Jazz, swing jazz, and saxophone (Theme 23) appeared more frequently in the 1960s albums, consistent with the success of vocal jazz and swing bands in the 1950s. This consistency across periods, styles, and listening experiences validates that our model discovers coherent themes.

Themes proportions for one album

Recall that an album is a mixture of themes, and therefore, the bag-of-tags representation of an album draws from different themes. Figure 11 shows the theme proportions for 21 by Adele and Speak Now by Taylor Swift. The theme proportions provide striking descriptions of these studio albums. The two albums draw on themes 32 and 54, but with different weights. Theme 32 is strongly associated with successful music of the 2000s, and theme 54 is associated with female vocalists, pop rock, and adult contemporary, as the two albums display a range of emotion—bitterness and moving forward—after a broken relationship. The two albums differ on several themes. For example, Speak Now is strongly associated with theme 14, modern country music. Themes 29 and 64 are strongly associated with Adele’s 21. Theme 29 refers to Motown, which is one of the influences of Adele’s previous album 19 and exhibits soul music inflections that are present in 21, and theme 64 refers to indie rock and pop of the 2010s.

Figure 11.

Theme proportions for Adele’s 21 and Taylor Swift’s Speak Now.

Major Label, Featured Artists, and Superstardom

Figure 12 exhibits the importance of major labels, superstardom, and collaboration among artists. We see a positive and increasing impact of major labels on album success over the years. This is not surprising given the greater expertise and resources of major labels.

Figure 12.

Estimated effects of major label, the number of featured artists, and superstardom.

The figure also shows a negative effect when multiple artists are featured in the album. The effect is also consistent across the years. Finally, artist superstardom, measured by the number of previous charting albums, positively impacts album popularity as in Bradlow and Fader (2001). Whereas these authors focused on movements of songs within a year, we examine how album success is impacted over the years. We find that this effect is nonlinear and is consistent for most years. It appears that the effect kicks in only after a threshold of artist popularity is reached and then wanes as the artist ages.

Having described the qualitative insights from our model, we now show how its different components can be used for a variety of marketing decisions.

Album and Playlist Scoring, Recommendation, and Design

We illustrate how the components of our model can be used for different managerial tasks. In selecting these applications, we emphasize the utility of the acoustic fingerprints and the user-generated tags. The applications include (1) forecasting the performance of new albums, (2) conducting album diagnostics and tuning, (3) recommending albums that are consistent with the musical styles associated with specific eras, and (4) designing contextual playlists.

Forecasting the Performance of New Albums

Consider a new album that is yet to be released. Our model can be used to predict its latent score and the probability $p (y^{*} \geq 0) = 1 - Φ [- F (x, a, a^{sd}, w) / σ]$ that it will appear on the charts ( $Φ$ denotes the cumulative distribution function for the unit normal distribution). To evaluate the forecasting performance, we reestimated our model and three competing models using data up to 2015. We then used them to predict the charting probabilities for the 751 albums (551 charted, 200 uncharted) in our data set that were released in 2016.

The models and the results are available in Table 6. The first three models are variants of our full model and differ in the types of data they use. Model NLD1 accounts for artists’ random effects and control variables. Model NLD2 adds genres to NLD1, and NLD3 adds the acoustic features to the previous model. The last model leverages all data modalities, including the tags. We used the state equation (Equation 6) to project forward the time-varying components of our model (i.e., the linear effects and the linear basis of the splines). We used the themes that are estimated on the training data and assigned each tag of the test albums to one of these themes in a semisupervised fashion (no new themes were considered).

Table 6.

Forecasting Performance of Our Model and Alternatives.

Model	NLD1	NLD2	NLD3	NLD-SHDP
AUC	.89	.89	.91	.92
RMSE	2.06	2.03	2.02	1.91

Table 6 reports the AUC based on the estimated charting probabilities and the RMSE of the estimated scores of the charting albums. Our model has the best predictive power among the four models, especially with a much lower RMSE value of 1.91. Finally, Table 7 shows how our model predicts in different prediction quartiles (Q1 is the lowest quartile and Q4 is the highest). The entries in the table are obtained by sorting the charting albums into four quartiles based on the predicted score and computing the fraction of these albums that also fall in the same quartile based on their actual scores. The entries along the main diagonal show that the model’s predictions are aligned with the actual scores.

Table 7.

Forecasting Performance of Our Model in Predicting the Different Quartiles of the Year-End Score.

Actual	Predicted
Actual	Q1	Q2	Q3	Q4
Q1	.39	.23	.19	.20
Q2	.37	.35	.20	.08
Q3	.17	.30	.31	.21
Q4	.06	.12	.30	.51

Album Diagnostics and Tuning

Our model can be used for album diagnostics, tuning, and balancing. For an illustration, we consider two albums released in 2014, Partners by Barbra Streisand and The Pye Anthology by Petula Clark. Our model correctly predicted that Partners, which has an estimated (actual) log-score of 7.82 (7.60), would appear on the Billboard 200 and that The Pye Anthology, which had an estimate score of −.34, would not.

Figure 13 shows the values of three acoustic components for the two albums and their contributions to album success.¹⁴ Both albums have low scores on mean speechiness (absence of spoken words in their songs) and high scores on average tempo. However, the two differ significantly on the mean valence. Partners has lower average valence (on average, sadder songs). This results in a gap of about 20 points on the original year-end scale between the two albums. Our model suggests that Clark’s album would have a higher probability of appearing on the charts if it had a lower average valence. The two albums have similar standard deviations for the acoustics. Both could benefit from more diversity in tempo and danceability. Additional details on these acoustical comparisons are available in the Web Appendix.

Figure 13.

Acoustic diagnostics of Barbra Streisand’s Partners and Petula Clark’s The Pye Anthology for musical success in 2014.

Scoring Playlists for Different Generations of Listeners

As noted previously, Holbrook and Schindler (1989) show that people’s preferences for popular music are formed in early adulthood. We can use the proposed model to predict the score for an album or playlist in different years and recommend it to listeners whose musical tastes match the periods in which it has high scores.

We used our model to score a playlist recommended by a website, TheMortonReport.com, that specializes in culture and entertainment. A weekly column on the website, “New Music for Old People,” recommends music to “fill the gap for those of us who were satiated musically in the ‘60s and then searched desperately as we aged for music we could relate to and get the same buzz from nowadaze” (Kooper 2015). Table 8 shows the playlist of songs recommended on April 17, 2015. All songs, other than Billy Swan’s “Don’t Be Cruel,” were released in the 2000s. We also scored three additional playlists, available on Spotify, that are hand-curated and recommended by another website, NewMusicforOldPeople.com, in 2015 and 2016. The purpose of this website is to recommend new songs that would appeal to Baby Boomers. Vince Dunbar, the website owner, says, “there is still a lot of music being made specifically to appeal to us, or at least with sensibilities we understand” (Dunbar 2015).

Table 8.

New Music for Old People Playlist.

Song Name	Artist/Group	Release Year
“Run”	Delta Rae	2015
“Honey”	The London Souls	2015
“I Can See for Miles”	Vanilla Fudge	2015
“It’s a Man’s Man’s Man’s World”	Gov’t Mule	2015
“Glimmer (Orchestral)”	Neil Young	2014
“Don’t Wait”	Mapei	2014
“Nice and Slow”	Max Frost	2013
“Long Haul”	Michael McDonald and Robben Ford	2013
“Tonight Insomnia”	Eye to Eye	2005
“Don’t Be Cruel”	Billy Swan	1973

We collected and used the information on the acoustic features to score all the songs in a playlist for all 54 years. Figure 14 shows that the estimated ranks for these four playlists indeed do peak in the 1960s and that these ranks decrease with time.

Figure 14.

Scores across different years for the “New Music for Old People” playlist and “New Music for Old Folks” playlists.

Playlist Compilation

Next, we show how our model can be used to compile playlists. To illustrate, suppose our task was to construct a playlist of five songs selected from Adele’s albums 19, 21, and 25. We could choose her five greatest hits. But our model suggests choosing a different mix of songs tailored to match the preferences of people whose musical tastes better match one or another time period.

We used the acoustic features of the songs to score all possible playlists consisting of five songs selected from the two albums. We scored the playlists separately for 1971 and 2016. Table 9 shows the highest-scoring playlists for each year, and Adele’s top five hits according to Billboard critics (Arnold 2021). The playlists have one song from Adele’s five greatest hits (“Rolling in the Deep”) and have only one common song (“Don’t You Remember”). Comparing the 1971 and the 2016 playlists, we observe that the 1971 playlist has higher average mean mode and marginally happier songs than the 2016 playlist.

Table 9.

Adele’s Top Five Billboard Songs and the Five Songs Generated by the Model for Listeners from Different Eras.

Billboard’s Top 5 Songs		Model-Generated Compilations
—		1971		2016
Album	Song	Album	Song	Album	Song
21	“Rolling in the Deep”	19	“Tired”	19	“Cold Shoulder”
19	“Chasing Pavements”	25	“Sweetest Devotion”	25	“I Miss You”
21	“Someone Like You”	21	“Rolling in the Deep”	25	“Million Years Ago”
25	“Hello”	21	“Don’t You Remember”	21	“Rolling in the Deep”
19	“Hometown Glory”	21	“One And Only”	21	“Don’t You Remember”

Contextual Recommendations

Tags contain information on the context in which a song or album is heard and the mood it evokes. We can use this information to recommend albums and playlists for particular contexts. Contextual playlists have become increasingly popular on platforms like Spotify (see Joven 2018). A context may be an activity (e.g., romantic) or a temporal event (e.g., Christmas).

We consider a hypothetical situation where a user queries a recommendation system by specifying tags from our vocabulary that describe a context. Our model can then be used to respond with a matching playlist or album. We assume that all the tags have the same importance and convert the query to a bag of 100 tags, where each tag appears with the same frequency. This results in a bag-of-tags representation that is similar to the one available in our data.

Then we extract the corresponding thematic representations by sampling the thematic allocations of each tag in the contextual query (without updating the thematic profiles of the training albums). We retrieve the thematic membership of the query after 100 iterations. Let $ρ_{ik}$ denote the weight of theme k for album i, and $ρ_{qk}$ the weight of theme k for query q. Let S denote a target set of albums. We use the Hellinger distance (Nikulin 2001),

H (i, q) = {(\frac{1}{2} \sum_{k = 1}^{K} {(\sqrt{ρ_{ik}} - \sqrt{ρ_{qk}})}^{2})}^{\frac{1}{2}},

to measure the closeness of album

i \in S

to query q and recommend albums with the smallest

H (i, q)

values in S. We illustrate the method for two contexts.

The first context considers a listener looking for a playlist for a romantic evening. Suppose the user selects the tags “romantic,” “smooth,” and “night.” The second context considers a listener looking for recommendations for a party. Suppose this user selects the tags “disco,” “dance,” “fun,” and “party.” Table 10 shows the recommendations obtained from our model for each context. The first query implies a theme proportions vector which places almost all its weight on two themes. The first of these themes has a proportion of .67 and relates to easy listening, jazz, and swing. The second theme has a probability of .33 and relates to R&B, soul, neosoul, smooth, and sexy music. We identified three albums that are thematically closest to this context in Table 10. These albums appear appropriate for this context.

Table 10.

Contextual Recommendations.

Context 1: Romantic, Smooth, Night
Themes (Top 5 Tags)	Theme Proportion
easy listening, jazz, swing, swing, jazz vocal	.67
rnb, soul, neosoul, smooth, sexy	.33
Closest Three Albums
Paul Young, Paul Young; Norman Connors, You Are My Starship; Mobb Deep, Free Agents
Context 2: Fun, Party, Disco, Dance
Themes (Top 5 Tags)	Theme Proportion
pop, electronic, dance, male vocalists, synthpop	.78
disco, industrial, country rock, industrial meta, covers	.22
Closest Three Albums
Kylie Minogue, Fever; RuPaul, RuPaul.ReWorked; The Smithereens, Blow Up

Incidentally, the second context also places most of its probability on two themes. The first theme relates to pop, electronic, and dance music and the second to disco and industrial music. The three albums thematically closest to this context in Table 10 are indeed appropriate for this context.

Conclusion

We developed a novel multimodal machine learning framework to model the success of creative products in terms of commonly available metadata, experiential attributes that capture the felt experience of consumers, and themes that summarize the semantic content of user-generated text. We then illustrated this framework using data on music albums from the past five decades. The different components of our modeling framework were specifically designed to assist in several managerial tasks of interest to marketers. The framework uses different types of Bayesian nonparametric components, including penalized splines for flexibly capturing the effect of acoustical fingerprints and a novel SHDP component to infer themes that are predictive of success. The model also includes dynamics that capture the evolving impact of the genres and acoustics over the years. Many aspects of our model are novel. For instance, the use of SHDPs is new to marketing, and our integration of different types of nonparametrics is a contribution to the wider statistical and machine learning literature streams.

We demonstrated the predictive superiority of our model by comparing its performance with several competing specifications. Together, these benchmark models span different sets of covariates and accommodate static, dynamic, and nonparametric effects. More importantly, in a series of applications we demonstrated why we need the different components of our modeling framework for a variety of marketing tasks. Specifically, we showed how the model can be leveraged to (1) recommend albums that are consistent with the musical styles associated with specific eras or generations of listeners, based on the nonlinear and dynamic components; (2) fuse different albums to create playlists for specific tastes; (3) recommend contextual playlists and albums based on queries that users may make to a recommendation engine, using the inferred themes; (4) forecast the performance of new albums, before they are released, using the time-varying coefficients; and (5) diagnose album performance and tune albums based on the acoustical features that may promote or inhibit success.

Our results also yielded several insights into the evolution of American popular music. Some of our findings relate to broad trends in the popularity and decline of different genres and the emergence of new forms of music. The results also uncovered how different acoustics have been instrumental in album success and how their relationship with success has waxed and waned over the years. The SHDP component of our model identified several themes. These themes, which are probability distributions over the tags, represent how listeners perceive and experience the albums. The themes emphasize different types of tags and together capture aspects of the album that are not tapped by the metadata and experiential covariates. The uncovered themes can be characterized as focusing on subgenres (e.g., soft rock, new soul, jazz vocal), particular eras (1970s, 1990s), artist details (male singer-songwriter, female vocalists), consumption contexts (e.g., holiday, Christmas), nostalgia (e.g., oldies, flashback alternatives), mood (e.g., smooth, chill out), and other aspects of music. Together, they offer a vivid summary of how listeners experience and perceive music.

Although some aspects of our model were constructed specifically for music, our model can also be used in other experiential contexts where multimedia data are available. These include the other creative contexts that we highlighted in Table 1. In all these contexts, it is important to capture the user’s felt experience so as to predict the success of different items, and textual collections can be leveraged to infer themes.

While we showcased several marketing applications of our model and obtained several substantive insights, it is also important to acknowledge some limitations of our research. Although we collected the best available data, we hope to leverage better data for future research. Specifically, actual album sales should improve the estimation and eliminate any aggregation bias present in our year-end score. Finally, it is important to quantify the interplay between the success of single songs and albums. Data that provide accurate dates of single releases and their commercial success in parallel with album success would be an excellent addition to this research. Methodologically, we used MCMC methods for inference because this allows for proper quantification of uncertainty. However, MCMC methods are difficult to scale. Future research can explore variational Bayesian alternatives and truncated approximations to the HDP for reduced computational time. Finally, for simplicity, we assumed that the variance of the error term for the Tobit model is static. This assumption could be relaxed in future research.

Our results describe how popular music has evolved over time, but although we accounted for artist and time-varying effects, the use of observational data means that we cannot fully assert causality about the effects of the covariates. While we have extensively validated our models on holdout data of different types, we have not tested our recommendations in a real-life setting. Although our model can be used to construct playlists for listeners with different types of preferences, we cannot recommend individualized playlists, as we do not have access to individual-level data. However, it is relatively straightforward to capture such sources of individual heterogeneity if such data were available. We modeled the themes and the acoustics additively in our framework. It is possible that they may have an interactive impact. Success can also depend on other covariates such as advertising, competing albums, and factors considered by previous researchers that we did not have access to. Although the introduction of the textual tags did not lead to significant improvement in the predictive performance of our model in our application, we expect that this could vary across data sets, and the different data modalities may vary in their relative contributions. Finally, the tags for the older albums (pre-2002) represent current perceptions. Unfortunately, we did not have access to contemporaneous perceptions, so we could not use them. We hope that future researchers will work toward alleviating some of these limitations and use our modeling framework in other experiential contexts outlined in Table 1.

Footnotes

Acknowledgments

This paper is based on the first essay of Khaled Boughanmi’s dissertation. We thank Rajeev Kohli for his insightful suggestions and valuable contributions to the development of this paper. We thank David Blei for his insightful comments on the Hierarchical Dirichlet Process and Michael Mauskapf for his valuable suggestions and comments about the music industry. We also thank the W. Edwards Deming Center of Columbia Business School for their generous financial support for this research.

Associate Editor

Eric Bradlow

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Online supplement:

Notes

References

Alexander

Peter J

. (1996), “Entropy and Popular Culture: Product Diversity in the Popular Music Recording Industry,” American Sociological Review, 61 (1), 171–74.

Anand

Narasimhan

Peterson

Richard A.

(2000), “When Market Information Constitutes Fields: Sensemaking of Markets in the Commercial Music Industry,” Organization Science, 11 (3), 270–84.

Ansari

Asim

Iyengar

Raghuram

(2006), “Semiparametric Thurstonian Models for Recurrent Choices: A Bayesian Analysis,” Psychometrika, 71 (4), 631.

Ansari

Asim

Mela

Carl F.

(2003), “E-Customization,” Journal of Marketing Research, 40 (2), 131–45.

Arnold

Chuck

(2021), “Every Adele Song Ranked: Critic’s Picks,” Billboard (May 5), https://www.billboard.com/articles/columns/pop/8448730/adele-songs-ranked.

Askin

Noah

Mauskapf

Michael

(2017), “What Makes Popular Culture Popular? Product Features and Optimal Differentiation in Music,” American Sociological Review, 82 (5), 910–44.

Bertin-Mahieux

Thierry

Eck

Douglas

Maillet

Francois

Lamere

Paul

(2008), “Autotagger: A Model for Predicting Social Tags from Acoustic Features on Large Music Databases,” Journal of New Music Research, 37 (2), 115–35.

Biyalogorsky

Eyal

Naik

Prasad

(2003), “Clicks and Mortar: The Effect of On-Line Activities on Off-Line Sales,” Marketing Letters, 14 (1), 21–32.

Blei

David M.

Andrew Y.

Jordan

Michael I.

(2003), “Latent Dirichlet Allocation,” Journal of Machine Learning Research, 37 (2), 993–1022.

10.

Bradlow

Eric T.

Fader

Peter S.

(2001), “A Bayesian Lifetime Model for the ‘Hot 100’ Billboard Songs,” Journal of the American Statistical Association, 96 (454), 368–81.

11.

Bradlow

Eric T.

Rao

Vithala R.

(2000), “A Hierarchical Bayes Model for Assortment Choice,” Journal of Marketing Research, 37 (2), 259–68.

12.

Bruce

Norris I

. (2019), “Bayesian Nonparametric Dynamic Methods: Applications to Linear and Nonlinear Advertising Models,” Journal of Marketing Research, 56 (2), 211–29.

13.

Bruner

Gordon C

. (1990), “Music, Mood, and Marketing,” Journal of Marketing, 54 (4), 94–104.

14.

Chung

Tuck Siong

Rust

Roland T.

Wedel

Michel

(2009), “My Mobile Music: An Adaptive Personalization System for Digital Audio Players,” Marketing Science, 28 (1), 52–68.

15.

Dai

Andrew M.

Storkey

Amos J.

(2015), “The Supervised Hierarchical Dirichlet Process,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 37 (2), 243–55.

16.

Datta

Hannes

Knox

George

Bronnenberg

Bart J.

(2017), “Changing Their Tune: How Consumers’ Adoption of Online Streaming Affects Music Consumption and Discovery,” Marketing Science, 37 (1), 5–21.

17.

Dew

Ryan

Ansari

Asim

(2018), “Bayesian Nonparametric Customer Base Analysis with Model-Based Visualizations,” Marketing Science, 37 (2), 216–35.

18.

DiMA (2018), “Streaming Forward: More Choices, Better Value,” annual report (March), https://dima.org/wp-content/uploads/2018/04/DiMA-Streaming-Forward-Report.pdf.

19.

Dowd

Timothy J

. (2004), “Concentration and Diversity Revisited: Production Logics and the US Mainstream Recording Market, 1940–1990,” Social Forces, 82 (4), 1411–55.

20.

Dunbar

Vince

(2015), “What’s This Site All About?” New Music for Old Folks (October 7), https://www.newmusicforoldfolks.com/single-post/2015/10/07/Whats-this-site-all-about.

21.

Elberse

Anita

(2010), “Bye-Bye Bundles: The Unbundling of Music in Digital Channels,” Journal of Marketing, 74 (3), 107–23.

22.

Eliashberg

Jehoshua

Jonker

Jedid-Jah

Sawhney

Mohanbir S.

Wierenga

Berend

(2000), “Moviemod: An Implementable Decision-Support System for Prerelease Market Evaluation of Motion Pictures,” Marketing Science, 19 (3), 226–43.

23.

Farquhar

Peter H.

Rao

Vithala R.

(1976), “A Balance Model for Evaluating Subsets of Multiattributed Items,” Management Science, 22 (5), 528–39.

24.

Giles

David E

. (2007), “Increasing Returns to Information in the US Popular Music Industry,” Applied Economics Letters, 14 (5), 327–31.

25.

Heinrich

Gregor

(2011), “Infinite LDA Implementing the HDP with Minimum Code Complexity,” technical note, http://www.arbylon.net/publications/ilda.pdf.

26.

Holbrook

Morris B.

Anand

Punam

(1990), “Effects of Tempo and Situational Arousal on the Listener’s Perceptual and Affective Responses to Music,” Psychology of Music 18 (2), 150–62.

27.

Holbrook

Morris B.

Hirschman

Elizabeth C.

(1982), “The Experiential Aspects of Consumption: Consumer Fantasies, Feelings, and Fun,” Journal of Consumer Research, 9 (2), 132–40.

28.

Holbrook

Morris B.

Schindler

Robert M.

(1989), “Some Exploratory Findings on the Development of Musical Tastes,” Journal of Consumer Research, 16 (1), 119–24.

29.

Iwanaga

Makoto

(1995), “Relationship Between Heart Rate and Preference for Tempo of Music,” Perceptual and Motor Skills, 81 (2), 435–40.

30.

Jovev

Jason

(2018), “Spotify: The Rise of the Contextual Playlist,” Medium (March 13), https://blog.chartmetric.io/spotify-the-rise-of-the-contextual-playlist-c6f2c26900f4.

31.

Juslin

Patrik N.

Laukka

Petri

(2004), “Expression, Perception, and Induction of Musical Emotions: A Review and a Questionnaire Study of Everyday Listening,” Journal of New Music Research, 33 (3), 217–38.

32.

Juslin

Patrik N.

Västfjäll

Daniel

(2008), “All Emotions Are Not Created Equal: Reaching Beyond the Traditional Disputes,” Behavioral and Brain Sciences, 31 (5), 600–621.

33.

Kim

Jin Gyo

Menzefricke

Ulrich

Feinberg

Fred M.

(2007), “Capturing Flexible Heterogeneous Utility Curves: A Bayesian Spline Approach,” Management Science, 53 (2), 340–54.

34.

Koelsch

Stefan

Fritz

Thomas

Cramon

D. Yves v.

Müller

Karsten

Friederici

Angela D.

(2006), “Investigating Emotion with Music: An FMRI Study,” Human Brain Mapping, 27 (3), 239–50.

35.

Kooper

(2015), “New Music for Old People: The London Souls, Neil Young, Gov’t Mule, Delta Rae and More,” The Morton Report (April 17), http://www.themortonreport.com/entertainment/music/new-music-for-old-people-the-london-souls-neil-young-govt-mule-delta-rae-and-more/.

36.

Krueger

Alan B

. (2005), “The Economics of Real Superstars: The Market for Rock Concerts in the Material World,” Journal of Labor Economics, 23 (1), 1–30.

37.

Lee

Jonathan

Boatwright

Peter

Kamakura

Wagner A.

(2003), “A Bayesian Model for Prelaunch Sales Forecasting of Recorded Music,” Management Science, 49 (2), 179–96.

38.

Lena

Jennifer C

. (2006), “Social Context and Musical Content of Rap Music, 1979–1995,” Social Forces, 85 (1), 479–95.

39.

Lena

Jennifer C.

Pachucki

Mark C.

(2013), “The Sincerest Form of Flattery: Innovation, Repetition, and Status in an Art Movement,” Poetics, 41 (3), 236–64.

40.

Yang

Ansari

Asim

(2013), “A Bayesian Semiparametric Approach for Endogeneity and Heterogeneity in Choice Models,” Management Science, 60 (5), 1161–79.

41.

Naik

Prasad A

. (2015), “Marketing Dynamics: A Primer on Estimation and Control,” Foundations and Trends in Marketing, 9 (3), 175–266.

42.

Nave

Gideon

Minxha

Juri

Greenberg

David M.

Kosinski

Michal

Stillwell

David

Rentfrow

Jason

(2018), “Musical Preferences Predict Personality: Evidence from Active Listening and Facebook Likes,” Psychological Science, 29 (7), 1145–58.

43.

Nikulin

Mikhail S

. (2001), “Hellinger Distance,” in Encyclopedia of Mathematics, Hazewinkel

, ed. Berlin: Springer, http://encyclopediaofmath.org/index.php?title=Hellinger_distance&oldid=16453.

44.

Ocasio

William

Mauskapf

Michael

Steele

Christopher W.J.

(2016), “History, Society, and Institutions: The Role of Collective Memory in the Emergence and Evolution of Societal Logics,” Academy of Management Review, 41 (4), 676–99.

45.

Papies

Dominik

van Heerde

Harald J.

(2017), “The Dynamic Interplay Between Recorded Music and Live Concerts: The Role of Piracy, Unbundling, and Artist Characteristics,” Journal of Marketing, 81 (4), 67–87.

46.

Peterson

Richard A.

Berger

David G.

(1975), “Cycles in Symbol Production: The Case of Popular Music,” American Sociological Review, 40 (2), 158–73.

47.

Pietroluongo

Silvio

(2012), “How We Chart the Year,” Billboard (December 14), https://www.billboard.com/articles/news/1481472/the-best-of-2012-the-year-in-music.

48.

Rajaraman

Anand

Ullman

Jeffrey David

(2011), Mining of Massive Datasets. Cambridge, UK: Cambridge University Press.

49.

RIAA (2018), “Mid-Year 2018 RIAA Music Revenues Report,” (accessed August 10, 2021), https://www.riaa.com/wp-content/uploads/2018/09/RIAA-Mid-Year-2018-Revenue-Report-News-Notes.pdf.

50.

Rossi

Peter

(2014), Bayesian Non- and Semi-Parametric Methods and Applications. Princeton, NJ: Princeton University Press.

51.

Rossman

Gabriel

(2012), Climbing the Charts: What Radio Airplay Tells Us About the Diffusion of Innovation. Princeton, NJ: Princeton University Press.

52.

Rubel

Olivier

Naik

Prasad A.

(2017), “Robust Dynamic Estimation,” Marketing Science, 36 (3), 53–467.

53.

Scherer

Klaus R

. (2004), “Which Emotions Can Be Induced by Music? What are the Underlying Mechanisms? and how can we Measure them?” Journal of New Music Research, 33 (3), 239–51.

54.

Schmitt

Bernd H

. (2010), Customer Experience Management: A Revolutionary Approach to Connecting with Your Customers. Hoboken, NJ: John Wiley & Sons.

55.

Serrà

Joan

Corral

Àlvaro

Boguñá

Mariàn

Haro

Martín

Josep

. Arcos (2012), “Measuring the Evolution of Contemporary Western Popular Music,” Scientific Reports, 2 (1), 1–16.

56.

Sethuraman

Jayaram

(1994), “A Constructive Definition of Dirichlet Priors,” Statistica Sinica, 4 (2), 639–50.

57.

Shively

Thomas S.

Allenby

Greg M.

Kohn

Robert

(2000), “A Nonparametric Approach to Identifying latent Relationships in Hierarchical Models,” Marketing Science, 19 (2), 149–62.

58.

Sun

Dongchu

Tsutakawa

Robert K.

Speckman

Paul L.

(1999), “Posterior Distribution of Hierarchical Models Using CAR(1) Distributions,” Biometrika, 86 (2), 341–50.

59.

Swets

John A

. (2014), Signal Detection Theory and ROC Analysis in Psychology and Diagnostics: Collected Papers. London: Psychology Press.

60.

Teh

Yee

Jordan

Michael

Beal

Matthew

Blei

David

(2005), “Sharing Clusters Among Related Groups: Hierarchical Dirichlet Processes,” Advances in Neural Information Processing Systems, 17, 1385–92.

61.

Tirunillai

Seshadri

Tellis

Gerard J.

(2014), “Mining Marketing Meaning from Online Chatter: Strategic Brand Analysis of Big Data Using Latent Dirichlet Allocation,” Journal of Marketing Research, 51 (4), 463–79.

62.

Tobin

James

(1958), “Estimation of Relationships for Limited Dependent Variables,” Econometrica, 26 (1), 24–36.

63.

Vehtari

Aki

Gelman

Andrew

Gabry

Jonah

(2017), “Practical Bayesian Model Evaluation Using Leave-One-Out Cross-Validation and WAIC,” Statistics and Computing, 27 (5), 1413–32.

64.

Vogel

Harold L

. (2014), Entertainment Industry Economics: A Guide for Financial Analysis. Cambridge, UK: Cambridge University Press.

65.

Wand

M.P.

Ormerod

J.T.

(2008), “On Semiparametric Regression with O’Sullivan Penalized Splines,” Australian & New Zealand Journal of Statistics, 50 (2), 179–98.

66.

Wedel

Michel

Zhang

Jie

(2004), “Analyzing Brand Competition Across Subcategories,” Journal of Marketing Research, 41 (4), 448–56.

67.

Zentner

Marcel

Grandjean

Didier

Scherer

Klaus R.

(2008), “Emotions Evoked by the Sound of Music: Characterization, Classification, and Measurement,” Emotion, 8 (4), 494–521.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.38 MB