Looking beyond the stars: A description of text mining technique to extract latent dimensions from online product reviews

Abstract

The purpose of this study is to contribute to the marketing literature and practice by describing a research methodology to identify latent dimensions of customer satisfaction in product reviews, and examining the relationship between these attributes and customer satisfaction. Previous research in product reviews has largely relied only on quantitative ratings, either stars or review score. Advanced techniques for text mining provide the opportunity to extract meaning from customer online reviews. By analyzing 51,110 online reviews for 1,610 restaurants via latent Dirichlet allocation, this study uncovers 30 latent dimensions that are determinants of customer satisfaction. Furthermore, this study developed measurements of sentiment and innovativeness as moderators of the effect of these latent attributes to satisfaction.

Keywords

attributes customer satisfaction latent Dirichlet allocation online reviews text analysis

Introduction

Customers dramatically increased their online activity in recent years (Hofacker, Malthouse, & Sultan, 2016). Customer reviews are an effective information source for customers to learn about a product without having to consume it first (Chevalier & Mayzlin, 2006; Hennig-Thurau, Gwinner, Walsh, & Walsh, 2003). Due to technology, the impact of customer reviews on customer behavior is more pronounced than ever. Both academic and business communities acknowledge the importance of online reviews (Breazeale, 2009; Pee, 2016). Customers perceive online reviews to be more trustworthy than traditional marketing (Chevalier & Mayzlin, 2006). Customers use reviews to reduce uncertainty about a product, estimate product quality (Reimer & Benkenstein, 2016) and inform them when buying a product (Lee, Jung, & Park, 2017).

Customer reviews are text information with high dimensionality and multiple latent interpretable topics underlying the texts (Heng, Gao, Jiang, & Chen, 2018). Advanced techniques for linguistic analysis provide the opportunity to extract meaning from reviews. This is valuable for businesses because an understanding of which attributes will enhance compliments or will lead to complaints is important to improve customer satisfaction (Ramanathan & Ramanathan, 2011). Reviews can provide a depiction of customers’ perceptions of products. Product quality improvement hints can be captured by analyzing users’ online data (Jia, 2019). Scientifically this is important because researchers should not only look at valence and volume of reviews but also examine the content of reviews (Srivastava & Sharma, 2017). However, extant research on the impact of reviews mainly focuses on numerical rations instead of textual reviews (Lee et al., 2017). Cabosky (2016) argues that although current market research studied extensively reviews, significant knowledge gaps still exists. Most research ignores text reviews because of their unstructured nature (Robson, Farshid, Bredican, & Humphrey, 2013). More recently, authors tried to broaden and deepen the understanding of how reviews can influence product performance. However, the applications of text-mining techniques to examine reviews in more depth has been lacking in the literature, partly because of the complexity of text analysis (Heng et al., 2018; Jia, 2019; Lee et al., 2017; Li, Mai, & Wu, 2018).

The studies’ objective was to investigate the relationship between the mention of product attributes in reviews and satisfaction and how this relationship is moderated by sentiment and innovation. Signaling Theory (Spence, 1973) and Familiarity-Liking Theory (Rindfleisch & Jeffrey, 1998) are the theoretical foundation of this study. The Signaling Theory argues that the product attributes expressed in the reviews signal customers acknowledgment and approval of these attributes which lead to satisfaction (Spence, 1973). The Familiarity-Liking Theory states that customers can be more satisfied because the exposure to the service (Sharma & Baoku, 2013). This empirical setting of this study is the restaurant industry but for other industries it could be equally important to study their customer reviews for the reasons described above.

The first contribution of this research is a methodological advancement by providing a technique to disentangle unstructured review texts to identify innovative efforts and sentiment expressed toward product attributes in reviews and check how attributes contribute to higher satisfaction. It provides a protocol for mining and correlating the unstructured data available online, which can be transferred to other industries (Jia, 2019).

The second contribution is that customers post thousands of reviews which generated a big data challenge research for business and academics (Hofacker et al., 2016; Singh et al., 2017). Our research uses text mining techniques to extract factors. Through the advanced techniques developed in natural language processing (NLP) and text mining, multidimensional dimensions can be found. However, this approach needs to be tested for improving the value for online reviews (Singh et al., 2017). Considering this, we use a machine learning approach for big data analysis to estimate the attributes mentioned in reviews, the innovativeness and sentiment.

This research has several practical implications. Big data in the form online reviews is of great interest to the discipline of marketing because online user reviews can influence both sales via customer decision-making and product quality improvement via organizations (Hofacker et al., 2016; Ruiz-Mafe, Chatzipanagiotou, & Curras-Perez, 2018; Singh et al., 2017). When customers are complaining or are excited about attributes, organizations can monitor this conversation (Hofacker et al., 2016). This research offers the opportunity to listen to customers by looking in more depth to which product attributes are positively and negatively valued and what customers find innovative about products. As a consequence, product development would become less risky because organizations can make more informed decision how to adapt product development based on the signals expressed by the customers toward attributes.

Literature review

Signaling theory

Hennig-Thurau et al. (2003) define e-WOM as “any positive or negative statement made by potential, actual or former consumers about a product or organization, which is made available to a multitude of people and institutions via the internet” (p. 39). Robson et al. (2013) offer an improved definition “any statement—positive, negative or neutral—add by potential, current or former stakeholders about a product, service, company or person, which is made available to a multitude of people, organizations or institutions, via a digitally networked platform” (p. 523). As a form of e-WOM, online reviews are peer-generated evaluations posted on websites (Ruiz-Mafe et al., 2018). The theoretical foundation to study the underlying mechanism explaining the relationship between the product attributes and satisfaction can be found within the Signaling Theory (Spence, 1973). This theory explains how people make decisions based on signals of quality, particularly when quality is difficult to ascertain. The lower the ability of the decision maker to evaluate all information, the more important signals will be (Spence, 1973). One could argue that it is hard for customers to process all product information because of information asymmetry (Spence, 1973) and information overload (Singh et al., 2017).

Information asymmetry arises because in markets, parties often have different information regarding transactions (Spence, 1973). Quality signaling is a solution for information asymmetry (Akerlof, 1970). Research suggests that customer reviews provide an informative signal of quality that encourage customers to make a purchase decision (Park & Nicolau, 2015). Furthermore, customers increasingly share their experiences of products which created an information overload on customers. It is impossible for customers to go through all reviews (Singh et al., 2017). Therefore, customers seek heuristic information cues to simplify the amount of information (Park & Nicolau, 2015).

In the current study we focus on the impact of the mention of attributes on satisfaction. This information cue not only conveys the existence of the product attributes thereby creating an awareness effect but also reflects a degree of product attribute quality. Reviews reveal customer’s detailed thoughts and experiences (Chevalier & Mayzlin, 2006). Therefore, this study examines the textual elements in reviews in the form of product attributes. An attribute is a characteristic that defines a product and affects a customer’s purchase decision (Keller, 1993). Attributes are key points about customers’ overall experience and influence whether their pre-purchase expectant can be met. Recently, studies begin to examine user-generated content to study the product attributes customers mention by using text mining techniques (Fan, Che, & Chen, 2017; Guo, Barnes, & Jia, 2017; Hu, Sian Koh, & Reddy, 2014; Salehan & Kim, 2016).

This study aims to link the mention of attributes to satisfaction. Customer satisfaction is defined as how products and services satisfy or surpass customer expectation (Farris, Bendle, Pfeifer, & Reibstein, 2010). In customer reviews, the relative importance of attributes is identified according to the intensity of the conversation about each (Guo et al., 2017). This study argues that when attributes are more frequently mentioned by customers, they will have a positive impact on satisfaction for two reasons. First, according to the Familiarity-Liking Theory (Rindfleisch & Jeffrey, 1998) an exposure effect can occur, which means that liking a stimulus increases when someone is repeatedly exposed to that stimulus (Zajonc, 1968). On repeated exposure, a linear increase of positive affect is produced due to the greater familiarity and reduced uncertainty. More experience with a product leads to more familiarity. When customers are more familiar with a product they will be more satisfied (Montoya, Horton, Vevea, Citkowicz, & Lauber, 2010).

Second, when customers use a product more often, they are better capable to articulate distinct attributes about it. Customer experience is a coalescing of meaning with behavior, thoughts, and feelings that occur during consumption. As customers’ product experience increases, their abilities to recognize attributes are enhanced. Greater brand experience is associated with understanding, enjoying, enhancing, and fostering of the brand (Weinberg, 2001).

The literature proposes that extensive mention of attributes affects customer behaviors and perception (Johns & Pine, 2002). However, there have been only a few studies examining the phenomenon of user generated content and aiming to identify latent dimensions (Floyd, Freling, Alhoqail, Cho, & Freling, 2014). Currently, an emerging stream of research in marketing managed to extract the attributes of products from online reviews. For example, authors examined the influence of topic expressed in online reviews on the sales performance in the ICT industry (Li et al., 2018). Ramanathan and Ramanathan (2011) concluded that there are critical attributes in the hotel industry. Guo et al. (2017) used LDA on reviews to estimate how different product attributes influenced satisfaction. Heng, Gao, and Jiang & Chen (2018) likewise concluded that latent topics influence review helpfulness. Based on previous research, this article proposes the following hypothesis:

H1. There is a positive relationship between the quantity of the mention of product attributes and overall customer satisfaction.

Sentiment

Sentiment is an attitude, thought, or judgment prompted by feeling (Fang & Zhan, 2015). The literature acknowledged the importance of examining sentiment expressed in reviews (Hu et al., 2014). Reviews often reveal emotions such as happiness, anger, criticism, and praise and customers can use these opinions to inform their decisions (Fan et al., 2017). Therefore, sentiment analysis became popular and widely used to research reviews (Fan et al., 2017; Hu et al., 2014; Lee et al., 2017; Salehan & Kim, 2016).

The extant literature suggests contradictory findings on how sentiment influences customers’ behavior. Some studies suggest that negative reviews influence customers’ purchase decisions more than positive reviews (Chevalier & Mayzlin, 2006; Pee, 2016). Other studies conclude differently and argue that positive reviews affect customers’ decision making (Vermeulen & Seegers, 2009). Salehan and Kim (2016) conclude that neutral reviews are more helpful. The study of Lee et al. (2017) concludes that the entropy level in reviews has a positive moderating effect on the relationship between reviews and sales.

Although previous literature studies the persuasive effects of customers reviews, to our knowledge no studies combined the mention of attributes with sentiment and the effect of these variables on satisfaction. This research aims to establish the effect of the combination of attributes and sentiment in reviews on customer satisfaction. If customer emotion toward product attributes is positive, this normally signals a higher satisfaction (Fornell, 1992). This study proposes that attributes must be linked with sentiment. It is not enough that people mention the attribute, they should also express a positive sentiment toward that attribute. Therefore, the second hypothesis is proposed:

H2. Sentiment score positively moderates the relation between product attributes and customer satisfaction, and a higher positive sentiment will lead a higher customer satisfaction.

Innovativeness and innovation

Product innovativeness is defined as the degree to which the product was new to an organization and reviewers (Olson, Walker, & Ruekert, 1995). There are mixed findings on the innovativeness performance relationship. Van Trijp and Van Kleef (2008) argue that disagreements in these findings may be due to the inadequate operationalization of product innovativeness. It is important to discriminate between innovativeness for organizations and customers. Customers evaluate product innovativeness in terms of mental models and habits that need to be altered (Calantone, Chan, & Cui, 2006). Customer responses to product innovativeness are determined by perceived risk involved in adoption of the innovation (Danneels & Kleinschmidt, 2000).

Innovativeness score can have an effect on product performance. Innovation is a vital activity that is important for almost all organizations to adopt (Ngo & O’Cass, 2013). Thus, when customers perceive something is new and innovative, customers would consider this as a positive signal that will lead to a higher level of satisfaction.

H3. The extent to which a product is considered innovative moderates the relation between product attributes and customer satisfaction, and a higher degree of innovativeness will lead a higher customer satisfaction.

Method

This article proposes a framework to extract latent dimensions from reviews. The framework in Figure 1 summarizes all procedures in this study, which can be used by researchers and business to identify latent dimensions and to explore the relationship between attributes and satisfaction, while considering sentiment and innovativeness as moderators.

Figure 1.

Framework for extracting latent dimensions and sentiment and innovativeness analysis.

Data collection

This study targets on the restaurant sector. The empirical setting is customer review website Yelp. Yelp has opened “Yelp Dataset Challenge” and introduced their real business dataset for research. The large dataset includes business, reviews, user, and check-in data in the form of separate JSON objectives. The authors used the JSON to CSV Converter (see json-csv.com). The dataset has a total number of 1,610 restaurants with 51,100 reviews across 7 states in the United States.

Latent dimension extraction

Data pre-processing

The pre-processing step was similar to those adopted in previous research (Zhao, Han, Meng, He, & Zhang, 2017), involving the tokenizing, lower-case each word, word stemming, removing the low-frequency words, and the common stopwords such as “the,” “we.” For example, the raw review is:

I like the food at this local place, but it was crowded and I have to wait the food for an hour to come out of the kitchen. Food was good but I am not sure why there was a long wait. But the environment is very nice.

After pre-processing, the review appears like:

like food, this local place but crowded have to wait food hour come out kitchen food good not sure why there long wait but environment very nice.

Dimension extraction

One of the contributions of this study is the extraction of latent dimensions affecting satisfaction by mining reviews. This study utilizes LDA which is the most common topic modeling method in NLP (Poria, Cambria, & Gelbukh, 2016). As an unsupervised machine learning approach, LDA is efficient to translate large-scale data into topics and gives the probability distribution of words. Besides, LDA compares the frequency of extracted dimensions based on customer experience. This article uses Stanford Topic Modeling Toolbox to extract latent dimensions via LDA.

As shown in Figure 2, there are three layers in LDA model. The inner layer, known as word level, has variables z and w which are sampled once for each word in each document. In the medium layer, θ refers to latent variables and sampled once in each document. The outer layer involves parameter α and β which are sampled once for the corpus. LDA assumes that the sequence of N words forms a review, which represents a “document” in the model, and a document w = (w₁, w₂, . . ., w_N), while M reviews consist of a corpus, D = {W₁, W₂, . . .., w_M}. Below are the generative steps for each review in a corpus (Blei, Ng, & Edu, 2003).

Choose N ~ Poisson (ξ).

Choose θ ~ Dir(α)

For each of the N words wn:

(a) Choose a topic Zn ~ Multinomial (θ) and

(b) Choose a word wn from p(wn zn, β), a multinomial probability conditioned on the topic zn.

In step 1, N represents the length of documents and Poisson in this step shows the length of the reviews distributed in each document. In step 2, α is the parameter of the Dirichlet prior on the pre-review topic distributions. The probability of k_th dimension (topic) occurs shows the relative importance of this dimension and can be represented as a k-dimensional Dirichlet random variable θ. In a given review, the probability density can be expressed by the following function

p (θ | α) = \frac{Γ (\sum_{i = 1}^{k} α_{i})}{\prod_{i = 1}^{k} Γ (α_{i})} \prod_{i = 1}^{k} θ_{i}^{α_{i} - 1}

(1)

In this function, parameter α is a k-vector with components α_i > 0, and Γ (x) is the Gamma function. In step 3, β can be regarded as a k-dimensional Dirichlet random variable in a given topic, θ is the joint distribution of a topic mixture, z is a set of N topics, and w is a set of N words. They are given by the following function

p (θ, Z, W | α, β) = p (θ | α) \prod_{n = 1}^{N} p (z_{n} | θ) p (w_{n} | z_{n}, β)

(2)

In which p(zn|θ) is θ for the unique i where $z_{n}^{i}$ = 1. We then can get the marginal distribution of a document

p (w | α, β) = \int^{} p (θ | α) (\prod_{n = 1}^{N} \sum_{z_{n}} p (z_{n} | θ) p (w_{n} | z_{n}, β) d θ)

(3)

Finally, the probability of a corpus can be obtained via taking the product of the marginal probability of single review

p (D | α, β) = \prod_{d = 1}^{M} \int^{} p (θ_{d} | α) (\prod_{n = 1}^{N_{d}} \sum_{z_{d n}} p (z_{d n} | θ_{d}) p (w_{d n} | z_{d n}, β) d θ_{d})

(4)

Figure 2.

LDA conceptual model.

Key attributes identification and sentiment and innovativeness analysis

Key attributes identification

This part used the processed text data in 3.1.1 and counted the word frequency. This study assumed that word frequency in reviews represents the importance of attributes (Tirunillai & Tellis, 2014). This article found that food, place, order, service, and product-type (pizza) are the most mentioned attributes in the dataset (see Figure 3), then applied “wordcloud” package in R to visualize the top 50 number of words (Figure 4).

Figure 3.

Top five most frequent words.

Figure 4.

Word cloud.

Sentiment and innovativeness analysis

This article furthermore focuses on sentiment and innovativeness in reviews. Sentiment analysis is a process to find the emotional tone behind a series of words, aiming to understand the attitudes, opinions, and emotions (Thelwall & Buckley, 2013). This article applied SentiStrength to analyze cleaned review dataset. SentiStrength estimates the strength of positive and negative sentiment in texts by using predefined sentiment word list (Thelwall, Buckley, Paltoglou, Cai, & Kappas, 2010). It is validated and tested by previous research (Salehan & Kim, 2016; Thelwall & Buckley, 2013). For example, a row review is:

I love you but hate the current political climate.

After being analyzed the output is:

I love [3] you but hate [-4] the current political climate. [sentence: 3, -4]

SentiStrength analyzes the text based on a 1–5 scale, “3” means this sentence has positive strength 3, and “–4” means negative strength 4. Therefore, the overall sentiment of this sentence is 3 + (–4) = −1.

There are four steps for innovativeness analysis. Step 1, based on the two words “new” and “innovation,” the authors developed a word list to capture all the words that customers use to describe something is new via The Synonym Finder (Rodale, 1986). Step 2 was about innovativeness word validation (Table 1). Two experts with at least 10 years of academic and business experience in restaurant management validated the word list. At this step, two experts randomly selected 200 reviews from the dataset and manually went through the reviews to add words that expressed innovativeness. Of the 96 words generated by The Synonym Finder, 28 were deleted and 5 new words were added. In total, 73 words were selected.

Table 1.

Innovativeness word list.

advanced, avant-garde, brand-new, breakthrough, contemporary, creation, creative, creativeness, creativity, cutting edge, developed, development, different, distinct, evolution, futuristic, imaginative, improved, improvement, innovation, innovative, innovatory, invention, inventive, inventiveness, leading edge, metamorphosis, mint condition, modern, modernism, modernistic, modernity, modernization, modernized, modification, mutation, neoteric, new, new- wrinkle, newest, newfangled, newfashioned, newness, novel, novelty, original, originality, origination, originative, progressive, radical, radical change, radically, rebuilt, recast, reconstructed, recreated, reformation, regenerated, remodeled, renascent, renewed, renovation, restyle, restyling, revolution, revolutionary, transformation, ultramodern, unhackneyed, unprecedented, up-to-date, way-out

After forming this list, the experts assigned scores to these words based on the degree of innovativeness in step 3. The score assignment process is similar to sentiment analysis which is based on a 1–5 scale. 1 means this word has little innovativeness, and 5 means this word has a high degree of innovativeness. To help further research this study listed the innovativeness words in Appendix 1. At the last step, the authors quantify innovativeness by replacing the sentiment word list with innovativeness word list in SentiStrength and got the innovativeness score of the sentences containing key attributes in each review.

Variables description

Independent variables

The independent variables are the five attributes in reviews. By counting the word frequency, the top five most important attributes food, place, order, service, and product-type (pizza) were found. Then counted the number of these five attributes in each review. For example, if a review says: “the food is super amazing, but the service is bad, I only recommend the food here.,” in this case, the food was counted 2, service 1, and place, order and product-type (pizza) are 0.

Dependent variable

The dependent variable is customer satisfaction which was measured by the online rating on a 1–5 scale. 1 represents the lowest and 5 the highest rating. The higher the rating is, the higher the satisfaction is (Guo et al., 2017).

Moderating variables

The moderating variables are sentiment and innovativeness of the five attributes in each review. For sentiment, can be varied from −5 to 5. For innovativeness, can be varied from 0 to 5, which means the higher the score, the higher degree of innovativeness these attributes.

Results

Dimensions of customer satisfaction

This study generated 30 topics with top 20 words in each topic with their relative weight. The naming process as follows: the authors first checked each topic, aiming to find the logical relationship between the most frequent words within this topic. For instance, as shown in Table 2, the topic name “location of restaurant” was based on the word “center,” weighted 11.10%, word “near,” weighted 4.90%, and word “street,” weighted 3.14%. The authors manually checked whether this topic makes sense to rest of the words. If a word did not fit the topic name, the labeling step of this topic would be restarted until the new topic fits all of the 20 words.

Table 2.

Topic identification example.

Topic	Relative weight	%	Topic	Relative weight	%
Topic 1: Location of restaurant			Topic 2: Return visits
center	17415.06905	11.10%	back	7224.328888	11.02%
near	7689.525599	4.90%	will	6816.536457	10.40%
street	4930.915713	3.14%	time	4104.57924	6.26%
walked	4199.470919	2.68%	definitely	2725.661719	4.16%
downtown	3996.062224	2.55%	again	2561.162446	3.91%
closed	3724.996414	2.37%	first	2154.07242	3.29%
time	3133.846717	2.00%	try	1776.764219	2.71%
minutes	3015.815286	1.92%	come	1769.481252	2.70%
bus	2822.231512	1.80%	next	1326.044388	2.02%
building	2489.885736	1.59%	visit	1318.799477	2.01%
park	2428.56044	1.55%	wait	965.1179432	1.47%
next	2336.336979	1.49%	coming	845.5947993	1.29%
across	2008.512956	1.28%	soon	828.7691615	1.26%
away	1830.419936	1.17%	went	773.3002577	1.18%
stop	1596.946975	1.02%	amazing	761.8073586	1.16%
far	1586.912572	1.01%	return	721.6755455	1.10%
busy	1369.52656	0.87%	delicious	713.9246118	1.09%
long	1254.499204	0.80%	sure	711.6708932	1.09%
wait	1213.57637	0.77%	going	693.2839421	1.06%
side	1182.09852	0.75%	excellent	664.3228997	1.01%

Figure 5 shows the top 30 most important dimensions (topics). Two of the dimensions show the overall perception of restaurant experience: amazing experience and satisficing. Amazing experience shows the whole experience is pleasant, while satisficing means a cognitive status in which expectation is just met. Four dimensions represent the degree of satisfaction or dissatisfaction, including regular customers, recommendation, poor reviews, and return visits. The rest dimensions represent 24 aspects of restaurant quality (e.g., long waiting time, location of restaurant, and music). Previous researchers have demonstrated that there are five most common categories in restaurants—food quality, price, service quality, location, and environment (Soriano, 2002), and this study builds on this classification and organizes 24 latent customer satisfaction dimensions into these categories (see Table 3).

Figure 5.

Extracted dimensions.

Table 3.

Classification of extracted dimensions.

Food (10)	Service (8)	Price (1)	Environment (4)	Location (1)
Dessert	Long waiting time	Poor value for money	Style and decoration	Location of restaurant
Poor flavor	Waitress service		Dinner atmosphere
Food ingredients	Menu and order		Restaurant environment
Pasta recipes	Suitable for party		Dinning view
Meat	Attentive staff
Pizza	Order take-away
Taste authenticity	Business hours
Wine	Music
Main food
Breakfast

To examine the validity of the analysis, this study compared the extracted dimensions with that from human analysis. Two researchers with experience in NLP and text mining manually identified latent dimensions. In total, 200 reviews were randomly selected. A t-test was applied to examine the difference between these 400 reviews and the whole sample. The result shows no signification difference in terms of star rating (t = 0.41, p > .1).

Then the Jaccard coefficient examined the overlap between dimensions extracted from LDA and that from human analysis (Table 4). N $(D i m_{l d a})$ represents the number of dimensions derived from LDA, and N $(D i m_{e x})$ represents the number of dimensions identified by the researchers

Table 4.

LDA analysis and rater validation.

Dimensions	LDA analysis	Researcher A	Researcher B
Location of the restaurant	√	√	√
Long waiting time	√	√	√
Style and decoration	√	√	√
Waitress service	√	√	√
Menu and order	√	√	√
Poor value for money	√	×	√
Dessert	√	√	√
Poor flavor	√	√	√
Food ingredients	√	√	√
Dinner atmosphere	√	√	√
Suitable for party	√	√	×
Pasta recipes	√	×	√
Attentive staff	√	√	√
Meat	√	√	√
Restaurant environment	√	√	√
Pizza	√	√	√
Dinning view	√	√	√
Order take-away	√	√	×
Business hours	√	×	√
Taste authenticity	√	×	√
Wine	√	√	√
Music	√	√	×
Main food	√	√	√
Breakfast	√	√	√
Freshness	×	√	×
Cleanliness	×	√	√
Location in restaurant	×	×	√
Noisy	×	√	√

LDA: latent Dirichlet allocation.

√ means included and × means not included.

J (D i m_{l d a}, D i m_{e x}) = \frac{| N (D i m_{l d a} \cap^{} D i m_{e x}) |}{| N (D i m_{l d a} \cup^{} D i m_{e x}) |}

(5)

The Jaccard coefficient is 0.74 and 0.78 for the researchers, respectively. Considering the nature of the task and the level of ambiguity the researchers faced when finding dimensions, above results prove that the LDA method is feasible and reliable to extract dimensions.

Description statistics and correlation

Table 5 presents the descriptive statistics and the correlation between variables. Among the reviews, the average star rating is 3.72. The mean sentiment score is 1.07 and that of innovativeness is 0.48. As can be seen from Table 6, there is a strong positive relationship between sentiment and customer satisfaction (r = .599, p < .01) and innovativeness (r = .033, p < .01) and customer satisfaction. This is important because a condition for moderation is that the moderating variable should have a significant effect on the dependent variable (Namazi & Namazi, 2016). In addition, all five attributes show a negative relationship with satisfaction (food: r = −.105, p < .01; place: r = −.041, p < .01; order: r = −.192, p < .01; service: r = −.068, p < .01; pizza: r = −.021, p < .01). Overall, the correlations between the independent variables do not above the r = .50 level. Therefore, multicollinearity is not a major concern here, and the independence of the constructs is verified. Since sentiment has an estimated effect of .53, which is higher than food, place, order, service, pizza, and innovativeness, it might be driving the significant relationships in the model. Therefore, we performed a robustness test to test moderation with a bias corrected bootstrap model (Hayes & Preacher, 2014). In Table 6, one can see that the interaction is still significant and not driven by sentiment.

Table 5.

Means, standard deviation, and correlation.

Variables	Mean	SD	1	2	3	4	5	6	7
1. Satisfaction	3.72	1.34
2. Sentiment	1.07	1.38	.599**
3. Innovativeness	.48	.83	.033**	−0.003
4. Food	.90	1.07	−.105**	−.089**	.098**
5. Place	.63	.95	−.041**	−.069**	.133**	.181**
6. Order	.52	1.04	−.192**	−.158**	.144**	.181**	.149**
7. Service	.48	.68	−.068**	−.040**	.053**	.248**	.055**	.100**
8. Pizza	.45	1.15	−.021**	−.048**	.084**	−.063**	.104**	.150**	−.023**

SD: standard deviation.

Correlation is significant at the .05 level (two-tailed).

Correlation is significant at the .01 level (two-tailed).

Table 6.

Estimation of fixed effects.

	Multilevel analysis				Robust coefficient with process v 3.0
	Parameter estimate	SE	T	Sig.	Parameter estimate	SE	T	Sig.
Constant	3.101	0.021	144.631	0	3.048	0.013	241.908	0
Food	−0.064	0.009	−6.701	0	−0.097	0.009	−10.438	0
Place	0.027	0.011	2.562	.01	0.008	0.011	0.792	.428
Order	−0.151	0.009	−15.938	0	−0.168	0.009	−17.795	0
Service	−0.026	0.014	−1.827	.068	−0.065	0.014	−4.457	0
Pizza	0.022	0.009	2.501	.012	0.018	0.008	2.157	.031
Sentiment	0.535	0.008	70.237	0	0.572	0.004	140.601	0
Innovativeness	0.092	0.01	8.927	0	0.071	0.006	10.692	0
Food × Sentiment	0.02	0.003	6.094	0	0.0339	0.0032	10.562	0
Place × Sentiment	−0.001	0.004	−0.222	.825	0.012	0.003	3.315	0
Order × Sentiment	0.009	0.003	2.746	.006	0.013	0.003	4.025	0
Service × Sentiment	0.006	0.005	1.171	.242	0.019	0.004	4.113	0
Pizza × Sentiment	0.015	0.003	4.616	0	0.017	0.003	5.368	0
Food × Innovativeness	0.006	0.005	1.241	.215	0.002	0.004	0.5	.616
Place × Innovativeness	−0.007	0.005	−1.438	.15	−0.005	0.005	−0.884	.375
Order × Innovativeness	0.013	0.004	2.805	.005	0.012	0.004	2.857	.004
Service × Innovativeness	−0.017	0.007	−2.376	.018	−0.024	0.007	−3.367	0
Pizza × Innovativeness	0	0.004	−0.111	.912	0	0.004	−0.143	.885
−2*Log-Likelihood	148125.73

Dependent variable: Satisfaction.

Test of hypothesis

The linear mixed model was applied to regress online rating on the variable described above. Fixed effects have the levels that are of primary interest, while random effects have the level that is not of primary interest, but rather are thought of a random selection from a larger set of levels (Barr, Levy, Scheepers, & Tily, 2013). Subject effects normally are considered as random effects.

Table 6 shows a significant relationship between the five attributes and satisfaction, although this relationship between attribute service and satisfaction is only marginally significant (p < .10). Food and order have a negative effect on overall customer satisfaction score, and each one unit rise in food is associated with .064 unit reduction in customer satisfaction, while one unit rise in order will result in a reduction of .151 in overall customer satisfaction score. The finding that the mention of food has a negative effect on satisfaction is the opposite of that we expected. This is why it is important to research the moderation effect of sentiment. Place and product-type (pizza) have a positive relationship with customer satisfaction. In sum, the results of the analysis partly support hypothesis 1.

Sentiment has a strong positive relationship with customer satisfaction (p < .001). One more sentiment will lead to 0.53 higher customer satisfaction. Innovativeness also shows a positive relationship with customer satisfaction (p < .001). Furthermore, sentiment moderates the effects of food (p < .001), order (p < .05), and pizza (p < .001). This is an interesting finding since we first concluded that there was a negative relationship between the mention of food and sentiment. However, when we look at the moderation effect of sentiment we conclude that this relationship becomes positive, just like we expected. However, the study does not find significant coefficients for the moderating effects of sentiment toward the effect of place and service. In addition, innovativeness indeed moderates the effect of order (p < .05) and service (p < .05), but the moderating effect on food, place, and pizza is not significant. Thus, the results only partly confirm Hypothesis 2 and 3. Figures 6 to 8 visualize the effect of food, order, and pizza, respectively on customer satisfaction with sentiment as moderator. Figures 9 and 10 visualize the effects of order and service respectively on customer satisfaction with innovativeness as moderator. Figure 9 shows a positive moderating effect, while Figure 10 shows a negative moderating effect of innovativeness. Random effects were presented in Table 7 which shows that the random restaurant-to-restaurant variance is .14. In individual review-to-review, differences with a variance 1.03, have a larger effect than restaurant-to-restaurant differences.

Figure 6.

Satisfaction affected by food and sentiment.

Figure 7.

Satisfaction affected by order and sentiment.

Figure 8.

Satisfaction affected by pizza and sentiment.

Figure 9.

Satisfaction affected by order and innovativeness.

Figure 10.

Satisfaction affected by service and innovativeness.

Table 7.

Estimates of covariance parameters.^a

Parameter		Estimate	Standard error
Repeated measures	Variance	1.032468	0.006543
Intercept [subject = business_id]	Variance	0.139857	0.009882

Dependent variable: Satisfaction.

Discussion

Theoretical implications

Online shopping is becoming customers’ first choice when shopping (Singh et al., 2017). The literature review discussed that there are studies investigated product attributes in reviews. However, no study explored the moderating role of sentiment and innovation in the relation between attributes and satisfaction.

This research first of all enriches the literature by clarifying how attributes, sentiment, and innovation influences customer satisfaction. Not all dimensions significantly affect satisfaction, and sentiment and the perception of innovativeness does not all lead to significant and/or positive overall satisfaction. This research offers a novel framework to extract latent customer satisfaction from reviews and analyze the effects of product attributes on customer satisfaction in any settings. This research adopted the Signal Theory (Spence, 1973) and the Familiarity-Liking Theory (Rindfleisch & Jeffrey, 1998) and showed that product attributes mentioned in the review text do signal (dis-)satisfaction. However, the weights how these different signals affect the overall satisfaction score are different.

Second, the findings reveal that attributes have an effect on satisfaction. This study proposed a framework for dimension extraction, and how to find the relationship between product attributes and customer satisfaction. Other researchers can apply this framework to extract dimensions. The levels of sentiment and innovativeness play significant roles in term of customer satisfaction. A more positive sentiment about food and order can lead to a higher customer satisfaction. Besides, compared to not discuss food, discuss food combined with innovativeness can lead to a higher customer satisfaction. In addition, compared to not discuss service, discuss service, combined with innovativeness will lead to a relatively higher customer satisfaction. Overall, sentiment can strongly influence customer satisfaction and the more positive sentiment is the higher customer satisfaction will be. The higher degree of innovativeness customers perceived, the higher satisfaction will be.

Third, this study proposed a list of words that customer used to express their perception of innovativeness and assigned weight to these words based on their degree of innovativeness. Fourth, this study has approved that LDA is an efficient approach for online review mining. The authors used a rich online dataset for dimension mining and approved LDA is a feasible and reliable method for topic extraction. A conceptual model was proposed to examine the relationship between attributes and satisfaction and explored the moderating effect of sentiment and innovativeness.

Managerial implications

The results of this research have important implications for managers. First, the huge amount of reviews has created an information overload among users and business (Hu et al., 2014). Managers could consider new contextual classifications for reviews in terms of attributes. This could lead to more accurate and useful online review classifications for customers’ decision making. Second, customer reviews are a prominent source of how customers see product attributes (Pee, 2016). It is necessary for managers to understand how their products are perceived by customers. Managers therefore can use reviews to leverage the popularity of the attributes of products. This study provided them with a method to do so.

Limitations and future research

Inevitably, this study has limitations. First, this study investigates a specific industry and national context. The replication of the study in different contexts is essential for future research. Second, an emerging area of interest is visual online reviews. Studies could research the effects of the visual content. Third, this study considered random effects and applied the linear mixed model to estimate these effects. Although the variance of different restaurants can be found, the reason behind this difference is still not clear. Fourth, this study only identified the five key attributes, but other important attributes were not included. This will cause biases, because the satisfaction is an overall perception, and the sentiment and innovativeness about other attributes (e.g., environment, price) also affect this perception. Further research can identify more attributes. Fifth, this research did not account for fake reviews. Even though Yelp has an algorithm to filter out fake reviews (Luca, 2011), future research could try to establish whether there are fake reviews on Yelp and research their effect on behavior.

Footnotes

Appendix

Appendix 1.

Innovativeness word list and scores.

advanced	4
avant-garde	5
brand-new	5
Breakthrough	4
contemporary	3
creation	4
creative	4
creativeness	4
creativity	4
cutting edge	3
developed	3
development	2
different	3
distinct	3
evolution	2
futuristic	4
imaginative	4
improved	3
improvement	3
innovation	3
innovative	3
innovatory	3
invention	4
inventive	4
inventiveness	4
leading edge	5
metamorphosis	5
mint condition	3
modern	3
modernism	3
modernistic	3
modernity	3
modernization	3
modernized	3
modification	1
mutation	2
neoteric	3
new	3
new wrinkle	4
newest	5
newfangled	4
newfashioned	3
newness	3
novel	5
novelty	5
original	5
originality	5
origination	5
originative	5
progressive	2
radical	5
radical change	5
radically	5
rebuilt	4
recast	4
reconstructed	4
recreated	4
reformation	4
regenerated	4
remodeled	4
renascent	4
renewed	3
renovation	3
restyle	4
restyling	4
revolution	5
revolutionary	5
transformation	5
ultramodern	5
unhackneyed	5
unprecedented	5
up-to-date	4
way-out	5

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Nelleke de Boer

References

Akerlof

G. A.

(1970). The market for “lemons”: Quality uncertainty and the market mechanism. The Quarterly Journal of Economics, 84, 488–500.

Barr

D. J.

Levy

Scheepers

Tily

H. J.

(2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68, 255–278.

Blei

D. M.

A. Y.

Edu

J. B.

(2003). Latent Dirichlet Allocation. Journal of Machine Learning Research, 3, 993–1022.

Breazeale

(2009). FORUM: Word of mouse. International Journal of Market Research, 51(3), 1–19.

Cabosky

(2016). Social media opinion sharing: beyond volume. Journal of Consumer Marketing, 33(3), 172–181.

Calantone

R. J.

Chan

Cui

A. S.

(2006). Decomposing product innovativeness and its effects on new product success. The Journal of Product Innovation Management, 23, 408–421.

Chevalier

J. A.

Mayzlin

(2006). The effects of word of mouth on sales: Online book review. Journal of Marketing Research, 43, 345–354.

Danneels

Kleinschmidt

E. J.

(2000). Product innovativeness from the firm’s perspective: Its dimensions and their impact on project selection and performance. The Journal of Product Innovation Management, 18, 357–373.

Fan

Z.-P.

Che

Y.-J.

Chen

Z.-Y.

(2017). Product sales forecasting using online reviews and historical sales data: A method combining the Bass model and sentiment analysis. Journal of Business Research, 74, 90–100.

10.

Fang

Zhan

(2015). Sentiment analysis using product review data. Journal of Big Data, 2, Article 5.

11.

Farris

Bendle

N. T.

Pfeifer

P. E.

Reibstein

D. J.

(2010). Marketing metrics: The definitive guide to measuring marketing performance. Upper Saddle River: Wharton School Publishing.

12.

Floyd

Freling

Alhoqail

Cho

H. Y.

Freling

(2014). How online product reviews affect retail sales: A meta-analysis. Journal of Retailing, 90, 217–232.

13.

Fornell

(1992). A national customer satisfaction barometer: The Swedish experience. Journal of Marketing, 56, 6–21.

14.

Guo

Barnes

S. J.

Jia

(2017). Mining meaning from online ratings and reviews: Tourist satisfaction analysis using Latent Dirichlet Allocation. Tourism Management, 59, 467–483.

15.

Hayes

A. F.

Preacher

K. J.

(2014). Statistical mediation analysis with a multicategorical independent variable. British Journal of Mathematical and Statistical Psychology, 67, 451–470.

16.

Heng

Gao

Jiang

Chen

(2018). Exploring hidden factors behind online food shopping from Amazon reviews: A topic mining approach. Journal of Retailing and Consumer Services, 42, 161–168.

17.

Hennig-Thurau

Gwinner

K. P.

Walsh

(2003). Electronic Word of Mouth: Motives for and consequences of reading customer articulations on the internet. International Journal of Electronic Commerce, 8, 51–74.

18.

Hofacker

C. F.

Malthouse

E. C.

Sultan

(2016). Big Data and consumer behavior: Imminent opportunities. Journal of Consumer Marketing, 33, 89–97.

19.

Sian Koh

Reddy

K. S.

(2014). Ratings lead you to the product, reviews help you clinch it? The dynamics and impact of online review sentiments on products sales. Decision Support Systems, 57, 42–53.

20.

Jia

(2019). Toward a better fitness club: Evidence from exerciser online rating and review using Latent Dirichlet Allocation and support vector machine. International Journal of Market Research, 61, 64–76.

21.

Johns

Pine

(2002). Consumer behaviour in the food service industry: A review. Hospitality Management, 21, 119–134.

22.

Keller

K. L.

(1993). Conceptualizing, measuring, and managing customer-based brand equity. Journal of Marketing, 57(1), 1–22.

23.

Lee

J. H.

Jung

S. H.

Park

(2017). The role of entropy of review text sentiments on online WOM and movie box office sales. Electronic Commerce Research and Applications, 22, 42–52.

24.

Mai

(2018). The effect of online reviews on product sales: A joint sentiment-topic analysis. Information & Management, 56, 172–184. doi:10.1016/j.im.2018.04.007

25.

Luca

(2011). Reviews, reputation, and revenue: The case of yelp.com. Retrieved from https://www.hbs.edu/faculty/Publication%20Files/12-016_a7e4a5a2-03f9-490d-b093-8f951238dba2.pdf

26.

Montoya

R. M.

Horton

R. S.

Vevea

J. L.

Citkowicz

Lauber

E. A.

(2010). A re-examination of the mere exposure effect: The influence of repeated exposure on recognition, familiarity, and liking. Psychological Bulletin, 143, 459–498.

27.

Namazi

N.-R.

(2016). Conceptual analysis of moderator and mediator variables in business research. Procedia Economics and Finance, 36, 540–554. doi:10.1016/S2212-5671(16)30064-8

28.

Ngo

L. V.

O’Cass

(2013). Innovation and business success: The mediating role of customer participation. Journal of Business Research, 66, 1134–1142.

29.

Olson

E. M.

Walker

O. C.

Ruekert

R. W.

(1995). Organizing for effective new product development: The moderating role of product. Journal of Marketing, 59, 48–62.

30.

Park

Nicolau

J. L.

(2015). Asymmetric effects of online consumer reviews. Annals of Tourism Research, 50, 67–83.

31.

Pee

L. G.

(2016). Negative online consumer reviews. International Journal of Market Research, 58, 545–567.

32.

Poria

Cambria

Gelbukh

(2016). Knowledge-based systems aspect extraction for opinion mining with a deep convolutional neural network. Knowledge-Based Systems, 108, 42–49.

33.

Ramanathan

(2011). Guests’ perceptions on factors influencing customer loyalty. International Journal of Contemporary Hospitality Management, 23, 7–25.

34.

Reimer

Benkenstein

(2016). When good WOM hurts and bad WOM gains: The effect of untrustworthy online reviews. Journal of Business Research, 69, 5993–6001.

35.

Rindfleisch

Jeffrey

(1998). Explaining the familiarity-liking relationship: Mere exposure, information availability, or social desirability? Marketing Letters, 9, 5–19.

36.

Robson

Farshid

Bredican

Humphrey

(2013). Making sense of online consumer reviews: A methodology. International Journal of Market Research, 55, 521–537.

37.

Rodale

J. I.

(1986). The synonym finder. Emmaus, PA: Rodale Press.

38.

Ruiz-Mafe

Chatzipanagiotou

Curras-Perez

(2018). The role of emotions and conflicting online reviews on consumers’ purchase intentions. Journal of Business Research, 89, 336–344.

39.

Salehan

Kim

D. J.

(2016). Predicting the performance of online consumer reviews: A sentiment mining approach to big data analytics. Decision Support Systems, 81, 30–40.

40.

Sharma

Baoku

(2013). Customer satisfaction in Web 2.0 and information technology development. Information Technology & People, 26, 347–367.

41.

Singh

J. P.

Irani

Rana

N. P.

Dwivedi

Y. K.

Saumya

Roy

P. K.

(2017). Predicting the “helpfulness” of online consumer reviews. Journal of Business Research, 70, 346–355.

42.

Soriano

R. D.

(2002). Customers’ expectations factors in restaurants: The situation in Spain. International Journal of Quality & Reliability Management, 19, 1055–1067.

43.

Spence

(1973). Job market signaling. The Quarterly Journal of Economics, 87, 355–374.

44.

Srivastava

Sharma

R. W.

(2017). Developing a model for studying the antecedents and effects of Word of Mouth (WoM) and e-WoM marketing based on literature review. Jindal Journal of Business Research, 6, 25–43.

45.

Thelwall

Buckley

(2013). Topic-based sentiment analysis for the social web: The role of mood and issue related words. Journal of the American Society for Information Science and Technology, 64, 1608–1617.

46.

Thelwall

Buckley

Paltoglou

Cai

Kappas

(2010). Sentiment strength detection in short informal text. Journal of the American Society for Information Science and Technology, 61, 2544–2558.

47.

Tirunillai

Tellis

G. J.

(2014). Mining marketing meaning from online chatter: Strategic brand analysis of big data using Latent Dirichlet Allocation. Journal of Marketing Research, 51, 463–479.

48.

van Trijp

H. C.M

van Kleef

(2008). Newness, value and new product performance. Trends in food science & technology, 19(11), 562–573.

49.

Vermeulen

I. E.

Seegers

(2009). Tried and tested: The impact of online hotel reviews on consumer consideration. Tourism Management, 30, 123–127.

50.

Weinberg

B. D.

(2001). Research in exploring the online consumer experience. ACR North American Advances, 28, 227–232.

51.

Zajonc

R. B.

(1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology Monographs, 9, 1–27.

52.

Zhao

Han

Meng

Zhang

(2017). Learning semantic representation from restaurant reviews: A study of Yelp dataset. Retrieved from http://memray.me/uploads/iconference17-representation-yelp.pdf