Is RateMyProfessors.com Unbiased? A Look at the Impact of Social Modelling on Student Online Reviews of Marketing Classes

Abstract

This article looks at how marketing student ratings of instructors and classes on online rating sites such as RateMyProfessor.com can be biased by prior student ratings of that class. Research has identified potential sources of bias of online student reviews administered by universities. Less has been done on the sources of bias inherent in a ratings site where those doing the rating can see prior ratings. To measure how student online ratings of a course can be influenced by existing online ratings, the study used five different prior ratings experiment conditions: mildly negative prior ratings, strongly negative prior ratings, mildly positive prior ratings, strongly positive prior ratings, and a control condition of no prior ratings. Results of this study suggest prior online ratings, both positive and negative, do affect subsequent online ratings and bias them. There are several implications. First, both negative and positive ratings can have an impact biasing subsequent ratings. Second, sometimes negative prior ratings must be strong in valence in order to bias subsequent ratings whereas even mildly positive ratings can have an impact. Last, this bias can potentially influence student course selection.

Keywords

teaching ratings social modelling student evaluation of teaching online ratings bias experiments methodology marketing education issues

Introduction

Online reviews and review sites have an important impact on consumers and businesses. Yelp alone had 115 million cumulative reviews by November 2016 (Smith, 2016). Consumers increasingly rely on electronic word-of-mouth (e-WOM) for their purchasing decisions, so ratings can have a major impact on sales (Chevalier & Mayzlin, 2006; Clemons, Gao, & Hitt, 2006; Liu, 2006). Similarly, in marketing education, online reviews can influence student choice of classes and instructors.

Some research has been done on the efficacy and potential sources of bias of online student reviews administered by universities (Estelami, 2015; Guder & Malliaris, 2010, 2013). Less research has been done on the impact of allowing students to rate instructors/courses after seeing prior ratings as on third party rating sites. Research has looked at RateMyProfessors.com, with some studies finding bias. For example, Legg and Wilson (2010) found that students who voluntarily went online to rate their professors gave them poorer evaluations than those who had been prompted to do so. On the other hand, others like Otto, Sanford, and Ross (2008) found that the scale items on RateMyProfessor.com, helpfulness, clarity, and ease, were not biased measures of student learning. This study seeks to shed light on another area of potential bias, the degree to which recent online student reviews on sources, such as RateMyProfessors.com, can influence subsequent reviews. Implications for the validity of online reviews are suggested.

Online Product Reviews and Valence

Online product reviews are an important source of information for consumers shopping both online and in non-online environments (Chevalier & Mayzlin, 2006). Such reviews are done by proactive online consumers who create large amounts of information that influence other consumers’ decisions. There are many sites, such as Amazon.com and Yelp, and many more specialized sites that act as guides for consumers for specific categories of products and services. Such reviews can be found in places as diverse as retailing sites online, message boards on websites (Chiou & Cheng, 2003), online social networks (J. Brown, Broderick, & Lee, 2007), and virtual online communities (Bagozzi & Dholakia, 2002). Online consumer reviews of products have become a starting point for many consumers in their search for products and in fact are actually useful in predicting sales of entertainment (Dellarocas, Zhang, & Awad, 2007).

Such online consumer reviews extend the scope and reach of WOM and they are in fact a great source of information for e-WOM, providing information as well as reviews and opinions about products and services (Chatterjee, 2001). WOM is in general more trusted than marketer-generated information, such as advertising, but it is limited by how many people individual consumers have contact with. By contrast, consumer online reviews have a potentially unlimited audience. As with in-person WOM, consumers have a variety of motives for writing online reviews that range from altruistic to self-interested (Hennig-Thurau, Gwinner, Walsh, & Gremler, 2004). Some of the reviews are generated by the marketers of the products. It is sometimes hard for consumers to know for sure how credible such e-WOM is from online reviews.

As a result of this anonymity, consumers may not trust online reviews as much as in-person WOM since they are usually not familiar with those writing them. Racherla, Mandviwalla, and Connolly (2012) found that greater perceived similarity and argument quality of reviewers will lead to greater trust by consumers. The valence of the review may also have an impact. One negative or positive review may not have an impact because it will tend to be attributed to the reviewer, but repeated valenced reviews will influence consumer views (Kim & Gupta, 2012).

Such valenced reviews also serve an informational purpose. Repeated good reviews will be viewed as a source of positive information about the product or service, suggesting it as a good means to fulfill various types of needs, just as numerous negative reviews serve as a warning to consumers about it. Negative information can lead to greater credibility of the review when the reviewer’s identity is known (Kusumasondjaja, Shanka, & Marchegiani, 2012).

Instructor Online Reviews

Just as products and services are rated, instructors are rated as well. There are teaching evaluations done by universities themselves that rate instructors and courses. Online surveys can save cost entering data for administration and are more convenient for students who can provide more extensive comments (Guder & Malliaris, 2010), but administering them raises new challenges, such as timing (Estelami, 2015) and response rates (Guder & Malliaris, 2013).

This research extends these findings by looking at what happens when instructor and course ratings are available online for students to see and rate their instructors. Certainly one benefit of allowing students to see instructor ratings online is that it will provide them information necessary to make good decisions. The information would also preempt less accurate information from rumors or other less professional sources.

Are Online Reviews Accurate?

Given the impact of online reviews/e-WOM, an important question is the issue of veracity. How accurate are these reviews? Are they subject to influences other than just the reviewer’s experience with the product or service?

There are external instructional ratings sites, such as RateMyProfessors.com, fitting into this category that provide students with the ability to both rate instructors and see the ratings of others. Such rating sites are commonly used by students. M. J. Brown, Baillie, and Fraser (2009) found that about 40% of the overall variance in university-administered student teaching ratings could be related to the variance in teaching ratings on RateMyProfessors.com. Valence is an important part of ratings, the vast majority are either positive or negative, which suggests that just as in product and service ratings, students come in with strong valenced feelings about their experiences in a class (Hartman & Hunt, 2013).

Sridhar and Srinivasan (2012) found that prior online ratings did have a moderating effect on the positive and negative aspects of a product or service experience, affecting the reviewer’s rating. This suggests that reviews can be influenced by previous reviews in ways that in turn influence overall perceptions. The question is how this process works. This article suggests two potential processes by which prior online reviews can affect a reviewer’s rating, modelling behavior and psychological reactance.

Social Modelling

On the one hand, there should be a general tendency for the valence of prior ratings to influence a reviewer’s rating in the same direction. Social modelling suggests that reviewers will use existing ratings as a cue for how they should rate the product. Research has found that families, peers, and other reference groups can have an influence on purchase decisions (Bearden & Etzel, 1981; Childers & Rao, 1992). In an actual experiment, Bevelander, Anschutz, and Engels (2010) found social modelling affected food purchases at a supermarket. What peers purchased in the store influenced the types of products purchased by subjects.

Research on polling finds a similar process. Polls that show majority support for one or another side of an issue can influence public opinion through what is known as the “bandwagon effect” (Marsh, 1985; Rothschild & Malhotra, 2014; Simon, 1954). For example, in an experiment, Marsh (1985) found that perceived public opinion had an influence on attitude toward abortion laws. Later, Rothschild and Malhotra (2014) found bandwagon effects changed opinion to conform to perceived consensus, especially for issues in which subjects did not have very strong prior opinions.

Similarly, social modelling suggests that the valence of ratings others have given to a product or service online should influence subsequent ratings. On an online course rating site, the comments and especially the rating of a course could provide information that would guide students providing new ratings. If there are predominantly negative ratings, there should be a negative influence on new ratings.

Evaluations of various aspects of the instructor and the course can also be influenced by the valence of the prior ratings. Evaluation of the helpfulness or clarity of the instructor or overall perceived satisfaction with such a course can be affected by negative information from ratings and also by positivity as well (Larsen, Smith, Kyle, & Cacioppo, 1998). Thus,

Hypothesis 1: Student online ratings of various aspects of an instructor and a course will be more negative when prior ratings are predominantly negative than when they are positive or when there are no prior online ratings.

On the other hand, if there are predominantly positive ratings, the ratings of online reviewers should be influenced to be more positive. Thus,

Hypothesis 2: Student online ratings of various aspects of an instructor and a course will be more positive when prior ratings are predominantly positive than when they are negative or when there are no prior online ratings.

There may also be circumstances in which an opposing process affecting the valence of online ratings interferes with the peer influence of social modelling. Sometimes raters are either not influenced by existing ratings or perhaps even influenced in the opposite direction by the valence of existing ratings. Comments on ratings sites mention “I don’t understand all the (positive/negative) comments” or how “unfair” are the existing comments.

Psychological Reactance

Reactance theory suggests that when freedom to make a certain choice or engage in a behavior is threatened, that threatened choice or behavior becomes more attractive (Lessne & Notarantonio, 1988). The classic example of reactance is increased desire in children when they are told they cannot have something (Rummel, Howard, Swinton, & Seymour, 2000). One of the key aspects of the reactance effect is that it is greater as the pressure to comply increases (Brehm & Sensenig, 1966). Fitzsimons and Lehmann (2004) found that unsolicited recommendations by experts contradicting a respondent’s initial impressions lead to reactance, which led to a behavioral backlash and behavior contradicting the advice. These findings suggest that prior online ratings that strongly contradict a reviewer’s initial impression will influence his or her rating in the opposite direction.

Applying this to student ratings of courses, the rating of students who see strongly negative or strongly positive reviews prior to rating may be more likely to experience reactance than if there were mildly negative or positive reviews. Rather than social modelling, these prior reviews may trigger a backlash, leading students to view strongly negative ones as “mean” or strongly positive ones as exaggerated. As a result, such ratings may have less of an impact than moderately valenced ratings. If there is reactance, the overall valence of respondents will similarly influence evaluation of all of the various aspects of the instructor and course, so

Hypothesis 3: Student online ratings of various aspects of an instructor and a course will be more positive when prior ratings are moderately positive than when they are strongly positive.

Hypothesis 4: Student online ratings of various aspects of an instructor and a course will be more negative when prior ratings are moderately negative than when they are strongly negative.

Study

This study examines the process of how valence of prior ratings affects subsequent ratings and tests the prior hypotheses. Specifically, it attempts to determine if students will model their ratings and evaluations of a course after those that rated previously. A total of 162 usable responses from undergraduate marketing students from a large public university in the southwestern United States were collected. There were 32 in the mildly negative ratings condition, 31 in the strongly negative ratings condition, 32 in the mildly positive ratings condition, 38 in the strongly positive ratings condition, and 29 in the no ratings control condition.

Respondents were asked to view a video clip, a little more than a minute in length, of a lecture for an advertising course. This method was used because video clips of even 30 seconds in length were found to be predictive of end-of-the-semester teaching evaluations, so this stimulus offered a realistic student evaluation of teaching (SET) stimulus (Ambady & Rosenthal, 1993). The exact same video was shown to respondents in all experimental conditions. Respondents were first given a manipulation check to see if they had viewed the video that asked them to identify the topic of the course, accounting, advertising, or management. Responses from respondents who failed to correctly identify the course were discarded. They were then asked to think about this course and lecturer and make evaluations.

Respondents were shown prior ratings that they were told were the latest compiled from those of students who had recently completed the questionnaire. These ratings ranged from 1 to 5, 1 = terrible and 5 = outstanding. Students in the mildly negative ratings section were shown prior ratings that averaged 2.75, and in the strongly negative ratings section the ratings averaged 1.5. Students in the mildly positive ratings section were provided with ratings that averaged 4.0, and in the strongly positive ratings section the ratings averaged 4.75. Students in the control condition were shown no ratings.

Students were then asked to provide their own rating of the class, giving it between 1 star (terrible) to 5 stars (outstanding), a rating system used on RateMyProfessors.com. They were then asked a measure of the percentage chance they would recommend this class to other business school students, commonly used to rate products and services (Kozak, Rimmington, & Kozak, 2000). The last of the introductory questions asked students to provide three words that they felt best described the class to provide qualitative information about their perceptions. This manipulation imitated the way the most recent ratings on online rating sites are the most visible to students.

Students then completed an online questionnaire. To be realistic, course-related measures were similar to those found in RateMyProfessors.com and were informed by instructor rating measures used in other educational research studies (Ackerman & Gross, 2010; Sidelinger & McCrowsky, 1997). There were three-item measures asking how easy (M = 4.15, α = .78), clear (M = 4.89, α = .92), and helpful (M = 4.80, α = .93) students felt the instructor was. There was also a measure of student satisfaction with the class (M = 4.85, α = .96). To examine discriminant validity, the dimensionality of the items was tested through confirmatory factor analysis. The analysis found a five-factor model to have the best fit. This five-factor model had a goodness-of-fit index of .93, a root mean square residual of .12, and total explained variance of 86%.

Measures related to perceptions of the instructor and class loaded cleanly onto five factors using varimax rotation. Questions a1 to a3 measured the helpfulness of the instructor with loadings of .71 to .73. Items a4 to a6 pertained to clarity of the instructor with loadings of .81 to .85. Questions a7 and a9 measured the degree to which students felt the instructor’s course would be easy with factor loadings of .71 to .88. Last, perceived satisfaction with the course, measured by Questions a10 to a12, had loadings of .86 to .88. These factors are displayed in the appendix.

Last, there were measures assessing the students themselves. There was a three-item measure of self-efficacy (Gist & Mitchell, 1992; Gist, Stevens, & Bavetta, 1991) regarding assessment of a business course (M = 5.72, α = .96), Questions b1 to b3. There was also a three-item scale (M = 3.18, α = .94) to measure students’ outlooks toward performance and ability. This scale included items for “A high score on this scale indicates that people believe that ones’ abilities are fixed and cannot change (an entity view) while a low score indicates the belief that abilities are malleable and can change (an incremental view)” (Dweck, 2005; Dweck & Leggett, 1988), Questions b4 to b6. Last, there was a three-item measure (M = 4.86, α = .82) of need for cognition (Chang & Yen, 2013; Haugtvedt & Petty, 1992), Questions b7 to b9.

These measures are displayed in the appendix. Afterward, students were debriefed regarding the nature of the study.

Results

A MANCOVA of prior ratings (“mildly negative,” “strongly negative,” “mildly positive,” “strongly positive,” and “none”) on the dependent variables was done to determine basic reactions of students to prior ratings they were exposed to. Self-efficacy regarding marketing was used as a covariate, allowing researchers to remove the bias that is caused by the variable of self-efficacy regarding marketing (Field, 2013). This analysis found significant differences for ratings as well as for several of the evaluations of the instructor and the course (F[5, 126] = 5.42, p < .01, Wilks’ Lambda = .84, partial eta-squared = .16). The results are displayed in Table 1.

Table 1.

Means for Prior Ratings (n = 162).

Measure	Mean for mildly negative ratings	Mean for strongly negative ratings	Mean for mildly positive ratings	Mean for strongly positive ratings	Mean for no ratings	Effect size (η²)	p
Course rating	3.50¹	2.97²	3.66¹	4.17³	3.48¹	.12	<.01
Chance of recommending	65¹	53²	71¹	79⁴	67	.11	<.01
Clarity of instructor	4.69¹	4.13²	5.22³	5.34³	4.83¹	.16	<.01
Helpfulness of instructor	4.69¹	3.84²	5.41³	5.24³	4.80¹	.15	<.01
Satisfaction with course	4.78¹	3.97²	5.22³	5.26³	5.07¹³	.10	<.01
Easy	3.78	3.84	4.09	3.47	3.48	.04	.39

Note. Means with completely different superscripts are significantly different from each other.

Impact on Ratings

Results of analysis of means for the chance that students would recommend the class to others found, on the one hand, that that means were significantly higher in the strongly positive condition and significantly lower in the strongly negative condition, but not significantly different between the mildly negative, mildly positive and control conditions ( $\bar{x}$ _{strong negative ratings} = 2.97, $\bar{x}$ _{mild negative ratings} = 3.50, $\bar{x}$ _{strong positive ratings} = 4.17, $\bar{x}$ _{mild positive ratings} = 3.66, $\bar{x}$ _{no ratings} = 3.48). These results are displayed in a line graph in Figure 1.

Figure 1.

Course rating by condition.

Similarly, the chance of recommending the course to others was significantly higher than the means in the other conditions in the strongly positive condition and significantly lower in the strongly negative condition, but not significantly different between the mildly negative, mildly positive, and control conditions ( $\bar{x}$ _{strong negative ratings} = 53, $\bar{x}$ _{mild negative ratings} = 65, $\bar{x}$ _{strong positive ratings} = 79, $\bar{x}$ _{mild positive ratings} = 71, $\bar{x}$ _{no ratings} = 67).

Impact on Evaluations of the Instructor and the Course

For the perceived ease of the course, the means for mild and strong positive prior ratings were not significantly different from each other but were significantly higher than the means for mild negative ratings. Perceived clarity of the instructor was highest when there were either mild or strongly positive ratings and lowest when there were strongly negative ratings. There were no differences between the mildly negative and control conditions ( $\bar{x}$ _{strong negative ratings} = 4.13, $\bar{x}$ _{mild negative ratings} = 4.69, $\bar{x}$ _{strong positive ratings} = 5.34, $\bar{x}$ _{mild positive ratings} = 5.22, $\bar{x}$ _{no ratings} = 4.83). Perceived helpfulness of the instructor was highest when there were either mild or strongly positive ratings and lowest when there were strongly negative ratings. There were no differences between the mildly negative and control conditions ( $\bar{x}$ _{strong negative ratings} = 3.84, $\bar{x}$ _{mild negative ratings} = 4.69, $\bar{x}$ _{strong positive ratings} = 5.24, $\bar{x}$ _{mild positive ratings} = 5.41, $\bar{x}$ _{no ratings} = 4.80). Last, perceived satisfaction with the course was lowest when there were strongly negative ratings, but were not significantly different between the mildly negative, both positive and control conditions ( $\bar{x}$ _{strong negative ratings} = 3.97, $\bar{x}$ _{mild negative ratings} = 4.78, $\bar{x}$ _{strong positive ratings} = 5.26, $\bar{x}$ _{mild positive ratings} = 5.22, $\bar{x}$ _{no ratings} = 5.07). These results provide support for Hypotheses 1 and but not Hypotheses 3 and 4.

Qualitative Results

The descriptive words students wrote about the course were counted by condition and tabulated in order from most frequent to least frequent. The five most frequently mentioned words for the mildly negative ratings condition, in order, were “boring, interesting, useful, difficult, informative.” The five most frequently mentioned words for the strongly negative ratings condition, in order, were “boring, easy, confusing, dull, not helpful.” The five most frequently mentioned words for the mildly positive ratings condition, in order, were “interesting, practical, hard, easy, informative.” The five most frequently mentioned words for the strongly positive ratings condition, in order, were “interesting, practical, informative, time consuming, insightful.” Last, the five most frequently mentioned words for the control condition, in order, were “informative, interesting, boring, interactive, common sense.”

Discussion

The findings of this study suggest that social modelling has a significant impact on online ratings of sites like RateMyProfessor.com, with a few visible positive or negative ratings biasing subsequent ratings. This is bad news for those concerned about the impact of those few students who would use such sites for revenge for a bad grade. Subsequent ratings will fall in line with such negative ratings and influence overall averages for a long time in a vicious cycle. On the other hand, a few good ratings can also have long-term impact on overall ratings as well in what could be thought of as a virtuous cycle. This suggests that online rating results can be gamed by either individual raters or by the individual being rated, resulting in significant inaccuracies of the results.

These results support findings of research in political polling, which find that polls not only reflect but can also influence public opinion through a bandwagon effect. Seeing polls that find a majority of people believe one way or another on a certain issue can influence others to fall in line as long as they do not have strong prior opinions about that topic. The magnitude of this impact can depend on the strength and presumably valence of prior opinions (Marsh, 1985; Rothschild & Malhotra, 2014).

A second important finding is that social modelling hay have a different impact depending on whether ratings are positive or negative. It was more often true that mildly negative ratings did not have a significant impact on the dependent variables and that only strongly negative prior ratings did. By contrast, for most of the dependent variables, both mild and strong positive prior ratings influenced responses. This suggests that those doing online teaching ratings may discount mildly negative prior ratings for whatever reason, maybe perceiving them as less informative, until they reach a certain level of negativity on which it has a big influence on their ratings. Respondent reaction to positivity in prior ratings was similar for the overall rating, no impact for mild but a strong impact for strong positivity, but even mild positivity influenced perceptions of the instructor. Perhaps there is a tendency to discount mild negativity as the norm in rating people, especially for students rating instructors after grades are turned in, but positivity is a believable cue that has more impact.

A third finding of this study is that, at least for evaluations of instruction, lecture, and people in general, the above-mentioned results are remarkably robust. The results were invariant to personality and outlook of student. Social modelling affected students in exactly the same way regardless of implicit theory, need for cognition, or self-efficacy about classes. Regardless of high or low levels of these personal attributes, student ratings were influenced by prior ratings, sometimes with large differences from the control condition.

This study also found that, outside of the ratings, various aspects of perceptions of classes or instructors can also be influenced merely by the positivity or negativity of the ratings of others. The perceived clarity, helpfulness of the instructor, and even satisfaction with the course are influenced by the valence of prior ratings. The more negative influence the prior ratings, the more difficult respondents perceived the course to be. Given that all of these aspects of a course are factors in student choice, prior ratings bias can have a significant influence on enrollment in a course.

Last, pretests suggest that the results of this study may be more applicable when people are not involved at all. A pretest asking for an evaluation of a new course syllabus, no lecture or person involved, found that when students saw unusually high or unusually low ratings, they did not react as much as when they saw less extreme ratings. On the other hand, even in this pretest, the ratings in the strong negative and positive prior ratings conditions were still lower and higher, respectively, than in the control conditions, suggesting that social modelling still has an impact even when there is no human aspect of what is being rated.

Limitations

This study is limited in that although students had time to make a judgment of the lecturer in the video clip, they have not had the time to make an emotional attachment over the course of a semester that may mitigate the effects of social modeling. Perhaps future research could find a way to examine the impact of prior ratings on the ratings of real classes and real instructors. Also, interviews or verbal protocols would be helpful to look more in-depth at the process.

This study is also limited in that subjects were not given the option to opt out of rating the instructor and class. It is likely that under some conditions, students would choose to just not provide their rating. Many instructors on RateMyProfessors.com are only rated by one or two people a semester at most. Future research needs to look at who rates on online rating sites and perhaps also at the influence of prior ratings on the propensity to make an online rating. Future research could also look at the influence of prior ratings on the propensity of students to rate on a rating site. Perhaps extreme ratings, especially negative ones, discourage them from rating a class. On the other hand, ratings in the middle may increase the propensity of students to do an evaluation on an online site.

Implications

This study has public policy implications for dissemination of information about marketing courses and instructors. Some business schools and universities provide student-evaluation information about courses online. The results of this study suggest that such evaluations may help combat the inaccuracies of third party rating sites, such as RateMyProfessors.com. Such university-administered SETs are collected from all students and are collected before course grades are turned in, which means that retaliation is not an issue, so they are going to be more accurate.

The results also have implications for the validity of third party rating sites that allow evaluation of marketing instruction. The negative implications are that teaching ratings sites have low validity in general and are easily influenced by factors that have no relation to actual perceptions. A few bad or malicious reviews can influence subsequent evaluations on that same site for some time to come. Conversely, marketing instructors can rely on past positive evaluations to boost ratings of future teaching performance.

These results have implications for research in online evaluation not just for education but also for products and services. Angry consumers or even competitors could cause significant damage to a firm’s reputation by posting a series of negative ratings. They could lower future evaluations for a period of time, which could in turn influence other evaluations as well, creating a domino effect. It is also possible that strongly negative ratings could prevent those who would otherwise post positive ratings from doing so. Conversely, a series of positive ratings could help a firm’s reputation above and beyond just the impact of those ratings alone.

These results also have implications for polling research in the impact of valence. It is likely that polls showing majority support for an issue or person may have a different impact on public opinion than those that show majority disagreement. As in online ratings, it is possible that negativity may require a higher threshold than positivity to create the bandwagon effect observed in that literature.

Last, this research also has implications for the relatively recent phenomenon of online mobs on social media and rating sites. It suggests that such attacks, often with hundreds of extremely negative ratings in a short period of time by those who may or may not have directly experienced what they are rating, can present an inaccurate view and do long-term damage to the reputation of a person or business. Results find that after significant numbers of negative ratings on an online rating site, subsequent ratings are not going to go back to normal after a while. For whatever reason, people will model, not resist, the online mob sentiment. Given these findings, maybe such sites might need to consider starting ratings anew every year or half year to combat such abuses.

Footnotes

Appendix

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Ackerman

D. S.

Gross

B. L.

(2010). Instructor feedback: How much do students really want? Journal of Marketing Education, 32, 172-181.

Ambady

Rosenthal

(1993). Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. Journal of Personality and Social Psychology, 64, 431-441.

Bagozzi

Dholakia

(2002). Intentional social action in virtual communities. Journal of Interactive Marketing, 16, 2-21.

Bearden

W. O.

Etzel

M. J.

(1982). Reference group influence on product and brand purchase decisions. Journal of Consumer Research, 9, 183-194.

Bevelander

K. E.

Anschutz

D. J.

Engels

(2010). Social modeling of food purchases at supermarkets in teenage girls. Appetite, 57, 90-194.

Brehm

Sensenig

(1966). Social influence as a function of attempted and implied usurption. Journal of Personality and Social Psychology, 35, 703-707.

Brown

Broderick

Lee

(2007). Word of mouth communication within online communities: Conceptualizing the online social network. Journal of Interactive Marketing, 21, 2-20.

Brown

M. J.

Baillie

Fraser

(2009). Rating RateMyProfessors.com: A comparison of online and official student evaluations of teaching. College Teaching, 57, 89-92.

Chang

C. T.

Yen

C. T.

(2013). Missing ingredients in metaphor advertising: The right formula of metaphor type, product type, and need for cognition. Journal of Advertising, 42, 80-94.

10.

Chatterjee

(2001). Online reviews: Do consumers use them? Advances in Consumer Research, 28, 129-133.

11.

Chevalier

J. A.

Mayzlin

(2006). The effect of word of mouth on sales: Online book reviews. Journal of Marketing Research, 43, 345-354.

12.

Childers

T. L.

Rao

A. R.

(1992). The influence of familial and peer-based reference groups on consumer decisions. Journal of Consumer Research, 19, 198-211.

13.

Chiou

J. S.

Cheng

(2003). Should a company have message boards on its web site? Journal of Interactive Marketing, 17, 50-61.

14.

Clemons

E. K.

Gao

G. G.

Hitt

L. M.

(2006). When online reviews meet hyperdifferentiation: A study of the craft beer industry. Journal of Management Information Systems, 23, 149-171.

15.

Dellarocas

Zhang

X. Q.

Awad

N. F.

(2007). Exploring the value of online product reviews in forecasting sales: The case of motion pictures. Journal of Interactive Marketing, 21, 23-41.

16.

Dweck

C. S.

(2005). Self-theories: Their role in motivation, personality, and development. Philadelphia, PA: Psychology Press.

17.

Dweck

C. S.

Leggett

E. L.

(1988). A social-cognitive approach to personality and motivation. Psychological Review, 95, 256-273.

18.

Estelami

(2015). The effects of survey timing on student evaluation of teaching measures obtained using online surveys. Journal of Marketing Education, 37, 54-64.

19.

Field

(2013). Discovering statistics using IBM SPSS statistics. Thousand Oaks, CA: Sage.

20.

Fitzsimons

G. J.

Lehmann

D. R.

(2004). Reactance to recommendations: When unsolicited advice yields contrary responses. Marketing Science, 23, 82-94.

21.

Gist

M. E.

Mitchell

T. R.

(1992). Self-efficacy: A theoretical analysis of its determinants and malleability. Academy of Management Review, 17, 183-211.

22.

Gist

M. E.

Stevens

C. K.

Bavetta

A. G.

(1991). Effects of self-efficacy and post-training intervention on the acquisition and maintenance of complex interpersonal skills. Personnel Psychology, 44, 837-861.

23.

Guder

Malliaris

(2010). Online and paper evaluations. American Journal of Business Education, 3, 131-137.

24.

Guder

Malliaris

(2013). Online course evaluations response rates. American Journal of Business Education, 6, 333-337.

25.

Hartman

K. B.

Hunt

J. B.

(2013). What RateMyProfessors.com reveals about how and why students evaluate their professors: A glimpse into the student mind-set. Marketing Education Review, 23, 151-161.

26.

Haugtvedt

C. P.

Petty

R. E.

(1992). Personality and persuasion: Need for cognition moderates the persistence and resistance of attitude changes. Journal of Personality and Social Psychology, 63, 308-319.

27.

Hennig-Thurau

Gwinner

K. P.

Walsh

Gremler

D. D.

(2004). Electronic word-of-mouth via consumer-option platforms: What motivates consumers to articulate themselves on the Internet? Journal of Interactive Marketing, 18, 38-52.

28.

Kim

Gupta

(2012). Emotional expressions in online user reviews: How they influence consumers’ product evaluations. Journal of Business Research, 65, 985-992.

29.

Kozak

Rimmington

Kozak

(2000). Tourist satisfaction with Mallorca, Spain, as an off-season holiday destination. Journal of Travel Research, 38, 260-269.

30.

Kusumasondjaja

Shanka

Marchegiani

(2012). Credibility of online reviews and initial trust: The roles of reviewer’s identity and review valence. Journal of Vacation Marketing, 18, 185-195.

31.

Larsen

T. A.

Smith

J. T.

Kyle

Cacioppo

J. T.

(1998). Negative information weighs more heavily on the brain: The negativity bias in evaluative categorizations. Journal of Personality and Social Psychology, 75, 887-900.

32.

Legg

Wilson

(2010). RateMyProfessors.com offers biased evaluations. Assessment & Evaluation in Higher Education, 37, 89-97.

33.

Lessne

G. J.

Notarantonio

(1988). The effects of limits in retail advertisements: A reactance theory perspective. Psychology and Marketing, 5, 33-44.

34.

Liu

(2006). Word of mouth for movies: Its dynamics and impact of box office revenue. Journal of Marketing, 70, 74-89.

35.

Marsh

(1985). Back on the bandwagon: The effects of opinion polls on public opinion. British Journal of Political Science, 15, 51-74.

36.

Otto

Sanford

D. A.

Ross

D. N.

(2008). Does RateMyProfessor.com really rate my professor? Assessment & Evaluation in Higher Education, 33, 355-368.

37.

Racherla

Mandviwalla

Connolly

D. J.

(2012). Factors affecting consumers’ trust in online product reviews. Journal of Consumer Behavior, 11, 94-104.

38.

Rothschild

Malhotra

(2014). Are public opinion polls self-fulfilling prophecies? Research and Politics, July-September, 1-10.

39.

Rummel

Howard

Swinton

J. M.

Seymour

D. B.

(2000). You can’t have that! A study of reactance effects & children’s consumer behavior. Journal of Marketing Theory and Practice, 8, 38-45.

40.

Sidelinger

R. J.

McCrowsky

J. C.

(1997, Winter). Communication correlates of teacher clarity in the college classroom. Communication Research Reports, 14(1), 1-10.

41.

Simon

H. A.

(1954). Bandwagon and underdog effects and the possibility of election predictions. Public Opinion Quarterly, 18, 245-253.

42.

Smith

(2016). By the numbers: 54 Amazing yelp statistics. Retrieved from http://expandedramblings.com/index.php/yelp-statistics/

43.

Sridhar

Srinivasan

(2012). Social influence effects in online product ratings. Journal of Marketing, 76, 70-88.