Abstract
Online consumer reviews significantly impact market performance as potential customers rely heavily on these reviews for consumer decision making. Accordingly, experienced online reviewers, or highly motivated reviewers who account for the largest attribution of reviews, are proposed to be an important part of the online reviewing ecosystem. This research examines experienced reviewers in the online communities. Using empirical data, this study found that experienced reviewers tend to behave as experts with the aim to achieve a common good with rating and selection attributes similar to critics. Hence, results showed that experienced reviewers leave lower ratings, have less extremity in their ratings, prefer sophisticated products but do not prefer popular products. The female experienced reviewers are less generous than novice female reviewers and their generosity decreases more dramatically than males in the rating propensity as they become experienced reviewers.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Online reviews are shared opinions of consumers via the Internet about various entities such as products, services, persons, or companies (Flanagin et al., 2011; Koh et al., 2010; Kunst & Vatrapu, 2019; Lin et al., 2017). These reviews are often considered more credible than commercial advertising or public relations (Furner & Zinko, 2017), and hence, potential consumers rely heavily on them in their purchase decisions (Filieri et al., 2018). According to a study (Fullerton, 2017), 93% of consumers reported that online reviews affect their purchase decisions and 68% of consumers are likely to pay 15% more for the same product or service if online reviews guarantee its quality. Therefore, online reviews have a substantial impact on the success or failure of the entities in the market (Chu & Kim, 2011; Cui et al., 2012; Vallurupalli & Bose, 2020; Zhu & Zhang, 2010). The impact of online reviews on sales has been illustrated in diverse business domains, such as ecommerce (Hong et al., 2017), local business (e.g., restaurant, hospital, hotel, etc.) (Luca, 2016), B2B business (McCabe, 2018), and film business (Lindbergh & Arthur, 2020).
Although online reviews are “shared” opinions, they do not necessarily represent the opinions of majority of consumers. In business, there is a phenomenon called the Pareto principle, also known as the 80/20 rule. It states that a few popular products (20%) create most market sales (80%) (Brynjolfsson et al., 2011). This phenomenon is found in online review communities, where a small number of highly motivated reviewers contribute a large number of online reviews. The presence of these experienced reviewers is well evidenced at Yelp.com, an online review website, where approximately 20% of the reviewers write 80% of the reviews (IESE Business School, 2015). Their presence is also evidenced on Amazon.com where a large portion of the online reviews are created by a small minority while more than 80% of the reviewers leave only one review at the website (Peddibhotla & Subramani, 2007; Woolf, 2014).
Due to the proposed importance of experienced reviewers, prior studies investigated their behaviors, affecting the success or failure of business. However, their results were mixed. Some studies investigated their motivations to contribute a large amount of reviews, arguing that they submit reviews to increase or decrease reputation (e.g., ratings) of products (Pinch & Kesler, 2011). Others argued that they submit reviews to gain compensation from retailers or manufacturers (Reichelt et al., 2014; Wu, 2019). For those reasons, some prior studies postulated that experienced reviewers tended to produce extremely positive or negative reviews (Pinch & Kesler, 2011). In contrast, other studies reported that experienced reviewers were likely to contribute balanced reviews, neither extremely positive nor extremely negative (Banerjee & Chua, 2018; Wang, 2010) and therefore have higher credibility in the communities (Banerjee et al., 2017). These conflicting findings cause confusion for business practitioners in various domains, who likely encounter experienced reviewers in their business. One of the possible reasons for the mixed results is an unverified assumption for the selection of review subjects and demographics of reviewers. The prior studies assumed that the experienced reviewers select the same products or services as novice reviewers, directly comparing average ratings between the two groups. This is a problematic assumption because the two groups may focus on different products for their reviews; for example, the novice group may prefer popular products, while the experienced group may prefer products that can satisfy their unique taste. In addition, some studies used selective samples with inadequate definitions on experienced and novice reviewers (Anderson & Simester, 2014; Banerjee & Chua, 2018). For example, Banerjee and Chua (2018) defined the novice group as reviewers submitting from one to ten reviews, the intermediate group from 45 to 54, and the experienced group from 91 to 100 reviews, ignoring the reviewers between the groups (e.g. reviewers submitting 55 to 90 reviews) or more than 100 reviews, who are highly experienced reviewers. Most notably, little prior research considered reviewers’ demographics such as age and gender in the comparison between experienced and novice reviewers, although they could affect review behavior (Leung & Yang, 2020; Mather et al., 2004). In particular, it is important to understand gender difference because some studies have shown that females are likely to contribute more reviews (Dunivin et al., 2020; Punj, 2013) and more likely to consider online reviews more seriously in their purchase decision (Abubakar et al., 2016; Freddie, 2018). Few studies investigated how experienced reviewers differ by gender in terms of rating score and extremity. This gap provides an opportunity to extend the literature of online and useful guidance for business practitioners who closely collaborate with experienced reviewers.
This research fills the aforementioned gaps in the extant experienced reviewer literature in the context of online movie review communities. The online movie community is selected because experienced reviewers have been shown to make a substantial impact on market performance of movies (Chintagunta et al., 2010; Ma et al., 2019; Zhang et al., 2020). In addition, online movie communities have been criticized for prevalence of unreliable reviews (Wilkinson, 2019), possibly contributed by experienced reviewers with strategic motivations. This is because the online movie communities have an environment better suited for such reviewers than ecommerce platforms, which require verification of purchase (e.g., user-created pictures, verified purchase) to submit reviews. Adopting empirical data including 211,197 reviews of 7642 users at Yahoo! Movies, in particular, this research addresses the following questions; (1) “do experienced reviewers select specific products for their reviews (e.g., products with sophistication)?” (2) “if so, do they have different review behaviors in their rating propensity (e.g., average rating, extremity in rating) from novice reviewers?”, and (3) “does their rating propensity differ by demographics (e.g., gender)? The findings of this study are expected to provide unique contributions to both academia and practitioners concerning experienced online reviewers. The rest of this paper is composed of the following sections. Section 2 discusses relevant studies to this research, particularly experienced online reviewers. Section 3 introduces major hypotheses of this study based on the extant literature. Section 4 discusses the dataset, research models to test the hypotheses, and results of statistical analysis. Lastly, Sect. 5 contains the conclusion of this study, contributions to both academia and industry, and limitations to be considered by future research.
Literature review
Experienced reviewers in online review platform
In the context of online review communities, experienced reviewers are highly important in that they are a small group of the population that contributes significantly larger reviews than novice reviewers (Peddibhotla & Subramani, 2007). They tends to provide not only a significant amount of information (i.e., heavy users) but also quality knowledge and information (i.e., experts) earlier than novice members (i.e., early adopters) (Peddibhotla & Subramani, 2007). Due to the importance of experienced reviewers (Aakash & Jaiswal, 2020), many prior studies examined them in online review communities. Although most of the studies focused on the relationship between their experience and perceived usefulness or helpfulness of their reviews (Choi & Leon, 2020; Fang et al., 2016; Liu & Park, 2015; Mudambi & Schuff, 2010; Racherla & Friske, 2012; Zhu et al., 2014), there are some studies on behavioral differences of experienced reviewers, which are the focus of this study.
One of the primary topics for experienced reviewer research is their motivations (Table 1). Past research investigated motivations of experienced reviews based on words in text reviews and the rating propensity of experienced reviewers (e.g., review valance, ratio of extreme ratings) compared to novice reviewers, which is a focus of this study. The motivations reported in the literature vary but can be summarized in two large categories: other-oriented and self-oriented motivations (Peddibhotla & Subramani, 2007). According to Bhattacharyya et al. (2020), reviewers with other-oriented motivations, such as social affiliation (Peddibhotla & Subramani, 2007) and altruism (Mathwick & Mosteller, 2017; Peddibhotla & Subramani, 2007), are the most dominant reviewer group. They contribute to online review communities for others who have a similar interest, providing neutral ratings and balanced information. They distinguish themselves from general reviewers who rate primarily to reward or punish the subjects (e.g., ecommerce sellers) with extreme rating scores (Lafky, 2014). Self-oriented motivations, which are inferred from their rating propensity and text reviews, include self-expression (Mathwick & Mosteller, 2017; Peddibhotla & Subramani, 2007), improvement of understanding of topics (Peddibhotla & Subramani, 2007), social image (Mathwick & Mosteller, 2017; Wang, 2010), publication of their work (McIntyre et al., 2016) and status seeking (Lampel & Bhalla, 2007; McIntyre et al., 2016; Wu, 2019). The self-oriented perspective supports the idea that experienced reviewers participate in the review process for their own satisfaction and enjoyment (Wu, 2019), although Lampel and Bhalla (2007) view status seeking as a social passion that generates the continued participation. In addition to the other-oriented and self-oriented motivations, some recent studies introduced strategic motivations. These studies argued that some experienced reviewers may have a strategic goal to generate a large amount of online reviews, such as encouraging (or discouraging) sales of specific products or services with extreme ratings and attaining compensation from the review activities (Reichelt et al., 2014; Wu, 2019). This situation can be complicated regarding experienced reviewers in the presence of incentives, as the buying public assume that the reviewers are regular customers (Owen, 2011). When manufacturers send top reviewers (e.g., influencers) products free of charge to solicit their reviews, a bias can be introduced undermining the credibility of the reviewers and the integrity of the process (Chow, 2013; Dvorak, 2011; Lee, 2020; Pinch & Kesler, 2011).
The presence of experienced reviewers with strategic motivations has brought researchers’ attention to their review behaviors. However, they reached mixed conclusions. Anderson and Simester (2014) found that the customers who reviewed a product without purchasing it are more experienced in terms of the number of contributed reviews and their reviews are significantly negative than reviewers with confirmed transaction. Pinch and Kesler (2011) reported that some Amazon customers, who are generally stakeholders of a certain product, have contributed numerous reviews to give extremely positive for their own but extremely negative rating for their rivals. In contrast, some other studies reported that experienced reviewers tended to be less extreme in their rating propensity than novice reviewers (Wang, 2010). A recent study of Banerjee and Chua (2018) investigated how online reviews of members in the IMDb online community change as they have more experience. They reported that reviewers in the novice stage, who have contributed less or equal to ten reviews, tend to give higher ratings to movies than the reviewers contributing reviews more than ten. Similarly, Costa et al. (2019) revealed that experienced reviewers, which are defined as incentivized reviewers in the study, tend to use more positive sentiments in their text reviews. Therefore, it is still inconclusive whether experienced reviewers are more negative or positive in their reviews than novice reviewers.
Summary of literature review
Although prior studies examined experienced reviewers, they primarily centered on whether they produce helpful reviews in ecommerce review platforms. There are some empirical studies on behavioral characteristics experienced reviewers in the online communities but they have the following limitations that this study attempts to address. First, the findings on the relationship between review experience and review propensity are mixed. Some studies revealed the experienced reviewers are more likely to be negative (Anderson & Simester, 2014; Banerjee & Chua, 2018) and extreme (Pinch & Kesler, 2011), while other reported that they are less likely to be negative (Costa et al., 2019) and extreme (Wang, 2010). Second, the empirical comparison between experienced reviewers and novice reviewers was not adequate in some studies, which can be one of the reasons for the inconclusive findings. For example, Banerjee and Chua (2018) used selective samples to compare the average ratings of different reviewer groups, adopting a simple t-test without including other influential factors. They defined online reviewers who submitted from one to ten reviews as at novice stage, from 45 to 54 at intermediate stage, and from 91 to 100 at expert stage, omitting reviewers who submitted the number of reviews between the stages (e.g., reviewers who submitted the number of reviews from 11 to 44 and from 55 to 90) and reviewers who submitted more than 100 reviews, who should have made substantial contributions to the review community. Third, the empirical studies simply assumed that the experienced reviewers evaluated the same products as the novice group and compared average ratings between two groups. However, the comparison is invalid if they had rated different products. For instance, if experienced reviewers mainly rated products in niche markets but novice reviewers did so popular products (Park & Yoo, 2018), the different rating propensity might have caused by the product difference, not by their rating propensity. Lastly, the empirical studies omitted important factors related to rating scores, such as age and gender of reviewers (Leung & Yang, 2020; Mather et al., 2004), product brand newness (Blythe, 1999; Choi et al., 2018), perceived quality of product (i.e., prior rating) (Lee et al., 2015), market popularity of product (Wang, 2010). Without considering these factors, the direct comparison between the two groups should lead to misunderstanding on the experienced reviewers.
Conceptual framework
This research proposes that experienced reviewers in online review communities are differentiated from novice reviewers by what they review, how they review, and why they review. Similar to professional critics, experienced reviewers are proposed to hold products to a higher standard and rate as such, induced by their expertise and knowledge. In terms of review topic selection, the experienced and novice groups may have a clear difference in diverse product domains such as movie (Sedgwick & Pokorny, 2014), music (Elvers et al., 2015), and video game (Santos et al., 2019). For example, while experienced reviewers prefer sophisticated products corresponding to their intellectual tastes, novice reviewers prefer products with market popularity, which may impose less risk in their decision (Sedgwick & Pokorny, 2014). Therefore, we empirically examine how experienced reviewers behave compared to novice reviewers, in terms of how they rate (e.g., average rating and extremity) and what they rate (e.g., sophisticated products vs. popular products) in the context of an online movie review community. Finally, we infer their motivations via how and what they review, as prior studies did. In addition, according to political science literature, where a certain minority group, particularly involving gender, was stressed to make a substantial difference (Dahlerup, 2006), female leader group has shown a different voting behavior. This implies that female experienced reviewers would be different in their review behavior (e.g., rating propensity). Therefore, we propose that gender differences affect the relationship between experience and rating propensity, which has not been discussed in the extant literature of experienced reviewers.
Rating propensity
In online review communities, one of the most popularly used rating methods is the star rating measurement, where reviewers can choose scores from one to five to evaluate products or services (Mudambi & Schuff, 2010). However, rating systems vary by platform. For example, Fandango uses the traditional five star single dimensional system with user text comment adjunct (Fandango, 2018), while Rotten Tomatoes, Metacritic, and IMDb have two rating systems for different groups or purposes. Yahoo! Movies, where the dataset for this study is from, uses a single dimensional rating scale similar to academic grading in the U.S. taking on the form of five letters with ± variation. Due to their unique characteristics, experienced reviewers are expected to illustrate different behavioral patterns in terms of rating scores and extremity in the ratings.
Rating propensity for rating score
Individuals can extend their understanding and knowledge on a certain area through contributing content in an online community with reviews (Lampel & Bhalla, 2007). Therefore, experienced reviewers are expected to have more knowledge or expertise about their topics. Major online review communities consider the number of reviews as a measure for the expertise of reviewers. For instance, Amazon.com identifies top reviewers based on the number of review entries and their overall helpfulness rated by the readers of their reviews (Chen et al., 2008). Extant academic literature also employed the number as one of the primary variables to estimate the expertise of online reviewers (Jennings et al., 2015; Liu et al., 2008), suggesting a positive relationship between the number of reviews submitted by the reviewers and their expertise.
The level of expertise of online reviewers are expected to influence their rating propensity, as reviewers with a higher level of expertise are typically more fastidious in their evaluation. Marketing literature concerning customer knowledge and market demandingness commonly indicates that as consumers are more informed and knowledgeable about products or services, they have more expectation and demand (House et al., 2005; Prahalad & Ramaswamy, 2004). In the movie industry, for example, frequent moviegoers tend to give lower ratings than infrequent moviegoers do due to their experience and knowledge, which introduces more demands on the movies (Chakravarty et al., 2010). Similarly, movie critics with expertise tend to give lower ratings than non-critics (Plucker et al., 2009). In summary, since experienced reviewers have more expertise, they would be likely to give a lower rating, introducing Hypothesis 1;
H1 Experienced reviewers are more likely to select a lower rating.
Rating propensity for review extremity
Extreme ratings, respectively representing a very satisfactory and a very dissatisfactory opinion, have been empirically examined and found to be positively or negatively related to sales growth. Therefore, the phenomenon is important to understand behavior of experienced reviewer behavior (Clemons et al., 2006). Extremity in rating propensity would distinguish experienced reviewers from novice reviewers. While one-time reviewers who write only one review at an online review community tend to be extreme because their motivation is to reward or punish the sellers of products or services (Feng et al., 2012), the experienced reviewers would be less likely to do so due to their different motivations considering social affiliation and altruism (Peddibhotla & Subramani, 2007). Therefore, they would be less likely to share extreme reviews because consumers perceive them as less credible and helpful than moderate ratings (Hunt & Smith, 1987; Mudambi & Schuff, 2010; Schmidt & Eisend, 2015). The extremity also does not correspond to self-oriented motivations such as social image (Wang, 2010) and self-expression (Peddibhotla & Subramani, 2007). Because they share personal thoughts in the online community to establish themselves as intelligent, fair, and good individuals, they would be less likely to choose drastic ratings, which may harm the desired social images. In addition, more knowledgeable consumers tend to be less extreme in their evaluation because the expertise of the consumers on products allows them to evaluate more diverse dimensions from an objective viewpoint (Sanbonmatsu et al., 1992; Sujan, 1985). Therefore, the following hypotheses are proposed:
H2a Experienced reviewers are less likely to select an extremely positive rating.
H2b Experienced reviewers are less likely to select an extremely negative rating.
Effect of gender on the relationship between experience and rating propensity
Gender difference has been popularly discussed in the extant online review studies. The studies commonly reported that female consumers are more influenced by online reviews (Abubakar et al., 2016; Awad & Ragowsky, 2008; Bae & Lee, 2011), negative reviews (Bae & Lee, 2011; Sotiriadis & Van Zyl, 2013), and inconsistent reviews (Zhang et al., 2014). As online review contributors, females tend to post more online reviews than males (Dunivin et al., 2020; Punj, 2013). However, there is little research to investigate how gender affects rating propensity of online reviewers, in particular female experienced reviewers.
In studies involving the relationship between gender and the level of generosity in behaviors, females were found to be more generous than males when their generosity caused little cost (Cox & Deck, 2006) or risk was low in their response (Eckel & Grossman, 2008). One explanation for this disparity is concerned with social expectation. Rand et al. (2016) showed that females have more altruistic tendencies and receive more negative feedback from society than males when they failed to be generous. In a regular social life, similarly, Mehl and Pennebaker (2003) found that females use significantly less negative words in their daily language, implying that females tend to be less aggressive but more generous. In the online review communities, further, Smith and Mangold (2012) reported that female consumers are less likely to write negative reviews on the internet than male consumers when they are disappointed with their purchase.
In the online review, rating propensity of experienced reviewers should differ by gender generosity of the reviewers. Although they have the same level of disappointment with a review subject such as products and services, more generous reviewers would select a higher rating score. As discussed, experienced reviewers should have higher expectation and demands concerning quality of their review subjects, leading to a negative relationship between experience and rating scores. However, based on the studies showing results of females’ generous behavior as well as more constant altruism (Baez et al., 2017; Mehl & Pennebaker, 2003; Rand et al., 2016), females are expected to exhibit more generous evaluations in their online reviews. The negative relationship between experience and the propensity to select a lower rating (Hypothesis 1) should differ by gender and is expected to be weaker for female reviewers. Concerning the extreme ratings, likewise, female reviewers should be more likely to choose the extremely positive but less likely the extremely negative due to the aforementioned attributes. Given the discussion above, the following hypotheses are introduced:
H3a Experienced female reviewers have a weaker tendency to select lower ratings than experienced male reviewers.
H3b Experienced female reviewers have a stronger tendency to select the extremely positive rating than experienced male reviewers.
H3c Experienced female reviewers have a weaker tendency to select the extremely negative rating than experienced male reviewers.
Product selection for review: product quality and market popularity
Experts and regular consumers often have different opinions on products (or services). Sometimes, products with hard criticism from industry experts make a huge market success (e.g., iPhone, Harry Porter, FedEx), implying different perspectives on product quality between experts and consumers. For movies, general moviegoers and professional movie critics tend to have different perspectives on artistic quality (Boor, 1992; Chakravarty et al., 2010). While general moviegoers often overlook the quality and consider the abstruse or boring, the critics tend to more pay attention to the quality that meets their intellectual tastes (Chakravarty et al., 2010; Lundy, 2010). In the video game industry, expert reviewers and amateur reviewers show differences in their evaluation of video games in terms of game genre, developer, and console platform. For example, the armatures tend to rate significantly higher for video games produced by Nintendo, which is more likely to develop friendly video games appealing a broad range of consumer groups, while experts do so for those produced by hard-core game developers, such as Blizzard and BioWare (Santos et al., 2019). Similarly, expert listeners of music tend to prefer more highbrow and sophisticated music, such as jazz and classical, while conventional listeners are more likely to enjoy folk, hard rock, house, and pop (Elvers et al., 2015). These differences indicate that industry experts prefer sophisticated products, which have more delicate, processed, and elaborated attributes (Herédia-Colaço & do Vale, 2018), while regular consumers may overlook the attributes in their product evaluation.
In the online review communities, experienced reviewers are expected to have a similar taste to that of industry experts, focusing on sophistication of products, in that both have strong intellectual motivations and high expectation. For instance, the experienced enjoy displaying their expertise and knowledge (Pinch & Kesler, 2011) as well as developing writing skills and enhancing understanding of topic (Peddibhotla & Subramani, 2007) by organizing and sharpening their ideas in the reviews. In term of social image, they want to establish themselves as intelligent and knowledgeable in their communities (Mathwick & Mosteller, 2017; Wang, 2010), which is an extension of their intellectual motivation. Therefore, the experienced reviewers are more likely to choose sophisticated products for their reviews, introducing the following hypothesis:
H4 Experienced reviewers are more likely to review products with sophistication.
According to Debenedetti and Larceneux (2011), professional movie critics tend not to highly evaluate movies with market success that ordinary moviegoers prefer. The critics may believe that such movies focus more on popular tastes but not on artistic quality, which is important in their evaluation. Likewise, fashion experts do not prefer mass produced items (Park & Yoo, 2018), because such items do not satisfy their unique taste or do not allow to present their expertise in the reviews. Given the aforementioned similarities to the critics, experienced reviewers would have the elite taste (Chakravarty et al., 2010), leading them to reject common products for their reviews. In terms of their desired social image, experienced reviewers would refuse to review popular products because they could not differentiate themselves as experts in their communities (Wang, 2010) as such products would provide less opportunity to project their expertise. Therefore, the following hypothesis is proposed:
H5 Experienced reviewers are less likely to review products with market popularity.
Figure 1 below summarizes the hypotheses proposed;
Empirical analysis
Data for analysis
This research adopts a dataset, titled “Yahoo! Movies User Ratings and Descriptive Content Information”, which is available at Yahoo! Research (https://webscope.sandbox.yahoo.com/catalog.php?datatype=r), a subsidiary research organization of Yahoo!. The organization provides a variety of datasets to support research of non-commercial users, such as academic scholars and scientists, after examining their personal information and research purposes. More details about the data request process are available at the Webcope website of Yahoo! Research (webscope.sandbox.yahoo.com). The dataset has been adopted in many studies, including analytic modeling research of Ebesu and Fang (2017).
The initial data provided by Yahoo! Research had six datasets about users’ ratings and their demographics and movie information (e.g., movie title/ ID/ year, GNPP, MPAA, awards nominated, etc.). We merged the datasets to prepare a dataset for this study. After handling missing values and errors, we had 211,197 movie reviews of 7642 users in the dataset. The top experienced reviewer contributed 1632 movie reviews to the community while the lowest level completed ten reviews. We sorted the 211,197 reviews and estimated average values of each variable by users to examine the proposed hypotheses. The average review number of each user is 27.636 and approximately top 23% of the reviewers contributed reviews more than average to the movie community. In terms of the number of reviews of an individual reviewer, the dataset illustrates the presence of experienced reviewers at Yahoo! Movies, suggesting that a small number of reviewers contributed a substantially large portion of reviews. As shown in Table 2, the top 5% contributed 29.5% of all reviews while the top 20% completed 53.7%, more than half of the reviews.
Figure 2 illustrates the overall curve fitting the distribution of the reviews for the top 2000 reviewers. It suggests that the growth of the contribution slows as the size of the group increases, indicating the large number of reviewers contributed considerably smaller amount of reviews to Yahoo! Movie than a few top reviewers did.
Construct operationalization
Using the variables in the dataset, we operationalized major constructs of this study. Table 3 summarizes the definitions of these variables. To measure experience of reviewers, we used the number of total reviews each reviewer contributed to Yahoo! Movie, operationalized as ReviewNumber. Although there are additional factors to describe experienced reviewers in online communities (e.g., quality of reviews), the number has been used as a key determinant for the group in the extant empirical studies on experienced reviewers (Banerjee & Chua, 2018; Buchanan et al., 2014; Nguyen et al., 2020; Peddibhotla & Subramani, 2007; Subramani & Peddibhotla, 2003; Wang, 2010). For example, Wang (2010) estimated experience of reviewers solely based on the number of reviews submitted to Yelp, Citysearch, and Yahoo! Local. Similarly, Peddibhotla and Subramani (2007) categorized reviewer groups by their experience given the number of reviews contributed to Amazon.com.
Rating score is operationalized as Rating. In Yahoo! Movie, they range from A + to F, which have 13 different rating scores. Thus, A + is converted to 13 while F is 1, which is the same conversion scheme as that of many prior studies on online reviews (Dellarocas et al., 2004; Duan et al., 2008; Simmons et al., 2011). Extremely positive rating is operationalized as ExtPositive, which stands for the most positive rating, 13 among the scores. We coded ExtPositive as 1 if the rating is 13, 0 otherwise. Whereas, we operationalized extremely negative rating is as ExtNegative, which refers to 1 in the score. If it is the lowest rating (i.e., 1), it is coded as 1 and 0 otherwise. Concerning constructs for product characteristics, we operationalized product sophistication as AwardsNominated, using the number of award nominations of the movies rated by a reviewer, which is known as a reasonable proxy for product (or service) sophistication (Bonner et al., 2003; Goldberg & Vashevko, 2013; Seng & Geertsema, 2018). Finally, market popularity is operationalized as GNPP, using Global Non-Personalized popularity (GNPP), which is a market popularity index used at Yahoo! Movies. Testing the interaction effect of female reviewers in the proposed hypotheses, we created a variable, Female. If a reviewer is female, it is coded as 1 and 0 otherwise (i.e., male).
Data analysis
The following five empirical models were constructed to test the proposed hypotheses. Each model examines the difference in terms of online review behaviors, including rating propensity and product selection for their review, given the set of control variables that affect the dependents. The controls include BirthYear, MovieYear, MPAA, AverageRating, and RaterNumber, which were barely considered in the extant literature of experienced reviewers. BirthYear is the year of birth of a reviewer. This is an inverse measure of demographic age of a reviewer. MovieYear indicates the release year of a movie, which is an inverse measure of newness of movies. As many prior studies suggested, newness is a key indicator of perceived product quality affecting online review valence and sales of the product (Choi et al., 2018). MPAA is Motion Picture Association of America (MPAA) ratings, indicating the suitability of a movie for certain age groups. As a movie has a high level of rating in the system, it generally has more inappropriate contents for young audience, which should influence rating scores for the movie. AverageRating means the average rating score of movies while RaterNumber does the total number of reviews for a movie. Each represents overall evaluation and interest of the entire community in a movie, potentially affecting review valence of each reviewer (Wang, 2010).
Model 1 testing H1 and H3a
Model 2 testing H2a and H3b
Model 3 testing H2b and H3c
Model 4 testing H4
Model 5 testing H5
Where i represents reviewer and k represents movie.
Model 1 tests Hypothesis 1 and 3a. It includes Rating as the dependent variable, representing the rating score of a reviewer. ReviewNumber is an independent variable to represent the total number of reviews contributed by a reviewer. It indicates the experience of the reviewer, testing Hypothesis 1, which is concerned with the relationship between the experience and the rating propensity of reviewers. The other independent variable in the model is ReviewNumber*Female, which is the interaction term of ReviewNumber and Female. It examines Hypotheses 3a, testing how the relationship between the experience and the rating differ by gender.
In Model 2 testing Hypothesis 2a and 3b, the dependent is ExPositive while the independents are ReviewNumber and ReviewNumber*Female. Each independent respectively examines the relationship between the experience and the tendency to give extremely positive rating and gender difference affect the relationship. Model 3 has the same independent variables as those of Model 2 but its dependent is ExtNegative. It tests whether the experience has to do with the propensity to select the extremely negative review valence and the interaction effect of gender, which are concerning Hypothesis 2b and 3c. Testing Hypothesis 4, Model 4 adopts AwardsNominated, which represents product sophistication, as its dependent. The model tests how a reviewer’s experience is related to the literary quality of movies selected for reviews. Whereas, Model 5 tests Hypothesis 5 concerning how the experience is associated with the market popularity of the movies.
Table 4 summarizes the descriptive statistics of the variables discussed and Table 5 illustrates their correlations. Overall, there is no significant correlation among the variables although GNPP and AwardsNominated have a relatively strong correlation (c.f., VIF’s of the models range between 1.20 and 1.79, indicating no significant multicollinearity). Since the dataset used in this study is unbalanced panel data without a time variable, which an individual reviewer has multiple review records on different movies, the models above are likely to have the violation of OLS (Ordinary Least Square) assumptions. Accordingly, we conducted a Breusch-Pagan test for the models (i.e., Model 1, 3, and 4) and found heteroscedasticity in the models. This suggests that OLS is not a reliable estimator for the models and an alternative estimator is necessary, such as a random effects model with Huber/White standard error correction. This is known as a more rigorous estimator than a random effects model to address heteroscedasticity in a large dataset when the model has heteroscedastic problem (Hayes & Cai, 2007). For Model 2 and 3 that have binary dependent variables (i.e., ExtPositive and ExtNegative), we adopted random effects logistic regression. This is a more rigorous estimator than logistic regression in panel data analysis, allowing to control for time invariant characteristics (Williams, 2009, 2018).
Estimation results
As aforementioned, we performed a random effects regression with Huber/White standard error correction for Model 1, 4, and 5 and a random effects logistic regression for Model 2 and 3. Due to the nature of the dependent variable of Model 1, which is bounded in the range between the two extreme ratings, we also performed random effect Tobit regression to check robustness of the results (Mudambi & Schuff, 2010). The hypothesis test results remain constant with those from a random effects regression with Huber/White standard error correction. Table 6 below summarizes major analysis results, including coefficients and P values of the variables.
In Model 1, the coefficient of ReviewNumberi (α1) is significant (P < 0.01) and negative. This indicates that as reviewers are more experienced, they tend to leave lower rating scores, supporting Hypothesis 1. Coefficient of the interaction term, Female (α2) is statistically significant (P < 0.01) and positive while ReviewNumber*Female (α3) is significant (P < 0.01) and negative. This suggests that although female reviewers tend to give a higher rating, the negative relationship between experience and rating scores is stronger in the female group, which is significant but in the opposite direction for Hypothesis 3a.
Model 2 tested the relationship between experience of online reviewers and a propensity for selecting the extremely positive score (i.e., A +) in their ratings. The coefficient of ReviewNumberi (β1) is significant (P < 0.01) and negative. This indicates that as reviewers are more experienced, they are less likely to choose the extremely positive score, corresponding to the prediction of Hypothesis 2a. Concerning Hypothesis 3b, both Femalei (β2) and ReviewNumberi*Femalei (β3) are statistically significant (P < 0.01) and negative. This implies that although the females are more likely to give the extremely positive score, the negative relationship between the experience and the propensity to choose the extremely positive is stronger in the female group, significant but opposite to the prediction of Hypothesis 3b.
Model 3 examined how the experience is related to the probability to choose the highly negative score (i.e., F) in their online review. The coefficient of ReviewNumberi (γ1) is significant (P < 0.01) and negative, illustrating that as more experienced online reviewers are less likely to select the extremely negative score, supporting Hypothesis 2b. Testing the interaction effect of gender (i.e., female) on the relationship between experience and the probability to select the extremely negative, the coefficient of ReviewNumberi*Femalei (γ3) is statistically not significant although Femalei (γ2) is significant (P < 0.01) and negative. This suggests that overall female reviewers are less likely to give the extremely negative score but this propensity is not unique enough to female experienced reviewers from male experienced reviewers, not supporting Hypothesis 3c.
The Coefficient of ReviewNumberi (δ1) in Model 4, which has AwardsNominatedk as its dependent, is significant (P < 0.01) and positive. It suggests that as more experienced, online reviewers tend to rate products with sophistication, supporting Hypothesis 4. Given that the coefficient of ReviewNumberi (ε1) in Model 5 is significant (P < 0.01) and negative, they are less likely to participate in reviewing products with higher market popularity, which is operationalized as GNPPk. Therefore, Hypothesis 5 is supported. Tables 7 and 8 below summarizes the hypothesis test results.
Additional analysis
In order to find more details about the test results, we conducted an additional analysis, adopting MANOVA (Multivariate Analysis of Variance) to examine the differences in the multiple dependent variables, including AvRating, ExtPositive, ExtNegative, AwardsNominated, and GNPP. Because the sample sizes of the experienced and novice reviewer groups are unequal, we conducted Box’s M test to check equality of covariance across the two groups. Not surprisingly, we found unequal covariance in the outcome variables, which is common to a large dataset. Therefore, we used Pillai’s trace, instead of Wilk’s lambda, to check the statistical significance of differences between groups (Tabachnick et al., 2001). We performed the analysis with different definitions on experienced reviewers, such as 15%, 20%, and 25% of top reviewers in terms of the number of reviews submitted to the community. As summarized in Table 9, the differences in the variables are statistically significant (p < 0.01) and correspond to the hypothesis test results for Hypotheses 1, 2a, 2b, 4, and 5. One of the most notable differences between the two groups is in AwardsNominated. The average number of award nominations of the movies selected by experienced reviewers is approximately twice larger than the movies selected by novice reviewers, indicating a clear taste difference between the two groups.
The gender differences in AvRating, ExtPositive, and ExtNegative were estimated, adopting the same grouping scheme for the previous MANOVA. Overall, female experienced reviewers tend to be stricter than male experienced reviewer groups (Table 9). The average ratings and the ratio of extremely positive rating of female experienced reviewers are consistently lower in the 15%, 20%, and 25% top reviewer groups. The ratio of extremely negative rating is higher than male reviewers in the same group and this difference is statistically significant (P < 0.01). This result is consistent with the overall findings concerning rating propensity of female experienced reviewers, which is less generous than male experienced reviewers, although it is different from the regression result for Hypothesis 3c.
Discussion
This research examines experienced reviewers in online review communities, who potentially impact product reputation and ultimately market performance as a small percentage of reviewers conduct more reviews than the rest of the reviewers. In terms of rating propensity, support for Hypothesis 1 indicates that the experienced tend to leave lower scores in the rating systems. This result corresponds to marketing literature illustrating a positive relationship between knowledge and market demandingness of consumers (House et al., 2005; Prahalad & Ramaswamy, 2004; Wikström, 1996) and therefore, it is more difficult to satisfy such demanding consumers. Since the experienced have higher expectations and demands on the movies attributed from their knowledge and expertise, they tend to choose lower ratings in their evaluation. Support for Hypothesis 2a and 2b suggests that the experienced tend to have less extremity in their review valence. Although this finding is contradictory to the literature that argued that experienced reviewers should have a strategic purpose to manipulate product reputation and sales (i.e., fake reviewers) (Luca & Zervas, 2016; Pinch & Kesler, 2011), this finding corresponds to the extant literature concluding that they tend to have motivations to help other people, including social affiliation and altruism, based on their neutral rating propensity. As such, the experienced reviewers should want to position themselves as personable individuals who are good, intelligent, and fair in the communities (Wang, 2010), and they would be less likely to choose extreme ratings, which contradicting the desired image. In addition, their altruistic motivations would promote them to choose a neutral rating because the community members generally perceive extreme ratings as less helpful in the communities (Hunt & Smith, 1987; Mudambi & Schuff, 2010; Schmidt & Eisend, 2015). This propensity also corresponds to “regression to the mean” of online reviews, suggesting that as a product has more reviews, its rating moves toward the neutral rating neither extremely positive nor extremely negative (Cloney et al., 2018; DellaPosta & Kim, 2016). As experienced reviewers contribute more reviews, the ratio of extreme ratings should decrease in their reviews.
The surprising findings regarding Hypothesis 3a, 3b, and 3c show not only that gender differences exist in online reviews, but also that there are gender differences in the relationship between experience and rating propensity, which are opposite to the extant literature suggesting higher generosity of females. Although it was found that females tend to give higher and more positive extreme rating scores corresponding to the extant studies reporting that females are more generous than the males in their evaluation (Cox & Deck, 2006; Eckel & Grossman, 2008; Smith & Mangold, 2012), the interaction of gender on the relationship between experience and rating propensity was both surprising and profound. Our findings showed that as female reviewers are more experienced, the negative relationship between experience and negative rating propensity actually becomes more substantial. An explanation for this finding could be related to gender, power, and online communities (Hemphill & Otterbacher, 2012). It is possible that power asymmetry in the online world affects the female experienced reviewers such that as they want their voices heard and adjust style in order to be better received (Hemphill & Otterbacher, 2012), task competence becomes a focus of the female experienced conveyed through objective success as an influence agent (Carli, 2001) outweighs the findings of the female sample as a whole and is significant enough to differ from the male experienced. Concerning the tendency to choose the extremely negative rating, females are less likely to do so than males, which corresponds to the generosity literature; however, gender (i.e., female) is not found to have a significant interaction effect on the relationship between experience and the tendency. This indicates that the negative effect of experience on the tendency does not differ by genders. This can be explained by literature on gender differences in online reviewers where reviews of well-known female members were shown to exhibit male characteristics (Hemphill & Otterbacher, 2012; Otterbacher, 2012). Support for Hypothesis 4 and 5 presents preferred products of experienced reviewers for reviews. They prefer products with sophistication in which they can demonstrate their expertise (Pinch & Kesler, 2011) and differentiate themselves from the novice who would not be interested in such products (Mathwick & Mosteller, 2017; Wang, 2010). For the same reasons, they are less likely select products with market popularity because reviewing popular products is less likely to satisfy the aforementioned motivations.
Theoretical contributions
This research enriches the literature of online reviewers, providing the following theoretical contributions. First, this study provides empirical evidence that the small number of the reviewers contribute most reviews to the community and they tend to have altruistic motivations, rather than strategic motivations given their rating propensity. Second, although some extant literature discussed the rating propensity of the experienced (Dellarocas & Narayan, 2006; Wang, 2010), few addressed the subject selection pattern. This study adds a novel dimension to the literature in terms of the pattern, illustrating that experienced reviewers prefer reviewing sophisticated products while they do not those with market popularity, due to the tendency which would have been induced by their expertise, and desired social image (Mathwick & Mosteller, 2017; Wang, 2010). In particular, this difference is found to be highly substantial in that the products reviewed by experienced reviewers have approximately twice more product sophistication than those by novice reviewers. Third, the finding of this study confirms the positive relationship between expertise and demandingness on products or services (House et al., 2005; Prahalad & Ramaswamy, 2004) by presenting that experienced reviewers who should be more knowledgeable tend to give lower ratings. Fourth, this study extends extant literature on experienced reviewers, adding knowledge about gender difference in the rating propensity. This was one of the first attempts to investigate how gender affects the propensity of experienced online reviewers, specifically showing a profound gender difference in rating propensity when a female reviewer happens to be experienced. It illustrates that the female experienced reviewers have less generous than the regular female members and their generosity decreases more dramatically than males as they are more experienced in terms of the amount of contributions. Lastly, this study provides a more comprehensive model for experienced reviewer research, including additional influential factors to determine online reviews, such as demographics of reviewers, newness of products, and average rating and market popularity of products, which were rarely considered in the extant literature but were found to have significant relationships with review behaviors in this study.
Managerial contributions
The findings of this study offer several managerial implications to practitioners. First, experienced reviewers in the online communities produce a substantial number of reviews. For example, 20% of the most experienced reviewers are found to contribute more than 50% of the entire reviews in Yahoo! Movies. This finding corresponds to the extant literature on Amazon.com (Peddibhotla & Subramani, 2007), CitySearch, Yahoo! Local, and Yelp (Wang, 2010), confirming the substantial impact of the experienced reviewers in the online review platforms. Knowing the influence of the experienced, online platform managers can take steps to increase the richness of the reviews; for example considering multidimensional rating systems (Chen et al., 2017). Some platforms (i.e., Rotten Tomatoes and Metacritic) have already acknowledged experienced reviewers and specified qualifications to meet these boundaries. Product manufacturers could consider partnerships with online platform managers to shape multidimensional rating criteria or provide system input; IMDb has initiated an IMDbPro feature for enhanced communication and collaboration between the industry and experienced reviewers. Second, this study verifies that the experienced in online communities would not have a strategic goal given their rating propensity and product selection pattern for review, even though the communities have a favorable environment to fake reviewers with strategic goals (Luca & Zervas, 2016). For example, if they had a strategic goal, they must have rated with extremely positive or negative ratings to manipulate the overall rating as they intended. However, the findings of this study suggest that they have a more neutral rating propensity. This may lessen the concerns of buying public and practitioners who have serious doubt about the reviews of the experienced (Owen, 2011) in particular, those in online movie communities (Wilkinson, 2019) Third, the findings of this study indicate that female reviewers are more generous than the males in rating propensity. However, the generosity decreases more dramatically as their experience increases and female experienced reviewers tend to be stricter in their rating propensity than male experienced reviewers. Therefore, practitioners may consider the difference in understanding their reviews or working with female experienced reviewers (e.g., influencers), being aware of the tendency opposite to the previous theory that females are more generous. Fourth, this study provides evidence that experienced reviewers are similar to professional critics in their rating propensity and product selection pattern. They tend to be stricter but to avoid extreme ratings, both extremely positive and negative, and prefer sophisticated products but do not popular products. These findings suggest that practitioners may use reviews of the experienced to understand the viewpoint of professional critics, separated from novice consumers. Understanding these trends can help online platform managers make business decisions regarding their rating systems; for example, Netflix replaced its five star rating system with a parsimonious ‘thumbs up, thumbs down’ system (McAlone, 2017) with the assumption that professional critic should not select extreme ratings and that general consumers are more likely to use the extreme ratings (i.e., thumbs up, thumbs down). This approach may hinder the participation of experienced reviewers, who play a major role in developing and maintaining the online review ecosystem with a large quantity of reviews as well as superior quality of knowledge about products, but should enable to collect pure ideas of novice consumers in the market.
Conclusion
This research has several limitations that future researchers may consider in designing their research on experienced online reviewers. First, this research was not able to consider time effect in the analysis due to lack of time information of each record in the dataset although timing of reviews must have an impact on overall ratings. For instance, when the difference between movie release date and review date is large (i.e., when reviewers rate old movies), the rating is more likely to be positive because consumers watch old movies when they have confidence that the movies certainly will be satisfactory after a careful selection process (Koren, 2010). As Lee et al. (2015) reported, in addition, it is important to consider the effect of previous ratings of others, which are influential to the following ratings. Although our empirical analysis included average rating of each movie to control such an effect, it was not possible to consider the ratings by time because of lack of time information in the data. The lack of time factor also does not allow considering the number of reviews created in a certain period time. For example, although some reviewers were experienced but not categorized as the experienced because they began to write the reviews later than other (and vice versa). Future research can include the time factor in their analysis to enhance the robustness of their findings. Second, this research did not include text reviews in the analysis, which can disclose additional differences of experienced reviewers from the novice. For example, the experienced could use less emotional terms but more objective information, such as market data and references, in building their ideas. Future research may find more behavioral differences between the experienced and the novice reviewers by analyzing text review data.
References
Aakash, A., & Jaiswal, A. (2020). Segmentation and ranking of online reviewer community: the role of reviewers’ frequency, helpfulness, and recency. International Journal of E-Adoption (IJEA), 12(1), 63–83. https://doi.org/10.4018/IJEA.2020010106
Abubakar, A. M., Ilkan, M., & Sahin, P. (2016). eWOM, eReferral and gender in the virtual community. Marketing Intelligence & Planning, 34(5), 692–710. https://doi.org/10.1108/MIP-05-2015-0090
Anderson, E., & Simester, D. (2014). Reviews without a purchase: low ratings, loyal customers, and deception. Journal of Marketing Research (JMR), 51(3), 249–269. https://doi.org/10.1509/jmr.13.0209
Awad, N. F., & Ragowsky, A. (2008). Establishing trust in electronic commerce through online word of mouth: An examination across genders. Journal of Management Information Systems, 24(4), 101–121. https://doi.org/10.2753/MIS0742-1222240404
Bae, S., & Lee, T. (2011). Gender differences in consumers’ perception of online consumer reviews. Electronic Commerce Research, 11(2), 201–214. https://doi.org/10.1007/s10660-010-9072-y
Baez, S., Flichtentrei, D., Prats, M., Mastandueno, R., García, A. M., Cetkovich, M., & Ibáñez, A. (2017). Men, women… who cares? A population-based study on sex differences and gender roles in empathy and moral cognition. PLoS ONE, 12(6), e0179336. https://doi.org/10.1371/journal.pone.0179336
Banerjee, S., Bhattacharyya, S., & Bose, I. (2017). Whose online reviews to trust? Understanding reviewer trustworthiness and its impact on business. Decision Support Systems, 96, 17–26. https://doi.org/10.1016/j.dss.2017.01.006
Banerjee, S., & Chua, A. Y. (2018). Tracing the growth of IMDb reviewers in terms of rating, readability and usefulness. 4th International Conference on Information Management (ICIM). https://doi.org/10.1109/INFOMAN.2018.8392809
Bhattacharyya, S., Banerjee, S., & Bose, I. (2020). One size does not fit all: Rethinking recognition system design for behaviorally heterogeneous online communities. Information & Management, 57(7), 103245. https://doi.org/10.1016/j.im.2019.103245
Blythe, J. (1999). Innovativeness and newness in high-tech consumer durables. Journal of Product & Brand Management, 8(5), 415–429. https://doi.org/10.1108/10610429910296028
Bonner, S. E., Walther, B. R., & Young, S. M. (2003). Sophistication-related differences in investors’ models of the relative accuracy of analysts’ forecast revisions. The Accounting Review, 78(3), 679–706. https://doi.org/10.2308/accr.2003.78.3.679
Boor, M. (1992). Relationships among ratings of motion pictures by viewers and six professional movie critics. Psychological Reports, 70(3 suppl), 1011–1021. https://doi.org/10.2466/pr0.1992.70.3c.1011
Brynjolfsson, E., Hu, Y., & Simester, D. (2011). Goodbye pareto principle, hello long tail: The effect of search costs on the concentration of product sales. Management Science, 57(8), 1373–1386. https://doi.org/10.1287/mnsc.1110.1371
Buchanan, K., Russo, R., & Anderson, B. (2014). Feeding back about eco-feedback: How do consumers use and respond to energy monitors? Energy Policy, 73, 138–146. https://doi.org/10.1016/j.enpol.2014.05.008
Carli, L. L. (2001). Gender and social influence. Journal of Social Issues, 57(4), 725–741. https://doi.org/10.1111/0022-4537.00238
Chakravarty, A., Liu, Y., & Mazumdar, T. (2010). The differential effects of online word-of-mouth and critics’ reviews on pre-release movie evaluation. Journal of Interactive Marketing, 24(3), 185–197. https://doi.org/10.1016/j.intmar.2010.04.001
Chen, P.-Y., Dhanasobhon, S., & Smith, M. D. (2008). All reviews are not created equal: The disaggregate impact of reviews and reviewers at amazon. com. SSRN. https://doi.org/10.2139/ssrn.918083
Chen, P.-Y., Hong, Y., & Liu, Y. (2017). The value of multidimensional rating systems: Evidence from a natural experiment and randomized experiments. Management Science. https://doi.org/10.1287/mnsc.2017.2852
Chintagunta, P. K., Gopinath, S., & Venkataraman, S. (2010). The effects of online user reviews on movie box office performance: Accounting for sequential rollout and aggregation across local markets. Marketing Science, 29(5), 944–957. https://doi.org/10.1287/mksc.1100.0572
Choi, H. S., & Leon, S. (2020). An empirical investigation of online review helpfulness: A big data perspective. Decision Support Systems, 139, 113403. https://doi.org/10.1016/j.dss.2020.113403
Choi, H. S., Ko, M. S., Medlin, D., & Chen, C. (2018). The effect of intrinsic and extrinsic quality cues of digital video games on sales: An empirical investigation. Decision Support Systems, 106, 86–96. https://doi.org/10.1016/j.dss.2017.12.005
Chow, L. (2013). Top Reviewers On Amazon Get Tons Of Free Stuff. NPR. http://www.npr.org/sections/money/2013/10/29/241372607/top-reviewers-on-amazon-get-tons-of-free-stuff
Chu, S.-C., & Kim, Y. (2011). Determinants of consumer engagement in electronic word-of-mouth (eWOM) in social networking sites. International Journal of Advertising, 30(1), 47–75. https://doi.org/10.2501/IJA-30-1-047-075
Clemons, E. K., Gao, G. G., & Hitt, L. M. (2006). When online reviews meet hyperdifferentiation: A study of the craft beer industry. Journal of Management Information Systems, 23(2), 149–171. https://doi.org/10.2753/MIS0742-1222230207
Cloney, M., Hopkins, B., Shlobin, N., & Dahdaleh, N. S. (2018). Online ratings of neurosurgeons: An examination of web data and its implications. Neurosurgery, 83(6), 1143–1152. https://doi.org/10.1093/neuros/nyy064
Costa, A., Guerreiro, J., Moro, S., & Henriques, R. (2019). Unfolding the characteristics of incentivized online reviews. Journal of Retailing and Consumer Services, 47, 272–281. https://doi.org/10.1016/j.jretconser.2018.12.006
Cox, J. C., & Deck, C. A. (2006). When are women more generous than men? Economic Inquiry, 44(4), 587–598. https://doi.org/10.1093/ei/cbj042
Cui, G., Lui, H.-K., & Guo, X. (2012). The effect of online consumer reviews on new product sales. International Journal of Electronic Commerce, 17(1), 39–58. https://doi.org/10.2753/JEC1086-4415170102
Dahlerup, D. (2006). The story of the theory of critical mass. Politics & Gender, 2(4), 511–522. https://doi.org/10.1017/S1743923X0624114X
Debenedetti, S., & Larceneux, F. (2011). “The taste of others”: Divergences in tastes between professional experts and ordinary consumers of movies in France. Recherche Et Applications En Marketing (english Edition), 26(4), 71–88. https://doi.org/10.1177/205157071102600404
DellaPosta, D., & Kim, M. (2016). The fickle crowd: Selective differentiation in cultural markets. Academy of Management Proceedings. https://doi.org/10.5465/ambpp.2016.257
Dellarocas, C., & Narayan, R. (2006). What motivates consumers to review a product online? A study of the product-specific antecedents of online movie reviews. WISE
Dellarocas, C., Awad, N., & Zhang, X. M. (2004). Using online reviews as a proxy of word-of-mouth for motion picture revenue forecasting. SSRN. https://doi.org/10.2139/ssrn.620821
Duan, W., Gu, B., & Whinston, A. B. (2008). Do online reviews matter?—An empirical investigation of panel data. Decision Support Systems, 45(4), 1007–1016. https://doi.org/10.1016/j.dss.2008.04.001
Dunivin, Z., Zadunayski, L., Baskota, U., Siek, K., & Mankoff, J. (2020). Gender, soft skills, and patient experience in online physician reviews: A large-scale text analysis. Journal of Medical Internet Research, 22(7), e14455. https://doi.org/10.2196/14455
Dvorak, J. C. (2011, June 29). Are Amazon Reviews Corrupt? PC Reviews. http://www.pcmag.com/article2/0,2817,2387749,00.asp
Ebesu, T., & Fang, Y. J. I. R. J. (2017). Neural semantic personalized ranking for item cold-start recommendation. 20(2), 109-131. https://doi.org/10.1007/s10791-017-9295-9
Eckel, C. C., & Grossman, P. J. (2008). Differences in the economic decisions of men and women: Experimental evidence. Handbook of Experimental Economics Results, 1, 509–519. https://doi.org/10.1016/S1574-0722(07)00057-1
Elvers, P., Omigie, D., Fuhrmann, W., & Fischinger, T. (2015). Exploring the musical taste of expert listeners: Musicology students reveal tendency toward omnivorous taste. Frontiers in Psychology, 6, 1252. https://doi.org/10.3389/fpsyg.2015.01252
Fandango. (2018). Movie Reviews. https://www.fandango.com/movie-reviews
Fang, B., Ye, Q., Kucukusta, D., & Law, R. (2016). Analysis of the perceived value of online tourism reviews: Influence of readability and reviewer characteristics. Tourism Management, 52, 498–506. https://doi.org/10.1016/j.tourman.2015.07.018
Feng, S., Xing, L., Gogar, A., & Choi, Y. (2012). Distributional footprints of deceptive product reviews. ICWSM, 12, 98–105.
Filieri, R., Hofacker, C. F., & Alguezaui, S. J. C. I. H. B. (2018). What makes information in online consumer reviews diagnostic over time? The role of review relevancy, factuality, currency, source credibility and ranking score. Computers in Human Behavior, 80, 122-131. https://doi.org/10.1016/j.chb.2017.10.039
Flanagin, A. J., Metzger, M. J., Pure, R., & Markov, A. (2011). User-generated ratings and the evaluation of credibility and product quality in ecommerce transactions. 44th Hawaii International Conference on System Sciences (HICSS). https://doi.org/10.1109/HICSS.2011.474
Freddie. (2018, February 8). Get The Insight – How Reading Online Reviews Differs By Age And Gender. Review Monitoring. https://reviewmonitoring.com/get-the-insight-into-how-shoppers-read-online-reviews-differ-by-age-and-gender/
Fullerton, L. (2017, March 27). Online reviews impact purchasing decisions for over 93% of consumers, report suggests. The Drum. https://www.thedrum.com/news/2017/03/27/online-reviews-impact-purchasing-decisions-over-93-consumers-report-suggests
Furner, C. P., & Zinko, R. A. (2017). The influence of information overload on the development of trust and purchase intention based on online product reviews in a mobile vs. web environment: an empirical investigation. Electronic Markets, 27(3), 211–224. https://doi.org/10.1007/s12525-016-0233-2
Goldberg, A., & Vashevko, T. (2013). Social boundedness as market identity evidence from the film industry. Stanford Graduate School of Business Stanford.
Hayes, A. F., & Cai, L. (2007). Using heteroskedasticity-consistent standard error estimators in OLS regression: An introduction and software implementation. Behavior Research Methods, 39(4), 709–722. https://doi.org/10.3758/BF03192961
Hemphill, L., & Otterbacher, J. (2012). Learning the lingo?: Gender, prestige and linguistic adaptation in review communities. Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work. https://doi.org/10.1145/2145204.2145254
Herédia-Colaço, V., & do Vale, R. C. (2018). Seize the day or save the world? The importance of ethical claims and product nature congruity. Journal of Business Ethics, 152(3), 783–801. https://doi.org/10.1007/s10551-016-3342-0
Hong, H., Xu, D., Wang, G. A., & Fan, W. (2017). Understanding the determinants of online review helpfulness: A meta-analytic investigation. Decision Support Systems, 102, 1–11. https://doi.org/10.1016/j.dss.2017.06.007
House, L., Lusk, J., Jaeger, S., Traill, W. B., Moore, M., Valli, C., Marrow, B., & Yee, W. M. (2005). Objective and subjective knowledge: Impacts on consumer demand for genetically modified foods in the United States and the European Union. AgBioForum, 7(3), 113-123.
Hunt, J. M., & Smith, M. F. (1987). The persuasive impact of two-sided selling appeals for an unknown brand name. Journal of the Academy of Marketing Science, 15(1), 11. https://doi.org/10.1007/BF02721949
IESE Business School. (2015, April 29). How To Make Online Reviews Work For You. Forbes. https://www.forbes.com/sites/iese/2015/04/29/how-to-make-online-reviews-work-for-you/#2d6ddaac3f98
Jennings, E., Arlikatti, S., & Andrew, S. (2015). Determinants of emergency management decision support software technology: An empirical analysis of social influence in technology adoption. Journal of Homeland Security and Emergency Management, 12(3), 603–626. https://doi.org/10.1515/jhsem-2014-0079
Koh, N. S., Hu, N., & Clemons, E. K. (2010). Do online reviews reflect a product’s true perceived quality? An investigation of online movie reviews across cultures. Electronic Commerce Research and Applications, 9(5), 374–385. https://doi.org/10.1016/j.elerap.2010.04.001
Koren, Y. (2010). Collaborative filtering with temporal dynamics. Communications of the ACM, 53(4), 89–97. https://doi.org/10.1145/1721654.1721677
Kunst, K., & Vatrapu, R. (2019). Understanding electronic word of behavior: Conceptualization of the observable digital traces of consumers’ behaviors. Electronic Markets, 29(3), 323–336. https://doi.org/10.1007/s12525-018-0301-x
Lafky, J. (2014). Why do people rate? Theory and evidence on online ratings. Games and Economic Behavior, 87, 554–570. https://doi.org/10.1016/j.geb.2014.02.008
Lampel, J., & Bhalla, A. (2007). The role of status seeking in online communities: Giving the gift of experience. Journal of Computer-Mediated Communication, 12(2), 434–455. https://doi.org/10.1111/j.1083-6101.2007.00332.x
Lee, Y., Hosanagar, K., & Tan, Y. (2015). Do I follow my friends or the crowd? Information cascades in online movie ratings. Management Science, 61(9), 2241–2258. https://doi.org/10.1287/mnsc.2014.2082
Lee, D. (2020). Amazon’s top UK reviewers appear to profit from fake 5-star posts. Ars Technica. https://arstechnica.com/gadgets/2020/09/amazons-top-uk-reviewers-appear-to-profit-from-fake-5-star-posts/
Leung, X. Y., & Yang, Y. (2020). Are all five points equal? Scaling heterogeneity in hotel online ratings. International Journal of Hospitality Management, 88, 102539. https://doi.org/10.1016/j.ijhm.2020.102539
Lin, C.-Y., Liaw, S.-Y., Chen, C.-C., Pai, M.-Y., & Chen, Y.-M. (2017). A computer-based approach for analyzing consumer demands in electronic word-of-mouth. Electronic Markets, 27(3), 225–242. https://doi.org/10.1007/s12525-017-0262-5
Lindbergh, B., & Arthur, R. (2020, September 4). Has Rotten Tomatoes Ever Truly Mattered? The Ringer. https://www.theringer.com/movies/2020/9/4/21422568/rotten-tomatoes-effective-on-box-office
Liu, Y., Huang, X., An, A., & Yu, X. (2008). Reviews are not equally important: Predicting the helpfulness of online reviews. 8th IEEE International Conference on Data Mining (ICDM), 443-452. https://doi.org/10.1109/ICDM.2008.94
Liu, Z., & Park, S. (2015). What makes a useful online review? Implication for travel product websites. Tourism Management, 47, 140–151. https://doi.org/10.1016/j.tourman.2014.09.020
Luca, M., & Zervas, G. (2016). Fake it till you make it: Reputation, competition, and Yelp review fraud. Management Science, 62(12), 3412–3427. https://doi.org/10.1287/mnsc.2015.2304
Luca, M. (2016). Reviews, reputation, and revenue: The case of Yelp.com. Harvard Business School NOM Unit Working Paper No. 12-016. SSRN. https://doi.org/10.2139/ssrn.1928601
Lundy, D. E. (2010). A test of consensus in aesthetic evaluation among professional critics of modern music. Empirical Studies of the Arts, 28(2), 243–258. https://doi.org/10.2190/EM.28.2.h
Ma, H., Kim, J. M., & Lee, E. (2019). Analyzing dynamic review manipulation and its impact on movie box office revenue. Electronic Commerce Research and Applications, 35, 100840. https://doi.org/10.1016/j.elerap.2019.100840
Mather, M., Canli, T., English, T., Whitfield, S., Wais, P., Ochsner, K., & Carstensen, L. L. (2004). Amygdala responses to emotionally valenced stimuli in older and younger adults. Psychological Science, 15(4), 259–263. https://doi.org/10.1111/j.0956-7976.2004.00662.x
Mathwick, C., & Mosteller, J. (2017). Online reviewer engagement: A typology based on reviewer motivations. Journal of Service Research, 20(2), 204–218. https://doi.org/10.1177/1094670516682088
McAlone, N. (2017, March 16). Netflix will kill its 5-star rating system in favor of 'thumbs-up, thumbs-down' — here's why. Business Insider. http://www.businessinsider.com/netflix-ditches-5-star-ratings-2017-3
McCabe, K. (2018, June 7). Consumer Reviews - 2018 B2B Sales & Marketing Report. Learn Hub. https://learn.g2.com/consumer-reviews
McIntyre, S. H., McQuarrie, E. F., & Shanmugam, R. (2016). How online reviews create social network value: The role of feedback versus individual motivation. Journal of Strategic Marketing, 24(3–4), 295–310. https://doi.org/10.1080/0965254X.2015.1095218
Mehl, M. R., & Pennebaker, J. W. (2003). The sounds of social life: A psychometric analysis of students’ daily social environments and natural conversations. Journal of Personality and Social Psychology, 84(4), 857. https://doi.org/10.1037/0022-3514.84.4.857
Mudambi, S. M., & Schuff, D. (2010). What makes a helpful online review? A study of customer reviews on Amazon.com. MIS Quarterly, 34(1), 185–200. https://doi.org/10.2307/20721420
Nguyen, J., Armisen, A., Sánchez-Hernández, G., Casabayó, M., & Agell, N. (2020). An OWA-based hierarchical clustering approach to understanding users’ lifestyles. Knowledge-Based Systems, 190, 105308. https://doi.org/10.1016/j.knosys.2019.105308
Otterbacher, J. (2012). Gender, writing and ranking in review forums: A case study of the IMDb. Knowledge and Information Systems, 35(3), 645–664. https://doi.org/10.1007/s10115-012-0548-z
Owen, L. (2011, June 24). What Shoppers Don’t Realize About Amazon’s Reviews. Gigaom. https://gigaom.com/2011/06/24/419-what-shoppers-dont-realize-about-amazons-reviews/
Park, M., & Yoo, J. (2018). Benefits of mass customized products: moderating role of product involvement and fashion innovativeness. Heliyon, 4(2), e00537. https://doi.org/10.1016/j.heliyon.2018.e00537
Peddibhotla, N. B., & Subramani, M. R. (2007). Contributing to public document repositories: A critical mass theory perspective. Organization Studies, 28(3), 327–346. https://doi.org/10.1177/0170840607076002
Pinch, T., & Kesler, F. (2011). How aunt Ammy gets her free lunch: A study of the top-thousand customer reviewers at amazon. com. Managing Overflow in Affluent Societies.
Plucker, J. A., Kaufman, J. C., Temple, J. S., & Qian, M. (2009). Do experts and novices evaluate movies the same way? Psychology & Marketing, 26(5), 470–478. https://doi.org/10.1002/mar.20283
Prahalad, C. K., & Ramaswamy, V. (2004). Co-creating unique value with customers. Strategy & Leadership, 32(3), 4–9. https://doi.org/10.1108/10878570410699249
Punj, G. N. (2013). Do consumers who conduct online research also post online reviews? A model of the relationship between online research and review posting behavior. Marketing Letters, 24(1), 97–108. https://doi.org/10.1007/s11002-012-9205-2
Racherla, P., & Friske, W. (2012). Perceived ‘usefulness’ of online consumer reviews: An exploratory investigation across three services categories. Electronic Commerce Research and Applications, 11(6), 548–559. https://doi.org/10.1016/j.elerap.2012.06.003
Rand, D. G., Brescoll, V. L., Everett, J. A., Capraro, V., & Barcelo, H. (2016). Social heuristics and social roles: Intuition favors altruism for women but not for men. Journal of Experimental Psychology: General, 145(4), 389. https://doi.org/10.1037/xge0000154
Reichelt, J., Sievert, J., & Jacob, F. (2014). How credibility affects eWOM reading: The influences of expertise, trustworthiness, and similarity on utilitarian and social functions. Journal of Marketing Communications, 20(1–2), 65–81. https://doi.org/10.1080/13527266.2013.797758
Sanbonmatsu, D. M., Kardes, F. R., & Herr, P. M. (1992). The role of prior knowledge and missing information in multiattribute evaluation. Organizational Behavior and Human Decision Processes, 51(1), 76–91.https://doi.org/10.1016/0749-5978(92)90005-R
Santos, T., Lemmerich, F., Strohmaier, M., & Helic, D. (2019). What's in a review: Discrepancies between expert and amateur reviews of video games on metacritic. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 1–22. https://doi.org/10.1145/3359242
Schmidt, S., & Eisend, M. (2015). Advertising repetition: A meta-analysis on effective frequency in advertising. Journal of Advertising, 44(4), 415–428. https://doi.org/10.1080/00913367.2015.1018460
Sedgwick, J., & Pokorny, M. (2014). The consumption characteristics of film. ResearchGate.
Seng, M. J. G., & Geertsema, J. (2018). Sharing practices, but what is the story? Exploring award-winning teachers’ conceptions of teaching. Higher Education Research & Development, 37(2), 254–269. https://doi.org/10.1080/07294360.2017.1373331
Simmons, L. L., Mukhopadhyay, S., Conlon, S., & Yang, J. (2011). A computer aided content analysis of online reviews. Journal of Computer Information Systems, 52(1), 43–55.
Smith, K. T., & Mangold, W. G. (2012). Using Online Reviews in Creative Selling. Baylor University. https://www.baylor.edu/business_new/kellercenter/doc.php/195402.pdf
Sotiriadis, M. D., & Van Zyl, C. (2013). Electronic word-of-mouth and online reviews in tourism services: The use of twitter by tourists. Electronic Commerce Research, 13(1), 103–124. https://doi.org/10.1007/s10660-013-9108-1
Subramani, M. R., & Peddibhotla, N. (2003). Contributing to document repositories-An examination of prosocial behavior. University of Minnesota, Minneapolis, MN.
Sujan, M. (1985). Consumer knowledge: Effects on evaluation strategies mediating consumer judgments. Journal of Consumer Research, 31–46. https://doi.org/10.1086/209033
Tabachnick, B. G., Fidell, L. S., & Osterlind, S. J. (2001). Using multivariate statistics. Pearsons, New York.
Vallurupalli, V., & Bose, I. (2020). Exploring thematic composition of online reviews: A topic modeling approach. Electronic Markets, 30(4) 1–14. https://doi.org/10.1007/s12525-020-00397-5
Wang, Z. (2010). Anonymity, social image, and the competition for volunteers: A case study of the online market for reviews. The BE Journal of Economic Analysis & Policy, 10(1), 1–34. https://doi.org/10.2202/1935-1682.2523
Wikström, S. (1996). Value creation by company-consumer interaction. Journal of Marketing Management, 12(5), 359–374. https://doi.org/10.1080/0267257X.1996.9964422
Wilkinson, A. (2019, May 23). Audience reviews on Rotten Tomatoes are easily manipulated. That’s about to change. Vox. https://www.vox.com/culture/2019/5/23/18637464/rotten-tomatoes-audience-verified-score-trolls-brigading-review-bombing
Williams, R. (2009). Using heterogeneous choice models to compare logit and probit coefficients across groups. Sociological Methods & Research, 37(4), 531–559. https://doi.org/10.1177/0049124109335735
Williams, R. (2018). Panel data 3: Conditional logit/fixed effects logit models. University of Notre Dame, 1–5.
Woolf, M. (2014, June 17). A Statistical Analysis of 1.2 Million Amazon Reviews. Max Woolf's Blog. http://minimaxir.com/2014/06/reviewing-reviews/
Wu, P. F. (2019). Motivation crowding in online product reviewing: A qualitative study of amazon reviewers. Information & Management, 56(8), 103163. https://doi.org/10.1016/j.im.2019.04.006
Zhang, K. Z., Cheung, C. M., & Lee, M. K. (2014). Examining the moderating effect of inconsistent reviews and its gender differences on consumers’ online shopping decision. International Journal of Information Management, 34(2), 89–98. https://doi.org/10.1016/j.ijinfomgt.2013.12.001
Zhang, H., Yuan, X., & Song, T. H. (2020). Examining the role of the marketing activity and eWOM in the movie diffusion: The decomposition perspective. Electronic Commerce Research, 20(3), 589–608. https://doi.org/10.1007/s10660-020-09423-2
Zhu, F., & Zhang, X. (2010). Impact of online consumer reviews on sales: The moderating role of product and consumer characteristics. Journal of Marketing, 74(2), 133–148. https://doi.org/10.1509/jm.74.2.133
Zhu, L., Yin, G., & He, W. (2014). Is this opinion leader’s review useful? Peripheral cues for online review helpfulness. Journal of Electronic Commerce Research, 15(4), 267.
Author information
Authors and Affiliations
Corresponding author
Additional information
Responsible Editor: Christopher Patrick Holland
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Choi, H.S., Maasberg, M. An empirical analysis of experienced reviewers in online communities: what, how, and why to review. Electron Markets 32, 1293–1310 (2022). https://doi.org/10.1007/s12525-021-00499-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12525-021-00499-8