FormalPara Chapter Overview

In this chapter, we consider changes in the loyalty of consumers over the last 25 years and the resulting implications. We find that customer loyalty, when measured as an estimate of customer retention, has increased substantially to the brands customers opt to engage with since 1994. Contrary to the many warnings of businesspeople and marketers, we find that Millennials are among the most loyal customers across the generational cohorts included in the ACSI data, behind only the rapidly dwindling Silent Generation in their loyalty. After considering industries with the strongest and weakest customer loyalty, we examine the service recovery paradox, the finding that customers who experience a problem with a good or service but receive highly effective complaint management end up with stronger-than-average loyalty, even stronger than those customers with a problem-free experience. Chapter 7 closes with a discussion of the future of customer loyalty measurement—and in some sense, measurement of the entire consumer experience—with an examination of recently popularized (but highly flawed) measurement methods.

FormalPara Key Conclusions

  • Contrary to the dire warnings by some business professionals of an impending “collapse in brand loyalty,” loyalty has not declined, and it has in fact increased dramatically, over the past quarter-century.

  • While Millennial consumers are often accused of rampant disloyalty to companies and brands, ACSI data show that Millennials are one of the most loyal generational cohorts.

  • The service recovery paradox, where companies manage failures and complaints very effectively, does in fact lead to stronger customer loyalty among consumers. Companies in most contexts should therefore implement and maintain effective complaint recovery systems.

  • Many popularized alternative measures of customer loyalty, like Net Promoter Score™, are highly flawed and should not be considered as viable alternatives to customer satisfaction and loyalty measurement.

7.1 Is Customer Loyalty Dying? Or Dead Already?

In many ways, consumers have more choice than ever before and more convenient ways of evaluating those choices before, during, and after the purchasing process. The Information Revolution, the internet, and the consequent rise of electronic commerce (e-commerce) have brought with them seemingly limitless alternatives for consumers, as well as other advantages. These new technologies provide consumers myriad new powers, such as the ability to more easily learn about alternative products, the ability to compare quality attributes and features between these more numerous options, compare prices, and then buy what they prefer from suppliers almost anywhere in the international marketplace. Under these new and more dynamic market conditions, worries among business professionals and others about the potential death of consumer loyalty are understandable. Put simply, consumers with more power and choice than ever before are better able to exhibit disloyalty and better able to abandon one company for another, resulting in reasonable concerns about the future of customer loyalty.

Yet worries about the death of customer loyalty in an era of greater choice are not necessarily universal and aimed at all consumers. Indeed, these worries have been focused on particular groups of consumers. For a variety of reasons, both related and unrelated to their behavior as consumers, the generations coming-of-age as the Information Age first began to truly materialize (Millennials) and then after its effects had already transformed society (Generation Z) have become the source of particular attention and consternation. As consumers, these two generations, we are often told, are fundamentally different than their predecessors in the Silent Generation, Baby Boomer, or Generation X cohorts. Millennials and Generation Z consumers either barely remember, or recall not at all, a time before the internet, smartphones, e-commerce, social media, and retail giants like Amazon.com.Footnote 1 While the behavior of Baby Boomers and Generation X consumers may have been changed by the Information Age, Millennial and Generation Z consumers have been shaped by it. These recent generations are, for far too many reasons to list here, the product of a radically new age and are fundamentally different in their ideas, opinions, and consumer behavior. Their consumer power is also growing quickly and requires the full, undivided attention of companies.

Just about the same time as we are required to send our book manuscript to the publisher for type-setting and publication, projections indicate that Millennials will surpass Baby Boomers as the largest generational cohort in the U.S. in 2019.Footnote 2 Since all individuals in both the Boomer and the Millennial groups are now adult consumers, with the youngest Millennials set to turn 23 in 2019, the latter will surpass the former as the largest group of adult consumers in the U.S.. But Millennials will not hold their title as “largest generational cohort” for very long. Projections suggest that Generation Z will pass Millennials as the largest cohort in total number later in 2020 as well, and as those consumers reach adulthood (which the youngest in Generation Z will do in 2028), their combined importance as consumers will be truly unmatched.Footnote 3 Consequently, we think our book on the reign of the customer is very timely, as a roadmap for customer-centric issues in an era of generational and dynamic market shifts. Together, Millennials and Generation Z consumers will soon dominate the economy—and thus the fates of most companies—over the next decade. And it is unlikely that all companies are fully ready for this onslaught of tastes, needs, and wants likely to be exhibited by Millennials and Generation Z consumers.

Taken together, all the aforementioned changes could be viewed as detrimental—and potentially deadly—to customer loyalty. Given that many companies rely heavily on loyal customers and repeat business to drive revenue and profitability, the potential effects of systematically less loyal customers, should this materialize, would be catastrophic.Footnote 4 Without being too technical, a company has two choices to have sustainable performance success: repeat customers and/or developing new products continually (the latter is a function of the product life cycle theory, albeit beyond the scope of this book). Needless to say, continually developing new products at a high rate to maintain success in the marketplace is typically more costly and ineffective than strategically working on and implementing measures to obtain repeat customer (i.e., customer loyalty).

Unfortunately, if all consumers have the ability to more intelligently and easily choose from among a larger number of alternatives, they are at a minimum also better able to exhibit disloyalty than ever before, regardless of their actual behavior. And if this behavior is centered in and most potently exhibited by the youngest and soon-to-be largest generational cohorts driving the economy, the effects could be even worse. This begs the question: Has the proliferation of choice and the Information Age, and the gradual emergence of new and different generational cohorts of consumers that exhibit different ideas and behaviors, resulted in the “death of brand loyalty,” or has customer loyalty actually improved over the last 25 years?

According to ACSI data and as we see in Fig. 7.1, aggregate national customer loyalty—measured here as an estimate of customer retention on a 0–100% scaleFootnote 5—has improved significantly over the last 25 years. Indeed, but for a few brief periods of decline, customers have gradually and consistently become more and more loyal since 1994. In 1994, estimated economy-wide customer retention posted an initial and relatively low score of 68.8%. By 2017, the retention estimate had increased 8.1 percentage points to 76.9%, an 11.8% gain. Near an all-time high, customers are now more likely to remain loyal to their chosen companies than at almost any point since 1994. In short, while customer satisfaction has increased moderately over the past 25 years, with bigger and more dramatic changes for expectations and value, customer loyalty and retention has leapt far higher than them all.

Fig. 7.1
figure 1

National customer retention, 1994–2017. (Source: Authors’ creation from American Customer Satisfaction Index data and methods)

Full size image

Yet the aggregate growth in customer retention notwithstanding, it is certainly still possible that trouble lies ahead. For instance, it is possible that the gains in customer retention from 1994 to 2017 are isolated to improvements only (or mostly) among the older generations, and that younger consumers—those Millennial and Generation Z consumers causing so much hand-wringing—will indeed turn out to be “problematic” customers. Are there differences across these generational cohorts that should make us worry about the death of customer loyalty in the near future, even if it has increased since 1994? Put differently, which generational groups of consumers are most loyal—those from the Silent Generation, Baby Boomers, Generation X, Millennials, or Generation Z? In Table 7.1, we answer these questions, analyzing the same aggregate sample of respondents across all companies and industries from the most recent available ACSI data, or nearly 175,000 consumer survey respondents answering about experiences with companies in 48 distinct industries across diverse economic sectors.

Table 7.1 Customer retention across generations
Full size table

As the ACSI data show, customer loyalty measured as an estimate of retained customers does not behave as popular opinion suggests. Indeed, there is very little differentiation in customer retention across the generations, and the differences that do exist mostly run contrary to the warnings being issued. Silent Generation consumers exhibit the highest customer loyalty at 78%, suggesting that the oldest consumers (at least as of 2017), those ranging in age from 72 to 89 years old at the time, are least likely to leave their current provider for a competitor and most likely to stay with their current company. Loyalty dips to 76% for Baby Boomers and 75% for Generation Xers, with consumers aged 53–71 and 37–52, respectively, somewhat less loyal than the oldest cohort in the sample. This decline in loyalty among Boomers and Gen Xers is perhaps not surprising, as these consumers also tend (in general and in the aggregate) to have greater wealth and access to resources, with most in the prime of their careers or early in retirement, and thus more financially able to switch between companies easily.

But something surprising happens when we get to the Millennial generation, the primary source of consternation among businesses needing customer loyalty for financial success. The customer retention estimate for this group rebounds to 77%, with Millennials having higher loyalty than either of the two generations preceding them, and nearly as high as their Silent Generation parents, grandparents, and great grandparents. In short, not only are Millennials not demonstrably less loyal than most of their predecessors, they indicate being more loyal than most of them, and in a statistical sense significantly so.

Importantly, however, customer loyalty does take a big dip with the generation following Millennials, a generation that also represents a source of concern for marketers, plunging down to 73% for the Generation Z cohort. This score is, by a significant margin, the lowest customer retention score. While the Generation Z finding could portend trouble ahead for companies vis-à-vis customer loyalty, it is important to note that in 2017, the year from which this sample was drawn, only a very small percentage of the Generation Z customers had actually reached adulthoodFootnote 6 and thus appear in the sample. As of 2017, the only Generation Z consumers in the sample were either 18, 19, or 20 years old, and thus this sub-sample of respondents is comparatively quite small and demographically narrower than the more than 20 years of data used as a comparison. Given this, it is too soon to declare Generation Z consumers as definitively “less loyal,” and until a larger proportion comes of age as adult consumers, we should withhold judgment on their loyalty behavior. Perhaps, though, the ACSI data provide some early warning signs for firms.

In sum, based on the analysis of customer retention across generations and the earlier finding that loyalty has not declined substantially between 1994 and the present, but has in fact improved substantially, it appears that the rumors of the “death of customer loyalty” have been greatly exaggerated. Loyalty still appears to be alive and well, and Millennials and Generation Z customers have not (or have not yet) killed it.

7.2 Which Customers Are Most Loyal?

Much like the other variables in the ACSI model that we have discussed thus far in the book, customer loyalty and retention might be expected to vary across industries, both in levels and in changes over time. While strongly correlated with and driven by customer satisfaction, customer loyalty is highly sensitive to both firm and industry effects, similar to the case of customer complaints that we described in Chap. 6. For example, customers can be highly satisfied with a good or service but indicate lower loyalty than satisfaction alone might predict due to a particularly high price point. Consumers are typically more satisfied with luxury goods but cannot always afford to buy them on every purchase occasion, leading to a disconnect between satisfaction and loyalty. Conversely, consumers are sometimes highly loyal to a good or service about which they have only lukewarm satisfaction (e.g., the fast food restaurant closest to a customer’s home). In many cases, strong loyalty despite lower satisfaction can be driven by some companies offering mediocre goods and services, but offering them at very low price points that consumers cannot afford or even want to reject.

Regarding exogenous influences, changes in income or employment situation, among other macroeconomic effects, can make consumers more or less loyal to certain brands than they would be otherwise, and these effects are likely to impact different industries differently. Cross-industry variance in levels and changes are thus to be expected, and this is confirmed to be the case for customer retention as well. This variance is visible in the results provided in Table 7.2, which shows industry-level customer retention rates from low to high in 2017, with changes over the previous ten years, a period during which the aggregate national customer retention rate increased 3.2%.

Table 7.2 Industry customer retention, ten-year changes, and growth rank
Full size table

The leader in customer loyalty is the supermarkets industry, with a customer retention estimate of 83%. Close behind, two nondurable goods industries, breweries (beer) and soft drinks, share second place with two retailers, health and personal care stores and internet retailers at 82%. The strong customer retention enjoyed by companies in all of these industries is understandable. Retailers like supermarkets and health and personal care stores are often chosen by consumers based on proximity to the consumer’s residence (similar to our fast food example earlier). While these consumers may stray from time to time and drive some distance to shop at an “exotic” alternative, most supermarkets and health and personal care stores enjoy strong “convenience-based” customer loyalty. Beers and soft drinks, on the other hand, are the type of products where there is a large variety of alternatives at similarly low prices, but for which consumers—through an evolving process of trial and error over time—often settle on a favorite and purchase it almost instinctively time and time again, a fact to which many bartenders across the country would attest.

At this juncture in the book, the worst performing industries should come as little surprise and almost deserve no further mention. Internet service providers come in near the bottom at 64% retention, only slightly better than their basement-dwelling cousins in Subscription TV, at 63%. While at one point the companies in these industries had near-monopoly power and little real reason to fear consumer dissatisfaction and disloyalty, due to government regulation and the importance of delivery infrastructure in these industries, as alternatives (to subscription TV, especially) finally arrive, many consumers are becoming “cord cutters” and leaving them behind at the first opportunity.

Regarding ten-year changes, the biggest gain in customer loyalty comes for airlines, increasing a whopping 20.3% over this period. This dramatic improvement in customer retention should not come as a shock either. The airlines industry has also gained the most over this ten-year period in expectations, quality, value, and satisfaction, and thus most of the companies in this industry are enjoying dramatic improvements in the loyalty of their customers. In turn, this is evidence that improvements in the experiences of consumers provide financial benefits for companies. The second largest gain in loyalty among the industries comes for wireless telephones, gaining 19.7% since 2008. As we saw in Chap. 5, the wireless telephones industry has also gained the second most in customer satisfaction over the past decade, and it has paid-off in more loyal customers.

The biggest drops in loyalty appear in cooperative energy utilities (−5.3%), and more interestingly, internet search engines and information (−4.8%). The latter industry has the second largest decline in satisfaction over this period as well. While Google is clearly the dominant player in this industry and has grown from a “smaller” $18 billion annual revenue company to a $140 billion revenue company over this period, or a more than 670% growth in revenue, it has done so through an aggressive advertising model that appears to have alienated some customers and caused them to seek alternatives and display disloyalty.

Leading in Loyalty

As a whole, the supermarket industry leads the measured industries in customer loyalty, and three individual supermarkets likewise lead in customer retention. Publix comes in first with a retention rate of 86%, with Aldi and HEB close behind and both at 85%. Amazon also scores in this top group at 85%, showing the power and endurance of one of the world’s largest companies to continue to grow rapidly even as innumerable smaller competitors seek to pick away at their customers.

At the very bottom and lagging behind in loyalty are—surprise, surprise—two internet service providers—Windstream at a mere 54% and Frontier even lower at 53%. Not only do these companies need to be particularly concerned about their ability to keep customers over the long term, but the industry as a whole should be worried. As real practical choice arrives for consumers in this space, they are likely to defect en masse.

7.3 The Service Recovery Paradox Is Real!

There has long been a suspected relationship between customer complaints, complaint management (i.e., complaint recovery or complaint handling) by a company, and a complaining customer’s continued loyalty to that firm. Because of this relationship, economic benefits are assumed to exist for a company that operates a complaint management system (e.g., a Customer Relationship Management or CRM system) that minimizes customer dissatisfaction and maintains loyalty (or even increases satisfaction and customer loyalty likelihood) among displeased customers. Indeed, a significant majority of the academic studies on the topic have found that complaint behavior itself is not fatal to a complainant customer’s satisfaction and loyalty, so long as firms manage and handle the complaints of customers very well.Footnote 7 As we briefly mentioned in the last chapter on customer complaints, most of the studies in this area take as their starting point the “service recovery paradox (SRP),” the notion that customers who experience a failure and complain to the company can remain as or even more satisfied, and as or even more likely to remain loyal, to the firm than non-complaining customers. Of course, this increased loyalty is dependent on complaints being managed really well by a firm.

But is the service recovery paradox real? After all, business professionals, marketers, and market researchers sometimes fall victim to sensational and convincing marketing truisms—such as the oft-repeated yet highly dubious claim that “every dissatisfied customer will tell 10 (or 15, or 20) people about their experience”—that are disconnected from any empirical evidence or apply only to some very limited contexts.

To confirm the existence of the service recovery paradox (SRP), we analyze a large sample of ACSI data across multiple industries and sectors and over the same, most recent ten-year period used throughout the chapter, to establish both the existence and the durability of this relationship, should it exist. The results are presented in Figs. 7.2 and 7.3. Figure 7.2 compares aggregate responses among consumers to the 1–10 scaled “repurchase likelihood” survey question discussed earlier, splitting the sample between those customers who did and those who did not complain. These results bring into sharp focus just how important customer complaints are to firms, revealing the substantial differences in loyalty intention between consumers who complain and those who do not. For example, while only 3% of non-complainants indicate that they are “very unlikely” to repurchase from the same firm again in the future, among complaining customers the number is five times higher at 15%. That is, across all economic sectors and consumer industries included in the ACSI data, a consumer who has complained is five times as likely to indicate that their next purchase for a good or service within that same category will almost certainly be with a new or different company. Similar results are observed on all of the lowest points on the scale from 1 through 4, with complaining customers four or five times more likely to pick these “disloyalty” options than non-complaining customers.

Fig. 7.2
figure 2

Customer retention among complainers and non-complainers. (Source: Authors’ creation from American Customer Satisfaction Index data and methods)

Full size image
Fig. 7.3
figure 3

Customer retention for non-complainers versus complainers with perfectly managed complaints. (Source: Authors’ creation from American Customer Satisfaction Index data and methods)

Full size image

Yet the results in Fig. 7.2 also show that a customer’s complaint behavior is not the sole determinant of loyalty intention. While 48% of non-complainants indicate that they are “very likely” to repurchase from the same firm again, the corresponding rate among complaining customers is 24%. That is, almost one-quarter of the customers that do have a problem significant enough to lead them to lodge a complaint with a company suggest that they will stay loyal to the same company in the future. But, if 15% of complainants are almost certain to defect, why are an even larger proportion—24% of complainants—almost certain to remain loyal? To be sure, brand loyalty, price competitiveness, limitations in alternatives, and similar explanations play some role. But the factor mostly responsible for dividing complaining customers into those who are almost certain to defect and those who are equally certain to remain loyal lies in successful complaint management by firms. This fact is confirmed in Fig. 7.3, which shows average repurchase likelihood among non-complaining customers and complaining customers who say their complaint was handled almost perfectly.

As we see in Fig. 7.3, the analysis of a large cross-sector and over time sample of ACSI data supports the existence of a service recovery paradox. For six of the seven economic sectors included in our sample (all but the manufacturing sector including both durable and nondurable goods and its underlying industries), complaining customers who have their complaint handled perfectly (i.e., rating the complaint handling experience a “10” on a 1–10 scale during surveying) give on average a higher repurchase or loyalty intention score than average non-complaining customers. The SRP “gap” is largest for transportation and warehousing (e.g., express parcel delivery) and finance and insurance (e.g., banks, credit unions, and insurance providers). In other words, for all sectors except manufacturing, “perfectly managed complainants” are significantly more likely to remain loyal than non-complaining customers.

In sum, the SRP and the imperative for firms to manage complaints effectively derived from it appears to be real. For most companies—or at least those that rely on customer loyalty to realize profitable growth—finding ways to handle complaints very well is necessary. While based on more complex analysis, however, we can also state definitively that the importance of complaint management to companies varies across industries, with some industries needing to more aggressively manage complaints than others.Footnote 8 This can be seen in Fig. 7.3, where complaint handling produces less pay-off in terms of increased loyalty likelihood than others. And because industry differences impact customers’ purchasing behaviors following complaint handling, the financial ramifications of firms’ complaint management efforts differ as well. Nevertheless, most firms interested in repeat business must seek to turn many complaining, unhappy customers into loyal ones, and must do so via effective complaint management.

7.4 Satisfaction, Loyalty, and Recommendation Are Different!

While we have closed most of the preceding chapters in the book with a discussion of the possible form consumer perceptions in these various areas might take in the near future, to close Chap. 7 we follow a different path. In this case, we discuss the future of the measurement of customer loyalty, and in some sense, the future of measurement of the customer experience as a whole. However, this discussion is not tangential to the future of customer loyalty; indeed, it bears directly on its future and that of other consumer perceptions. That is, because how companies measure consumer insights often has a dramatic impact on how they perform and improve (or fail to improve) in providing positive consumer experiences, this discussion is an important one that cannot be ignored. When combined with recent and troubling trends in customer loyalty measurement, the significance of this metric discussion is greater than ever before.

Over the past decade or so, a handful of researchers, other marketing professionals, and even CEOs and related business leaders, have come to question traditional consumer experience and customer satisfaction measurement. The arguments and justifications of those in this group vary. For some, consumer experience and satisfaction measurement systems are valuable, but are simply too costly, complex, and time-consuming to undertake, especially in an era when the prevalence of consumer surveying has grown exponentially, response rates have declined dramatically, and data collection has become more difficult.Footnote 9 For others, measurement of different key performance indicators (KPIs) is simply deemed preferable for assorted reasons. A minority of these latter doubters have gone so far as to suggest that customer satisfaction is “worthless” and reject the need for its measurement or management entirely.Footnote 10 According to these more extreme contrarians, customer loyalty—and not customer satisfaction—is all that matters, and thus loyalty is all that should be measured. Perhaps the most influential example of this type of thinking comes from advocates of the “Net Promoter Score” (NPSTM).

First outlined in-depth in the book The Ultimate Question—published in 2006 but actually an expanded version of an earlier, shorter 2003 article published in the Harvard Business Review—NPS supporters argue that measurement of customer satisfaction is an unnecessary waste of resources and that all companies really need to do is measure a proxy for and correlate of customer loyalty: how likely the consumer is to recommend the company/brand to others.Footnote 11 Because likelihood to recommend is both strongly associated with the individual’s own loyalty intentions and behaviors, along with the ability to promote and attract new customers to the company through recommendation, so the argument goes, it alone is the one number companies need to measure and understand, the “one number they need to grow.”

Described briefly, the NPS survey question asks a group of respondents (i.e., a sample of a company’s customers) how likely they are to recommend the company to a friend or colleague (on a 0–10 scale). The NPS metric is calculated from the resulting data as the percentage of the sample of respondents that are “promoters” (those that answer 9 or 10 on the scale) minus the percentage of the sample that are “detractors” (those that answer from 0 to 6), with the difference between the two reflecting the “net” promoter score. (Those respondents giving a 7 or 8 on the scale are ignored as “neutral” or “passive” respondents, neither likely to promote nor to detract, and thus assumed to be silent about their experience.) The resulting statistic—which theoretically ranges from −100 (all detractors) to 100 (all promoters)—serves as the NPS for a company or other organization. And to be sure, over the past ten years or so, many companies have come to accept the advantages of NPS and the claims of its advocates. A large number of high-profile Fortune 500 companies have used or are currently using NPS, and anyone working in market research or consulting would have likely come across NPS at some point.

Why has NPS become so popular and been adopted by so many companies? NPS is, at its core, a rejection of traditional market research via consumer surveys and statistical methods that many companies have long struggled with. The NPS metric abandons all of this, conflates word-of-mouth (recommendation) and customer loyalty—two metrics already important to and measured by many companies—in favor of a simplistic, single-question approach. Then, and critically, its advocates claim that companies with strong NPS scores enjoy greater revenue growth and profitability than those with low scores, and that NPS is a better predictor of firm financial growth than any other metric. For these reasons alone, the idea is appealing to many. Indeed, a metric that promises to be a stronger driver of financial performance than any other, but that is also simple to measure, does not require a long survey, does not require substantial data collection efforts, does not require complex statistical methods, and can be understood by non-statisticians within companies sounds too good to be true. The problem is, however, that the promises made by NPS and its supporters are too good to be true.

The problems with NPS begin with how the metric is calculated, transforming a 0–10 scaled variable arbitrarily into a three-category variable. Why, for instance, should we assume that respondents answering 9 or 10 are definitely going to promote the company or brand, while those answering an 8 will do absolutely nothing? Are those respondents giving an “8” on the scale really that different than those that give a “9,” in terms of their future behaviors, so much so that we should assume the “8s” will stay totally silent while the “9s” become active, frequent, boisterous promoters of the company and its brands? Conversely, are those respondents answering a 0 versus a 6 all really the same, in terms of their likelihood to speak negatively about a company? Should a company that finds that 50% of its customers give it a “10” while the other 50% give it a “6” really have the same NPS score as one where 50% of its customers give it a “9” and the other 50% a “0,” resulting in a dramatically lower mean score on the raw “likelihood to recommend” survey variable? These and many similar oddities used to calculate NPS cause the experienced market researcher to question the validity of the metric out of the gate.

Yet much has already been written about these issues and about the methodological and statistical shortcomings of NPS that result from transforming an 11-point variable (0–10 scale) into a three-category variable.Footnote 12 What are the real, practical implications of the imprecision of NPS, in terms of its sensitivity to differences and thus the meaningfulness of insights gleaned from it? To find out, we examined a sample of data from the ACSI from 2017, the same sample examined and described previously covering customers/survey respondents of nearly 400 companies in 48 different economic industries. To get at the usefulness of NPS, we compared ACSI’s 0–100 customer satisfaction variable with the NPS variable from the same sample. Using this data, we calculated company-level mean scores, standard errors, and 95% confidence intervals (CIs) for both variables. The results of this analysis are included in Table 7.3 below.

Table 7.3 NPS , ACSI, and statistical precision
Full size table

The 95% CI is the most important statistic in Table 7.3. A cornerstone of inferential statistics, a 95% CI is interpreted as the probability of observing the same results 95 times if you were to draw 100 random samples for the same variables, within a margin of error. The CI is that margin of error. These results show that, on average, we can expect an NPS score to vary randomly 9.1 points across samples. In other words, a company that sees its NPS at 32 and 41 across two separate samples may only be seeing random noise instead of a real difference. By comparison, the ACSI satisfaction variable varies only 2.3 points on average. Normalizing these estimates for the different sizes of the scales (−100 to 100 vs. 0–100), the results show that while the ACSI variable should be expected to randomly vary about 2.2% on its scale between samples, the NPS score varies about 4.5%, or more than twice as much variance. This larger random noise exists for NPS when compared to the ACSI variable precisely because of the way in which the metric is calculated and in how respondents are arbitrarily assigned to the underlying NPS categories. Likewise, any additional analysis done on the NPS variable—such as any type of correlation or regression analysis—is much more likely to produce unreliable results because of this same random variance.

The source of the much greater random error in NPS—the transformation of the variable from its original 0–10 scale into three arbitrary categories—also complicates any practical efforts for companies working to improve their NPS. That is, assuming firms seek to improve their NPS by maximizing “promoters,” the data becomes even more error-prone and even less reliable. Across the same sample of companies analyzed, the “promoter” group shows average normalized variance of 5.7%. Thus, even a very large boost in the proportion of promoters for a company—from 55% to 60%, for example—may represent nothing more than random noise, rather than the effects of any actions taken.

Finally, because the average sample size of completed interviews per company in the analysis is reasonably large (more than 425 respondents per company), it is important to note that at smaller sample sizes the random variance in NPS will increase dramatically, making interpretation of the metric even more difficult. For instance, holding all else constant in the analysis, but changing the company-level sample sizes to a smaller but not at all uncommon 100 interviews per company instead of 425, the 95% CI for NPS increases to 14.8, meaning that an NPS of 46 and one of 32 might not actually be meaningfully different (for sake of comparison, the same sample size change would take the 95% CI for the ACSI variable to only 3.8 from 2.3).

Based on the comparisons, it is unsurprising that NPS’ central claim, the argument that first garnered attention and resulted in its adoption by many companies—that the metric is the strongest predictor of firm revenue and profitability growth—has been repeatedly disproven.Footnote 13 The NPS metric is not the strongest predictor of growth, and its failure to predict growth is directly related to the random noise created when calculating it. The logic is simple to understand, as we have illustrated in this section. But the statistical and econometrics rationale go much deeper as well. Specifically, as we also stated in Chap. 6, simply measuring customer satisfaction as a function of how likely a customer is to recommend a good or service to another potential customer does not capture the full variance of satisfaction or loyalty, nor its predictive impact, level, and change over time. The noise in the NPS data has serious implications. As a very coarse-grained example, any company leader would be hard pressed to justify the chance that revenue for the year, for example, would be 11% different just by chance (or an 11% drop in stock price). Why would a company then be accepting an absolute data difference 14.8% for NPS versus 3.8% for ACSI? That margin of error is too large.

But even beyond the ability of the NPS metric to predict growth, it must be noted that it is never advisable to disconnect customer satisfaction, customer loyalty, recommendation, or any other outcome variable from the broader context in which these perceptions emerge. Doing so ignores vital information. For example, there is typically a very strong (though neither perfect nor consistent) positive statistical relationship between customer satisfaction and loyalty; as satisfaction increases, so too does loyalty. But the linkage between the two measures varies both across different industries and over time. In practice, this means that satisfaction can matter more or less to loyalty in some industries rather than others, and that this relationship can shift for a single industry or company based on other external factors (such as competition in the market, macroeconomic conditions) over time. Thus, tracking not just the scores but also the relationship between the two metrics at regular intervals is critical.

Moreover, most high-quality and action-oriented market research measures not only customer satisfaction and loyalty, but also the key drivers of satisfaction (i.e., predictive influencing factors) for the company and its products and services, with the goal of better understanding how these can be manipulated and improved to increase satisfaction and loyalty. For example, a bank may certainly want to know its customers’ satisfaction, loyalty, and propensity to recommend the bank to others, but it must also measure how things like number of branches or ATMs, quality of customer service personnel, quality of the website, and so forth are viewed by consumers, and how these variables impact both satisfaction and loyalty. Only with this information can the company make efficient improvements in the attributes that matter most to customers and thereby most effectively improve their experiences.

Across Chaps. 3, 4 and 5, we discussed in detail the dynamics of customer satisfaction over the last 25 years. In particular, in those chapters we focused on the fact that satisfaction improvements appear to have been driven almost entirely by improved consumer perceptions of value, while consumer perceptions of quality have been unchanged. Quality, as we said earlier, has more potential power in driving satisfaction but has been limited in doing so because of the flatness of the perceptions of quality of the last 25 years. Based on this data, we concluded that the Information Age has allowed companies to pass efficiency-driven cost savings on to consumers, lessening the need for investments in quality, and that this, therefore, is how companies have gone about providing higher satisfaction to consumers. But could there be another parallel development responsible for this trend, for the lack of improvement in quality and companies relying on price to boost satisfaction? Based on the analysis of NPS, another explanation for lagging consumer perceptions of quality over the last 25 years, and particularly the small decline in quality over the last ten years, should be considered.

Could it be that popular but highly flawed and uninformative metrics like NPS have failed firms, leading them to misunderstand the importance of quality to improved satisfaction—or even how to improve quality at all? Are companies wasting time and resources chasing the noise endemic to NPS, time, and resources that would be far better deployed on solid measurement that can help realize real changes that will improve the customer experience? These possibilities must be considered. After all, low-quality metrics often lead to poor decision-making, and are sometimes worse than no metrics at all. Moving forward, companies that want to compete on more than price, that want to improve their customers’ experiences in an efficient way that also improves perceptions of quality relative to competitors and drives business their way, are far better served relying on concise, well-designed, and reliable consumer surveys and statistical models in doing so (a one-question NPS survey or a 410-question JD Power survey are not the answers—the NPS for the reasons discussed and the JD Power satisfaction assessment for survey fatigue reasons, where the quality of the data the respondents provide deteriorate to the point of being unreliable and invalid). Absent this, the stagnant quality consumers perceive from the economy may continue, and ultimately lead to weaker satisfaction and economic troubles, both for firms and national economies.