Keywords

1 Introduction

Customer Experience (CX) is a central focus of service management literature viewing customer evaluations as an outcome of interactions between customers, employees, systems and processes in a service context (Bitner et al., 1997). Organizations have customers regardless of whether they are internal or external and regardless of being called guests, members, patients or clients. It is well established that facilitating a meaningful customer experience is essential to achieving competitive advantage (Bolton et al., 2014; Homburg et al., 2017; Verhoef et al., 2009), greater revenue and greater employee satisfaction (Rawson et al., 2013).

It is not surprisingly therefore that facilitating and managing great customer experiences is among the top priorities of CEOs around the world (Dixon et al., 2010; Toman et al., 2013; Lemon & Verhoef, 2016). Forrester (2016) found improving customer experience to be the top priority for over 70% of businesses (Flavián et al., 2019). Some would even say that experience is everything in service management (Clarke & Kinghorn, 2018). Indeed, there is evidence to suggest that a significant number of customers are prepared to walk away after just one bad experience. Certainly competition is tough and customers can take their business elsewhere. So it is critical that organizations know what their customers value in the customer experience. Getting to know what customers want in their experience with an organization, as well as what they don’t want is vital in service management. When customers feel they are getting an experience that they value, they are likely to be loyal, say positive things about the organization, even advocate and continue buying, often buying more, from that organization. In short, customers are looking for meaningful, authentic human experiences without the frustrations so often associated with interactions between humans and machines (Clarke & Kinghorn, 2018).

While some organizations think they know what their customers value and so they focus on designing the customer experience in terms of what they think is best for the organization, most firms are interested in better understanding what their customers think about their experiences and how the firm can turn customer experience from good to great. We know that small details can make big differences (Bolton et al., 2014). Many organizations spend vast amounts of money, sometimes millions of dollars, in an effort to get to know their customers better and understand what is important in the customer experience. But many firms are not very good at listening to their customers and this isn’t from want of trying. Millions of dollars are spent on collecting information. Yet, organizations report that they are not satisfied with the answers they are receiving from their customers. It appears that the problem is not a lack of effort on the part of the organization, but rather that the tools that are widely in use still today are imprecise—not measuring what they are supposed to be measuring. Knowing what to measure, how to measure it, in order to gain rich insights that matter to customers through multiple data sources, and especially what to do with open-ended feedback has not been clear until now (Zaki et al., 2021).

2 Conceptualizations of Customer Experience (CX)

Before exploring a widely employed range of practical tools used to measure customer experience, we will first consider some key conceptualizations of customer experience. This next section provides an overview of a range of definitions of customer experience (CX) highlighting key elements. As shown in Table 1, several definitions have been offered on CX. Frow and Payne (2007) are one of the first to define customer experience as holistic, comprised of multiple touchpoints in a journey. This notion of a journey over time is echoed by Neslin et al. (2006) and McColl-Kennedy et al. (2019). There is general agreement among researchers that a customer’s perception of his/her experience is holistic in nature, involving multiple internal and subjective responses to interactions with an organization (Meyer & Schwager, 2007; Schmitt et al., 2015). Customers respond to a range of stimuli in the service environment and this is acknowledged by Lemon and Verhoef (2016) who highlight that CX is comprised of the customer’s cognitive, affective, emotional, social and sensory elements. Voorhees et al. (2017) underscore that the customer experience takes place throughout many interactions, including multiple “moments of truth” that influence customer outcomes. This conceptualization is consistent with the view that customer experience is a process (Grönroos, 1998; Rawson et al., 2013), comprised of interactions and activities across multiple touchpoints. Homburg et al. (2017) define customer experience as the evolvement of a person’s sensorial, affective, cognitive, relational, and behavioral responses to a firm or brand by living through a journey of touchpoints along pre-purchase, purchase, and post-purchase situations and continually judging this journey against response thresholds of co- occurring experiences in a person’s related environment. McColl-Kennedy et al. (2019) elaborate on the interactions and activities, identifying key elements of the customer experience as comprising value creation elements (resources, activities, context, interactions and customer role) and both customer discrete emotions and cognitive responses at touchpoints across the journey. Becker and Jaakkola (2020) suggest that customer experience should be viewed as non-deliberate, spontaneous responses and reactions to particular stimuli provided by a firm. Indeed, Rawson et al. (2013) emphasize the importance of viewing the customer experience as a “complete experience” taking into account the experience on the way to purchase all the way through to after purchase.

Table 1 Illustrative examples of conceptualizations of customer experience (CX)
Full size table

3 Traditional CX Tools Are Too Blunt

Historically, organizations have used customer satisfaction metrics to measure customer experience. However, widely used customer satisfaction metrics often fail to reveal what customers really think and feel about the service experience. In the digital era, organizations need to take deep dives into the data if they are serious about understanding what their customers value to gain rich insights into what is wrong from their customers’ perspectives, and importantly what needs to change in order to provide seamless, meaningful experiences (Zaki et al., 2021).

Among the most popular tools, used widely by organizations, are satisfaction and loyalty surveys, as well as Net Promoter Scores (NPS). These tools provide numeric scores. As such, they give the impression that they are precise, accurate measures. At best, they can be regarded as blunt instruments which tell us very little about what customers are really thinking, feeling and doing. At worst, they are misleading.

Net Promoter Score (NPS) provides the percentage of customers who would recommend a given organization to their friends and family. NPS was developed by Frederic Reichheld (2003) to measure loyalty. But it is reportedly the most commonly used customer experience metric because it is simple and easy to use (Morgan, 2019). Most firms still focus on one question “How likely are you to recommend this company to a friend or colleague?” on a scale of 1 to 10. It is overly simplistic and is really a measure of positive word of mouth rather than customer experience.

Customer satisfaction measures are relatively easy to administer and can be used to produce impressive-looking graphics and they are generally based on large quantities of data. But the sheer volume of data does not mean that the results ensure insights into what customers really value. Our research shows that relying on these scores alone can be misleading masking serious problems with the business. Not only is quantitative surveying more resource-intensive, customers are also finding filling out surveys increasingly intrusive and are becoming less inclined to participate (Morgan, 2019; Holmlund et al., 2020). Another critical weakness is that they cannot pick up customer emotions. By masking significant customer dissatisfaction, firms can lose customers without knowing why (Zaki et al., 2021).

Interestingly, organizations use many qualitative approaches, such as focus groups, interviews or by manually reading and analyzing open-ended comments from their customers as part of a survey. However, organizations typically do not delve deeply into the free text comments that customers provide in these qualitative approaches. This is because analyzing thousands and thousands of comments “by hand” is not only time-consuming and labor intensive, it is also difficult to categorize the comments into useful themes. Therefore, open-ended feedback that firms receive is often ignored (McColl-Kennedy et al., 2019). If used at all, organizations have traditionally grouped the open-ended free text comments into two broad, overly simplified categories (1) positives (“compliments”) or (2) negatives (“complaints”). When this occurs, organizations lose a great deal of valuable information that potentially can offer insights into why customers think and feel the way they do. McColl-Kennedy et al. (2019) found that organizations can pick up on a third category labeled “suggestions” to listen to customers’ ideas to improve the experience. These could be suggestions to improve processes such as ways to reduce wait times, improve communication between frontline employees and customers, communications internally within the organization, using mobile apps to provide information in real time to customers and enable them to provide feedback easily through their phone or tablet.

Another metric used to measure customer experience is the customer effort score (CES) (Morgan, 2019). CES measures how much work customers have to do through an interaction with the brand or organization. It is typically measured by asking customers “How much effort did you have to put in to resolve the issue?” on a scale from Very Low Effort to Very High Effort. This metric may help firms to determine customer friction points and find ways to create a more seamless experience.

The churn rate is another metric used by organizations (Ascarza & Hardie, 2013; Morgan, 2019). The churn rate tracks how many customers discontinued doing business with an organization over a particular period of time. The thinking behind this metric is that customers will not leave the organization if they’re having a good experience. Churn rate is calculated by dividing the number of customers lost during the timeframe by the number of customers at the beginning of the timeframe. In essence the churn rate is the opposite of the retention rate.

4 Multiple Metrics Are Recommended

But it is in the open ended free text where customers can best articulate what they do not like (and like) about their customer experience. They can also elaborate on why this is the case, providing context which is very important for a full understanding. None of this can be obtained through simple numeric scores. While traditionally it has been time-consuming to classify and make sense of these comments, it is in the free text comments that customers express their true feelings (McColl-Kennedy et al., 2019), and these turn out to be a much more reliable predictor of their behavior than the boxes they have checked. In a similar vein, Becker and Jaakkola (2020) call for the development of new dynamic measurement approaches.

Many customers today use smart, real-time digital devices, including mobile apps which enable firms to collect more precise real-time data about their customers’ journeys. In fact, an unprecedented volume of textual data generated from a wide range of sources and formats such as news items, industrial reports, online chatter, surveys, interviews, blogs, scripts and notes are available to organizations and it is expected that the number and complexity of these qualitative data documents will only increase in the future (Zaki & McColl-Kennedy, 2020). By 2025 the International Data Group predicts that there will be 163 ZB of data globally, with around 80% of business-relevant information originating from unstructured forms, primarily text (Techrepublic, 2017). Consequently, large amounts of data, including textual data such as actual comments from customers, from Twitter, Facebook, customer blogs, as well as the more traditional online and telephone surveys are generated at many touchpoints across the customer journey. Clearly, firms need to review and re-think the approach they are taking to measure customer experience.

5 New Digital Technologies Offer Useful Ways to Measure and Manage CX

Due to digital advances, organizations have access to a vast array of data about what customers think about the organization’s products and services, much of which is in free text form (Zaki, 2019). For example, textual data such as verbatim comments from customers are now generated across the customer journey. User-generated content and free text feedback contain excellent sources to delve into the customer’s views (Tirunillai & Tellis, 2014) providing insights into what customers really think about specific pain points throughout the customer journey. A recent study has indicated that AI and machine learning in the management toolkit has shown increasing implementation, with approximately a 10% increase in use year on year since 2018 (Moorman, 2021). It is expected that AI and machine learning use will increase by 20% in the next three years. Although AI has yet to be widely adopted by marketers and customer experience managers it offers great promise as it enables organizations to mine huge datasets and extract meaningful insights from customers about what they value and do not value in CX.

Text mining can be used to extract customer views from unstructured comments (Pang & Lee, 2008). For instance, Xiang et al. (2015) applied text mining to customer reviews to understand the relationship between customer experience and satisfaction. Culotta and Cutler (2016) used a social network mining model to analyze multiple Twitter datasets in order to investigate how strongly consumers associate with different brands.

Text mining and other emerging technologies such as AI offer potentially more effective ways to measure and manage customer experience (Zaki et al., 2021). An important learning from McColl-Kennedy et al. (2019) is the need to connect both qualitative data and quantitative data to enable organizations to gain rich insights. Through this approach the authors were able to identify seven root causes of problems for the customer for a complex B2B service highlighting distinct opportunities for improving the CX. In that specific setting root causes identified were: capability, communication, parts, price value, process adherence, quality and service capacity (McColl-Kennedy et al., 2019). Their model enabled the identification of what was influencing and responsible for each root cause. For instance they found the “parts” root cause centered around parts being unavailable (resources), problems around customer activities picking up parts, issues with customers not being able to collect parts on weekends (context), shortcomings in interactions between the employees and customers regarding the ordering and delivery of spare parts and what the role of the customer was and what it should be.

6 Need to Think Multi-channels Across Physical, Social and Digital

Assessment at the various touchpoints contribute to the overall customer experience across the customer journey. At times these may be in one channel, such as face to face. At other times the channel may be virtual, such as online shopping. Prior work has demonstrated that customers may utilize different channels for different aspects of their customer journey.

For instance, some may purchase in one channel but seek post-purchase assistance in another (De Keyser et al., 2015; Verhoef et al., 2009).

Further, it is important to think not only about physical channels, consideration of digital increasingly needs to be incorporated in customer experience measures as well as new forms of social interactions. New technologies are changing the way organizations interact with their customers and transforming the customer experience (Lemon, 2016; Van Doorn et al., 2017). AI, robots and virtual reality are already playing a role in the customer experience. These new ways of engaging with customers will not completely replace face-to- face encounters but increasingly they will operate alongside them. Managers will need to understand customer experiences across the digital, physical and social realms, and design services and facilitate experiences accordingly.

As outlined by Bolton et al. (2018), organizations need to think about the customer experience in terms of all three realms—the physical, digital and social realms, and not in isolation but viewing them as connected. That is how customers view them. Edvardsson et al. (2010) also highlighted the importance of social, as well as interactions between customers and between customers and employees, and the role of technology, in addition to the physical elements of the servicescape. Bolton et al. (2018) argue that customer experience can be conceptualized within a three dimensional space—low to high digital density, low to high physical complexity and low to high social presence—yielding eight octants.

Digital technologies can be used to design optimal and seamless customer experiences (Flavián et al., 2019). Managers are encouraged to think more broadly about the advantages of using different channels, such as virtual reality (VR), augmented reality (AR) and mixed realities (MR) in order to provider richer experiences for their customers (Brynjolfsson et al., 2013; Verhoef et al., 2015).

7 A Robust Conceptual Framework Is Required

In order for these new technologies, such as AI to be useful, researchers and managers need to apply a conceptual framework as the technology does not automatically provide the deep insights required. Villarroel Ordenes et al. (2014) proposed a framework comprising three elements of the customer experience (1) activities; (2) resources and (3) context. They used a linguistics-based text mining approach to automate sentiment analysis of customer feedback in the context of carpark and transfer services at a UK airport. Their model captured customer activities and resources, company activities and resources, and customer sentiment (complaints and compliments) demonstrating how certain features of linguistics-based text mining, such as dictionaries and linguistic patterns, can be used to analyze textual customer feedback.

Baxendale et al. (2015) take an integrated view of customer experience, highlighting the importance of understanding multiple touchpoints, interactions at the touchpoints and modeling the valence of the customer’s affective response at the respective touchpoints along the customer journey. Further, they develop and implement a new tool designed to collect real-time customer experience data for selected consumer goods.

McColl-Kennedy et al. (2019) building on and extending Villarroel Ordenes et al. (2014)’s work developed a CX framework that takes into account the customers’ perspective as the starting point using both qualitative and quantitative data. McColl-Kennedy et al.’s (2019) CX framework directs machine learning to provide meaning from the unstructured customer data. It works by classifying the data according to the following components: (1) customer touchpoints, (2) value creation elements, (3) emotions and (4) cognitive responses. McColl-Kennedy et al. (2019) followed the six-step established approach of Chapman et al. (2000) and Hevner et al. (2004) and applied advanced text mining techniques to two years of customer feedback in a complex B2B heavy asset service setting and tested it on two additional datasets.

In the next section a step-by-step guide for practitioners to apply and measure customer experience using AI, is summarized below and illustrated in Fig. 1.

Fig. 1
A diagram exhibits the cycle of six steps. Business understanding, data understanding, data sampling, applying the framework, model development, and model validation are observed clockwise from the top.

Step-by-step guide for practitioners to apply AI to measure CX

Full size image

8 Practical Guidelines for Practitioners

Step 1 is Business Understanding. Here the aim at this stage is to understand in depth the organization and its various services and products. It is recommended that field research be carried out, including for instance interviews with customers and frontline employees as well as managers. Shadowing employees across all the key touchpoints was undertaken by McColl-Kennedy et al. (2019) and is highly recommended as it enables observation of processes, practices and interactions to be observed first hand. For example, they interviewed 34 employees and 20 customers to understand the customer experience from the respective informants’ perspectives.

Step 2 is the Data Understanding phase. This step involves building and testing of the customer experience analytic developed from interrogation of a dataset of longitudinal customer experience responses obtained from a survey administered by a third-party market research firm. McColl-Kennedy et al. (2019) adapted and extended the linguistic text mining approach introduced by Villarroel Ordenes et al. (2014). In addition to obtaining the quantitative scores for customers’ ratings of 12 questions (10-point scale from “Very Satisfied” to “Very Dissatisfied”)—overall satisfaction, repurchase, referral, resource availability, responsiveness, communication, service completion duration, preparation, service quality, invoice timeliness and invoice accuracy, they analyzed responses to the free text question. They collected two years of survey data from a large B2B firm.

The third step is Data Sampling. Following established practice, McColl-Kennedy et al. (2019) used a random sample of 100 comments from the dataset in the training stage to provide rich text for data understanding and pattern development (Singh et al., 2011). These comments were divided into separate sentences. Two coders independently classified each comment following the conceptual framework. Macros and linguistic pattern rules were developed and applied to the conceptual framework. Resulting patterns were then mapped to the root causes. This is a very important stage as it enables an organization to understand the root causes and identify opportunities to improve CX by taking steps to address the problems.

The fourth step is Applying the Conceptual Framework. Here, for example, the coders manually annotated each sentence in terms of (1) touchpoints, (2) all value creation elements—resources, activities, context, interactions and customer’s role, (3) discrete emotions and (4) cognitive responses. A judge was employed when disagreement was encountered. Using a fine-grained approach enables text mining algorithms to capture specialized vocabulary used by customers. This offers a better way to identify pain points that matter to customers than the general linguistics-based text mining applications (Villarroel Ordenes et al., 2014) that are expected to be too coarse to capture important details that matter to the specific customers (Bolton et al., 2014).

The fifth step is Model Development. To develop their text mining model, McColl-Kennedy et al. (2019) used text mining techniques such as Part of Speech (POS) to capture different forms of speech (e.g., verbs, nouns) and they developed patterns using macros and linguistic pattern rules applied to the conceptual framework. This step is essential to enable the text mining model to map automatically the customers’ verbatim words to the four CX dimensions (touchpoints, value creation elements, emotions and cognitive responses). They evaluated and extended the dictionaries as appropriate. New concepts and patterns were developed and the researchers iterated back and forth and then mapped each element to root causes, enabling the firm to identify opportunities to improve the customer experience.

The sixth and final step is Model Validation. McColl-Kennedy et al. (2019) employed five different tests: (1) a manual linguistics validation; (2) a second dataset validation; (3) a second firm validation; (4) feedback from the customer experience team at the focal organization and (5) a CHAID analysis.

By using chi-square automatic interaction detection (CHAID) classification technique, McColl-Kennedy et al. (2019) were able to predict to what extent customers were satisfied with the customer experience and generate meaningful insights. For example, this technique enables a firm to identify critical touchpoints from the customer’s perspective, including potentially new touchpoints previously unknown, to understand what really matters to the customer at each touchpoint, map each touchpoint to its root cause, that is, the specific firm action or strategy, and finally to take specific actions to improve the experience at each touchpoint, as well as the overall CX.

Their model is able to uncover customers who are at risk of leaving the firm, even customers who give high satisfaction scores (or NPS scores). Customers with high satisfaction scores normally would be viewed by an organization as “satisfied”, or those with high NPS scores would be deemed “very likely to recommend”, and therefore not identified by the firm as requiring attention. However, McColl-Kennedy et al. (2019) demonstrate that these customers are voicing their concerns in the free text comments and require follow-up by the organization to address their concerns. Relying solely on the numeric scores gives an incomplete picture of the true feelings of the respective customers.

Further, the text mining model enabled an entire “hidden” segment of supposedly highly satisfied customers to be identified. Analysis showed that 42% of customers who give scores of 9.5 and above (out of 10) actually complained. Customers who give scores between 7 and 9.4 (44%) complain too. Complaints from customers who gave satisfaction scores of 7 or greater were often ignored by organizations despite accounting for a significant portion of sales. Sales figures indicated that when these customers’ concerns were not addressed sales went down markedly. For instance, one so called “satisfied” customer reduced purchases from over $200,000 to less than $2000. A key takeaway is that ignoring small details that can be identified through the text analytics model, can mean big losses for firms.

9 Conclusion

This chapter has highlighted the importance of using multiple metrics to measure and manage customer experience. New technologies, including AI and text mining, offer organizations today with an efficient way of delving deeply into what customers are really saying, thinking and feeling about their experience with the organization and how their experience could be improved, and importantly what things should be improved in the experience.

Attention was drawn to the need to develop a conceptual framework to guide the text mining such as that developed and tested by McColl-Kennedy et al. (2019) and a practical step-by-step guide for organizations to implement was provided. Technology per se does not fix customer experience problems but it can be an enabler of better customer experience provided the technology is guided by a sound conceptual framework. We encourage both researchers and practitioners to re-evaluate their approach and the measures they are currently using in their attempt to better measure and manage the customer experience. We also encourage researchers and practitioners to consider using the new AI technologies that can enable free text comments provided by customers, in real time across the range of channels and over multiple touchpoints, to be more easily analyzed. The bottom-line customer experience is a key differentiator in today’s highly competitive world.