A multi-stage satisfaction index estimation model integrating structural equation modeling and mathematical programming

Abstract

In this study, a satisfaction index estimation model is proposed integrating structural equation modeling and mathematical programming methods with fuzzy customer data. Firstly, a deep literature survey is conducted in this field of study. Then, a new model is proposed by taking into consideration gaps in the literature. The estimation model is composed of five stages and first stage is building conceptual model in which measurement and latent variables are introduced. At the second stage, a fuzzy evaluation method is developed for decreasing subjectivity in customer data. At the third stage, for measurement variables that are directly observed, a measurement model is developed with Linear Structural Relations. In the solution of the measurement model maximum likelihood algorithm is used. In the solution of structural model that is composed of latent variables that are not directly observed, a mathematical estimation model is developed in this study at the fourth stage. Mathematical model is coded in ILOG Cplex Optimization Studio. In the mathematical model that minimizes estimation errors, structural relations and measurement variable weights (precedence coefficients) are defined as constraints. At the fifth and last stage, index scores are calculated with mathematical model outputs. Application of the model is carried out in public sector at a local government service point. In the application model, service quality, innovation, communication, satisfaction and cost perception dimensions are used. Application results are discussed for both measurement and latent variables in detail. The results of model we developed are also compared with an alternative model outcomes and we show that we achieve optimum estimation capability with minimum estimation errors.

Introduction

For an effective management of customer relationship, an institution must measure and evaluate customer satisfaction with causes and effects in a systematic way. Customer satisfaction index (CSI) is a systematic cause-and-effect model of advanced customer satisfaction analysis. CSI models are used by several private and public institutions for developing key customer strategies throughout the world. Index values are based on predictions of customer evaluations. An effective customer satisfaction analysis provides significant advantages for companies especially in gaining competitiveness. In order to reach these objectives primarily companies need to identify and analyze their customers. In this respect, effective communication and commitment to customers and changing market conditions is of great importance to increase the level of customer satisfaction.

CSI is a standard of customer satisfaction measurement that is used as national and individual purposes. CSI models are used for comparing customer satisfaction performance of industries, sectors, companies and government institutions. CSI models are mainly focused on the basic objectives of economic returns, economic stability, welfare and economic outputs (Grigoroudis and Siskos 2004). CSI Analysis is used for measuring customer satisfaction levels, taking counter actions for the low satisfaction points and improving high satisfaction points. When the customer becomes the focus of organization and if this organization gains more satisfied customers, then high satisfaction contributes in both internal and external processes of a company (Aktepe et al. 2015). CSI models, have attracted hundreds of firms and institutions in the world since 1992 starting with Sweden Customer Satisfaction Barometer. Today, in private and public sector, a lot of countries use these models and conduct their strategies according to index results by listening to the voice of customers with scientific models.

CSI is a systematic approach that analyses causes and effects of customer satisfaction with mathematical estimation models. CSI, beyond customer satisfaction measurement with classical methods, produces index value between 0 and 100. The index score shows company and sector performance of satisfaction level. In competitive structure of business life, it becomes a significant indicator of performance of company and products. The index models give us wide range of opportunities for developing key strategies of customer satisfaction.

Method of satisfaction evaluation is very significant for extracting key strategies of the model (Kwong and Bai 2002; Risdiyono 2013; Usmanij et al. 2013; Sun and Kim 2013; Wang et al. 2015). Therefore, in this study, first we categorize CSI estimation methods used in the literature. This classification enabled us to develop a novel approach for CSI estimation by finding out gaps in the previous models. In the literature CSI models are mainly based on cause-and-effect design and developed with various methods. We classify the related literature into four groups which are: Statistical estimation methods, linear programming, non-linear programming and fuzzy index estimation methods.

Statistical estimation methods

The root of CSI models dates back to Swedish Customer Satisfaction Barometer (SCSB)-first introduced in 1989-developed by Fornell (1992). As a second significant progress, American Customer Satisfaction Index (ACSI) was brought in this literature by Fornell et al. (1996)-first introduced in 1994. ACSI is the most widely used model of CSI in private sector (both for manufacturing and service institutions) throughout the world and it is the most cited model among satisfaction index models in the literature. In these two first models, satisfaction is modeled with antecedent and consequent latent variables of satisfaction using structural equation modeling (SEM). In ACSI model (Fornell et al. 1996), customer expectations, perceived quality and perceived value are antecedents of satisfaction. Customer complaints and customer loyalty are modeled as consequences of customer satisfaction. After that various applications of other national CSI models are carried out by Kristensen et al. (2000) and Ciavolino and Dahlgaard (2007) developing European Customer Satisfaction Index (ECSI). In ECSI models, different from SCSB and ACSI, “Image” latent variable was included in the structural equation model of satisfaction and “customer complaints” was removed from the model. Another study is carried out by Turkyilmaz and Ozkan (2007). In this study they use partial least squares algorithm developed by Chin (1998) and develop an index for mobile phone brands. In this study, differently from ACSI and ECSI the cause-and-effect relationship between customer expectations and perceived value is removed the model, asserting a weak relationship between these latent variables. Cause-and-effect approach of these satisfaction models were programmed as a SEM and solved with partial least squares method which is based on statistical modeling approach.

Linear programming approaches

Another method that attracted the authors is linear programming which differs from SEM approach. CSI estimation with linear programming is conducted in two different ways:

• MUSA method and its extensions Grigoroudis and Siskos (2002), develop a new perspective on CSI models and propose a linear programming (LP) model, called as multicriteria satisfaction analysis (MUSA). This approach differs from previous methodologies as using ordinal values of customer evaluations which aims to decrease subjectivity of customer evaluations. In this study, ordinal regression equations are transformed to an LP model. MUSA method is based on following principles: Customers’ global satisfaction depends on a set of variables and an additive formula is used in order to aggregate partial evaluations in a global satisfaction measure. Improving MUSA method, Angilella et al. (2014) develop multicriteria customer satisfaction analysis with interacting criteria (MUSA-INT). In this study authors include positive and negative interactions among satisfaction criteria. MUSA-INT has two different characteristics from MUSA method: In MUSA-INT different ordinal scales can be used and it uses a set of utility functions.

• Data envelopment analysis (DEA): Another LP approach was developed by Bayraktar et al. (2012) using DEA. In this study, latent variables of ECSI are categorized as input and outputs customer satisfaction and final efficiency scores of different companies are compared as satisfaction scores.

Non-linear programming methods

Türkyılmaz (2007), develops an artificial neural network model for customer satisfaction index estimation of which application is carried out for mobile phone brands in Turkey. Al-Nasser et al. (2011) use non-linear principal component analysis and factor analysis for customer satisfaction index estimation. They state loyalty, complaint, expectation, image and service quality as the main CS factors of their model. In their study, they develop Jordan Customer Satisfaction Index (JCSI) model.

Fuzzy index estimation approach

Liu et al. (2008) develop a fuzzy inference system for satisfaction index estimation. In this study, they develop a system approach for CSI model with input and outputs. They propose a fuzzy inference method for customer satisfaction index estimation. ECSI model is used for an application in e-commerce. However, the model and application of this approach is very limited.

The major deficiency of literature efforts on CSI is the lack of maintaining minimized customer subjectivity and minimum estimation errors at the same time. Therefore, in this study we propose a new programming model for estimating CSI with fuzzy customer evaluations minimizing estimation errors for more reliable and robust customer strategies. The estimation model brings significant contributions in this field of study. With the help of the model, we can find weights of measurement variables of a latent variable with minimized errors which is a key success factor in producing reliable indexes. In addition the model enables us to find coefficients of prediction equations that contribute to extend evaluation of index results. The model is also tested with data of a comprehensive survey and application results are included. The 5-stage model that we develop has following novel properties:

• With this study we bridge the gap in decreasing subjectivity of customer evaluations on satisfaction points and we have a more objective evaluation advantage of customer satisfaction. We accomplish this by using a fuzzy evaluation method. We call this method as pessimistic–optimistic approach which is discussed in the next section of the paper.

• A significant contribution of our study is that it is step-by-step designed. Structural model is created independently from confirmatory factor analysis (CFA). This brings many advantages; we create confirmatory factor analysis with reflective blocks and we can calculate weights of latent variables with separate formative blocks. These are very significant characteristics for customer strategy development.

• Another innovation point of the study is minimizing model estimation errors by developing a non-linear programming model with fuzzy customer evaluations. With the help of this model, we accomplished minimum model errors. We measure estimation strength of the model with an objective function that minimizes Type-I (e) and Type-II (r) estimation errors which are defined in the “Application of the model in a service system” section of the study.

• A further extension of our study is developing a new post-analysis method. After CS index calculations we evaluate index results, obtained at different time intervals, and make conclusions about increase and decrease in scores by using coefficients of determination.

The paper is organized as follows: In the next section we discuss the 5-stage model we developed for index estimation. Here we discuss index calculation method proposed for calculating index scores and structure of the model. In the “Application of the model in a service system” section of the study, application of the model in a local government point is presented. There is detailed information about application and outcomes of the model here. Here we also compare results of the model with an alternative model which enables readers make inferences about results. In the “Interpretation and post-analysis of Satisfaction Index results” section we interpret the results of application and conduct a post-analysis on index scores. Finally, we conclude with discussing significant aspects of the model and key customer strategies according to results.

Methodology: multi-stage satisfaction index estimation model

In this section, methodology of CSI estimation model is explained step by step. We develop a 5-stage methodology for CSI estimation (Aktepe 2015). This is summarized in Fig. 1 below.

Fig. 1

Multi-stage satisfaction index estimation model

Creating conceptual model (1st stage)

The start point of the model is development of conceptual (theoretical) model of satisfaction index. Conceptual model contains latent and measurement variables that are used to evaluate customer satisfaction and related factors. With conceptual diagram, we present dimensions of the model. Each dimension in the model is called as latent variable. Latent variable is a hidden factor of satisfaction related dimensions. A group of measurement variables that are grouped together constitutes a latent variable. Measurement variable, on the other hand, is an observed factor of satisfaction model. The survey question of application is example of a measurement variable.

The conceptual model is created with expert views and literature survey. Experts are asked which dimensions must be included in the related model and correspondingly a thorough literature survey is carried out. After that alternative models are created and finally best-fit model is decided after statistical confirmatory factor analysis. Below in Fig. 2, how we create conceptual model of the study is shown.

Fig. 2

Method of creating conceptual model

Fuzzy pessimistic–optimistic approach (2nd stage)

In this stage we discuss fuzzy pessimistic–optimistic approach we developed for more objective customer evaluations. This method is discussed in step-by-step below.

Development of fuzzy scale

At the second stage, first we create a fuzzy scale of customer evaluations for data collection. Fuzzy logic, firstly introduced to the literature by Zadeh (1965), has been used for many decision making problems. Binary logic—in other words—classical logic is based on certainty theory. However, real life is quite uncertain by its nature (Şen 2004). Customer satisfaction can also be considered as a vague concept and may be defined, for example, as the degree of customer happiness that a customer experiences with a company’s product or service and it is a function of the gap between (Woodruff and Gardial 1996). Customer feelings, customer decisions on degree of satisfaction do not depend on precise information and fuzzy in nature. Customer evaluations are complicated and based on linguistic variables (Şen 2004). For eliminating this vagueness and for having more objective results fuzzy scales are used in decision models (Shen et al. 2001; Chen 2005; Liu et al. 2008). Li (2013), shows drawbacks in classical Likert scaling in questionnaires and proves superiority of fuzzy scales in his study. He uses Consensus Analysis and proves that using fuzzy scale instead of classical Likert scales brings the advantages of capturing the lost information, regulating the distorted information and providing more accurate measurement.

A triangular fuzzy number is shown as $\tilde{A}=(a_{1},a_{2},a_{3}$). Here, $a_{1}$ shows the minimum value that $\tilde{A}$ can take, $a_{2}$ represents the most probable value and $a_{3}$ represents the maximum value. Membership function of a triangular fuzzy number defined like this is depicted in Fig. 3.

Membership function of a triangular fuzzy number is defined as follows (Eq. 1):

$$\begin{aligned} \mu _{\tilde{A}}(x)=\left\{ \begin{array}{ll} 0, &{}\quad x<a_{1} \\ {\frac{x-a_1}{a_2-a_1},} &{}\quad a_1\le x\le a_2 \\ {\frac{a_3-x}{a_3-a2},} &{}\quad a_2\le x\le a_3 \\ 0, &{}\quad x>a_3 \\ \end{array}\right. \end{aligned}$$

(1)

In this study, we also develop a linguistic scale for determining the degree of satisfaction due to reasons discussed above. Using a fuzzy scale for customer evaluations enabled us to get better results in consistency coefficient statistics. These statistics are shown in the application section of the study. In Fig. 4, the fuzzy scale that we use in our research is shown. Here we use triangular fuzzy numbers.

Fig. 3

Triangular fuzzy number

Each number in the set has $\mu _{\tilde{A}}(x$) membership function as shown on the figure. If the membership function is 0, then variable x does not belong to related set, if it is 1, then variable x is absolutely in the related set. The values between 0 and 1, show the membership degree of x.

Fig. 4

Fuzzy scale

A linguistic variable is one whose values are words or sentences in a natural language (Moon and Lee 2005). In our study, we presented linguistic expressions such as very low, low, medium, high and very high for customer satisfaction levels.

Defuzzification of data by pessimistic–optimistic combination method

At the 3rd stage of the study, we conduct CFA with crisp data. Before developing measurement and structural models, we develop a defuzzification method for defuzzifying linguistic customer evaluations. Customer replies to satisfaction index survey questions are the fuzzy input data for model. In measurement and structural models, crisp data are used. The pessimistic–optimistic combination approach that we develop is used here for defuzzification, shown in Fig. 5. This method is based on $\alpha$-cut approach. We consider both pessimistic and optimistic behavior of the customer. Pessimistic value ($P_{i}$), in Eq. 2, is defuzzified from the left side of fuzzy number and optimistic value ($O_{i}$), in Eq. 3, is defuzzified from the right side. Defuzzified value ($Q_{i}$) is calculated as geometric mean of $P_{i}$ and $O_{i}$ as shown in Eq. 4.

$$\begin{aligned} P_i= & {} a_1 \cdot \alpha +a_2 \cdot (1-\alpha ) \end{aligned}$$

(2)

$$\begin{aligned} O_i= & {} a_3 \cdot \alpha +a_2 \cdot (1-\alpha ) \end{aligned}$$

(3)

$$\begin{aligned} Q_i= & {} \sqrt{P_i \cdot O_i } \end{aligned}$$

(4)

Fig. 5

Pessimistic–optimistic combination shown on a triangular fuzzy number

Creating measurement and structural models (3rd Stage)

Creating measurement model

After achieving defuzzified values of customer evaluations by pessimistic–optimistic combination method, CFA is used to create measurement model and finding out latent variables. Latent variable is hidden dimension of satisfaction model and measurement variable is observed factor of a latent variable. CFA is carried out for grouping measurement variables ($x_{11}$ to $x_{nm}$) to latent variables ($x_{1}, x_{2}, {\ldots },x_{m}$) with reflective blocks as shown in Fig. 6. In reflective blocks, each measurement variable is associated with latent variable separately.

CFA is a statistical technique used to verify the factor structure of a set of observed variables. CFA allows the researcher to test the hypothesis that a relationship between observed variables and their underlying latent constructs exists (Schumacker and Lomax 1996). After customer data are defuzzified, variables that measure latent variables are determined with CFA. CFA helps us to find which measurement variable represents a latent variable more effectively. CFA is used to estimate the model parameters and examine the factor structure. It is built by using the maximum likelihood estimation method developed by Chou and Bentler (1996) which is the most commonly used approach in SEM. Performance of CFA model is checked with overall model fit indices in LISREL software (Aktepe et al. 2015). To build CFA model SIMPLIS language of LISREL 8.80 (Jöreskog and Sörbom 1993) software is used in this study. If all goodness-of-fit statistics defined in the studies of Bentler and Bonett (1980), Byrne (1998), Jöreskog and Sörbom (2006), and Çokluk et al. (2012) are achieved, then structural model is created in the second part of 3rd stage.

Fig. 6

A sample representation of measurement model

Fig. 7

A sample representation of structural model

Creating structural model

After determining latent variables with CFA, structural model is created with independent and dependent latent variables. Structural model shows the cause-and-effect relations among latent variables. In structural model, formative blocks are used. In formative blocks, all of the measurement variables in a dimension are associated with latent variable. A sample structural model is shown in Fig. 7. Here, x is independent latent variable. $y_{1l}$ represents Type-I dependent variable which is affected by only one latent variable and $y_{2j}$ represents Type-II latent variable which is affected by more than one latent variable. $\hbox {a}_{\mathrm{i}}, \hbox {b}_{\mathrm{i}}, \hbox {c}_{\mathrm{i}}, \hbox {d}_{\mathrm{i}}$ and $\hbox {f}_{\mathrm{i}}$ represent path coefficients of measurement variables of independent latent variables, $w_{1ls}$ represent path coefficients of measurement variables of Type-I dependent latent variables and $w_{2jp}$ represent path coefficients of measurement variables of Type-II dependent latent variables. $\gamma _{1}$ represent path coefficient from independent latent variable to Type-I latent variable, $\gamma _{2 }$represent path coefficient from independent latent variable to Type-II latent variable and finally $\beta$ represent path coefficient from Type-I latent variable to Type-II latent variable.

Note that measurement model is built as reflective blocks in LISREL. However, in structural model there is a transformation from reflective blocks to formative blocks. This transformation enables us to show cumulative effects in mathematical model that is discussed in the next sub-section.

Solution of structural model with mathematical model (4th stage)

As the 4th stage of methodology, in order to minimize estimation errors and finding weights, coefficients of the structural model, a mathematical model is built. With the help of this model, weights of Type-I and Type-II latent variables, which are critical in assessment of satisfaction results, can be found with minimum estimation errors. The indices, decision variables, objective function and constraints of the model are defined below:

Indices

i::

customer indice,

m::

the number of customers included in the analysis,

n::

the indice of independent latent variable,

l::

the indice of Type-I dependent latent variable,

j::

the indice of Type-II dependent latent variable,

$\hbox {c}_{1}$::

the number of Type-I latent variables,

s::

the number of measurement variables that are bound to Type-I dependent latent variable,

$\hbox {c}_{2}$::

the number of Type-II latent variables,

p::

the number of measurement variables that are bound to Type-II dependent latent variable,

$\hbox {c}_{3}$::

the number of independent latent variables.

Variables

$x_{in}$::

the value of independent variable in nth dimension for customer i,

$y_{1il}$::

the value of Type-I dependent variable in lth dimension for customer i,

$\hbox {y}_{\mathrm{2ij}}$::

the value of Type-II dependent variable in jth dimension for customer i.

Decision variables

$e_{ij}$::

Type-II estimation model errors,

$r_{il}$::

Type-I estimation model errors,

$a_{1n}, b_{1n},a_{2n}, b_{2n}$::

Estimation model coefficients of independent latent variable,

$a_{3l}, b_{3l}$::

Estimation model coefficients of Type-I dependent latent variable,

$v_{ls}$::

Weights of Type-I dependent latent variable,

$w_{jp}$::

Weights of Type-II dependent latent variable.

The objective function of mathematical model is shown in Eq. 5. With this objective function, Type-I and Type-II estimation model errors are minimized. The average mean squared estimation error is minimized with this function. This function comprises the whole estimation error of the model. It minimizes total estimation error.

$$\begin{aligned} \min .\left( {\sum _{i=1}^m {\left( {\sum _{j=1}^{c_2 } {(e_{ij} ^{2})\Big /m} +\sum _{l=1}^{c_1 } {(r_{il} ^{2})\Big /m} } \right) } } \right) \Big /(c_1 +c_2 )\nonumber \\ \end{aligned}$$

(5)

Weight constraints are shown in Eqs. 6 and 7 for Type-I and Type-II variables respectively. The sum of weights for a latent dependent variable is equal to 1 according to these constraints. This equality is not used for independent variables.

$$\begin{aligned} \sum _{s=1}^{c_1 } {v_{ls} }= & {} 1 \quad \forall l \end{aligned}$$

(6)

$$\begin{aligned} \sum _{p=1}^{c_2 } {w_{jp} }= & {} 1 \quad \forall j \end{aligned}$$

(7)

Weight inequality constraints for Type-I and Type-II variables are shown in Eqs. 8 and 9 respectively. These constraints enable us to assign different weights for decision variables v and w.

$$\begin{aligned}&v_{ls}^{g_1 } \ne v_{ls}^{h_1 } {\left( {g_1 ,h_1 \le c_1 \wedge g_1 \ne h_1 } \right) } \end{aligned}$$

(8)

$$\begin{aligned}&w_{jp}^{g_2 } \ne w_{jp}^{h_2 } \left( {g_2 ,h_2 \le c_2 \wedge g_2 \ne h_2 } \right) \end{aligned}$$

(9)

In the mathematical model, separate constraints are defined for each structural relation in the previously defined structural model. Constraints listed below are named as structural constraints which are very significant for transformation of structural blocks to mathematical constraints. The structural constraints that are built considering relations in 3rd stage, are shown in Eqs. 10 and 11 for Type-I variables and in Eqs. 12 and 13 for Type-II variables.

$$\begin{aligned}&-\sum _{n=1}^{c_3 } {\left( {a_{1n} \cdot x_{in} +b_{1n} } \right) } -r_{il} \le -\sum _{l=1}^{c_1 } {\left( {v_{ls} \cdot y_{1il} } \right) } \quad \forall i \end{aligned}$$

(10)

$$\begin{aligned}&\sum _{n=1}^{c_3 } {\left( {a_{1n} \cdot x_{in} +b_{1n} } \right) } -r_{il} \le \sum _{l=1}^{c_1 } {\left( {v_{ls} \cdot y_{1il} } \right) } \quad \forall i \end{aligned}$$

(11)

Equation 11 is the absolute of Eqs. 10 and 12 is the absolute of Eq. 11. This property enables us having positive values for decision variables $e_{ij}$ and $r_{il}$.

$$\begin{aligned}&-\sum _{n=1}^{c_3 } {\left( {a_{2n} \cdot x_{in} +b_{2n} } \right) } -\sum _{l=1}^{c_1 } {\left( {a_{3l} \cdot y_{1il} +b_{3l} } \right) } -e_{ij} \nonumber \\&\quad \le -\sum _{j=1}^{c_2 } {\left( {w_{jp} \cdot y_{2ij} } \right) } \quad \forall i \end{aligned}$$

(12)

$$\begin{aligned}&\sum _{n=1}^{c_3 } {\left( {a_{2n} \cdot x_{in} +b_{2n} } \right) } +\sum _{l=1}^{c_1 } {\left( {a_{3l} \cdot y_{1il} +b_{3l} } \right) } -e_{ij} \nonumber \\&\quad \le \sum _{j=1}^{c_2 } {\left( {w_{jp} \cdot y_{2ij} } \right) } \quad \forall i \end{aligned}$$

(13)

Finally sign constraints are defined below in Eq. 14.

$$\begin{aligned} c_1 ,c_2 ,c_3 ,g_1 ,h_1 ,g_2 ,h_{1} ,t\in Z^{+} \quad w_{jp} ,v_{ls},>0 \end{aligned}$$

(14)

The mathematical model built in 4th stage to estimate satisfaction index has several characteristics. The objective function enables us doing minimum mistake in estimation. Therefore we can obtain optimum results in terms of error minimization. Constraints of the model are linked each other which maps all cause and effect relations in a structural model. In a sense, it is a mathematical programming reflection of conceptual and structural models.

Fig. 8

Dimensions in the conceptual model

Calculation of satisfaction index scores

After comleting 4th stage, satisfaction index scores are calculated in the 5th stage. Index scores are normalized between 0 and 100. It is calculated with below equations using mathematical model outputs. Coefficients of independent variable ($a_{1n}, b_{1n}, a_{2n}, b_{2n}$) and weights of dependent varibles ($v_{ls}, w_{jp}$) are used in index calculation. Index score is calculated for each type of latent variable. The formula for independent variable index, $I_{0}$, is in Eq. 15, the formula for Type-I dependent variable index, $I_{1l}$, is in Eq. 16 and the formula for Type-II dependent variable index, $I_{2j}$, is in Eq. 17.

$$\begin{aligned} I_0= & {} \sum _{n=1}^{c_3 } {\left( {(\bar{{x}}_{in} )\cdot ((a_{1n} +b_{1n} +a_{2n} +b_{2n} )/c_3 )} \right) } \cdot \frac{100}{u} \nonumber \\\end{aligned}$$

(15)

$$\begin{aligned} I_{1l}= & {} \sum _{l_s =1}^s {(\overline{y_{1il} } \cdot v_{ls} } )\cdot \frac{100}{u} \end{aligned}$$

(16)

$$\begin{aligned} I_{2j}= & {} \sum _{j_p =1}^p {(\overline{y_{2ij} } \cdot w_{jp} } )\cdot \frac{100}{u} \end{aligned}$$

(17)

u is the maximum defuzzified value of fuzzy scale which is calculated in the 1st stage. This value is used for normalization of index scores between 0 and 100.

Application of the model in a service system

Application of the model is carried out with data of a local government satisfaction survey. The survey was carried out in the scope of a scientific project supported by The Scientific and Technological Research Council of Turkey (TUBITAK). The survey is conducted with 400 respondents. There are 15 questions in the original survey.

Conceptual model of local government satisfaction index

First of all, views of an expert team are evaluated for creating conceptual model of application. The expert team is composed of ten experts (one local governor, two industrial engineers, two academic members, two sociologists, two econometrists, one statistician who is at manager position in National Statistical Institute and one expert from National Productivity Center). We conducted face-to-face interviews with each expert and gathered their opinions attentively. In addition we benefited from literature studies that we discuss in the “Introduction” section of the study. These intensive efforts are synthesized and we produced alternative models.

The alternative models are tested with pilot applications and final conceptual model is determined after this thorough processes. Model dimensions in the final conceptual model are explained with Fig. 8.

Data collection and defuzzification of application data with pessimistic–optimistic approach

In this section, we discuss data collection process and defuzzification of application data.

Data collection and explanations on data set

The application is professionally carried out in a local government point in Turkey. Application data are collected with face-to-face interviews with financial and expertise support from The Scientific and Technological Research Council of Turkey-TUBITAK. The items in questionnaire used for application is depicted in “Measurement model of application” section. The sum of target population is 195.595, involving 31 different neighborhoods. Sample size is calculated with following equality (Yazıcıoğlu and Erdoğan 2004) given in Eq. 18.

$$\begin{aligned} n=\frac{N\cdot p\cdot q\cdot z^{2}}{p\cdot q\cdot z^{2}+(N-1)\cdot d^{2}} \end{aligned}$$

(18)

Here n is sample size, N is population size (195.595), p is observe ratio of research units, q is calculated as $1-p$. z represents z-value (it is 1.96 in 95 % confidence interval) and d is sampling error ratio. In the application, the parameters are determined as follows: N: 195.595, p and q: 0.5, z: 1.96 and d: 5 %. With these parameter values sample size (n) is calculated: 383. In addition, there is a practical way of calculating sample size in survey researches. There exists a program in this link: http://www.surveysystem.com/sscalc.htm. With this program same value of sample size (383) is calculated and verified. In our application we decided to apply satisfaction survey to 400 citizens ($n+17$), due to probable missing or some inconsistent data causing from wrong understanding of subjects.

Table 1 Distribution of sample size to neighborhoods with Pre-Sampling Unit

Sampling is carried out systematically in three stages. First stage is Pre-Sampling Unit and determined as follows: Preferred sample size (400) is distributed to 31 different neighborhoods according to their population percentage in total population (195.595). Sample size for each neighborhood ($n_{h}$) is calculated with Eq. 19. Here, $n_{h}$ represents sample size of a neighborhood, n is sample size (400) $N_{h}$ represents the population of each neighborhood and finally N is total population size (195.595). The Secondary Sampling Unit is households. Households are chosen randomly in each neighborhood by using random numbers. This is important point of application for ensuring enough heterogeneity (Fink and Kosecoff 2005). Final Sampling Unit is determined from subjects aged above 18 and randomly according to gender (Table 1).

$$\begin{aligned} n_h \cong \frac{n}{100}\cdot \frac{N_h }{N} \end{aligned}$$

(19)

After survey application, only one respondents’ data are removed from analysis because of blank answers. Total effective sample size is 399. Table 2 provides information about sample characteristics. Of the 399 total number of respondents, 184 (46 %) were male and 215 (54 %) were female customers. Average age is 29.5. This gender and age composition is a reasonable representation of the citizens in an average size local government in Turkey. In addition, the majority (78 %) of the respondents had a high school degree or higher, which we believe is another important characteristic of the customer group who can make reasonable evaluations of satisfaction and loyalty questions in the survey. The reliability of the data is checked by conducting reliability analysis in SPSS statistical package (SPSS 2007). Most reliability scores were within the suggested levels ($>$.70) in the literature.

Table 2 Sample profile

Now we want to give information about data set we used for application. The final data set used in the study is composed of totally 399 customer replies to 15 survey questions. For application analyses we have a linguistic data matrix of 399 rows $\times$ 15 columns. A linguistic scale is used (From “Very Unsatisfied”, “Unsatisfied”, “Neutral”, “Satisfied”, “Very Satisfied”). The linguistic data set is then converted into fuzzy data set of 399 rows $\times$ 45 columns. For each linguistic statement a triangular fuzzy number is generated (3 $\times$ 15 $=$ 45 columns totally). After that, we produce 11 different defuzzified data sets with different $\alpha$-cut levels. Final data set is chosen according to highest Cronbach alpha value among these 11 different crisp data sets. Below in Fig. 9, we summarize these processes.

Fig. 9

The process of achievement of final data set

Data of application are collected with a professional survey application. First part of the survey is composed of demographical variables which were summarized in Table 2 above. Second part of the survey is composed of customers’ replies to questions. Questions are formed as measurement variables listed in “Measurement model of application” section below. Variable types used in the questionnaire form are summarized in Table 3 below:

Table 3 Data types in the data set of application

Defuzzification process

In the survey application, respondents’ replies are collected with fuzzy linguistic scale given in Fig. 4. After data collection, fuzzy customer data are defuzzified with pessimistic–optimistic combination method discussed in the previous section. Here, various $\alpha$-cut levels are used for defuzzification and best $\alpha$-cut level is found by testing consistency of defuzzified data sets using SPSS statistical analysis software.

According to Nunnally (1967, 1978) and Murphy and Davidshofer (1998); Cronbach Alpha value must be between 0.90 and 0.95 in wide-ranging survey application research. Table 4 provides information about consistency values of some defuzzified data sets. After several trial-and-error, best $\alpha$-cut level is determined as 0.25 with best Cronbach alpha value. So, in the next sub-sections, data defuzzified with 0.25 $\alpha$-cut level are used for applications.

Table 4 Cronbach alpha values of some defuzzified data-sets

Measurement and structural models of application

At the 3rd stage of application, measurement model is created with CFA using SIMPLIS commands in LISREL software. After that, structural model is created.

Measurement model of application

We conduct CFA by using LISREL 8.80 (Linear Structural Relations) software created by (Jöreskog and Sörbom 1993, 1996). LISREL is a statistical language that interfaces with statistical applications. We use SIMPLIS codes in LISREL application for CFA application.

In local government sector application, firstly in accordance with the opinion of experts, questions between 1 and 5 are associated with service quality, questions between 6 and 8 are associated with innovation, questions 9 and 10 are associated with citizen satisfaction, questions between 11 and 13 are associated with communication and questions 14 and 15 are associated with cost perception. The measurement variables and associated latent variables are depicted in Table 5. While designing measurement variables and associated latent ones, local government dynamics are thoroughly examined by expert team. After pilot survey applications, the best combination is determined as shown in below table.

Table 5 Measurement variables of application and associated latent variables

CFA model is developed to test measurement model created by experts. CFA is used to estimate the model parameters and verify the latent variables of prediction. The measurement model is estimated using the maximum likelihood estimation method in LISREL which is the most commonly used approach in SEM (Orel and Kara 2013). In Fig. 10, standardized solution result of final CFA model is shown.

Fig. 10

CFA model of application in LISREL

Table 6 Goodness-of-fit statistics for measurement model of application

Fig. 11

Structural model of application

Table 7 Values of decision variables obtained as a result of mathematical model

In CFA model, there are five dimensions of model: Service Quality (SQ), Innovation (I), Citizen Satisfaction (CS), Communication (C) and Cost Perception (CP). Measurement variables S1, S2, S3, S4 and S5 are in dimension SQ; S6, S7 and S8 are in dimension I; S9 and S10 are in dimension CS; S11, S12 and S13 are in dimension C and S14 and S15 are in dimension CP. The CFA model created produces acceptable and good results according to statistical fit indices. The threshold values for good fit statistics is defined in the studies of Bentler and Bonett (1980), Bryne (1998), Jöreskog and Sörbom (2006), Çokluk et al. (2012). Statistical fit results are shown in Table 6 and all of them show that we have a good-fit statistical model. Another significant factor of statistical fit is t-values calculated automatically by LISREL. t-values for each path coefficient must be higher than 1.96 in for an acceptable model. In our model, the lowest t-value is 9.12 in the CFA model. This also shows the statistical power of model.

Structural model of application

Both goodness-of-fit statistics and t-value results show that we obtained statistically good level of model. Determination of latent variables and measurement variables are completed. Now, at the second part of 3rd stage, we develop structural model of application depicted in Fig. 11. In this Figure, we see relationships among latent variables. These relations will be solved and assessed by mathematical model developed in this study discussed in the next sub-section.

Solution of structural model of application with mathematical model

In this stage of application, first, structural model of application is transformed into mathematical model. For solving mathematical model, equations defined in Chapter 2.4, Equations between 5–14 are used. The mathematical model is coded in IBM ILOG Cplex Optimization Studio 12.6.0.0 software. The model can find optimum value under defined constraints. The value of objective function, weights of measurement variables and coefficients of independent latent variable is shown in Table 7. Values listed below are used for interpretation of results in the concluding sections.

According to model results, objective function is found as 0.459. This value is minimum cumulative value of estimation error. This shows the cumulative effect of estimation errors. In order to calculate an estimation error value for a dimension, cumulative estimaton errors of all causes and effects of this dimension are taken into consideration. We can see it in Eqs. 10, 11, 12 and 13 in the mathematical model. The model finds the optimum value for estimation error under defined constraints. We find minimum cumulative structural estimation error as 0.429 for CS; 0.456 for SQ; 0.673 for C and 0.555 for CS dimensions. In addition we can calculate v and w which are vectors for explaining weights for Type-I and Type-II measurement variables.

Fig. 12

Index scores for each dimension of application model

Fig. 13

An alternative comparison model of structural relations

Calculation of index scores of application

With the help of the model developed, index scores for each dimension can be calculated using Eqs. 15, 16 and 17 in “Calculation of satisfaction index scores” section (Fig. 12).

After the application, General Citizen Satisfaction Index is calculated as 45,125 over 100. Highest score is communication index with 57,535 and lowest score is 41,524 for Service Quality dimension. Innovation index is calculated as 47,876 and finally Cost Perception index is 43,634. An index takes a value between 0 and 100 (with the help of Eqs. 15, 16, 17).

Comparison of results with maximum likelihood method

In this part of the study, mathematical model (4th stage of the whole model) results are compared with another method. This is maximum likelihood estimation technique. This method was previously used for building CFA in this study. On the other hand, here, it is used for a different purpose. Here, the 4th stage of the model is developed in LISREL using maximum likelihood estimation algorithm in order to compare our model with it. By using same defuzzified data set with same latent variables, the structural model is solved in LISREL. For developing this comparison model, structural model shown in Fig. 11, is developed as a path diagram with reflective blocks in LISREL with SIMPLIS syntax (which is a tool in LISREL). Below in Fig. 13, we show structural model developed in LISREL. All goodness of fit statistics are in acceptable limits.

Fig. 14

Comparison of index results

Fig. 15

Comparison of estimation errors

Fig. 16

Weights of measurement variables in dimension SQ (from higher to lower)

Then new index scores are calculated by using mean of defuzzified customer replies and path coefficieints of new model. Estimation errors of this second model is calculated using cumulative error values in structural equations generated by LISREL. After that, we compare results of two models. This comparison enables us to show that we acquire minimum estimation error results with our optimization model.

Below we show application results of our mathematical model compared to maximum likelihood model in Fig. 14.

In order to understand which index result is more reliable, we compare mean absolute estimation error values of each approach. Below in Fig. 15, we show comparison of model estimation errors. Results show that mathematical model we developed produces very lower level of estimation errors in each dimension compared to results found by maximum likelihood outcomes in LISREL.

Here we compare mean absolute estimation errors of structural equations of our model (which were given in Table 7) with mean absolute estimation errors of structural equations of maximum likelihood algorithm built in LISREL. Mean absolute estimation error can take a maximum value of $u\times 5$ which is approximately 25 (u is used in Eqs. 15, 16 and 17 and 5 is the number of dimensions). It is better if it is close to 0. Our mathematical model produces optimum values for estimations.

Interpretation and post-analysis of satisfaction index results

The results of satisfaction index model developed in this study present several insights for customer strategies. We define two different strategy type with model outcomes. First we can prioritize customer requirements with Type-I and Type-II latent variable weights. Second, we analyze relationships among latent variables with determination coefficients.

Prioritizing latent variables and customer needs with mathematical model outcomes

Here, we prioritize dimensions from lowest index score to highest index score and also customer needs for each dependent dimension separately. Next to each measurement variable, we write weights of each measurement variable in parentheses. Higher value in the parentheses means it has more priority than others. This sorting enables us to see priority of customer needs and local government must firstly deal with prior requirements.

Service quality (SQ) dimension (Index Score: 41,524)

Five measurement variables in this dimension are sorted according to weights or importance degree in Fig. 16 ($v_{1i}$):

Cost perception (CP) dimension (Index Score: 43,634)

Two measurement variables in this dimension are sorted according to weights or importance degree in Fig. 17 ($v_{3i}$):

Fig. 17

Weights of measurement variables in dimension CP (from higher to lower)

Fig. 18

Weights of measurement variables in dimension CS (from higher to lower)

Citizen satisfaction (CS) dimension (Index Score: 45,125)

Two measurement variables in this dimension are sorted according to weights or importance degree in Fig. 18 ($w_{2i}$):

Communication (C) dimension (Index Score: 57,535)

Three measurement variables in this dimension are sorted according to weights or importance degree in Fig. 19 ($v_{2i}$):

Fig. 19

Weights of measurement variables in dimension C (from higher to lower)

Post-analysis of satisfaction index results

Calculation of satisfaction index scores and using them for customer strategies is a continuous effort. It is significant to maintain continuity of such efforts. In this point, interpretation of decrease and increase in index scores becomes important. With this conscious, in our study we include calculation of effect ratio of latent variables on each other in the model. This is calculated with coefficient of determination ($\hbox {R}^{2}$) shown in Eq. 20. Here $y_{i}$ is observed value for each customer, $\hat{{y}}_i$ is estimated value and $\bar{{y}}$ is average value. $\hbox {R}^{2}$ is a statistics that reflects what percentage of variance is caused by which latent variable. The remaining percentage of variance not explained by other latent variables is caused by other factors. It takes values 0 and 1. If it is close to 1, the model is better in explaining the variance.

$$\begin{aligned} R^{2}=\frac{\sum {(\hat{{y}}_i } -\bar{{y}})^{2}}{\sum {(\hat{{y}}_i } -\bar{{y}})^{2}+\sum {(y_i -\hat{{y}}_i } )^{2}} \end{aligned}$$

(20)

$\hbox {R}^{2}$ values of effects of each latent variable (dimension) in the structural model is calculated using SPSS software. $\hbox {R}^{2}$ values calculated are presented in Table 8.

Table 8 Structural relations in the model and interpretation of relations

Conclusion and discussions

When we analyze studies carried out on CSI in the literature, we discover that there are four significant points that we accomplished in our CSI estimation efforts.

• First, CSI estimations are based on customer evaluations and subjectivity of customer evaluations must be decreased. Therefore, in this study, we use a fuzzy evaluation method based on pessimistic and optimistic approach which produces better results in decreasing subjectivity of customer evaluations according to reliability comparison of scales. This is discussed and showed in the second section of the paper.

• Second significant contribution of our study is that structural model is created independently from confirmatory factor analysis. This brings many advantages: We create confirmatory factor analysis with reflective blocks and we can calculate weights of latent variables with separate formative blocks.

• Third, we minimize model errors by developing a non-linear programming model. We accomplished lowest model estimation errors. In this study, with the help of mathematical index estimation method we have several advantages over literature efforts. We can calculate weights of latent variables independently from confirmatory factor analysis outcomes. We can minimize estimation errors with a non-linear mathematical model and obtain better results than previous studies.

• Fourth, a step-by-step estimation model is created in 5 stages. By this approach, cumulative effect of estimation errors is not further transferred to next steps. This is a novel insight for CSI literature.

Application of the developed model was performed in a local government service point. There are two different strategies for each dimension of calculated index score. One of these is ranking of measurement variable weights and second strategy is investigating the effects of latent variables. As a result of the application, the lowest index score is found as 41,524 out of 100 for Service Quality dimension. One of the recommended strategies to improve service quality is to improve measurement factors according to their importance rate. Ranking by importance of measurement variables for service quality dimension: a) Social and cultural activities, b) Water, sewerage and solid waste services, c) Transportation services, d) Roadside, asphalt, pavement, e) Sub-structure and super-structure services and finally, Park and green field, building audit, housing and retrofit services . According to application results, 66.5 % of the change in service quality dimensions can be explained by the innovation dimension. Another strategy to increase the quality of services is also innovative practices. In this regard, developed recommendations for developing new service applications are:

i.:

To popularize the usage of the technological opportunities with new applications for mobile device and website at the local government,

ii.:

E-municipality applications should bring into conformity with the citizen profiles.

In second place, dimension of cost perception has 43,364 point out of 100. The ranking by importance level of measurement variables for dimension of cost perception: a) Rationality of utilizing public funding, b) Sufficiency of services in return taxes paid. According to application results, 86.1 % of the change in cost perception factors can be explained by the dimension of citizen satisfaction index. Developed recommendations to increase the dimension of cost perception scores are as follows:

i.:

Meeting the expectations of citizens for gathering information about the services offered with confidence,

ii.:

Giving priority to citizens requirements with a professional management approach.

In third place comes citizen satisfaction index for the public sector with 45,125. Ranking measurement variables of in citizen satisfaction index: a) General satisfaction level, b) Fulfillment of expectations. According to application results, 90.4 % of the change in citizen satisfaction index can be explained by the service quality and innovation dimensions. Suggestions which are developed to increase the satisfaction index score are:

i.:

Improving the level of quality of service with considering importance ranking in this size,

ii.:

Becoming open to change and innovation through the use of new technologies.

Innovation ranks fourth with 47,876 points. Innovation, which is independent and hidden variable, is not affected by another variable in the model. Measurement variables in innovation dimension: a) Equal and just management, b) Reachability of citizens to mayor and workers and c) Openness to change and innovation.

Finally, the communication dimension’s index is calculated with highest score: 57,535. Ranking of measurement variables in communication dimension: a) Sufficiency of personnel and background of management for dealing with citizens, b) awareness of citizens for municipality services, c) attitude and behavior of manager and workers. According to application results, 86.3 % of the change in communication can be explained by service quality dimension. The recommendations proposed to increase the communication score:

i.:

Service providers should be polite while communicating with citizens,

ii.:

To comply with the legal procedures for resolving disputes.

Additionally, when we just consider the index score itself, one reference point to compare the index score calculated is American public sector index (ACSI government model, 2015). Satisfaction index is not calculated in the public sector on a regular basis in many other countries. ACSI government index was 66.9 in 2011, 68.4 in 2012 and 66.1 in 2013. In local government service point that we performed our application, satisfaction index score was found as 45,125. When we compare this score to American scores, we see that model produces a consistent index score because the institution we carried out application is below Turkey’s and the world average in terms of quality and general satisfaction.

In future studies, before conducting satisfaction index analysis a classification study can be carried out for segmentation of respondents. Chen et al. (2007), for example, emphasize importance of customer segmentation by presenting concrete reasons. And customer-focused systems are closer to success compared to other systems as Wang et al. (2015) emphasize in their study. Second, for comparing and interpreting index results achieved in different time periods, change management (Ayhan et al. 2015) models can be used. It is a new approach used in production systems and methodology can be adapted to service systems in order to see reflections of change in index scores by time. In addition, the model can be applied to other private and governmental sectors together with some changes in measurement variables.