1 Introduction

Higher education is the spine of development and economic growth of any country. Given that the academic institutions are responsible for capacity building required for a country’s long term plans, the educational system in particular, is one of the pillars on which a country depends on to increase its productivity and thus efficiently execute its strategic plans. Engineering education is the activity of teaching knowledge and principles related to diverse professional practices of engineering. It includes the initial education for becoming an engineer, and any advanced education and specialization that may follow. Engineering education goes along with additional examinations and supervised training as the requirements for a professional engineering course. Since its independence from the British Rule in 1947, India has conceived the importance of engineering education for its ultimate economic development. Future prosperity and quality of life of the common people of India solely depend in large measure on the incorporation of superior skill, intelligence and added value into its products and services while establishing a sound basis for a sustainable global environment. Professional engineers can play important roles in creating high quality employment, establishing new ventures, reorganizing existing processes, and developing new products and services. The basis for excellence in engineering profession is basically the excellence in the system of engineering education at undergraduate, graduate and postgraduate levels. It is very essential that the technical and engineering education system evolves effectively to meet the changing needs of Indian society. With emergence of Japan as a growing economic power in 1980s, a new concept of economic growth known as ‘globalization’ has been established. Since then, the economic focus has started shifting to Asian countries. In 1990s, with liberization and opening of the Indian economy, a new chapter in the history of globalization has begun. From the beginning of 21st Century, India has occupied a place of prominence in the comity of nations specially in terms of world trade. The last decade has witnessed an era of joint ventures, mergers and acquisitions of international companies by the Indian industrialists. Even at the time of global economic recession during 2008–09, India stood strong with its stable economy. It has become the world leader in information technology, software and manufacturing industries primarily due to its inherent mental capabilities on one hand, and the theoretical and analytical education training imparted to its youth on the other. The faculties in Indian engineering institutions are now recognized to be scientifically strong by the international standards, but actually far from the best possible in the contribution they can make to the initial and continuing education of the professional engineers for effective industry practice. India should commit itself to a continuing role in promoting engineering education of suitable content and quality, in cooperation with engineering faculties, universities, industrial houses, professional and technical societies, and governments. India has enormous potential to be a global technology leader. The Indian economy has been growing at a rapid rate. The Indian industry has also become globally competitive in several sectors and can increase its global market share. The most critical issue for the future success of the Indian industry is the growth of engineering education in India.

Nowadays, India has also moved towards a long way from the hierarchical society and its exclusive educational system which were imbibed from its cultural and social heritage, as well as its majestic past. The thrust of the envisaged changes in the society calls for knowledge for all and a shift towards an inclusive knowledge-based society. One of the major objectives of National Knowledge Commission set up by the Prime Minister of India in 2005 was also to address the issue of Indian education system to make it a true knowledge-based society. As envisaged by the Indian intellectual academicians around 1,000 years ago, education is a never ending journey from less illumination to more illumination. Education is the expression of perfections already in man. Initially, the focus of engineering education system in India was only to produce engineering graduates to implement, operate and manage the growing Indian industries that mainly relied on imported technology. Subsequently, as the economy grew, there emerged a need for technology development, and then for research and development. The engineering institutions those were primarily set up for undergraduate teaching started emphasizing on research, and evolved postgraduate level and doctoral programs. Indian engineering education has witnessed a tremendous growth over the few decades, both in number of students and number of institutions. Many world class technological institutions, including Indian Institutes of Technology (IITs), National Institutes of Technology (NITs) and some reputed Indian universities with so many engineering departments have already been established to provide the cutting edge knowledge to their students. It is now an established fact that since its independence, India has produced a large number of competent, qualified engineers who have contributed to the success of many Indian companies and industries. Some of the best technical and engineering minds in the world were trained in India’s renowned engineering institutions. A large number of engineering graduates has made an impact in the corporate world internationally. But, unfortunately, Indian engineering education is still a very exclusive pursuit. The engineering education segment has been an overregulated sector so far. So, there is a need to experiment and innovate to bring the academic sector and its allied areas within the reach of the common people. The average qualities of Indian engineering colleges/universities and their respective departments along with the graduated students have become a matter of imagination [29]. As the primary concern and requirement of today’s industrial houses are innovative and quality engineers in each and every facet of specialization, therefore, performance evaluation and ranking of various engineering departments of Indian technical institutions/universities have become a major research issue. Also, performance evaluation always plays an important role in assessing the organizational ability to achieve targets. As the number of technical institutions is increasing rapidly in India, performance efficiency should be used as one of the criteria to ensure that the expected level of efficiency is met. Engineering departments should focus on teaching, research and innovative practices so as to achieve the institutional objectives [4]. The performance of an engineering department as well as quality of its outputs contributes immensely to achieve a certain level of efficiency. Without monitoring the performance efficiency methodically, it would not be possible to estimate the optimal amount of resources that should be instrumented to various engineering departments. In a wider view, performance efficiency level would generally be an indicator in comparing the ranking of engineering departments globally. A systematic performance measurement is thus vital for any institution to cultivate an environment of continuous improvement and make certain about successful implementation of its strategic plans. There are very few available literatures focusing on the strategies adopted in developing performance measurement models for academic departments. In recent years, some researchers have analyzed performance and efficiency of academic departments using data envelopment analysis (DEA) approach.

Lopes and Lanzer [21] examined the productivity and quality of academic departments in a university using DEA model to simulate a process of cross-evaluation between the departments. The DEA results in the dimensions of teaching, research, service and quality were modeled as fuzzy numbers, and then aggregated through a weighted ordered aggregator to generate a single index of performance for each department. Köksal and Nalçaci [19] used DEA for measuring the relative efficiencies of academic departments of an engineering college. The related input and output criteria were identified, and measured utilizing an academic personnel performance measurement scheme. The discriminating power of DEA was also improved while developing a dual DEA-Charnes, Cooper and Rhodes (DEA-CCR) model coupled with assurance region constraints. Martin [22] applied DEA methodology to assess performance of some departments of the University of Zaragoza in Spain. The evaluation criteria consisted of both teaching and research activities of the departments, and the results revealed that the departments having better teaching and research activities had shown satisfactory performance. Tauer et al. [31] computed the technical and allocative efficiencies of 26 academic departments in the College of Agriculture and Life Sciences at Cornell University using DEA model. Kao and Hung [18] applied DEA to assess the relative efficiency of the academic departments at National Cheng Kung University in Taiwan. The outputs considered were total credit hours, publications and external grants; and the inputs utilized by the departments were personnel, operating expenses and floor space. Tyagi et al. [32] evaluated the performance efficiencies of 19 academic departments of IIT Roorkee (India) employing DEA technique. Technical, pure technical and scale efficiencies were computed for each of the departments, and the reference sets for inefficient departments were also identified. Overall performance, research performance and teaching performance were assessed separately using sensitivity analysis. Rayenim and Saljooghi [28] analyzed efficiency and benchmarking of university departments using DEA approach. Benchmarking was used to detect mistakes of inefficient departments to become efficient and learn better managerial practice. Agha et al. [2] applied DEA to assess the relative technical efficiency of academic departments of Islamic University in Gaza. The inputs considered were operating expenses, credit hours and training resources, while the outputs were number of graduates, promotions and public service activities. The potential improvements and super-efficiency were computed for inefficient and efficient departments respectively. Alwadood et al. [5] proposed the application of DEA method to measure the performance efficiency of academic departments in a public university of Malaysia. Two inputs and three outputs strongly influencing the efficiency of those academic departments were selected. The academic departments with perfect efficiency score were considered as the benchmark departments. Derlacz and Parteka [10] employed DEA model for evaluating efficiency of academic departments at a public university by providing a single measure of efficiency for each academic unit. Agasisti et al. [1] investigated the trade-off among four research outputs, e.g. quantity (publications), quality (citation indexes), funds obtained through research grants and applied research funds obtained through external orders. DEA methodology was subsequently employed to measure the efficiency of 69 academic departments located in Italy. Aziz et al. [6] illustrated the application of DEA approach to measure the relative efficiency of 22 academic departments of a public university in Malaysia. The input variables considered were number of academic staff, number of non-academic staff and yearly operating expenses, while number of graduates for the year, total amount of research grant received for the year and number of academic publications by the faculty members were considered as output variables. Al-Shayea and Battal [3] investigated the efficiency of 18 faculties in Qassim University of Saudi Arabia. Using number of students enrolled, number of teachers and staff as inputs, and total number of students with a bachelor’s degree and number of research publications as outputs, an output-oriented DEA model was applied with variable return to scale to estimate efficiency scores of the alternative departments.

From the literature survey, it is observed that few attempts have been made by the past researchers on performance evaluation of academic departments using DEA models, but no venture has yet been put to compare the relative performance of various engineering departments using any of the multi-criteria decision-making (MCDM) methods. However, DEA-type analysis has long been criticized for lacking in statistical power and being deterministic [8]. Olson and Vu [24] also addressed the disadvantages of DEA models in the presence of outliers. Thus, in this paper, an attempt is made to evaluate and rank 16 engineering departments of a reputed Indian university using a combined approach consisting of decision-making trial and evaluation laboratory (DEMATEL), entropy and compromise ranking methods. This paper also surveys the uniqueness of these MCDM approaches and identifies a set of important criteria for evaluating the performance of the considered engineering departments. The causal relationship and interaction between those criteria are determined, and the most critical criterion having significant influence on the remaining criteria is also identified. The related digraph is then developed to illustrate the influential network for this performance evaluation problem. The adopted multi-criteria approach can assist the decision makers to directly evaluate the criteria according to their influential degree while providing a ranking preorder to all the 16 engineering departments.

The rest of this paper is structured as follows. Section 2 presents the mathematical concepts of DEMATEL, entropy and compromise ranking methods. Section 3 illustrates an example to validate the proposed integrated model. The results and discussions are also presented in this section. Section 4 concludes the paper with final comments.

2 Mathematical models of DEMATEL, entropy and compromise ranking methods

In this section, the basic concepts of DEMATEL method are presented to establish the relationship structure in an evaluation problem. Entropy approach is used to determine criteria weights, and a compromise ranking method is applied to prioritize and rank the considered engineering departments.

2.1 DEMATEL method

DEMATEL method is used to perceive intricate relationships and build a network relation map between criteria. It was mainly developed by the Battelle memorial association of the Geneva research center [11, 12] to study complicated world problems concerning about race, hunger, environmental protection, energy etc. It is based on a concept of pair-wise comparison of decision-making attributes (alternatives, criteria). The attributes are compared with respect to relative influence of each over the other. The main objective of DEMATEL method is to directly compare the interaction relationship between different variables of a complicated system to determine direct and indirect causal relationships and influence levels between the variables. A visual structural matrix and a causal diagram are developed to express the causal relationships and influence levels between the considered variables, and assist in making appropriate decisions. The evaluation criteria are usually comprised of many complicated aspects, including financial and non-financial, as well as qualitative aspects, and are either directly or indirectly mutually related. DEMATEL method is based on the notion of digraphs, which can separate the considered criteria into cause and effect groups to visually observe the inside of a complex system/problem. It assumes a system containing a set of components C = {C1,C2,…,Cn}, with pair-wise relations that can be evaluated. In order to apply DEMATEL method, this paper adopts the following six steps [9, 13, 30]:

  1. Step 1:

    Generation of the direct-relation matrix (A)

    At first, the decision makers or experts evaluate the relationship between the sets of paired criteria to indicate the direct effect that they believe each i th criterion exerts on each j th criterion, as indicated by an integer scale (score) ranging from 0 to 4, representing no influence (0), low influence (1), medium influence (2), high influence (3) and very high influence (4). Then, as a result of these evaluations, the initial data is obtained as a direct-relation matrix (A) which is in the form of an n × n matrix, in which the individual element (a ij ) denotes the degree to which i th criterion affects j th criterion and n denotes the total number of criteria.

    $$ A=\left[\begin{array}{cccccc}\hfill 0\hfill & \hfill {a}_{12}\hfill & \hfill \dots \hfill & \hfill {a}_{1j}\hfill & \hfill \dots \hfill & \hfill {a}_{1n}\hfill \\ {}\hfill {a}_{21}\hfill & \hfill 0\hfill & \hfill \dots \hfill & \hfill {a}_{2j}\hfill & \hfill \dots \hfill & \hfill {a}_{2n}\hfill \\ {}\hfill \dots \hfill & \hfill \dots \hfill & \hfill \dots \hfill & \hfill \dots \hfill & \hfill \dots \hfill & \hfill \dots \hfill \\ {}\hfill \dots \hfill & \hfill \dots \hfill & \hfill \dots \hfill & \hfill \dots \hfill & \hfill \dots \hfill & \hfill \dots \hfill \\ {}\hfill {a}_{n1}\hfill & \hfill {a}_{n2}\hfill & \hfill \dots \hfill & \hfill {a}_{nj}\hfill & \hfill \dots \hfill & \hfill 0\hfill \end{array}\right] $$

    The digraph portrays contextual relationships between the elements of a system, as shown in Fig. 1. For example, an arrow from element b to element a signifies that b affects a, and the influence level is 4.

    Fig. 1
    figure 1

    A typical digraph

    Full size image
  2. Step 2:

    Development of the normalized direct-relation matrix (X)

    Once the direct-relation matrix is developed, the normalized matrix (X) is obtained using Eqs. (1) and (2). Each element in matrix X ranges from 0 to 1.

    $$ X=k.A $$
    (1)

    where

    $$ k=\frac{1}{\underset{1\le i\le n}{ \max}\left({\displaystyle \sum_{j=1}^n{a}_{ij}}\right)},\kern0.24em i,j=1,2,\dots, n $$
    (2)
  3. Step 3:

    Computation of the total-relation matrix (T)

    The total-relation matrix (T) is determined using Eq. (3), in which I denotes the identity matrix. The element t ij represents the indirect effects that i th criterion has on j th criterion, and the matrix T reflects the total relationship between each pair of system criteria.

    $$ \begin{array}{c}\hfill T = {{\left[{t}_{ij}\right]}_n}_{\times n},i,j = 1,2,\dots, n\kern30em \hfill \\ {}\hfill T=X+{X}^2+{X}^3+\dots .+{X}^k=X\left(I+X+{X}^2+\dots +{X^k}^{-1}\right)\left[\left(I\hbox{--} X\right){\left(I\hbox{--} X\right)}^{-1}\right] = X\left(I\hbox{--} {X}^k\right){\left(I\hbox{--} X\right)}^{-1}\hfill \end{array} $$

    Then,

    $$ \begin{array}{c}\hfill T=X{\left(I\hbox{--} X\right)}^{-1},\ \mathrm{when}\kern0.5em k\to \infty, {X}^k = {{\left[0\right]}_n}_{\times n}\hfill \\ {}\hfill T=X{\left(I\hbox{--} X\right)}^{-1}\kern11em \hfill \end{array} $$
    (3)
  4. Step 4:

    Determination of the sums of rows and columns of matrix T

    In the total-relation matrix T, the sum of rows and sum of columns are represented by vectors D and R, as derived using Eqs. (4) and (5) respectively.

    $$ {D}_i={\left[{\displaystyle \sum_{j=1}^n{t}_{ij}}\right]}_{n\times 1}={\left[{t}_i\right]}_{n\times 1},\;i=1,2,\dots, n $$
    (4)
    $$ {R}_j={\left[{\displaystyle \sum_{i=1}^n{t}_{ij}}\right]}_{1\times n}={\left[{t}_j\right]}_{n\times 1},\;j=1,2,\dots, n $$
    (5)
  5. Step 5:

    Set a threshold value (α)

    This threshold value (α) is obtained from the average of the elements in matrix T, as computed using Eq. (6), where N is the total number of elements in matrix T. This calculation is aimed to eliminate some minor effect elements in matrix T.

    $$ \alpha =\frac{{\displaystyle \sum_{i=1}^n{\displaystyle \sum_{j=1}^n\left[{t}_{ij}\right]}}}{N} $$
    (6)

    Since matrix T provides information on how one factor affects another, it is necessary for the decision maker to set up a threshold value to filter out some negligible effects. While doing so, only the effects greater than the set threshold value are chosen and shown in a digraph.

  6. Step 6:

    Development of a causal diagram

    The horizontal axis vector (D k  + R k ) named ‘prominence’ is computed by adding D to R while k = i = j = 1 which reveals how much importance the criterion has. Similarly, the vertical axis (D k – R k ) named ‘relation’ is calculated by subtracting D from R, which divides the criteria into a cause group and an effect group. If D i is the sum of i th row in matrix T, then D i summarizes both direct and indirect effects given by i th criterion over other criteria. Similarly, R j indicates the sum of j th column in matrix T, and it shows both direct and indirect effects given by j th criterion over other criteria. If j = 1, it indicates the total effects given and received by i th criterion. Thus, (D k  + R k ) shows the degree of importance that i th criterion plays in the entire system. In the contrary, (D k – R k ) determines the net effect that i th criterion contributes to the entire system.

    Generally, when (D k – R k ) is positive, the criterion belongs to the cause group. Otherwise, if (D k – R k ) is negative, the criterion belongs to the effect group. Therefore, a causal diagram can be developed while mapping the dataset of (D k  + R k ,D k – R k ), providing valuable insight for making decisions. Therefore, the decision maker can use the causal relationship of the variables and their interaction influence levels to identify the driving variables of the core problem in a complicated system, and plan for suitable decision to solve the problem in accordance with attribute type and influence level. Figure 2 presents the procedural steps as involved in DEMATEL method.

    Fig. 2
    figure 2

    Process algorithm of DEMATEL method

    Full size image

2.2 Entropy method

Entropy weights are the measures of uncertainties in the information formulated using probability theory. It indicates that a wide distribution represents more uncertainty than does a sharply peaked one. Entropy weights measure the information content in the attribute values of the alternatives, thereby evaluating each attribute’s usefulness in detecting differences in the data. For example, if an attribute has the same value for each of the alternatives, then that attribute provides no information to distinguish the alternatives. On the other hand, an attribute that has different values for each alternative has high information content, and is useful when comparing and contrasting the alternatives. The steps of calculating criteria weights using entropy method are enlisted as below [17].

  1. Step 1:

    Normalize each criterion of the decision matrix.

    $$ {r}_{ij}=\frac{x_{ij}}{{\left[{\displaystyle \sum_{i=1}^m{x}_{ij}^2}\right]}^{1/2}}\kern0.5em \left(i = 1,2,\dots, m;j = 1,2,\dots, n\right) $$
    (7)

    where x ij is the performance measure of i th alternative with respect to j th criterion, r ij is the normalized value of x ij , m is the number of alternatives and n is the number of criteria.

  2. Step 2:

    Calculate the entropy value (ej) of each criterion.

    $$ {e}_j=-k{\displaystyle \sum_{i=1}^m\left({r}_{ij}\times \log \kern0.24em {r}_{ij}\right)} $$
    (8)

    where k = 1/(ln m) is a constant that assures 0 ≤ e j  ≤ 1.

  3. Step 3:

    Determine the objective weight (wj) for each criterion.

    $$ {w}_j=\frac{1-{e}_j}{{\displaystyle \sum_{i=1}^m\left(1-{e}_j\right)}} $$
    (9)

    These objective weights can be treated as the criteria weights in any decision-making problem.

2.3 Compromise ranking method

Opricovic [25] and Opricovic and Tzeng [26] mainly established the concept of compromise ranking method, popularly known as VIse Kriterijumska Optimizacija kompromisno Resenje (VIKOR) method. The basic concept of VIKOR method lies in defining the positive and negative ideal solutions. The positive ideal solution indicates the alternative with the highest value, while the negative ideal solution denotes the alternative with the lowest value. It was basically introduced as a multi-criteria ranking tool, based on the particular measure of closeness to the ideal solution using linear normalization procedure. It focuses on selecting the best alternative from a set of feasible alternatives in presence of mutually conflicting criteria while determining a compromise solution. It provides a maximum group utility for the ‘majority’, and a minimum of individual regret for the ‘opponent’. The compromise solution is a feasible solution, which is the closest to the positive ideal solution and farthest from the negative ideal solution. The following multiple attribute merit for compromise ranking is developed from the L p -metric used in the compromise programming method.

$$ {L}_{p,i}={\left\{{\displaystyle \sum_{j=1}^n{\left({w}_j\left[{\left({x}_{ij}\right)}_{\max }-{x}_{ij}\right]/\left[{\left({x}_{ij}\right)}_{\max }-{\left({x}_{ij}\right)}_{\min}\right]\right)}^p}\right\}}^{1/p}\kern0.5em 1\le p\le \infty; \kern0.5em i=1,2,\dots, m $$
(10)

In VIKOR method, L 1,i and L ∞,i are used to formulate the ranking measure. The procedural steps for VIKOR method are enlisted as below [25, 26].

  • Step 1: Identify the major decision criteria for the given problem and shortlist the alternatives on the basis of the identified criteria.

  • Step 2: a) After shortlisting the alternatives and development of the decision matrix, determine the best, (x ij )max and the worst, (x ij )min values of all the criteria.

    b) Calculate the values of E i and F i .

    $$ {E}_i={L}_{1,i}={\displaystyle \sum_{j=1}^n{w}_j\left[{\left({x}_{ij}\right)}_{\max }-{\mathrm{x}}_{\mathrm{ij}}\right]/\left[{\left({x}_{ij}\right)}_{\max }-{\left({x}_{ij}\right)}_{\min}\right]} $$
    (11)
    $$ {F}_i={L}_{\infty, i}={\mathrm{Max}}^{\mathrm{m}}\mathrm{of}\left\{\kern0.5em {w}_j\left[{\left({x}_{ij}\right)}_{\max }-{x}_{ij}\right]/\left[{\left({x}_{ij}\right)}_{\max }-{\left({x}_{ij}\right)}_{\min}\right]\;\right\}\kern1em j=1,2,\dots, n $$
    (12)

    For non-beneficial criteria, Eq. (12) can be rewritten as follows:

    $$ {E}_i={L}_{1,i}={\displaystyle \sum_{j=1}^n{w}_j}\left[{x}_{ij}-{\left({x}_{ij}\right)}_{\min}\right]/\left[{\left({x}_{ij}\right)}_{\max }-{\left({x}_{ij}\right)}_{\min}\right] $$
    (13)

    c) Calculate P i value.

    $$ {P}_i=v\left(\left({E}_i\hbox{--} {E_i}_{\min}\right)/\left({E_i}_{\max}\hbox{--} {E_i}_{\min}\right)\right)+\left(1\ \hbox{--} v\right)\left(\left({F}_i\hbox{--} {F_i}_{\min}\right)/\left({F_i}_{\max}\hbox{--} {F_i}_{\min}\right)\right) $$
    (14)

    where E i max and E i min are the maximum and minimum values of E i respectively, and F i max and F i min are the maximum and minimum values of F i respectively. The parameter v is introduced as weight of the strategy of ‘the majority of attributes’ (‘the maximum group utility’). The value of v is usually set by the decision maker, ranging between 0 and 1.

    Practically, if the decision maker assumes v > 0.5, he/she gives more importance to the first term in Eq. (14) and hence, to the global performance of the alternative in respect to the whole of the criteria. While using a v value smaller than 0.5, he/she gives more weight to the second term that is related to the magnitude of the worst performance exhibited by the alternative with respect to each single criterion. When both these aspects are considered equally relevant, v = 0.5 should be used.

    d) Arrange the alternatives in ascending order, according to P i values. The best alternative is that one having the minimum P i value.

    The VIKOR method is an effective MCDM tool, specifically applicable to those situations when the decision maker is not able, or does not know to express his/her preference at the beginning of the decision-making process. The computational procedure of VIKOR method is quite simple, and it offers a systematic and logical approach to arrive at the best decision. The obtained compromise solution can be accepted by the decision maker because it provides a maximum group utility of the ‘majority’ and a minimum individual regret of the ‘opponent’. The compromise solutions can be the base for negotiations, involving the decision maker’s preference on criteria weights. The VIKOR results depend on the positive ideal solution, which stands only for the given set of alternatives. Inclusion (or exclusion) of an alternative can affect the VIKOR rankings of the new set of alternatives. The main advantage of VIKOR method as compared to any other MCDM methods is that the final performance score in VIKOR is an aggregation of all criteria, their relative importance, and a balance between total and individual satisfaction. The compromise solution as provided by this method can be the groundwork for negotiations, involving the decision maker’s preference on criteria weights.

3 Performance evaluation of engineering departments in an Indian University

An extensive, high quality network of universities is an essential building block for any country that aspires to be an economic powerhouse. For India, which models itself as a knowledge economy, development of such a network is even more critical. India is still in short of a world class university system. Unlike any industrial house, whose performance can be assessed by looking at different quantitative variables, like revenue and profit, measuring relative success of any educational institution is more complex. It is a perceived fact that evaluation and ranking of various engineering departments of any university/institute is not an easy task to perform. Also, it is very cumbersome to determine the relative performance scores of the departments in an absolute sense. There may be a lot of influential criteria which may directly or indirectly affect the performance of the engineering departments. The results may be dependent and sensitive to the selection of proper criteria. In a broader sense, factors that influence efficiency of each engineering department need to be identified, and must be relevant and directly aligned with the goals and objectives of the department. This paper considers the basic objectives of a university in order to identify and classify those factors that influence the performance efficiency of the engineering departments. In this paper, an attempt is made to rank 16 engineering departments of Jadavpur University (JU), an Indian university of international repute. It is a premier autonomous public state university located in the heart of Kolkata city, in the state of West Bengal (WB), India. JU is at present an internationally recognized premier university of the country. It is an urban university with a global perspective. JU is an outcome of the nationalist movement initiated in the early 20th century by the intellectuals and thinkers of WB [20, 23]. The Governor of the state of WB is its Chancellor. However, the daily administration is handled under the leadership of the Vice Chancellor (VC), who is appointed by the Governor of WB. The VC is assisted in his duties by the Pro Vice Chancellor who is de facto a senior faculty member of the university. Within the university’s administrative structure, the highest body is the university court, consisting of members elected by the staff members of the university from amongst themselves as well as members of the alumni. Primarily, JU has four distinct heritage faculties of studies, e.g. Faculty of Arts, Faculty of Science, Faculty of Engineering and Technology (FET), and Faculty of Interdisciplinary Studies, Law and Management. FET offers courses in traditional engineering as well as technology-based disciplines, with Bachelor/Master of Engineering/Technology/Pharmacy/Architecture programs spread over 16 departments, including Architecture, Chemical Engineering, Civil Engineering, Computer Science and Engineering, Construction Engineering, Electrical Engineering, Electronics and Telecommunication Engineering, Food Technology and Biochemical Engineering, Information Technology, Instrumentation and Electronics Engineering, Mechanical Engineering, Metallurgical and Material Engineering, Pharmaceutical Technology, Power Engineering, Printing Engineering and Production Engineering. Admission to various engineering departments in JU is highly competitive, as it consistently ranks amongst the best universities in India. To facilitate interdisciplinary learning and research in diverse fields, there are also a number of other schools and centers for study. Some of the major research ventures undertaken by these schools include the pioneering work done by the School of Environmental Studies in highlighting the presence of arsenic in groundwater in countries, like India and Bangladesh, and the development of the first alcohol-based car by the School of Automobile Engineering. While most Indian engineering institutions excluding IITs emphasize on undergraduate teaching, JU is primarily recognized for its research activities. JU had been ranked first among the top engineering and technology universities in India [27]. After IITs, it had been ranked 6th in research output in India for its engineering faculty, and ranked 2nd in 2009 in India for the number of research papers published [14]. Also, in the year 2009, FET had been ranked 10th in India (Mint C-Fore), 12th in India (Outlook); and 10th in India in 2011 (Career360 2011) according to various surveys [14].

As already discussed, education system, specially engineering education contributes a major role for the development of a nation. In any academic institution, teachers and students are the two main leaders without which it can never be survived. Teachers are the most important assets of an educational institution and good teachers provide the good quality education among the students. Innovative practices can increase the quality of performance of an institute. However, selection of different attributes for evaluating engineering departments using any performance appraisal model is a difficult task. There are at least two difficulties in selecting those attributes. One is the accessibility and availability of data, while the other is the measurement of departmental quality, as there is a lack of a common base for comparing the quality of research works and other related activities. Relying on the literature and available information, faculty/student ratio (FS) based on the present faculty strength and student intake in an department, research publications in terms of number of papers/books published (RP), number of successful graduate and postgraduate students (SS), number of doctoral theses completed (DT), financial infrastructure in terms of yearly costs related to teaching and non-teaching staffs (TN), and departmental operating costs/student in a year (OC) are identified as the key performance indicators for evaluating and ranking of 16 engineering/technological departments of JU. The evaluation criteria as mentioned above are all self-explanatory. OC of each department is the ongoing operational expenses, such as administrative cost, research expenditure and stationery cost as associated with the department. All the related criteria values, except TN and OC are accumulated for the time period spanning from 2008 to 2012 [15, 16]. Except TN and OC criteria, all the remaining four criteria are of beneficial type, i.e. higher values are always desirable. The information regarding TN and OC are based on the financial year 2012 [7].

Among the 16 engineering departments in JU, the department of Architecture was established in 1964. Since its inception as a constituent department in Engineering Faculty in year 1965, only Bachelor of Architecture was the course that was offered for teaching. The Master of Architecture course in ‘Housing and Urban Design’ was started from the year 1981. The establishment of Chemical Engineering dates back to 1921 when a course of study was offered for the first time at the Bengal Technical Institute, the predecessor of JU. This department is one of the oldest departments in chemical engineering in the world. This department commemorated Diamond Jubilee in 1982 and Platinum Jubilee in 1996. Civil Engineering department started in the year 1956. Since the last few decades, nearly 5,000 graduates of this department have been serving the professional society globally. Apart from teaching and research, this department also organizes symposia and short-term courses around the academic year. Academic activities in the area of electronic digital computers were started in JU under the umbrella of Electronics and Telecommunication Engineering (ETCE) department in the early 1960s, and it got a big boost in 1962, when the Indian Statistical Institute (ISI), Kolkata and JU jointly undertook a project to design and build ISIJU-1 computer which was commissioned in 1966. Several students who have passed out successfully from this department are holding significant positions at prestigious centers, like NASA etc. In the middle of the year 1988, ETCE department was trifurcated into three separate departments, e.g. ETCE, Computer Science and Engineering, and Instrumentation Engineering. Construction Engineering department is comparatively new, being established as late as in 1989. It was concurrently set up with the second campus of the university in Salt Lake. The primary endeavor of this department is to impart application-oriented engineering education to the undergraduate students in the field of construction management. Electrical Engineering department is one of the oldest departments of JU, being established in the year 1906. Over the years, it has grown into a department of national repute and importance. Food Technology and Biochemical Engineering department was established in the year 1964 providing only undergraduate courses, whereas, the master degree level courses were initiated 3 years later. Department of Instrumentation and Electronics Engineering started in the year 1966 and is now functioning from the second campus of JU, at Salt Lake. This department at present is conducting two undergraduate courses. Mechanical Engineering department is functioning since 1906 and is one of the oldest departments that was established under the National Council of Education, Bengal. During the days of freedom struggle, several luminaries who graduated from this department served as industrialists and fulfilled the nationalist aspirations. At present, it is the largest department under the FET. This department is actively engaged in various research projects with many national and international organizations. Metallurgical and Material Engineering department was established in 1956 with the purpose of imparting excellent training to students to make them able to join several industries as qualified engineers, dealing with production and processing of metals, alloys, foundries, rolling mills and fabrication shops. The focus of postgraduate degree is mainly fixed to train qualified metallurgical engineers for state of art technologies at different Indian metallurgical industries. Pharmaceutical Technology department was established in the year 1963, as the only one of its kind in Eastern India at that time. It has well equipped research laboratories with modern and sophisticated analytical as well as production instruments and equipment. A good number of research scholars and scientists are presently working with this department. Power Engineering department was established under the name of Power Plant Engineering in the year 1989 at the Salt Lake Campus of JU with a view to produce specialized engineers in the field of power generation, transmission and distribution. Although printing technology is becoming increasingly important, both in the country and abroad as an academic discipline, the fact that the subject of printing could be included in the FET was never thought of in India until it was established in JU in 1988. This department at present is offering the Bachelors degree in Printing Engineering with an annual intake of 25 students. Production Engineering department was established in 1980 and is comparatively young, but has already been able to earn national and international reputation specially during the late 80s. Since then, there has been rapid advances in manufacturing technologies, with computer controlled processes and management information systems that effectively enhance a specialized training in manufacturing and production that cater to the growing requirements of industries both in India and abroad.

Thus, the decision matrix as required for effectively evaluating the performance of engineering departments in JU consists of 16 departments and six evaluation criteria, as shown in Table 1.

Table 1 Decision matrix for performance evaluation of 16 engineering departments
Full size table

Since selecting the best engineering department of any university/institute along with its performance appraisal is truly a complex multi-criteria problem, it is not appropriate to assume the elements within the evaluation system to be independent. As, all of the six identified evaluation criteria are deemed to be significant and indispensable, hence, it becomes essential to find out the important criteria for the performance appraisal system and measure the relationships between those criteria. To achieve this, DEMATEL method is applied for capturing the profound relationships between those evaluation criteria causally and visually. Following the DEMATEL steps as enlisted in Section 2.1, the relationships between different criteria are scored using an integer scale, as explained earlier. Once the relationships between those criteria are measured, the initial direct-relation matrix (A) is developed, as shown in Table 2. It is a 6 × 6 matrix, obtained by pair-wise comparisons in terms of influences and directions between the considered criteria.

Table 2 Initial direct-relation matrix
Full size table

From the matrix A of Table 2, the normalized direct-relation matrix (X), as given in Table 3, is obtained, employing Eqs. (1) and (2). Then, using Eq. (3), the total-influence matrix (T) is derived, as provided in Table 4. Now, the sum of rows and sum of columns as represented by vectors D and R respectively are computed employing Eqs. (4) and (5), and are shown in Table 5. The causal diagram, as exhibited in Fig. 2, is then developed by mapping the dataset of Table 6. The (D + R) and (D – R) values of Table 6 represent the total influence levels and net influence levels for different criteria respectively, where the positive values indicate that it influences other criteria more than any other criterion influences it and the negative values denote that it is significantly influenced by the other criteria. Table 6 indicates that FS criterion has the largest net influence level, followed by TN and OC in this performance evaluation problem.

Table 3 Normalized direct-relation matrix of criteria
Full size table
Table 4 Total-relation matrix of criteria
Full size table
Table 5 Computation of vectors D and R
Full size table
Table 6 Total and net effects for each criterion
Full size table

Now, looking at the causal diagram of Fig. 3, it is clear that the six considered evaluation criteria are visually divided into cause and effect groups. The cause group consists of three criteria, i.e. FS, TN and OC, and the effect group contains the remaining three criteria (RP, SS and DT). It is quite obvious that FS, TN and OC criteria are the main driving factors for RP, SS and DT. Among these six criteria, FS is identified as the most important criterion because it has the highest intensity of relation to the other criteria for having maximum D + R value; moreover, it is also the most influencing factor due to its maximum D – R value. Thus, FS criterion plays a major role in this performance evaluation problem of engineering departments, and it has the greatest effect on the other criteria. On the contrary, SS criterion is greatly influenced by the other criteria, having the lowest negative value of (D – R). The threshold value (α) is now derived from the average of elements in matrix T, as 0.3277. The values of t ij in Table 4, which are greater than α (0.3277), are shown as t ij *, which present the interaction between two criteria, e.g. as the value of t 12 (0.5049) > α (0.3277), an arrow in the diagraph, as shown in Fig. 4, is directed from FS to RP. Thus, this digraph portrays the contextual relationships between the elements of the considered performance appraisal model.

Fig. 3
figure 3

DEMATEL causal diagram of criteria

Full size image
Fig. 4
figure 4

DEMATEL diagraph for performance evaluation problem

Full size image

For the purpose of performance evaluation of 16 engineering departments in JU using VIKOR method, at first, the best and the worst values of all the criteria are identified from the developed decision matrix of Table 1. The relative normalized weights of the six different evaluation criteria are determined using entropy method as w FS = 0.1919, w RP = 0.1002, w SS = 0.1566, w DT = 0.1470, w TN = 0.2065 and w OC = 0.1977, and these criteria weights are used for all the subsequent analyses. Now, the values of E i and F i are calculated using Eqs. (11) or (13) and (12) respectively, as given in Table 7. This table also exhibits the values of performance scores (P i ) for v = 0.5 and the compromise ranking of 16 engineering departments. The candidate departments are then arranged in ascending order, according to their P i values. The best performer amongst the 16 engineering departments is Electrical Engineering department (A6), followed by Mechanical Engineering department (A11). Information Technology department (A9) is the worst performer in this evaluation process. Using VIKOR method, the total compromise ranking list of all the 16 engineering departments is obtained as A6 > A11 > A4 > A12 > A16 > A13 > A7 > A14 > A3 > A2 > A10 > A1 > A5 > A15 > A8 > A9. From the decision matrix of Table 1, it is observed that Electrical Engineering department outperforms the other engineering departments of JU with respect to higher values of SS and DT, and lower value of OC criteria. On the other hand, the main reason behind the underperformance of Information Technology department is its very low FS value, although it has very attractive values for TN and OC criteria. It is identified that this department has very few full time faculty staffs and a good number of faculty positions remains still vacant.

Table 7 E i , F i and P i values of various engineering departments
Full size table

A graphical view of the Pareto analysis results based on P i values of the 16 engineering departments in JU is presented in Fig. 5. It is observed from this figure that engineering departments A6, A11, A4, A12, A16, A13, A7 and A14 (with a cumulative P i of 80–100 %) can be considered as the benchmarks for departments A3, A2, A10, A1, A5, A15 and A8 (which form the second group with a cumulative P i of 20–80 %) and this second group can be adjudged as an improvement target for engineering department A9 (cumulative P i of 0–20 %).

Fig. 5
figure 5

Pareto analysis of P i values for 16 engineering departments

Full size image

Table 8 and Fig. 6 show the compromise rankings of 16 engineering departments for two extreme values of v = 0.1 and v = 0.9 as obtained using VIKOR method. It can be concluded that in both the cases, the best (Electrical Engineering) and the worst (Information Technology) choices of engineering departments do not change, although the rankings of the intermediate departments deviate slightly.

Table 8 Rankings of engineering departments for different v values
Full size table
Fig. 6
figure 6

Rankings of 16 engineering departments for different v values

Full size image

4 Conclusions

The primary mission of engineering institutions is to discover and communicate knowledge to the society. As the number of students enrolling in various engineering disciplines in India is increasing day by day, therefore, it becomes essential to evaluate those departments with respect to their suitability and applicability in context of modern educational system. Hence, the main objective of this paper is set to resolve the issue of evaluating and ranking such engineering departments of an Indian university. This paper uses DEMATEL method to analyze and explain the interaction relationships and impact levels between the evaluation criteria. DEMATEL also helps to develop the causal relationship diagram, dividing the considered criteria into cause and effect groups. Based on the results, it is recommended that the university administration should focus on maintaining proper faculty/student ratio according to the guidelines of different accrediting bodies and invest more in infrastructural development to attract the best talents as its teachers. It is also recommended to motivate its academic staffs to focus more on student results and international research publications. A successful education system is that which is able to uphold the status of teaching and research by attracting the worldwide best talents to its teaching fraternity, and providing lucrative professional development opportunities comparable to other professions requiring similar levels of qualification and expertise. It is also the responsibility of the university administration to provide teachers the opportunities to communicate across classrooms and cultures by strengthening its management system and work procedures. VIKOR method is adopted here to aggregate the performance measures under different criteria into an overall performance score for each department. However, the evaluation criteria should be selected carefully because they play a vital role in performance evaluation and subsequent ranking of the departments. The adopted approach considers a strategic decision-making platform for both national and international educators and educational administrators in devising appropriate criteria for designing academic program reviews for various departments. The findings from this paper offer valuable insights on different attributes which significantly contribute to departmental efficiencies so that inefficient departments can focus on those attributes to improve themselves. From the Pareto analysis, it is observed that the inefficient departments have to reduce their operating expenses, and should adopt the benchmark policies and techniques of the Pareto-efficient departments with respect to faculty/student ratio, infrastructure development and international publications to become efficient. In other words, at least eight departments, including Architecture, Construction Engineering, Food Technology and Biochemical Engineering, Information Technology, Instrumentation and Electronics Engineering, and Printing Engineering can improve their performance. The adopted combined evaluation framework is more suitable than the traditional MCDM approaches to solve problems with different degrees of effects between criteria and it has high potential for practical applications in evaluating performance of engineering departments. It is also generic in nature and can easily be applied to solve any type of multi-criteria performance evaluation problem.