The Multicriteria Electre III Model Applied to the Evaluation of the Placement of University Graduates

Abstract

During the last years the Italian university system has been undergoing a reform process regarding issues of governance such as the progressive financial autonomy of the University (art. 5, law 537/24 December 1993) and the reshape of the academic curricula (law 509/99 and 270/04). The most obvious result of this complex process has been the challenging attempt to manage a necessary change in a context of limited financial resources. In order to encompass the composite features of an efficient and effective management of resources, features of high relevance for the full accomplishment of the reform benchmarks, one has to take into account the academic organisation’s objectives, so that to use tools and methods able to support decisions and, thus, allowing a rationalization of the decision-making processes.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Preliminary Remarks

During the last years the Italian university system has been undergoing a reform process regarding issues of governance such as the progressive financial autonomy of the University (art. 5, law 537/24 December 1993) and the reshape of the academic curricula (law 509/99 and 270/04). The most obvious result of this complex process has been the challenging attempt to manage a necessary change in a context of limited financial resources. In order to encompass the composite features of an efficient and effective management of resources, features of high relevance for the full accomplishment of the reform benchmarks, one has to take into account the academic organisation’s objectives, so that to use tools and methods able to support decisions and, thus, allowing a rationalization of the decision-making processes.

The rationalization of the processes unavoidably implies both their own evaluation and the evaluation of the objectives at stake. To this end, analytical models based on decisions have been developed over the last decades and represent a solid starting point for our analysis.

Thus, we have decided to apply multi-criteria methods to the university environment in order to evaluate groups of graduates from the point of view of their placement on the labour market. Moreover, we share strong beliefs that there might be similarities between the academic reality and the business area where previous studies had tested similar methods. However, this is the second attempt in the university arena, following the previous evaluation of the degree programmes in two faculties of the University of Palermo in 2005 [5]. These methods could, for instance, identify those degree programmes that need supplementary support for increasing their performances or, on the contrary, award the outstanding ones. The analysis is structured in three parts. The first one describes the data used for the analysis. The second one describes the multi-criteria method and the model of outranking applied to the university graduates environment. The third part introduces the results of the previous analysis. The final chapter provides several concluding remarks aiming to stress out the advantages and limits of the employed methodology. In the end, a follow-up agenda is briefly introduced.

2 The Data

This analysis uses the data collected by STELLA (Statistiche in tema di laureati e lavoro) post-graduation placement survey conducted by CILEA on the 2006 University of Palermo graduates one year after their graduation [2]. The survey aims to evaluate the graduate’s global satisfaction considering the whole period he/she attended the university. Thus, it provides the Italian universities with an update database of their graduates aiming to analyse the job opportunities, the level of satisfaction regarding the academic curriculum, the students’ knowledge, competencies and skills, the convergence degree between academic curricula and labour market requirements, the overall degree of satisfaction regarding the quality of the academic curriculum, etc.

Based on CATI method, the survey used questionnaires with five distinct sections. With a broad grasp, the first section regards all the graduates of the new system and focused on general information on the graduates’ academic curriculum, their mobility experiences, their job experiences during their studying and, more in general, information concerning their families social background. The last question was a filter aiming to identify the students’ post-graduation path (the employment status was divided in two categories: employed graduates and job seeking graduates; the non-employed status was divided in two categories: the graduates who continued their education and other categories). The second, third and forth parts focused on the employed and job-seeking graduates (these two categories were gathered in the global Workforce category) and those continuing their studies. In the final part of the survey, the graduates expressed general opinions and thoughts on the university system.

Based on this survey, the analysis took into account exclusively the 3 years long university degrees belonging to all the groups of disciplines,¹ codified as such:

The main findings can be summarised as such.

From a socio-economic point of view, almost all the groups of disciplines include graduates rated with a medium-high social background. Concerning the social background, defined taking into account the median value of a variable obtained from analysing both the parents’ profession and their education degree (see the metho-dological note in [2]), the survey shows a relationship between this variable and job and university success [2]. As an exception, the Health and Physical Education group is characterised by a medium-low social background (Table 12.1).

a 1:	Agrarian studies	a 9:	Sciences of Education
a 2:	Architecture	a 10:	Literature
a 3:	Chemistry and Pharmaceutics	a 11:	Linguistic studies
a 4:	Economy and Statistics	a 12:	Medicine
a 5:	Health and Physical Education	a 13:	Political and Social studies
a 6:	Geo-biology	a 14:	Psychology
a 7:	Law studies	a 15:	Science
a 8:	Engineering

Table 12.1 Several characteristics of the graduates by group of disciplines, social background, effective length of the studying period, graduation grades, and percentage of satisfied graduate students with their academic curriculum

Concerning the length of their university studying, students in Medicine followed by students in Health and Physical Education tend to graduate faster than the other categories. At the other extreme, the Scientific group tends to graduate later than the others. In addition, not only the students in Medicine and Health and Physical Education are graduating faster, but they also have the highest final grades (respectively 107.8 and 107.6 – Table 12.1), followed by the Architecture (107.3). The lowest final grades are registered among the graduates in Chemistry and Pharmaceutics (100.3).

By excluding, on the one side, the Health and Physical Education group and, on the other, the Linguistic studies group whereas less than 50% of graduates tend to be satisfied with their academic curricula (respectively 40% and 48.4%), for the most part (more precisely over 70%), the graduates in Engineering, Economy and Statistics, Science, Agrarian Studies, Medicine and Law Studies declare themselves satisfied with their academic curricula.

Concerning their employment status, only one third of the graduates has a job one year after their graduation (30.9% equivalent to 30% of the male students and 31.4% of the female students). The gender issue does not seem to be a discriminant for the job placement after graduation. According to their groups of disciplines, strong differences can be seen when it comes to the path followed by the graduates in short-term degrees.² Thus, over 80% of the Medicine students (81.5%) have a fast track to the job market immediately after their graduation, while less than 20% of the graduates in Geo-Biology (10%), Engineering (18.1%) and Literature (19.6%) have a smooth insertion on the job market (Table 12.2). As illustrated by Table 12.2, by excluding the Medicine graduates, only the graduates in Health and Physical Education (60%) and those in Chemistry and Pharmaceutics (53.9%) are employed beyond the threshold of 50%. Still, while the graduates in Medicine and Chemistry and Pharmaceutics have usually full time contracts (respectively 68.7% and 41.9%), the graduates in Health and Physic Education have a more precarious situation, with part-time or occasional contracts. Decisional autonomy and responsibility are the main characteristics of the graduates in Agrarian studies (83.8% of them report their decisional autonomy and 68.9% of them report being responsible for other employees – Table 12.2). Similarly, over 70% of the graduates in Science (77%), Medicine (75.6%), Architecture (73.5%) and Economy and Statistics (71.2%) report having a decisional autonomy. The lowest level of decisional autonomy characterise the graduates in Health and Physical Education. Concerning the responsibility in relation to other employees, with the exception of the graduates in Agrarian studies, 50% of the graduates in Architecture report having a certain level of responsibility. None of the graduates in Chemistry and Pharmaceutics or in Psychology reports having a responsibility position in their job.

Table 12.2 Several characteristics of the graduates by group of disciplines, type of job (A employed, B full time and for undetermined length of tie contracts), decisional autonomy (C), degree of responsibility (D), employed on the Sicily labour market (E) and medium monthly net income (F)

For the most part, beyond the affiliation to specific groups of disciplines, most of the graduates are employed in Sicily: over 90% of the graduates in Geo-Biology (100%) Psychology (96.4%) and Architecture (92.70%). Percentages lower than 70% are registered among the graduates in Engineering (66.7%), Health and Physical Education (66.7%) and Science (65.5%) (Table 12.2).

By taking into account the monthly income of the employed graduates, only the graduates in Medicine, Chemistry and Pharmaceutics and Engineering earn more than 1,000 euros (Table 12.2), while the graduates in Literature, Psychology and Health and Physical education earn less than 600 euros monthly. In order to evaluate the opinions reported by the employed graduates regarding the coherence between study and employment, the academic curricula adequacy to the current employment and the overall satisfaction with their work, the analysis takes into account the median value of an indicator built in accordance with the graduates’ remarks. Expressed on a scale from 1 to 4, these opinions have been successively transformed on a scale from 0 (“not satisfied at all”) to 10 (“completely satisfied”) [1]. Based on Table 12.3, according to the three analysed aspects, the graduates in Medicine are among the most satisfied (coherence study-employment 7.5, academic curricula adequacy 7.5, overall satisfaction 9).

Table 12.3 Opinions reported by employed graduates (median values) regarding the coherence between study and employment, the adequacy of the curriculum to the current employment requirements and overall satisfaction with their job description

The graduates in Health and Physical Education are the only other group reporting a higher level of overall satisfaction (9.5). Nevertheless, the same group reports lower levels of satisfaction concerning the study-employment relation (5.5) and academic curricula coherence (3.5). Beside the graduates in Medicine, five other groups of disciplines report a positive evaluation of the three analysed aspects: the graduates in Agrarian Studies, Architecture, Science of Education, Linguistic Studies and Science. Concerning the coherence between study and employment, opinions under-stating a satisfactory level characterised the groups of Law Studies (3.5), Literature (4), Chemistry and Pharmaceutics (5), Health and Physical Education, Geo-Biology Group, Social and Political Sciences and Psychology (5.5). Concerning the academic curricula adequacy to the current job requirements, under-stating opinions were reported by graduates in Health and Physical Education (3.5), Literature (4), Geo-Biology (4.5), Economy and Statistics, Law Studies, Social and Political studies and Psychology (5), Chemistry-Pharmaceutics and Engineering (5.5). Without any differences in terms of belonging group of disciplines, all the interviewees reported an opinion at least satisfactory with the current job.

3 The Multicriteria Electre III Model

Several evaluation approaches aiming to identify the “best” possible solution refer to the utility theory which implies the existence of a univocal utility function. Tracing the decisional aspect back to the maximization of a utility function raises problems for the decision-maker since it does not take into account the different dimensions, the various points of view and the diverse objectives [3]. The optimisation paradigm had been abandoned in various areas of the theoretical research and is regularly criticised by the literature. For example, according to Herbert Simon (1978 Nobel Price for economy), a promoter of a critical discourse on the topic, this is not the “best” alternative one has to achieve (objectively, it might also be impossible to do it), but one should aim to identify those alternatives that “satisfy” a certain number of requirements explicitly defined (the model satisfying choice of H. Simon [11]).

In line with the above, if one intends to analyse a problem, taking into account the various aspects of the issue and its features, it is necessary to adopt a method that replaces the “optimal solution” with a group of “efficient solutions”. According to this approach, defined as a multi-criteria approach, the final solutions depend on the initial conditions identified by the decision-maker him/herself. These decisions must, therefore, be defined and “justified”. According to this criterion, the general approach to a decisional problem consists in using the information together with the opinions expressed by the decision-maker in order to establish a compromise or, in other words, to help the decision-maker to choose the alternative more coherent with his/her structure of preference [7]. In general, there is no possible decision (a solution for the problem or an action to be undertaken) which is simultaneously the best choice from all the points of view considered as being relevant for dealing with the decisional problem. There is, instead, one set of solutions, generally numerous, that provides a logical framework for the choice of a “compromise” solution between the problems and the values that inspire the evaluator.

In the early 1960s, the field of the operative research laid emphasis on the need to take into account a multiplicity of criteria, sometimes in conflict with each other, in order to provide a solution. The solution in cause did not have anymore the characteristics of the “optimal” solution typical for mathematic programs; although it still was an admissible solution, given that, by substituting a single objective to optimize with a plurality of objectives, sometime in conflict with each other, there were not anymore the logical and mathematical conditions for guaranteeing the existence of an optimal solution.

The multi-criteria analysis is still a young theoretical approach, illustrating one set of diversified methodologies which are not yet homogenized in a common theoretical framework. The most recent research fills the gap between the empi-rical aspects and the theoretical systematization. The multi-criteria analysis integrates the following basic components: the actions and their related criteria the decision-maker(s) and the possible support for the data elaboration, the decision rule (rule used for ordering the alternatives according to the information received and the decision-maker’s preferences). The decision procedure generally debouches into the choice between various elements that the decision-maker examines and evaluates according to specific criteria. These elements are considered actions or alternatives and compose the cluster A of actions among which the decision-maker has to operate his/her choice. The definition of A not only depends on the specific problem that has to be solved and the subjects involved in the decision-making process, but strongly interacts with the modelization of the preferences, the definition of the criteria, the enunciation of the problem and, last but not least, the choice between the supporting methods that are applied. Criteria are measured on each action. A criterion can directly provide indications regarding the level of a criterion; in certain cases, a criterion can have a correspondent characteristic. Thus, there might be a characteristic (a set of characteristics) that, indirectly, provides information concerning that criterion.

Among the multi-criteria methods supporting the decision process, special attention is due to the outranking method developed for dealing with problems of choice (the best action among various alternatives), of classification (assignment of actions to more classes which characteristics are known) and of ordering (construction of an order of preferences linked to the set of possible actions). These methods aim to build a relation between the actions, a so called outranking relation, and to use this relation for supporting the decision-maker in dealing with the specific problem.

In all the methods of outranking, pairs of potential actions are confronted on each individual criterion in order to establish if one of the two actions is preferable to the other or if there is no difference at all. The challenge behind the aggregation of the results of these confrontations is dealt by building a relation of outranking (S), understood as the union of elementary relations of indifference (I), light preference (Q) and heavy preference (P). Furthermore, the method takes into account the lack comparability between actions (N), different from the indifference since caused by the existence of contrasting preferences on various criteria making impossible to establish which of the actions is better, knowing that they are not the same.

One can say that action a outranks action a′ (aSa′) if, according to what it is known regarding the preferences of the decision-maker and the quality of the evaluations of the actions, “there are sufficient reasons for considering that a is at least as good as a′ and there are not good reasons for refuting this statement”. The outranking is based on the principle of concordance/discordance,³ in other words on testing the existence of the concordance of criteria in favour of an action instead of another and on controlling that there are not situations of strong discordance among the evaluations able to challenge (the veto issue) the concordance. All the outranking methods provide a structure in steps, in which one is focused on confronting two by two individual criteria and on the aggregation of these results with the outranking modelization (through tests or elaboration of indices of concordance and discordance). The next step uses the outranking relations to reach a final result, by adopting a procedure in order to make operative a coherent decision rule in dealing with the decisional issue. There are various outranking methods the choice among different methods is motivated by indications connected to the nature of the available data and, thus, the criteria that can be used, to the precise decision rule to be made operative, to the presence/absence of thresholds.

Two main families form the category of outranking methods: the methods Electre, oriented towards the choice (Electre I) or towards the ordering (Electre II, III and IV), and the methods of selection/segmentation, dealing with the problem of classification (as Electre Tri). The methods Electre (Elimination Et Choix TRaduisant la REalité), developed by Roy and his collaborators from the University Dauphine – Paris, starting with late 1960s, distinguish themselves by the confronted issues (choice for the first one, ordering for the others), the nature of the data and, thus, the type of criteria (criteria for the first and second, with cardinal scales for the first one and cardinal or ordinal scales for the second; pseudo-criteria are, instead, used for the others together with cardinal scales with thresholds) and the outranking modelization procedure [9].

The analysis carried out in this study uses the multi-criteria decision method Electre III [9]. This approach makes it possible to take into account the imprecision and uncertainty with which the characteristics are often evaluated and, meanwhile, avoid that a solution, unacceptable for on single requirement, can prevail on other; this is obtained by applying a veto threshold to the comparison between two solutions for each considered evaluation criterion.

The third version of the model represents the first attempt of fuzzy outranking relation in the literature and it goes back to 1978 [9]. This method differentiates itself from Electre I and II mainly because it uses pseudo-criteria, in other words criteria to which are associated elements of informative and preferential uncertainty and implies, at the first stage of the method, a fuzzy outranking relation by associating to each relation between ordinate pairs of actions a characteristic function $\delta(a,a')$ that expresses the degree of credibility of the outranking relation.

According to Electre III model, the user has to employ the data (alternatives and criteria) and the decision-makers’ preferences. These preferences are defined according to a weight and three threshold values for each criterion. The weight associated to each criterion corresponds to a coefficient of relative importance that represents one of the most delicate parts of the model since it is the most direct and explicit expression of the decisional preferences and can relevantly influence the results of the method. The thresholds correspond to values that are introduced for limiting two types of risks: the risk of considering distinct two situations corresponding to conditions and values which are very close and substantially equivalent and the risk of not encompassing preferential situations as different. In particular, the indifference threshold (q _j ) refers to the smallest difference, among the values of the criterion j, to which the decision-maker attributes a meaning in terms of indifference. For example, if the difference between two groups of disciplines equals to two points related to the graduating average degree and the indifference threshold for this criterion equals to 3, then the two groups are, in fact, indifferent to this criterion. Only a difference beyond 3 is considered relevant. The preference threshold (s _j ) expresses the minimal difference, among the values of the criterion j, to which the decision-maker attributes a meaning in terms of narrow preference. For example, if the difference between two groups of disciplines equals to 5 points related to the graduating average degree and the preference threshold established by the decision-maker for this criterion equals to 4, then the group of disciplines with the highest degree will be preferred to the other. The veto threshold (v _j ) expresses the minimal difference, among the values of the criterion j, beyond which the decision-maker considers that the gap between the scores is not anymore balanceable by the performances of the other criteria. For example, if the group of discipline A surpasses the group of discipline B by 8 points, related to the graduating average degree, and the veto threshold for this criterion is established by the decision-maker to 5, then B cannot outranks A, whatever the relative value of the other characteristics might be.

Let $A=\left\{a_1: i\in I\right\}$ a finite set of alternatives, evaluated by a family of pseudo-criteria $g=\left\{g_j: j \in J\right\}$, then on the scale E _j of each criterion 3 thresholds are defined (q _j , s _j , v _j ):

$$0 \leq q_{\!j} \leq s_{\!j} \leq v_{\!j}$$

((1))

respectively, indifference threshold, preference threshold and veto threshold. To each criterion is assigned a weight so that to obtain a vector of normalized weights $p=\left\{p_j: j \in J \right\}$, such as

$$\forall j \quad 0 \leq p_j \leq 1 \quad \hbox{and} \quad \sum\limits_{{\textrm{j}} \in {\textrm{J}}} {{{\textrm{p}}_{\textrm{j}}}}= 1$$

((2))

During the first step, the model Electre III is based on the introduction of marginal indices of concordance and discordance for each criterion $j \in J$ and can be summarised as follows:

For each pair of alternatives $(a,a')$ and for each individual criterion, the marginal concordance index is defined according to the comparison between the evaluation differences $g_j (a) - g_j (a')$ and the thresholds q _j and s _j , distinguishing the cases when the criterion is increasing (the judgment on the alternative improves as the criterion value increases) and decreasing (the judgment on the alternative worsens as the criterion value increases).

If the criterion is increasing, then

$$\left. \begin{array}{l} \\ -\, \forall j \in J \ \textrm{if} \ g_j (a)\geq g_j (a'), \ a \ \hbox{outranks the action} \ a' \ \textrm{marginally},\\ \quad a S_j a' \Rightarrow c_j (a,a')=1;\\ -\, \hbox{if} \ g_j (a') \leq g_j (a)+q_j \Rightarrow c_j (a,a')=1 \ \hbox{the two alternatives are}\\ \quad \hbox{indifferent;}\\ -\, \hbox{if} \ g_j (a') \geq g_j (a)+s_j \Rightarrow c_j (a,a')=0 \ \hbox{the alternative} \ a' \ \\ \quad \hbox{outranks the action} \ a;\\ -\, \hbox{if} \ g_j (a)+q_j <g_j (a')<g_j (a)+s_j \ \hbox{an interpolation has to be}\\ \quad \hbox{performed and it is possible to say that the alternative} \ a'\\ \quad \hbox{``weakly'' outranks the alternative} \ a. \ \hbox{By considering, among}\\ \quad \hbox{the possible interpolations, a linear interpolation, then:}\\ \quad c_j (a,a')= \frac{s_j - (g_j (a')-g_j (a))}{s_j -q_j}\\ \\ \end{array} \right\}$$

((3))

If, instead, the criterion is decreasing then:

$$\left. \begin{array}{l} \\ -\, \forall j \in J \ \textrm{if} \ g_j (a)\geq g_j (a'), \ a \ \hbox{outranks the action} \ a' \ \textrm{marginally},\\ \quad a S_j a' \Rightarrow c_j (a,a')=1;\\ -\, \hbox{if} \ g_j (a') \geq g_j (a)-q_j \Rightarrow c_j (a,a')=1 \ \hbox{the two alternatives are}\\ \quad \hbox{indifferent;}\\ -\, \hbox{if} \ g_j (a') \leq g_j (a)-s_j \Rightarrow c_j (a,a')=0 \ \hbox{the alternative} \ a' \ \\ \quad \hbox{outranks the alternayive} \ a;\\ -\, \hbox{if} \ g_j (a)-s_j <g_j (a')<g_j (a)-q_j \ \hbox{an interpolation has to be}\\ \quad \hbox{performed and it is possible to say that the alternative} \ a'\\ \quad \hbox{``weakly'' outranks the alternative} \ a. \ \hbox{By considering, as}\\ \quad \hbox{always, the linear interpolation, then:}\\ \quad c_j (a,a')= \frac{g_j (a')-(g_j (a)-s_j)}{s_j -q_j}\\ \\ \end{array} \right\}$$

((4))

In this way, a concordance matrix for each criterion is obtained; the elements of each matrix are the concordance indices among all the alternatives’ pairs according to the considered criterion.

A similar reasoning concerns the marginal discordance indices, but in this case the veto threshold is introduced.

If the criterion is growing, then:

$$\left. \begin{array}{l} \\ -\, \hbox{if} \ g_j (a') \leq g_j (a)+s_j \Rightarrow d_j (a,a')=0 \ \hbox{the two alternatives are}\\ \quad \hbox{indifferent;}\\ -\, \hbox{if} \ g_j (a') \geq g_j (a)+v_j \Rightarrow d_j (a,a')=1 \ \hbox{the alternative} \ a' \ \\ \quad \hbox{outranks the alternative} \ a;\\ -\, \hbox{if} \ g_j (a)+s_j <g_j (a')<g_j (a)+v_j \ \hbox{an interpolation has to be}\\ \quad \hbox{performed and it is possible to say that the alternative} \ a'\\ \quad \hbox{``weakly'' outranks the alternative} \ a. \ \hbox{By considering the linear}\\ \quad \hbox{interpolation, then:}\\ \quad d_j (a,a')= \frac{(g_j (a')-g_j (a))-s_j}{v_j -s_j}\\ \\ \end{array} \right\}$$

((5))

If, instead, the criterion is decreasing, then:

$$\left. \begin{array}{l} \\ -\, \hbox{if} \ g_j (a') \geq g_j (a)-s_j \Rightarrow d_j (a,a')=0 \ \hbox{the two alternatives are}\\ \quad \hbox{indifferent;}\\ -\, \hbox{if} \ g_j (a') \leq g_j (a)-v_j \Rightarrow d_j (a,a')=1 \ \hbox{the alternative} \ a' \ \\ \quad \hbox{outranks the alternative} \ a;\\ -\, \hbox{if} \ g_j (a)-v_j <g_j (a')<g_j (a)-s_j \ \hbox{an interpolation has to be}\\ \quad \hbox{performed and it is possible to say that the alternative} \ a'\\ \quad \hbox{``weakly'' outranks the alternative} \ a. \ \hbox{By considering the linear}\\ \quad \hbox{interpolation, then:}\\ \quad d_j (a,a')= \frac{(g_j (a)-g_j (a'))-s_j}{v_j -s_j}\\ \\ \end{array} \right\}$$

((6))

Once the J concordance matrices and the J discordance matrices are obtained (they are I×I matrices), one proceeds with the calculation of the I×I aggregated concordance matrix, whose elements are the weighted sum, with the weights initially assigned to each criterion, of the marginal concordance indices:

$$c(a,a')=\sum\limits_{j \in J} {{p_j}{c_j}(a,a')}$$

((7))

By employing the aggregated concordance matrix and the discordance matrices one calculates the credibility outranking matrix, whose elements are obtained as illustrated below:

$$\left. \begin{array}{l} \\ \hbox{if} \ \forall j d_j (a,a')=0 \Rightarrow \delta(a,a')=c(a,a')\\ \\ \hbox{if} \ \exists j \in J: d_j (a,a')>0 \Rightarrow \\ \\ \qquad - \,\hbox{if} \ d_j (a,a') <c(a,a') \Rightarrow \delta(a,a')=c(a,a')\\ \\ \qquad - \,\hbox{if} \ \exists j^* \in J^* \subseteq J: d_j (a,a') \geq c(a,a') \Rightarrow\\ \\ \qquad \qquad \Rightarrow \delta(a,a')=c(a,a')\times\prod\limits_{j^* \in J^*}{\left(\frac{1-d_{j^*}(a,a')}{1-c(a,a')} \right)}\\ \\ \end{array} \right\}$$

((8))

Thus, the final order is established, i.e. the global classification of the alternatives. To this end the distillation algorithm⁴ is used. One introduces a discrimination threshold $s(\delta)$, that is the maximal discrepancy between two credibilities, so that they can be still considered within the same order of magnitude. The distillation algorithm allows to extract from the credibility matrix the alternatives that will belong to the classification. Two distillation algorithms are applied: a descending and an ascending one. Descending distillation selects at first the best alternatives to end the process with the worst ones. On the contrary the ascending distillation selects first the worst alternatives to end the process with the best ones. Two complete pre-orders are therefore found on all the alternatives.

Within the credibility matrix, the maximum degree of credibility δ ₀ is established for the extraction of the alternatives, equal to:

$$\delta_0 = \max\limits_{(a,a') \in A^k} \delta(a,a')$$

((9))

that is the maximum among the values $\delta(a,a')$ at the k-th step (A ^k is the credibility matrix at the k-th step); this determines a “value of credibility”, and only the values $\delta(a,a')$ that are close enough to δ ₀ will be considered. Hence, the discrimination threshold $s(\delta)$ is subtracted and, thus, δ ₀′ is calculated:

$$\delta_0' = \delta_0-s(\delta)$$

((10))

the first level of separation is calculated δ ₁, according to the set A ^k:

$$\delta_1= \left\{ \begin{array}{l} \\ \max\limits_{(a,a') \in \varOmega}\delta(a,a'), \ \textrm{where} \ \varOmega=\left\{(a,a')|\delta(a,a')< \delta_0'\right\}\neq\O\\ \\ 0, \ \hbox{if} \ \varOmega =\O\\ \\ \end{array} \right.$$

((11))

The qualification score $q^\delta(a)$ of each action $a \in A$, where A is a finite set of alternatives, is defined as the number of actions that are outranked by the action a _i minus the number of actions outranking it, i.e.:

$$\left. \begin{array}{l} \\ q_A^\delta(a)=p_A^\delta(a)-d_A^\delta(a) \ \textrm{where}:\\ \\ p_A^\delta(a)=|\left\{ a' \in A : \delta(a,a')>\delta \right. \ \textrm{e} \ \left.(\delta(a,a')-\delta(a',a))>s(\delta)\right\}|\\ \\ d_A^\delta(a)=|\left\{ a' \in A : \delta(a',a)>\delta \right. \ \textrm{e} \ \left.(\delta(a',a)-\delta(a,a'))>s(\delta)\right\}|\\ \\ \end{array} \right\}$$

((12))

The descending distillation algorithm classifies the actions according to the maximal classification, following the rule:

$$q^+=\max\limits_{a \in A^k} q^{\delta_1}(a)$$

((13))

and the following A ^k subset is obtained:

$$D_1^+=\left\{a \in A : q^{\delta_1}(a)=q^+\right\}$$

((14))

where $D_1^+$ is the first distillate from above and each class $C_k^+$ will be built from above starting from this distillation unit. If $D_1^+$ contains only an action, then $C_k^+=D_1^+$ and the above procedure is repeated on all the remaining actions for the next iteration. Otherwise, the algorithm is applied to all the $D_1^+$, generating, in this way, a sub-distillation until only one action will be left. The procedure is repeated starting from $A^{k+1}$ and finishes when all actions in A have been attributed to a class. As previously, the result is a descending distillation. In the distillation from below, the procedure is similar to the previous one but the selection is done according to the minimal qualification rule:

$$q^-=\min\limits_{a \in A^k} q^{\delta_1}(a) \qquad D_1^-=\left\{a \in A : q^{\delta_1}(a)=q^-\right\}$$

((15))

In this case, $D_1^-$ is the first distillation unit from below and each class $C_k^-$ will be built from below. Once obtained the two pre-orders $P(A)^+$ and $P(A)^-$ according to the distillation algorithms, the final order is established. The procedure to define the final order suggested by Schärlig [10] is an “intersection”, according to the set theory, based on the following three rules. Firstly, an action cannot be ahead of another action in the final order, unless it is ahead of it, in one of the two preliminary-orders $P(A)^+$ or $P(A)^-$ and ahead of it or ex equo in the other one. Secondly, two actions cannot be ex equo in the final order unless they belong to the same class in both classifications (from below and from above). Thirdly, two actions are incompatible in the final order if one is ahead the other one in a classification (from below or from above) and is behind it in the other one. The result can be represented as a graph.

4 Groups of Disciplines Ranking

By putting together the data provided by the first part of the analysis, the second paragraph of this chapter aimed to identify the variables (criteria) to be used in the Electre III method. The method has been applied to various groups of disciplines according to: g₁, graduation degree; g₂, duration of effective studying period; g₃, employment percentage; g₄, percentage of full time and undermined length of time contracts; g₅, monthly net income; g₆; coherence study-work; g₇, academic curricula adequacy to the current job; g₈, (overall) satisfaction with the current job; g₉, percentage of employed graduates with decisional autonomy; g₁₀, percentage of employed graduates with job responsibilities; g₁₁, social background⁵; g₁₂, percentage of graduates satisfied with their degree; g₁₃, percentage of employed graduates working in Sicily.

If we consider the code of the groups of disciplines and the criteria associated to them then we have the performance matrix that is reported in the next page.

Performance matrix by group of disciplines
	g1	g2	g3	g4	g5	g6	g7	g8	g9	g10	g11	g12	g13
a1	103.3	1.7	24.2	16.3	696	6.5	7.0	6.5	83.8	68.9	3	73.9	83.8
a2	107.3	1.5	30.1	16.9	957	6.5	7.0	6.5	73.5	50.4	4	64.6	92.7
a3	100.3	1.5	53.9	41.9	1,139	5.0	5.5	8.0	55.6	0.0	3	58.1	88.9
a4	103.2	1.7	23.7	14.3	767	6.0	5.0	8.5	71.2	25.1	3	76.8	89.1
a5	107.6	1.2	60.0	0.0	417	5.5	3.5	9.5	33.3	33.3	2	40.0	66.7
a6	105.7	1.6	10.1	5.0	726	5.5	4.5	6.0	49.2	32.1	4	65.2	100.0
a7	102.4	1.6	20.7	11.4	655	3.5	5.0	7.0	46.8	17.6	3	70.1	78.1
a8	104.0	1.6	18.1	13.9	1,069	6.0	5.5	7.5	63.8	28.4	4	78.2	66.7
a9	102.2	1.4	47.8	17.1	671	6.0	6.5	8.0	63.2	11.4	3	52.3	73.0
a10	106.7	1.5	19.6	6.2	548	4.0	4.0	7.0	66.7	30.8	4	62.1	81.4
a11	105.2	1.6	34.1	16.2	678	6.0	6.0	7.5	47.1	25.5	4	48.4	87.4
a12	107.8	1.1	81.5	68.7	1,169	7.5	7.5	9.0	75.6	33.8	3	73.4	79.9
a13	105.8	1.5	31.0	18.3	819	5.5	5.0	7.5	60.8	34.1	3	68.6	74.0
a14	104.4	1.5	21.0	3.9	505	5.5	5.0	6.0	46.0	0.0	3	67.9	96.4
a15	103.4	1.7	24.7	19.8	980	6.0	6.0	7.0	77.0	46.0	3	74.7	65.5

	g1	g2	g3	g4	g5	g6	g7	g8	g9	g10	g11	g12	g13
Directiona	C	D	C	C	C	C	C	C	C	C	C	C	C
Weight	0.11	0.06	0.13	0.06	0.06	0.11	0.06	0.11	0.06	0.06	0.06	0.06	0.06
Threshold (q)b	2	0.1	5	5	100	1	1	1	5	5	1	5	5
Threshold (s)c	4	0.2	15	15	200	2	2	2	15	15	2	15	15
Threshold (v)d	6	0.4	25	25	600	3	3	3	25	25	3	25	25

1. ^aC = growing, D = decreasing.
2. ^bThreshold of indifference.
3. ^cThreshold of preference.
4. ^dThreshold of veto.

The matrix of the weights and thresholds correlated to the criteria is, instead, the following (see (1) and (2)):

The weights and the veto thresholds have been attributed according to the characteristics of the territory both from the point of view of the labour market and the university’s offer [4]. In particular the veto thresholds are quantified after the explorative analysis of STELLA data and, also, in accordance of the evidences expressed from governmental and job market authorities often participant in round tables on this subject.

The weights were an individual expression of preference expressed by the referee of STELLA initiative (see http://stella.cilea.it) [7].

Starting from the matrix of preference and taking into account the pre-established thresholds for each criterion, marginal concordance and discordance indices among all the potential pair of alternatives, have been calculated according to each criterion following the (3), (4), (5), and (6).

Based on the matrices of marginal concordance and by taking into account the weights initially established by the decision-maker, a matrix of aggregated concordance is built (Table 12.4). Its elements are given by the weighted sum of the indices of marginal concordance (see (7)).

Table 12.4 Matrix of aggregated concordance indices

The elements of the matrix of aggregated concordance are then used together with the matrices of marginal discordance for calculating the outranking indices of credibility (Table 12.5), the starting point of the final ordering.

Table 12.5 Outranking credibility indices

By establishing a discrimination threshold $s(\delta)=10$ such as the minimum significative difference between two credibility indices and by applying to this matrix the distillation algorithm in a ascending and descending approach, two pre-orders can be found in (16) and (17).

According to Schärlig [10], the complete final order results from the two pre-ordering intersection (Fig. 12.1). The final order consists in observing for each alternative how this relates to others in the two pre-order.

$$\begin{array}{lll} &&P(A)^+ = \left\{ a_{12} \right\} \succ \left\{ a_2 \right\} \succ \left\{ a_1 \right\} \succ \left\{ a_4,a_{13} \right\} \succ \left\{ a_{11} \right\} \succ \\ &&\succ \left\{ a_3,a_6,a_8 \right\} \succ \left\{ a_{10} \right\} \succ \left\{ a_9 \right\} \succ \left\{ a_{15} \right\}\left\{ a_5,a_7,a_{14} \right\} \end{array}$$

((16))

$$\begin{array}{lll}&&P(A)^- = \left\{ a_{12} \right\} \succ \left\{ a_{2} \right\} \succ \left\{ a_{1} \right\} \succ \left\{ a_3,a_4,a_5 \right\} \succ \\ && \succ \left\{ a_9,a_{15} \right\} \succ \left\{ a_8,a_{13} \right\} \succ \left\{ a_6,a_{10} \right\} \succ \left\{ a_{11} \right\} \succ \\ && \succ \left\{ a_{14} \right\} \left\{ a_7 \right\} \end{array}$$

((17))

Fig. 12.1

Representation of the final ordering with $\alpha, \beta, \gamma, \delta, \epsilon$ paths

The lack of comparison between certain groups generated 5 different rankings ($\alpha, \beta, \gamma, \delta, \epsilon$), following different paths after the Economy and Statistics group and reuniting with the Psychology group. The group of disciplines a ₁₂ belongs to the first position in both pre-orders and consequently can be at the top of the final pre-order. Being the second in both pre-orders, a ₂ follows immediately afterwards and so on. The alternatives a ₃ and a ₁₃ outrank each other, thence, non-comparable, just as a ₃ does not compare with $a_{11}, a_{13}$ besides observing that a ₃ does not compare with a ₅, a ₉, $a_{15}, a_{11}$ does not compare with a ₃, a ₅, a ₉, a ₁₅, a ₈, a ₆ e a ₁₀, and so on.

According to this ordering, the Medicine group is better valued, followed by Architecture, Agrarian Studies and Economy and Statistics. Despite the fact that the groups of Chemistry and Pharmaceutics and Political and Social Studies place on the fifth position, the groups are incomparable considering that the partial pre-order P(A)⁺ a ₁₃ outranks a ₃ while in the pre-order P(A)⁻ a ₃ outranks a ₁₃. The ranking contains other incomparable groups: Health and Physical Education, Linguistic Studies and Literature, groups that mutually outrank in the two pre-orders. The last positions in the ranking are occupied by the Psychology and Law Studies group. The explanation behind the two groups’ last positions may be linked to the fact that, according to most of the variables taken into account, the two groups occupy the last positions and strongly differentiate themselves from the others (Sect. 12.2).

5 Final Remarks

The results of the method employed in this chapter shed lights on a fundamental organisational issue, allowing thus the stakeholders to understand if the University has succeeded in transferring to the students its academic curricula and simultaneously maintain a high level of attention on the high education system’s needs both at a national and local context. The knowledge and skills developed by the graduate and his/her placement result from both the individual commitment and the efficiency of the academic curriculum. The measurement of the efficiency of the academic curriculum is a delicate issue, mainly if the applied method doesn’t take into account the complexity of the argument. This method has, of course, various weaknesses such as the subjectivity of the criteria weighting, the establishment of the thresholds and impossibility of “measurement” of the distance between the alternatives in the final ordering. Nevertheless, various strong points can be mentioned: the multi-criteria approach encompasses various aspects directly linked to the topic in question, the possibility to weight the criteria and to establish thresholds provides the method with a major flexibility and, thus, it can be adapted to various needs and requirements. By establishing weights and thresholds, this standard method facilitates the decision-making process without ambiguity and, last but not least, the methods easily adapt to statistic softwares (for example R).

Our approach, adapted to the university arena, aims to provide a ranking of the groups of disciplines according to the graduates’ placement. This is the first step for providing the universities a strategic tool able to guarantee a qualitative improvement of the system. More specifically, it aims at providing a support for those in charge with the planning of the various degrees and the guiding of the graduates on the job market. Thus a follow-up agenda of this analysis might imply the development of strategic actions for the improvement of the quality in synergy with the available resources and the preferences of the stakeholders, among which the most relevant are the graduates and the enterprises. Our analysis lays the basis for further developments such as the enterprises opinions on the graduates and, thus, a critical overview of the professional efficiency according to the graduates’ specific academic curricula.