Keywords

1 Introduction

A multicriteria (MC) model can be used to easily express multiple visions of the same problem and synthesize knowledge elements that include quantitative data and intangible values. Moreover, MC methods can deal with a high heterogeneity of model components, without reducing their richness, and can facilitate an easy and direct comprehension of the people who are involved in any way, as decision makers and stakeholders, or at least as proponents of specific visions or of detailed knowledge of some problem elements or domain expertise. MC models and methods can be used in a communication context, even when the problem is not well defined and the main aim is to acquire and structure knowledge, rather than to choose a solution or implement a clear problem statement of ranking or sorting (Norese 2016a). They can be used to identify specific new points of view or to modify already expressed ones, and even to better formulate a decision problem.

MC applications to Public Administrations (PA) have been proposed in literature in relation to different possible decision and problem contexts. A structured visualization, which distinguishes the main complexities MC applications have to cope with, was proposed in (Norese 2016b) by means of a theoretical framework. A specific typology was described, in terms of a new and unstructured decision problem situation, internal to the PA decision system (i.e., a system that includes decision makers and decision structures, with rules and formal relationships with other actors in the decision process), with participants from the involved organization units or with specific expertise in relation to the decision problem situation. Decision aiding activities, in relation to these situations, are often oriented toward defining, activating or improving a new policy or internal procedure (see, for instance, Bana e Costa 2001; Norese 2009; Merad et al. 2013). The Multicriteria Decision Aiding (MCDA) methodology (see the EURO Working Group MCDA website “http://www.cs.put.poznan.pl/ewgmcda/”) adopts a constructivist approach, where the model as constructed, the concepts and the procedures constitute a communication and reflection tool that allows the participants in the decision process to carry forward a process of thinking and to talk about the problem (Genard and Pirlot 2002).

In the policy analysis field, the process is characterized by a cycle of design, testing, implementation, evaluation and review of public policies (Tsoukias et al. 2013). In the 1990s, the British Government defined policy making as a learning process that should be studied, analysed and monitored in order to obtain new evidence that could be used to build future policies. MCDA plays an important role in policy making processes that allocate tangible or intangible public resources. In general, these processes involve a single organization, with different institutional levels and sometimes with different departments. In rare cases, the organizational nature of the decision system is more complex (Norese and Torta 2014).

When a problem situation is new and unstructured, a monitoring action should be associated to each action implementation, but the aims of the monitoring and future use of the acquired data cannot be clearly defined, because of a total lack of previous experience or well tested reference procedures (Norese 2010).

In these situations, a structuring and a visualization of the main aspects facilitate communication between knowledge sources and the involved actors. Methodologies in the field of Problem Structuring Methods (Rosenhead and Mingers 2001) can be used to actively support public authorities during the preliminary phases of complex decision processes, when their uncertainties have to be analysed and reduced, the decision problem has to better formulated and structured, and the feasibility of each action has to be verified. The structuring and visualization potentialities of these methodologies can also play an important role in the monitoring processes of new and unstructured decision problems. Moreover, their potentialities can be multiplied through an integration with MCDA. MC models can be used to transparently synthesize knowledge and allow possible decisions to be formulated and analysed. MC methods can be used not only to aid decision making, but also to describe how a decision system could deal with a problem and which elements of a preference system could be elicited and used to understand the consequences of a policy implementation.

A new and unstructured decision problem, and the monitoring context in which it was formulated, are proposed in the first section, while the second section describes the knowledge acquisition process and the adopted methodological approach. The third section presents the development and the use of two models in the evaluation process, and the last section deals with the applications of MC methods, which were proposed in the monitoring context as examples of a formal and transparent use of acquired knowledge and information elements.

2 The Context

Urban communities and cities in Europe are currently the focus of an intense debate, at both a political and an institutional level, which has identified them as protagonists of a process of redesigning strategic development toward sustainable, smart and inclusive growth models. Cities create “a great combination of new business types of cooperation and employment opportunities with a strong social dimension” (European Commission 2013). The concentration of social and environmental problems and pressure on local welfare systems and on economy are problems that can easily be recognized in urban areas, but, at the same time, the potential of the cities as fields of transformation and laboratories of technological and social innovation can also be recognized. In this context, the city of Turin is making an effort to disseminate a culture of social interaction, aimed at co-designing development policies, in order to stimulate new forms of entrepreneurship in the citizens to respond effectively to local needs. The goal is to transform innovative ideas into new services, products and solutions which, at the same time, create economic and social value for the region and the community.

When the Municipality of Turin decided to invest in social innovation, it involved several organizations from the social economy and non-profit contexts, as well as public and private incubators, in a Public Program and in a network (Turin Social Innovation—TSI) that had the aim of connecting people, organizations and ideas in the field of social innovation. TSI was created in 2013 to promote and disseminate a social innovation culture, as a stimulus to explore new markets, and to promote and support new forms of entrepreneurial projects in a synergic and collaborative economy environment. One of the initiatives of the Public Program, FaciliTO Giovani,Footnote 1 which was elaborated and applied for the first time in January 2014, was to aid social innovation start-ups, through financial support and accompanying measures, in the development of the technical, economic and financial feasibility aspects of their projects. In 2015, the Municipality activated a monitoring process, and the Social Economy Office (SEO) of the Chamber of Commerce, a TSI member, was asked to participate in the process and, in particular, to evaluate the social impact of the funded start-ups. SEO set up a team to analyse several aspects of the monitoring process and to participate in meetings with the municipality. The invitation to evaluate the social impact was discussed and criticized and, eventually, it was refused, above all because the team felt that only some months of project implementation were not sufficient to produce a social impact. The team underlined that the definition of social innovation dynamics cannot be generalized easily, but the presence of some specific elements could indicate a tendency of the projects and the social entrepreneurs of going in the direction of an effective social innovation.

A different kind of involvement of SEO in the monitoring process was proposed: acquiring and using not only financial and other quantitative data, but also knowledge elements and intangible values, in order to evaluate the different attitudes of the start-ups to produce social innovation in the first steps of project implementation. The proposal was accepted, and a working group was created. The group involved the authors: Fabrizio Barbiero, who represented the Municipality of Turin and the FaciliTO Giovani Council; Laura Sacco, who represented SEO as the coordinator of its activities; Laura Corazza and Maria Franca Norese, who contributed with different competences (studies on the development of a shared economy, experiences in social innovation, studies and applications in the MC evaluation, decision aid and problem structuring fields) and specific technical and methodological support from two different University Departments.

Different methodological approaches were adopted and integrated to analyse the behaviour of the start-ups and to evaluate whether they were able to address social needs, in their specific fields, and develop business projects for an inclusive and sustainable economy.

The documentation about each funded project and start-up (above all referring to their initial business plans) was analysed with the aim of organizing a set of interviews with the members of the start-ups, but first with the members of four incubators which, as TSI partners, had accompanied and oriented each start-up to obtain funds. The acquired elements of knowledge were structured and discussed in the working group and then oriented toward two different aims: (i) to help the TSI promoters to better understand whether they had moved and were still moving well toward the promulgation and fulfilment of specifications that characterize the idea of promoting social innovation, or whether it was necessary and possible to introduce some modifications and improvements; (ii) to construct a pilot project that should be general enough to be applied to other situations.

Some cognitive maps were created to include all the acquired knowledge elements, in relation to the FaciliTO Giovani project (above all criticisms, positive judgements and improvement proposals). These elements were illustrated and discussed to facilitate the Municipality monitoring and decision processes. The other elements, in relation to the behavior of the start-ups, were used to evaluate their propensity to produce social innovation.

Logical graphs were elaborated to synthesize and visualize information about the social innovation network each start-up had created (Hermans and Thissen 2009). A pilot multicriteria model was then structured to evaluate the social innovation comprehension of each start-up and the ability of each start-up to implement its social innovation project. The results of the working group were then proposed and discussed with the FaciliTO Giovani committee, in relation to the Municipality monitoring and decision processes.

The adopted approach and its results are presented in the next sections, starting from the inquiry and its main results, which are presented in the second section. The different methodological approaches and their structuring of the acquired elements of knowledge are dealt with in the third section. The work is concluded with some remarks on the different possible uses of the results, in this decision process and for future use.

3 The Knowledge Acquisition Process

The analysis started with an examination of the FaciliTO procedure documentation, and above all of the evaluation criteria of the procedure steps; the former was used to select projects and their access to an initial entrepreneurial support, in terms of an accompanying action and a small quantity of money, and the latter to decide on their access to the financial facilitation process. The role of the FaciliTO committee, which had initially been created to include all the involved actors, was analysed and directly observed by means of working group participation in some committee meetings.

Documentation and data about each funded project and start-up company were then acquired and analysed, to obtain more detailed information about who the companies were and what the history of their ideas was, as well as to organize a set of interviews. The analysed documentation included the situation of the companies when had been accepted for the financial support in the first year of FaciliTO, their business ideas and the business plans the companies had prepared, together with the incubators, in order to obtain financial support at the end of the second step. The main elements of the business plans (the nature of the project, the social and innovation aspects, the positioning of the new idea on the market) were schematized in order to use them to start the interviews. When there were any confused elements in the business plans, they were underlined, for each project, in order to clarify them during the interviews.

Before these interviews, the four incubators that had given entrepreneurial support to the companies after the first project selection step were contacted to describe their involvement in the FaciliTO procedure, and to establish their approach to helping the companies better define their business idea, in terms of social innovation. At that point, a general framework was created for the interviews, which were oral and conducted without a tape recorder in order to create a friendly environment in which the interviewees could express their opinions freely. Moreover, the original framework was adapted each time to the attitude of the interviewees, in order to enlarge specific aspects of interest and allow them to give more details. Each interview was conducted by two people in order to follow the lines of discussion without losing any important concepts the interviewees were proposing or explaining, and each interview lasted about one hour and a half. The results of the interviews were accurately written down and sent, by e-mail, to the start-ups so that they could check the content. In some cases, some parts of the text were changed and/or integrated by the start-ups, and some of the interviewers’ doubts were clarified.

3.1 A Cognitive Mapping Approach to Knowledge Structuring

The texts of the interviews were analysed, structured by means of a cognitive mapping approach (see Norese and Salassa 2014) and used to understand the visions and actions adopted by the start-ups to produce social innovation. The analysis of each interview included a coding of each expressed concept in information cells. A clustering approach was then activated on the coded sentences of all the interviews to identify a possible structure of themes (or topics or main concepts) which, in some cases, were deliberately introduced during the interview, but in other cases often emerged freely, without prompting from the interviewer.

Five main themes were identified: definition of the perceived social needs, of the updated business plan elements, positive opinions or criticisms of the FaciliTO procedure, FaciliTO improvement proposals, descriptions of their social networks (the subjects who could be influenced by the new idea or who could influence the idea and the project) or hypotheses on how their social networks could be created.

All the collected opinions and proposals about the FaciliTO procedure were organized in cognitive maps that can be defined as logical graphs, in which groups of concepts are connected on the basis of relationships of a different kind and can be used to identify specific aspects that require attention and processing or better explanations. The maps were analysed by the working group and then described to and discussed with the FaciliTO committee. The first two clusters were used to revise and complete the project description schemata and were used, together with the components of the last cluster, to start the evaluation process that is described in the next section.

4 The Evaluation Process

After the first interviews, it became evident that the projects were very different, in terms of content, aims and implementation procedures. The main differences between them not only concerned the nature of the innovative idea and the complexity of its implementation, but also the perception of the importance that should be given to the actors who could facilitate a social innovation project to be developed and social needs to be satisfied. The European Commission (2013) guide to social innovation states that “social innovation can be defined as the development and implementation of new ideas (products, services and models) to meet social needs and create new social relationships or collaborations” (page 6). For this reason, the first stage of the evaluation process was oriented toward analysing the completeness and quality of the social innovation network that each start-up created in the first steps of the project development.

Only at that point, was the second stage activated to formally use the knowledge acquired during the interviews and synthesized to a great extent in the social innovation networks, in order to analytically evaluate the different attitudes of the social entrepreneurs to produce effective social innovations.

4.1 The Social Innovation Networks

During the first interviews, the different descriptions of the relationships activated by the social entrepreneurs with possible actors of their social innovation projects were synthesized and visualized in very simple graphs, which became richer and clearer whenever their structure was proposed in a new interview, to obtain information on the networking of each specific social innovation. The general framework of this logical and visual representation of the social innovation networks was defined step by step, and when the structure of this logical graph became stable, each network that resulted from an interview with a funded start-up was sent, by e-mail, to the appropriate company to test the visualization effect of this tool and to check the quality of the working group interpretation of their network descriptions. The reactions of the start-ups were positive, and in just a few cases did they propose a change to include new relationships or new actors.

The structure of this logical graph includes nodes, which denote the actors and their roles in the social innovation network, and arcs that explain the nature of the different relationships between an enterprise and the actors involved in the social innovation project.

Individual or organizational actors may be human or non-human (the terminology that was proposed in the Actor Network Theory (Law 2007) to define and analyse the role of technologies or events in the processes). The actors’ roles in these social innovation projects were indicated by the entrepreneurs during the interviews.

The general framework divides these roles into three categories (see Fig. 1). The first includes human and non-human knowledge sources that can help the start-ups to better define the features of their project ideas. These sources may be taken from literature, research institutes, incubators or from people with professional competences in the specific ambit of the project, but also competitors and production or distribution partners, international events, such as fairs and exhibitions, or events that TSI proposes with a knowledge mobilization aim. The second category comprises potential clients and/or end users who have a direct relationship with the start up, but also commercial agents, or old and new media, which become communication channels that enable the diffusion of a new idea. The last category is composed of “social actors” (in general associations or organizations that express social needs) that may be essential for a better definition of the social needs and the generation of a market for the specific social innovation idea. In some cases, they are directly involved in the innovation project, in others they are included in the social innovation network to bridge the gap between an innovation project and the social needs that have to be satisfied.

Fig. 1
figure 1

General framework of the social innovation network

Full size image

The arcs that explain the nature of the different relationships can indicate mono or bi-directional interactions, which may become more specific (cooperation in the use of knowledge, knowledge acquisition or knowledge transfer, partnership activation, identification of new social needs and so on). In some cases, communication difficulties or interruptions of these relationships can be underlined in the logical graph, together with an absence of communication and of the relationships that are considered essential for the project.

The social actor networks that were elaborated for each start-up (two of which are proposed in Fig. 2), were analysed by the working group and then presented to the FaciliTO committee. The committee appreciated the clear visualization of the differences between the funded start-ups, in terms of their behaviours during the first steps of their project implementations.

Fig. 2
figure 2

Two different social innovation networks

Full size image

4.2 The Multicriteria Evaluation Model

The logical structure of an MC model includes the strategic aspects of the problem (or model dimensions) and their analytical formalization in criteria pertaining to the different related factors. Two main aspects were proposed during the interviews and they could be used to describe the propensity of a funded start-up to generate social innovation. The first is a cognitive aspect, that is, the start-up’s comprehension of the complex concept of social innovation, and the second is an operational aspect, namely, the start-up’s capability to implement a social innovation project. Several different knowledge elements were proposed, during the interviews, in relation to these two model dimensions. They were expressions of specific points of view and attitudes or descriptions of implementation actions and their consequences. Each proposal was analysed and the structured whole analysis was synthesized and formalized as criteria.

The logical Comprehension of the complex concept of social innovation dimension is dealt with in the MC model by means of two criteria. The first is Awareness, which proposes the idea that comprehension starts from the perception of the relational characteristics of each social innovation. However, this perception is easier in some situations than in others where the complexity of the project implementation is high and the enterprise that implements this project is new and does not have a clear vision or knowledge of the specific complexity characteristics. For this reason, the need for new relation activation, to reduce complexity and uncertainty and facilitate social innovation, may not be clearly perceived in these situations. Instead, awareness should be evaluated as being very poor when the same limited perception and comprehension of the basic elements of social innovation are present in enterprises with sufficient expertise that deal with less complex project implementations. The second criterion of the first dimension is Knowledge mobilization. Knowledge and expertise are resources that may be present in a funded start-up and need to be improved in relation to the new project, or have to be acquired and used by means of oriented actions. This criterion evaluates how well knowledge is mobilized and used to improve the comprehension of the complex concept of social innovation.

Two criteria are included in the model in relation to the second logical dimension, that is, Capability to implement a social innovation project. The first criterion is Quality of the social innovation network that the start-up has created, in terms of multiplicity and nature of the activated relations and presence of social actors. The second criterion is Quality of the results that can be generated from the activated relationships, in relation to the definition of the social needs and the verification of the validity and feasibility of the project idea.

The criteria are associated with different scales. In two cases, the evaluation states of the ordinal scales result from documented combinations of values (see Tables 1 and 2). The ordinal scale of the Knowledge mobilization criterion includes only three evaluation states, whose meanings are described hereafter, together with the criterion. The evaluations of the last criterion, Quality of the results, could be expressed in terms of the different levels of importance of the possible results and of the time available to attain them. However, the differences between the analysed implementation processes, in terms of time, were fund to be minimal, and the definition of the different levels of importance of each result was considered a topic that needed to be defined in later phases of the monitoring process. Therefore, the adopted evaluations were only linked to the different kinds of achieved results. The scales and evaluation states of the four criteria are described hereafter in detail.

Table 1 Ordinal scale of the awareness criterion
Full size table
Table 2 Ordinal scale of the quality of the network criterion
Full size table

Awareness

The evaluations of the Awareness criterion are the result of a combination of two aspects (Complexity of the project implementation and Perception of the relational nature of the social innovation) and their values.

Complexity of the project implementation is related to the nature of the project and to the experience of the enterprise, which could either be “new”, that is, created specifically for the FaciliTO funding project, or “old”, i.e. created, and sometimes incubated, before FaciliTO had been set up. Each funded project was different, but the nature of the project, in terms of implementation complexity, could be divided into four different kinds of social innovation project: New service (in general activated by means of Internet technology), New product, Evolution of an existing Service or Evolution of an existing Product. The four situations were ordered, in terms of decreasing complexity, in New product, New service, Product evolution and Service evolution. A logical combination of the different kinds of project with the conditions of new or old enterprise generated four ordered project complexity states that satisfied the conditions of the projects when funded by FaciliTO. These states are described in Table 1.

A good perception of the relational nature of social innovation is underlined in the social innovation network by the presence of social actors in relation with the start-up and of cooperation relations with possible clients and/or final users in the use of the acquired knowledge (Good-G). If the need for relations with certain identified social actors is recognized, but no relationship has been activated with them, perception is Limited (L), and becomes Very Limited (VL) if no social actors are present or have been identified. The combination of these aspects generated an ordinal scale of ten Awareness evaluation states (see Table 1) ranging from 1 (VL perception in conditions of relatively simple project implementation) to 10 (G perception in conditions of very complex project implementation).

Each combinatorial approach generates a scale, whose values/evaluation states have to be defined in relation to the specific decision problem and together with the decision makers. In this case, there was a maximum number of different states of twelve, and they were defined by the number of input components (four situations of complexity combined with three levels of perception), while the final number was the result of a shared analysis of the problem situation.

Knowledge Mobilization

The knowledge sources can be of a different nature, and their identification and a cooperative relationship with them could have been used to mobilize knowledge. The Knowledge mobilization criterion distinguished three levels of mobilization, which were expressed by means of three evaluation states. Mobilization is classified as Reach (R) when multiple knowledge sources are identified and a cooperative action with them is activated to acquire, use and improve knowledge. Mobilization is classified as Limited (L) when a cooperative knowledge acquisition and use action is only oriented toward a single source. Knowledge mobilization is classified as Minimal (M) when it is only oriented toward the analysis of literature and/or the competitor operations.

Quality of the Network

The quality of the social innovation network that each start-up had organized can be evaluated in terms of the presence and, if possible, multiplicity of involved social actors, an aspect that was combined with the nature of the relations that were activated. Each network is different, because each one had to be created in relation to a specific social innovation idea.

Four clearly different situations were considered in the model, in relation to the funded projects: social actors are Absent (A) in the network; some possible social actors have been identified, but no relationships have been activated (F, for Future involvement of identified actors); only One typology of social actors has been activated in the network (O) and Multiple typologies of social actors are involved (M).

Three kinds of non-generic relations were recognized in the analysed networks: Knowledge acquisition or transfer, Cooperation in the use of knowledge and Identification of new social needs. Some of the relations were found to almost always be activated, while others were activated more rarely, but all the kinds of relations were activated in each reach network. Four different network completeness levels were distinguished in the model: each kind of relation (EK), only two kinds (2 K), only one kind (1 K), and no kind (NO). When the two aspects and their characteristics were combined, four combinations were found to be impossible, while the others generated an ordinal scale that included values ranging from 1 to 8 (see Table 2).

Quality of the Results

The activated relationships generated different results during the project implementation process. The interviews identified the following results: (a) identification of new social needs; (a′) improved definition of social needs and the requirements; (b) validity and/or feasibility verification of a project idea; (b’) verification without results; (c) development and management of the relationships with possible end users. Some results may have been more consistent with the aims of the FaciliTO project, but it was not possible to distinguish their different levels of importance, and therefore the number of achieved results was used to evaluate the Quality of the results criterion.

5 An Application

The model was tested in relation to a small group of enterprises and their social innovation projects. Table 3 synthesizes the evaluations of eight start-ups that had been funded in the first year of the FaciliTO project. The same incubator had been involved during the first phase of accompanying measures for the development of the technical, economic and financial feasibility of these projects. The evaluations, in relation to the four criteria, arose from the elements of knowledge that were acquired in the interviews and were used to describe the social innovation networks.

Table 3 Structural elements of the MC model
Full size table

This application was developed above all to demonstrate how a visualization of the network characteristics can be translated into an evaluation model and how an MC model can facilitate a transparent visualization of the differences between the propensities of start-ups to generate social innovation.

Table 3 facilitates a first reading, which underlines how start-ups a1 and a6 show a clearly better propensity than start-ups a2 and a7, because the first group presents the best values in almost all the criteria and the second group the worst ones. The other four start-ups are in intermediate positions. Another reading of the evaluation model can divide the set of start-ups into two groups, the efficient group (or Pareto optimal solutions) and the non-efficient group. Start-ups a2, a4, a5, a7 and a8 are not efficient, because they are dominated by at least one other start-up, i.e. another start-up is equal or better than the analysed start-up as far as each criterion is concerned. The only efficient start-ups are a1, a3 and a6. Therefore, a monitoring process could lead to the activation of actions to improve the limited propensity of some start-ups, above all that of start-ups a2 and a7, and/or to analyse the possible reasons for their limited propensity in the accompanying activities phase and/or in the selection process.

A different approach could be adopted in relation to a problem situation that requires a ranking of the different start-ups (classification problem statement), for example to identify which accompanying actions produced the best propensities to produce social innovation. Another situation could require the assignment of each start-up to a pre-defined category (sorting problem statement), which is associated with a specific management and control action in a monitoring process, to maximize the results when a new procedure has to be activated.

In these situations, the Table 3 model should include other parameters that the problem situation and its actors can propose: weights, which distinguish the criteria in terms of relative importance, and parameters, which translate the nature and risks of a specific decision, for the decision makers, into formal terms, or reduce a negative impact on the result when uncertainty is associated with data and/or evaluations. Structure, components and parameters allow specific MC methods to be applied to an MC model, in order to produce rankings or assignments to ordered categories (Roy 1996).

The limited dimensions of the analysed case (only four criteria that could have almost the same importance and eight start-ups that were evaluated in relation to scales that present a limited uncertainty) can be used to demonstrate how two MC outranking methods, ELECTRE II and ELECTRE Tri, can be used to facilitate decisions (Roy 1990, 1996).

ELECTRE II (Roy and Bertier 1973) was the first ELECTRE method designed specifically to deal with ranking problems. It is now only used in rare situations (to rank actions when no uncertainty is associated with the evaluations), but it is still an interesting option because the complete development of a method application can be described, without the aid of a SW tool, and used to explain the logic of an outranking method. ELECTRE Tri (Roy and Bouyssou 1993; Yu 1992) is a sorting method that is used for many different decision problems and which may easily be associated with different visions of how a problem can be dealt with.

5.1 ELECTRE II

The ELECTRE II method is an outranking method that can be used to deal with the problem of ranking a set of actions from the best option to the worst (Figueira et al. 2005) in the classification problem statement. Like the other ELECTRE methods, ELECTRE II includes two phases: construction of an outranking relation, S, whose meaning is at least as good as, followed by a procedure that applies a decision rule that is consistent with the specific decision problem and is used to elaborate recommendations from the results obtained in the first phase.

The ELECTRE II method is applied to an MC model whose components are: A, a complete set of actions ai ∈ A; a family J of consistent criteria gj ∈ J, which associates, to each ai ∈ A, its evaluation, gj(ai) ∈ E, in relation to a specific criterion gj and its scale E, and inter-criterion parameters.

5.1.1 First Phase of ELECTRE II

The outranking relation S is a binary relation that is used to model preferences between couples of actions. Considering two actions, a and a′, four situations may occur: aSa′ and not a′Sa, i.e., aPa′ (a is strictly preferred to a′); a′Sa and not aSa′, i.e., a′Pa (a′ is strictly preferred to a); aSa′ and a′Sa, i.e., aIa′ (a is indifferent to a′); not aSa′ and not a′Sa, i.e., aRa′ (a is incomparable to a′). If one of the P or I situations is verified, there is outranking. If neither P nor I are verified, there is incomparabily, R, a preference relation that is useful to account for situations in which the decision maker is not able to compare two actions. The ELECTRE II method can only be applied if each criterion is a true-criterion, for which there is strict, or net, Preference for each difference between evaluations and Indifference for the same evaluations. The outranking relation is based on the concordance-discordance principle, which involves declaring that an action is at least as good as another if a “majority” of the criteria supports this assertion (concordance condition) and if the opposition of the other criteria does not generate “too strong” reasons (non-discordance condition). An outranking relation is constructed with the aim of comparing, in a comprehensive way, each pair of actions (a, a′), and the concordance—discordance principle is implemented in ELECTRE II by means of two tests that verify concordance and non-discordance conditions.

Concordance Test

An action a can outrank an action a′, aSa′, if a sufficient majority of criteria are in favor of this assertion. The concordance condition can be defined as follows: the concordance index C(aSa′) has to be at least equal to a concordance level c, and C(aSa′) has to be at least equal to C(a′Sa), in order to consider only conditions of preference and not of indifference. In order to make this definition operational, the criteria are partitioned into J+, which includes the criteria in favour of the first element of the couple (a, a′), J = (when the evaluations of a and a′ are equal) and J−, the criteria in favour of the second element of the couple (a, a′). The weights pj of the criteria included in J+, J= and J− are synthesized in P+, P= and P−.

$$ {\mathbf{P}}^{ + } ({\mathbf{a}},{\mathbf{a}}^{{\prime }} ) = \sum {{\mathbf{pj}}_{{\mathbf{j}} \in {\mathbf{J}}^{ + }}} $$
$$ {\mathbf{P}}^{ = } ({\mathbf{a}},{\mathbf{a}}^{{\prime }} ) = \sum {{\mathbf{pj}}_{{\mathbf{j}} \in {\mathbf{J}}^{ = }}} $$
$$ {\mathbf{P}}^{ - } ({\mathbf{a}},{\mathbf{a}}^{{\prime }} ) = \sum {{\mathbf{pj}}_{{\mathbf{j}} \in {\mathbf{J}}^{ - }}} $$

These weights are used in the concordance test:

$$ {\mathbf{C}}({\mathbf{a}},{\mathbf{a}}^{{\prime }} ) = \frac{{{\mathbf{P}}^{ + } ({\mathbf{a}},{\mathbf{a}}^{{\prime }} ) + {\mathbf{P}}^{ = } ({\mathbf{a}},{\mathbf{a}}^{{\prime }} )}}{{\sum {{\mathbf{P}}_{{\mathbf{j}}} } }} \ge {\mathbf{c}}({\mathbf{level}}\,{\mathbf{of}}\,{\mathbf{concordance}}) $$
$$ {\mathbf{P}}^{ + } ({\mathbf{a}},{\mathbf{a}}^{{\prime }} ) \ge {\mathbf{P}}^{ - } ({\mathbf{a}},{\mathbf{a}}^{{\prime }} ) $$

Non Discordance (or veto) Test

When the concordance condition holds, none of the criteria in the minority should oppose the assertion aSa′ too much. In order to make this definition operational, a set of discordance Dj* is created to include couples of values (e, e’) that are considered too discordant (e is “too much” worse than e’) in relation to the J* criteria, which can activate the discordance test (the test can be activated in relation to all the criteria, but also in relation to just some of them). If (a, a′) is a couple of actions and their evaluations are

$$ {\mathbf{g}}_{{{\mathbf{j*}}}} ({\mathbf{a}}) = {\mathbf{e}}\quad {\mathbf{and}}\quad {\mathbf{g}}_{{{\mathbf{j*}}}} ({\mathbf{a}}^{{\prime }} ) = {\mathbf{e}}^{{\prime }} $$

for at least one of the J* criteria, a does not outrank a′, even though the concordance test for the couple (a, a′) has been passed.

5.1.2 Application of the Two Tests to an MC Model

The two model dimensions shown in Table 3, that is, Comprehension of the complex concept of social innovation and Capability to implement its project of social innovation, may have a different importance that indicates Capability as the most important (55% of the total importance) and Comprehension as strategic but less important (45%). Therefore, the relative importance pj of the four criteria shown in Table 3 is linked to the different importance of the model dimensions. These parameters are essential to apply the concordance test. Other parameters have to be defined to activate the non-discordance test: a set of discordance Dj*, which includes couples of values logically in discordance, in relation to situations, and criteria J*, where a very bad evaluation of an “interesting” action can generate a risky decision, when another action presents a very good evaluation. In this case, there are three J* criteria, while the discordance test is not activated in relation to the Knowledge mobilization criterion, because the logic distance between the three evaluation states is not so high (Table 4).

Table 4 MC model
Full size table

The last parameter that has to be defined is the concordance level. The Concordance condition is modelled in ELECTRE II in order to take into account the notion of embedded outranking relations. There are two embedded relations: a strong outranking relation, which is used in the first phase of the method and generates the input for the second phase, and a weak outranking relation, which is used only in the second phase of the method, when there are actions with the same merit. The strong and weak relations are built thanks to the definition of two concordance levels, cs and cw, where cs > cw. The suggested values for cs and cw are cs = 3/4 and cw = 2/3, and both have to be included in the [0.5; 1-min pj] interval.

The results of the first phase of ELECRE II are synthesized in Table 5, where the eight start-ups are compared (56 comparisons), and the columns J+, J= and J− indicate the criteria (or more precisely their identification numbers) that are partitioned in the three groups. The concordance test is expressed in the two columns (P+ ≥ P–) and (P+ + P=), and when P+ is less than P−, the second part of the test is not useful (the concordance test is not verified) and is therefore not activated. The P+ + P= values are expressed and compared with the concordance level cS, which in this case is 0.76, that is, slightly more than ¾, because the concordance indices are very high for several couples of actions. The cw concordance level, which is used in the second phase, is 2/3.

Table 5 First phase of the ELECTRE II applicatiom
Full size table

5.1.3 Second Phase of ELECTRE II

The outranking relation S, which is constructed in the first phase, can be represented by an outranking graph, where the actions are the nodes and the oriented arcs indicate the presence of an outranking relation between two nodes (see Fig. 3). The second phase activates two iterative procedures on the graph to produce two preorders (i.e. orders that accept an element in joint position with others in some classes). The first procedure is oriented toward identifying, at each iteration, a sub-set of actions that follow the “the best actions are not outranked” rule (ascending procedure), and the second procedure actions that follow the “the worst actions do not outrank any other action” rule (ascending procedure).

Fig. 3
figure 3

Outranking graph

Full size image

If the graph does not include circuits, at least one action is consistent with the procedure rule at each iteration. When only one action is consistent with the rule, it is assigned to a preorder class and eliminated from the graph. When more than one action is identified by the rule, a weak outranking relation is applied, by means of a weak concordance level, cw, to the sub graph that includes the identified actions. The same rule is then applied to the sub graph.

At the end of the second phase, the intersection of the two preorders produces the result, that is, a final partial graph (some remarks on the analysis of these graphs have been proposed in Norese et al. 2016)

Application to the Second Phase of ELECTRE II

In the second phase, the descending (P(A)+) and ascending (P(A)−) procedures are applied to the outranking graph shown in Fig. 3 (which is without circuits). Each arc represents one of the outranking relations that were modelled in the first phase, in relation to the concordance level cS = 0.76.

P(A)  +  (descending procedure, to create a ranking from the best to the worst)

The actions that are not outranked are identified at each iteration.

Iteration 1: A1 = A

D1 = {a3, a6}

D1 includes the two actions that are not outranked. The weak outranking relation is activated in order to distinguish between the actions. It adopts the weak concordance level cW = 0.67 in the concordance test, in relation to the sub-graph which only includes the actions of D1. The weak outranking relation can distinguish between the actions: a6 is the only action that is not outranked, and only this action is therefore assigned to the first class, C1+, of the descending pre-order.

C1+ = {a6}

Iteration 2: A2 = A1\C1+ =  {a1, a2, a3, a4, a5, a7, a8}

D2 = {a3, a4}

C2+ = {a4}

Iteration 3: A3 = A2\C2+ =  {a1, a2, a3, a5, a7, a8}

D3 = {a3, a1}

C3+ = {a3, a1}

In this case, the weak outranking relation cannot distinguish between the two actions, which are assigned to the same class together. After the fourth iteration, the outranking graph is completely changed (see Fig. 4) and only includes four actions.

Fig. 4
figure 4

The outranking graph after the fourth iteration

Full size image

Iteration 4: A4 = A3\C3+ = { a2, a5, a7, a8}

C4+ = {a8}

Iteration 5: A5 = A4\C4+ =  {a2, a5, a7}

D3  =  {a2, a5}

C3+ = {a5}

Iteration 6: A6 = A5\C5+ =  {a2, a7}

C6+ = {a2}

Iteration 7: A7 = A6\C6+ = {a7}

C7+ = {a7}

A8 = A7\C7+ =  Ǿ → |A8| = 0 STOP

P(A) + (sequence of the classes from the best to the worst) = {a6}, {a4}, {a1, a3}, {a8}, {a5}, {a2}, {a7}

P(A) – (ascending procedure, to construct a ranking from the worst to the best)

The actions that cannot outrank any other action are identified at each iteration.

Iteration 1: A1 = A

C1− = {a7}

Iteration 2: A2 = A1\C1− = {a1, a2, a3, a4, a5, a6, a8}

D2 = {a2, a5}

C2− = {a2}

Iteration 3: A3 = A2\C2− = {a1, a3, a4, a5, a6, a8}

C3− = {a5}

Iteration 4: A4 = A3\C3− = {a1, a3, a4, a6, a8}

D4 = {a1, a8}

C4− = {a1, a8}

Iteration 5: A5 = A4\C4− = {a3, a4, a6}

D5 = {a3, a4}

C5− = {a3}

Iteration 6: A6 = A5\C5− = { a4, a6}

C6− = {a4}

Iteration 7: A7 = A6\C6− = { a6}

C7− = {a6}

A8 = A7\C7+= Ǿ → |A8| = 0 STOP

P(A) − (sequence of the classes from the worst to the best) = {a7}, {a2}, {a5}, {a1, a8}, {a3}, {a4}, {a6}

The two preorders are similar and their intersection proposes, as final result, a ranking in which the sequence is

{a6}, {a4}, {a3}, {a1}, {a8}, {a5}, {a2}, {a7}

5.2 Sorting Problem Statement and ELECTRE Tri

In a sorting problem, each element of a set A (or an evolving set A(t)) of candidate actions is considered independently from the others, in order to determine its intrinsic value, an absolute judgement that is not influenced by the performance of the other candidates (Figueira et al. 2005).

Each candidate has to be assigned to one of the pre-existing categories, whose typical elements can be defined by levels of adequacy/urgency/priority/risk/…, or by reference profiles that express local/general norms/standards, or management and control activities that have to be arranged. The assignment results are expressed using the absolute notion of “assigned” or “not assigned” to a category, “adequate” or “not adequate” to some norms, and “similar” or “not similar” to a reference profile that represents a quality level, an activation level of a plan or a control action.

Each category (or segment or class) is conceived in order to receive certain potential actions that conform with the assignment norms—which include reference actions and assignment procedures—that characterize the category. These assignment norms are not always made explicit or completely formalized in the decision systems. In this sense, a sorting problem has to be faced through a three-step procedure: the first step includes modelling/validation activities of the assignment norms, the second includes the exploitation of the outranking relation procedure and the third the assignment to categories procedure.

Each action is evaluated in relation to a family J of consistent criteria and compared with a set of reference actions, or profiles, that have been evaluated on the same criteria. These reference actions, which can be typical elements of the categories or bounds that distinguish the categories, have to be indicated in the first step of each sorting procedure, and defined in relation to the problem and therefore to the chosen method.

The ELECTRE Tri method was specifically designed to sort a set of actions A, evaluated on the basis of criteria J, into a set of predefined and ordered categories (classes or groups), denoted here by Ch. The assignment of a given action, a, to a certain category, Ch, results from the comparison of the action, a, to the profiles bh-1 and bh that define the (lower and upper) limits of the categories. The outranking relation is built in order to enable a comparison of an action a with a profile b.

The model (Table 3) and the ELECTRE Tri method could be used in the monitoring process, in relation to the problem described in the second section, to assign each start-up to a different “need of control” category. In this case, the application of the method has not been described, because the aim of the paper is only to underline the different problem vision that this approach can make explicit.

The situation is described logically in Fig. 5, where one action (continuous line) is included completely within category C1 (need for an immediate control action) and another action (dotted line) is included in category C3 (control action is not required) for a most of the criteria, but with an evident discordance (a bad performance in relation to the last criterion), which could require an investigation action to better understand the strange and perhaps risky situation. This methodological approach is particularly consistent with the aims of a monitoring process, in terms of both easy visualizations of local policies and of an analytic assignment of each action to a category, an assignment that is absolute, i.e. independent of the other action assignments, and directly connected to the formal expression of a policy.

Fig. 5
figure 5

Categories and action assignment

Full size image

6 Conclusions

MCDA proposes tools that facilitate communication in decision processes and activate a process of thinking, in relation to the several components of a problem situation.

These tools are models, procedures and methods, but also concepts, which can be used with different meanings in debates involving opposing viewpoints that have to be clarified and shared, or which present different meanings, in the involved knowledge fields, that have to be harmonized.

MCDA facilitates a shared definition of concepts in models of a different nature and can integrate different models in a unified, formal and procedural approach.

Intangible values can be explicitly included in MC models and used to facilitate decisions. PA in general and social innovation, in particular, can benefit from these MCDA features.

In this case, cognitive maps and actor networks were built and integrated in an MCDA approach that involved the actors of Turin Social Innovation in relation to the monitoring of a Public Program.

The immediate and easy visualization of these tools was appreciated by both the Program Committee and the evaluated start-ups. The possibility of analysing activities and events, in the social innovation context, and of expressing values, in terms that are analytical but neither quantitative nor financial, were judged positively by the involved organizations.