Keywords

1 Introduction

Several types of operations research methods have been developed to help decision-makers evaluate transport projects. A common method to do this is Multiple-criteria decision analysis (MCDA) or Multiple-criteria decision-making (MCDM), ranking or sorting different alternatives based on at least two criteria [8]. MCDM has become more and more popular as it allows to evaluate different kinds of criteria (and not only economical ones). However, in practical transport cases, more than just one individual or group of individuals, called stakeholders, are involved, which can significantly influence or be influenced by the result of the decision [10]. Crucial is thus to incorporate different points of view from several stakeholders into such an analysis. As the result, it can reveal the preferences of different stakeholders, hence allowing easier and clearer decision-making.

MAMCA, an extension of traditional MCDM methods, was proposed for transport project evaluations [18]. During the decision-making process, different stakeholders are taken into account. The concept of stakeholder is involved at the early stage of the evaluation, which leads to a better understanding of the objectives for different stakeholders. MAMCA successfully reflects the preferences of every individual stakeholder and expresses their concerns. It has been applied in various domains, especially in the field of mobility and logistics [1]. MAMCA was used in different scenarios such as evaluating transport policy measures [4] and transport technologies [19]. It has also proven itself as a useful methodology in transport-related decision making [2].

To facilitate the application of the MAMCA methodology, a web tool was developed, called MAMCA software [12]. Since 2016, the MAMCA software has helped decision-makers in different sectors to gain a better understanding of the MAMCA methodology and support them with decision-making. However, as time goes by, the limits of the original MAMCA software were exposed, mainly in the form of the difficulty of extending functions and outdated programming technology. Thus, new software is required to be developed to help MAMCA adapt to fast-paced technology changes, and capable of the situation that massive stakeholders can participate in the evaluation.

In this paper, we will first introduce the MAMCA methodology in Sect. 2. Section 3 presents the new MAMCA software and its distinct features. Finally, we will discuss the future directions made possible by the new MAMCA software in Sect. 4.

In order to present the features and illustrate visualizations of the software, a didactic last-mile case in the supply chain will be taken as an example.

1.1 Supply Chain Management Case Study

The case study entitled “The last-mile in the supply chain” is a fictive case study, but corresponds to real dilemma situations regarding home deliveries. It is aimed to gain insight into the extent to which different alternatives for the last mile of a supply chain for home deliveries contribute to the interests of the different stakeholder groups involved. As the stakeholder groups hold different priorities into different criteria, a multi-actor view is needed to show the different points of view of the stakeholder group. The list of alternatives and the criteria of the stakeholder groups are shown in Tables 1 and 2.

Table 1. The alternatives in the supply chain management case
Full size table
Table 2. The criteria of stakeholder groups in the supply chain management case
Full size table

2 MAMCA Methodology

The steps of a classic MCDM process include the problem statement, alternatives and criteria definition, alternatives screening, scores determination, scores analysis, and drawing of conclusions [22]. Unlike classical MCDM methods, MAMCA takes stakeholder analysis to identify stakeholder groups after defining alternatives. Each stakeholder group can have different criteria tree [17]. In Fig. 1, the overall methodology of MAMCA is shown.

Fig. 1.
figure 1

The methodology of MAMCA [16]

Full size image

In the first step, the potential alternatives to solve the problems are defined. The decision-makers need to identify and classify the alternatives in terms of different scenarios, policy measures and so on. In the second step, stakeholder analysis is taken to identify the stakeholders/stakeholder groups. It is a crucial step in MAMCA as for each stakeholder (group) there is a different criteria tree. An in-depth understanding of each stakeholder group is needed. Next, criteria and the corresponding weights are chosen and defined for each stakeholder group. One or more indicators for each criterion need to be constructed in step four. The indicators can be used to measure each alternative, providing the scale for the judgment.

In step 5, the overall analysis is taken within stakeholder groups. Any MCDM methods can be used to assess the alternatives. The Group decision support methods (GDSM) are well suited in this step such as Preference ranking organization method for enrichment evaluation (PROMETHEE) [3], analytic hierarchy process (AHP) [11]. There is no conflict between stakeholder groups and groups. The final evaluations and results of every stakeholder group will only be confronted at the end of the analysis.

The results of the analysis are presented in step 6. Additionally, a sensitivity analysis can be performed to check the robustness of the results. For each stakeholder (group), the multi-criteria analysis reveals their respective criteria and favored solutions, while the multi-actor multi-criteria analysis indicates the comparison of the different points of view of every stakeholder (group), which supports the decision-maker in making the final decision. Eventually, the actual implementation of the decision chosen is taken. The information collected from the previous steps helps the decision-maker to define the implementation paths.

3 The New MAMCA Software

To fulfill the need of MAMCA assessment with an interaction interface, MAMCA software was developed. However, the studies on MCDM increased every year, the innovated methodologies emerge and evolve fast [20]. The original version of MAMCA cannot integrate more MCDM methods because of the limitation of extendibility. In the workshop, it took time to introduce the MAMCA methodology and the MCDM method will be used in the evaluation. An efficient and simple MAMCA procedure is sought to speed up the workshop. Additionally, the higher capacity number of stakeholders for analysis is asked, to maximize the participation involvement. In order to make the evaluation within a stakeholder group with a large number of participants feasible, extending the MAMCA software as a mass participation tool is needed. By doing this, it is possible to get more opinions from a large stakeholder group like citizens.

Thus, a new version of MAMCA software with high extensibility has been developed to integrate new information technologies and visualizationsFootnote 1. It is written in the software stack of MongoDB, Node.js, Express, React (MERN) [24].

3.1 The Evaluation Steps and Visualizations

The new MAMCA software follows the evaluation structures of MAMCA methodology. In a MAMCA project assessment, the software divides it into 6 steps, which include alternatives identification, stakeholder group identification, criteria definition, criteria weight allocation, alternative evaluation, discussion and results.

After creating a MAMCA project, the project manager is able to define new alternatives, as well as modify and remove them. After defining alternatives, stakeholder groups are identified. Each stakeholder group is described according to the objectives they have regarding the alternatives. These objectives are the criteria used to evaluate the impact of scenarios on stakeholders’ support. With these three first steps, the project manager has designed the architecture of the MAMCA projects. Data are then collected in the next steps to run the analysis.

In the fourth step, each criterion is weighted. The project manager or the stakeholders can manually allocate weights. Still, other allocation methods are proposed in the software. The stakeholders can choose the pairwise comparison, that they indicate their preference intensities for pairs of criteria.

Stakeholders can also use Direct Rating [7]. All criteria will be rated on a 100-point scale. The most important criterion will be given by the highest number. All other criteria are then rated in comparison to the most important one. The rated scores will be normalized. Suppose there is a set of criteria in one stakeholder group, calling \(F = \{f_1, f_2, ..., f_m\}\). \(W = \{W_1, W_2,..., W_m\}\) is the set of given priority scores for the criteria, and \(w = \{w_1, w_2, w_m\}\) is the normalised criteria weights set. The final weight of criterion k will be:

$$\begin{aligned} w_{k} = \frac{W_k}{\sum _{j=1}^m W_{j}} \end{aligned}$$
(1)

In the fifth step, the stakeholders should evaluate the alternatives based on their criteria. Currently, two additional methods are available: AHP developed by Saaty and Simple Multiattribute Rating Technique (SMART) [6]. If AHP is chosen, pairwise comparison is conducted between alternatives.

If SMART is chosen, the preferences of the alternatives can be rated on a 10-point scale. Suppose one stakeholder has to evaluate ta finite set of alternative \(A = \{a_1, a_2,...a_n\}\) The performance score of \(P_i\) of alternative \(a_i\) will be calculated by means of weighted sums [9]:

$$\begin{aligned} P_i = \frac{\sum _{j=1}^m p_{ij}w_{j}}{10} \end{aligned}$$
(2)

Where \(p_ij\) is the performance score of alternative \(a_i\) on the criterion \(f_j\), \(w_j\) is the weight of criterion \(f_j\). The final performance score is divided by 10 in order to keep the score ranges from 0 to 1.

Once all stakeholders in one stakeholder group finished evaluating, the final performance score in the stakeholder group will be calculated in arithmetic mean. Say there are h stakeholders in a stakeholder group \(X = \{X_1, X_2, X_3..., X_h\}\). The set of final scores F is thus:

$$\begin{aligned} F = \lbrace F_i = \frac{\sum _{k=0}^h P_{ik}}{h}; i = 1, ..., n \rbrace \end{aligned}$$
(3)

Finally, after evaluation, the results are visualized. The new version distinct itself from the previous one, introducing lines with different marker symbols. This allows the lines to be more easily distinguished from one another, as well as to offer greater accessibility for black-and-white prints or color-blind readers. The Multi-Actor view as shown in Fig. 2 represents the final scores on different alternatives for each stakeholder group. The lines stand for the alternatives. It is easy to see that different stakeholder groups have different preferred alternatives. This chart represents the value of the MAMCA: it depicts clearly the support of each stakeholder for different solutions.

Fig. 2.
figure 2

The multi-actor analysis

Full size image

The sensitivity analysis is integrated into the evaluation and weight chart. As shown in Fig. 3, The project manager is able to change the weights of criteria in any stakeholder group, hence allowing to check the robustness of the results. As shown in the figure, by clicking the button top-right corner, the project manager can check the weights allocation and evaluation results from different stakeholders in the stakeholder group “Local Authorities”.

If there is more than one stakeholder in the stakeholder group, the box plot of the weights’ difference can be shown when the project manager wants to check the average result of one stakeholder group. As shown in Fig. 4, the box plot of each weight indicates the difference of the weights allocation from different stakeholders. This visualization is especially beneficial when there is a large number of stakeholders in one stakeholder group. This allows the project manager to know if stakeholders are more controversial about the importance of some criteria while having an agreement on other criteria. For example, in Fig. 4, it can be seen that there is bigger deviance in the weight allocation of criterion “Quality of life”, and a less deviance in the weight allocation of the criterion “Network optimization”.

Fig. 3.
figure 3

Actor sensitivity analysis

Full size image
Fig. 4.
figure 4

Average result of the stakeholder group “Local Authorities”

Full size image

3.2 New Features in the Software

Besides the change of software stack, other major changes were made.

High Effective Technologies. The first major change of the software is the replacement of web services. Web services are means to exchange data and information over the network. By building the web services, the frontend of the software and backend can be separated. The web services will communicate from the frontend and the backend of the software. The web services can be built based on two styles, the previous version of MAMCA software relying on Simple Object Access Protocol (SOAP). However, the other style, Representational State Transfer Protocol (REST), which was defined later, has a better throughput and response time. It has the definite advantage over the SOAP style [13].

Another major change of technology is the programming language. Java and PHP are used in the previous version of MAMCA software, which is robust and secure. Oppositely, the new MAMCA software is written in JavaScript, both frontend and backend. With the help of JavaScript, it is possible to make MAMCA a single-page application, that is, a web interface composed of individual components which can be reloaded independently [21]. So there will be no need for reload of the entire page, which can save more resources for the software. The data transaction between the frontend and backend is through JavaScript Object Notation (JSON). As a lightweight data carrier, it is human-readable and efficient. [15] The final result is that the new software has less response time than the previous one. To test the performance of the new software, a controlled trial was taken between new MAMCA software and the previous version of MAMCA software. The same project was chosen, and pairwise comparison was taken in the same stakeholder group “Local Authorities” to weigh the criteria. Network traffic was captured during the weighing, resources loaded and response time was recorded as shown in Table 3. The previous MAMCA software sent 49 requests to the server and loaded 1.2 MB resources in total. Oppositely, the new MAMCA software only sent 2 requests to the server and loaded 1.9 KB resources also with much faster response time.

Table 3. The performance of two versions of MAMCA software
Full size table

New Database Structure. In the previous version of MAMCA software, the relational database is used. MySQL is used as the database management system. In the new MAMCA software, MongoDB is chosen, which is a NoSQL database. It is a document-oriented database, and the data is stored in JSON-like documents. It is more flexible than the SQL as it allows the different structure or fields. Talking about the performance of the database, MongoDB has higher reading and writing speed than the conventional SQL [14]. Furthermore, as a document-oriented database, the data of one project saved in MongoDB is not distributed in different database tables anymore, which is easier to collect and analyze for further data analysis.

3.3 Enhanced “Participation” Concept

It is necessary to get an idea of the needs and objectives of the stakeholders, that’s the reason to develop the MAMCA software. The new MAMCA software is easier to involve more stakeholders in the decision-making process. And in this software, it can have an easier, faster way to evaluate and better comprehension. We did this, by the integration of the SMART method which is a very straight forward way to evaluate alternatives. The new participation system also improves the interaction experience between participants.

The Integration of SMART Method. It was observed during MAMCA workshops that the evaluators most often spent a lot of time to understand the theory of the MCDM method. Also, it was time-consuming when they did the pairwise comparison if there are many criteria. Thirdly, for many it was still perceived as a black box. That’s why SMART is integrated in the software. As the oldest, simplest and most used MCDM method, the reason to apply this method into software is that stakeholders will be able to understand how their input is used to calculate preference scores, which is more unlikely in PROMETHEE and AHP. In contrast to AHP, there is no issue of stakeholders having to perform lots of pairwise comparisons. Another advantage of SMART is that the overall performance scores can be meaningfully interpreted, instead of being a dimensionless index that is only meaningful in comparison to other scores.

Comparing to AHP, SMART sacrifices accuracy and sensitivity for its simplicity. Because of the subjective nature of technique, SMART is not consistent in contrast to the pairwise comparison. It is not suggested to use SMART method to make the final decision but a way to get insight into the objectives with different alternatives in a short time [23]. With SMART, the participants can save more time to comprehend the meaning of the performance score, and understand the importance of the presence of other stakeholders in the group: as shown in (2), it is easy to know the different weight allocation on criteria and the different preference on alternatives from other stakeholders will affect the final score of one alternative.

Easier Interaction Between Participants. The project manager and invited stakeholders can have a better experience in communication and comparison in the new software thanks to the new participation system. It helps facilitate the MAMCA evaluation, especially in a workshop. The Fig. 5 shows how the new participation system works.

Fig. 5.
figure 5

The new participation system in the MAMCA software

Full size image

After identifying the stakeholder groups in a MAMCA project, the project manager can invite stakeholders to the project through the email invitation. As the dashed circles in the Fig. 5 indicates, it is optional that the project manager and stakeholders can identify alternatives together. The stakeholders can also define the criteria of their groups with guidance from the project manager.

The project manager can coordinate the works of evaluators. For example, normally the weight allocation of criteria is more subjective than the evaluation of alternatives. The stakeholders can allocate weights based on their priority. Though when they evaluate the alternatives, they may need help from the experts. They can discuss the consensual performance scores of alternatives in the stakeholder group. The project manager can put the scores they discussed in the evaluation table. After that, the stakeholders are able to use the same performance scores from the project manager with one click of a button.

During the evaluation, the project manager is free to check the weight allocations and alternative evaluations from stakeholders. Also, as mentioned before, after the evaluation, the participants can check the average stakeholder group result. Stakeholders in the same stakeholder group are able to check the result of others. The project manager can do the sensitivity analysis, in order to reach the consensus among all. The new software expresses the differences of MAMCA from the other MCDM methodology: it searches the win-win solutions by taking the different points of view from stakeholders’ accounts. The new software can help stakeholders understand the impact on each other.

4 Discussion and Future Work

The motivation for developing the new software is to make the MAMCA methodology more understandable and more accessible for the participants in the project. The software is refined for ease of use and reliability and is especially suitable for the evaluation in workshops. The integration of the SMART method allows participants to understand the evaluation steps, hence being more transparent.

Because of the characteristic of the development stack, the new MAMCA software is easy to extend functions, which means there can be more features to integrate into the future works.

Improvement on the Concept of “Participation”. The first refinement for further work is to improve the concept of “Participation”. Currently, the stakeholders can discuss the weight differences of the criteria, compare the performance scores they give to alternatives. In the end, the concept should be finished in a closed loop: a consensus-seeking mechanism is thus needed. Doan and De Smet developed an alternative weight sensitivity analysis based on linear programming (MILP) [5]. It can be applied in the MAMCA methodology to offer a consensus between different stakeholders by taking the inverse optimization point of view.

Integration of Other MCDM Methods. As any MCDM method can be used in the MAMCA methodology, especially the GDSM-method as they are able to cope with the stakeholder concept [17], other MCDM methods such as PROMETHEE can be integrated into the software thanks to easy extendibility of the software. By increasing the available methods the users have more freedom to choose suitable methods. For example, evaluators can use PROMETHEE as they provide different preference functions which suitable for different scenarios, or they can choose AHP because of its consistency.

Development for Mass Participation. Because of the flexibility and high performance of the new database in the software, it is prepared for mass participant involvement analysis. A stakeholder group such as citizens is able to include massive amounts of participants with different behaviors and preferences. Subgroups within one stakeholder group can be clustered based on their evaluation or preferences. A model will be designed to analyze and classify this large amount of data.

5 Conclusion

In this paper, the new MAMCA software was introduced to better support the decision-making process of the stakeholders. As the new interaction tool for MAMCA methodology, it follows the evaluation structures of the methodology with a simple and clear user interface. It is aimed to have a better performance in workshop settings. The SMART method is integrated to make the participants focus on understanding the meaning of their scoring instead of spending time to comprehend the theory of the MCDM method. The software enhances the concept of participation during the evaluation. Besides the representative result visualizations, sensitivity analysis and box plots of weight allocations within stakeholder groups are developed. The participants can have a better understanding of the influence of their behaviors and preferences.

The MAMCA software is designed as a tool to understand and analyze the role and input of stakeholders in strategic processes. It can be seen as a transition tool as participants learn to look at the decision problem in a new and more empathetic way. The uniqueness of MAMCA lies in the multi-actor evaluation, as stakeholders learn to see how other stakeholders might have other goals and criteria. In the evaluation process, the stakeholder is aware of the presence of the other stakeholders. There is a learning loop for the stakeholders. The stakeholder can have a better understanding of each other’s position, which makes a stakeholder group more prone to search common solutions, to reach the consensus. The idea is that the habits of one individual should be altered, however not in an imposed way, but rather in a voluntary way. In addition to this, we should be aware that individual behavior is not happening on an island. In the end, the MAMCA software is not a tool to make the decision for the participants, but a tool to help them to understand and analyze the role and input of themselves in strategic processes.