1 Introduction

The mining industry in Quebec is a major contributor to the socioeconomic development of the nation through the exportation, job creation and technological progress. In spite of this contribution, the reputation of the Quebec mining sector has been compromised by certain unenviable factors and unfortunate events (Rodon and Lévesque 2015). This situation has motivated several players to undertake the transition of this industry towards sustainable development. Several elements explain the perceived need for such a change. These include new and constraining legislation regarding the environment and occupational health and safety as well as increasing public awareness of the negative impact that the industry can have in these realms (Jenkins 2004).

Current models of the socioeconomic impact of industries present limitations that raise questions about the reliability of the evaluations they provide. In the case of the mining industry, the criteria and indicators of impact on sustainable development often appear to be poorly matched with the setting being studied (Petrov et al. 2013). Another concern is the suitability of the method used to measure the performance of the mine. The challenge of controlling the subjectivity of expert judgments continues to undermine the reliability of evaluations. In spite of much research, the calculations in current use have yet to reduce this subjectivity to any significant degree (Su et al. 2010). The association of different types of indicator (quantitative or qualitative) is not always considered, and this has complicated their coherent integration into calculation models (Petrov et al. 2013).

This article reports the findings of a research project divided into two parts. In part one (Gueye et al. 2020), aspects of the social and economic impact of mining in Quebec as well as criteria and indicators relevant to the evaluation of its transition towards sustainable development were identified. The goal of part two was to develop a socioeconomic impact evaluation model based on the study of models currently used. This model will be built into a decision-aid tool for promoters of mining projects in Quebec, to help them adjust better to the expectations of various groups involved (stakeholders, communities) and thereby ensure greater overall success of future mining projects.

This article is structured as follows: The research problem is presented in Sect. 2, the research methodology in Sect. 3, the results in Sect. 4, a discussion in Sect. 5 along with the limitations of the study, and finally a conclusion.

2 Research problem

The research focuses on four problems encountered in attempting to evaluate the impact of decisions and actions in the mining sector: (1) limitations of impact evaluation methods applied to sustainable development, (2) problems associated with using criteria and indicators of sustainable development, (3) constraints associated with legislation and standards, (4) conflicts between promoters and local communities. Figure 1 summarizes these problems and their underlying elements.

Fig. 1
figure 1

Elements of the research problem

Full size image

2.1 Limitations of socioeconomic impact evaluation models

A survey of tools used to analyse the socioeconomic impact of mining has revealed various problems affecting the quality of the evaluations. These problems stem from the subjectivity of judgments by stakeholders as well as poor grasp of interactions between various sources of impact. Another problem slowing the development of these tools concerns their enablement in specific settings. None of the tools examined has been designed for Canadian mines or for Quebec mines in particular. This reason alone is already sufficient justification for developing an impact evaluation tool for identifying deficiencies in the transition of mines in Quebec towards sustainability.

2.2 Problems associated with using criteria and indicators of sustainable development

The choice of sustainable development criteria and indicators is an important step in the evaluation of the social and economic impact of a mine, one that continues to confound experts. The chosen indicators do not always fit the goals of the evaluation or reflect the impact, and may lead to biased interpretations. There is the additional problem of matching indicators with the industrial setting. Some mines are located in isolated regions (e.g. the arctic), and the indicators in general use in the mining sector may not reflect the impact actually felt in such regions. In practice, different indicator classifications abound in spite of guides and terms of reference used to facilitate their organization and use in the mining industry in general. Some experts rank them on the basis of dimensions of sustainable development, whereas others prefer the subdivision of mining activities or the decision level (strategic, tactical or operational) as a basis. Uniformity in the ranking of sustainable development indicators in the mining sector would facilitate comparisons of evaluation results. Proper selection and ranking of criteria and indicators are expected to improve the reliability of the proposed model and hence its credibility as a decision-aid tool.

2.3 Constraints associated with legislation and standards

In Quebec, sustainable development legislation and standards are becoming more and more demanding. Mining activities are carried out within a rather rigid framework comprising over 100 laws, regulations and standards (AMQ 2016a). In enforcing these, the balance between the interests of promoters and local residents is frequently disregarded (Amos and Audoin 2009). This imbalance shows up also in litigation between mining promoters and other stakeholders. In view of these criticisms, a new model of socioeconomic impact would help the transition of this industry towards greater sustainability. Its application should facilitate the identification of weaknesses in the industry and of opportunities to improve the current standards.

2.4 Conflicts between promoters and local communities

The local community is a strategic stakeholder in the evaluation of the social and economic impact of mining activities. This is apparent in the connexion between the social acceptability of projects and the quality of the relations between the promoters and the local community (Bergeron et al. 2015). The review of the literature shed light on the reluctance of local communities facing mining projects. Mining companies maintain that public opposition affects the likelihood that their projects will receive governmental approval (Wilson and Green 2013). Local communities are manifesting their will to control the natural resources in their regions. In spite of efforts by the mining industry to improve its relations with local communities, a tendency to neglect commitments persists in the practices of some promoters in the field (Bergeron et al. 2015). The development of a new model of social and economic impact evaluation could resolve this problem at least partly, by shedding light on the perceptions that are threats to harmonious relations. This work will be based primarily on the presumption that all stakeholders are (or can be) involved in such evaluations.

3 Research methodology

The research comprises three steps. A review of the literature was carried out in order to present the state of the art with regard to the research subject matter. Suitable criteria and indicators of sustainability for evaluating the socioeconomic impact of a Canadian mine are then presented. The final step is devoted to structuring the proposed model of socioeconomic impact evaluation. The three steps are outlined in Fig. 2.

Fig. 2
figure 2

Steps of the research methodology

Full size image

3.1 Review of the literature

The results of the review of the literature are described in detail in a previous article (Gueye et al. 2020). The survey and summary of the factors having social and economic impact provided a clear vision of their nature in the context of mining in Quebec. Identification of criteria and indicators provided an overview of opportunities for improvement in sustainability. Focused on mining during the years 2005 through 2017, the survey and critical analysis allowed identification of factors affecting the reliability of the current performance evaluation models. All of the above should allow the development of a more reliable model for mines in Quebec.

3.2 Identification of criteria and indicators relevant to sustainability

The criteria and indicators must provide information on performance with regard to the identified social and economic impact. The steps leading to their determination are summarized in Fig. 2.

3.2.1 Correspondence between socioeconomic impact and sustainability indicators

The social and economic impact of mining, as identified in the review of the literature, is multiple and specific to each mine. This study was therefore limited to mines in Canada, to make the impact and the elements associated with its evaluation easier to identify and to determine their relevance by comparing them with criteria and indicators of sustainability drawn from scientific articles and terms of reference (Gueye et al. 2020).

The criteria and indicators of sustainable development used on an international scale are numerous. Among those found in guides and terms of reference, a family corresponding to social and economic impact factors relevant to the Quebec mining industry can be identified (Gueye et al. 2020).

3.2.2 Codifying the selected criteria and indicators

The criteria and indicators are codified in order to organize the data and facilitate their utilization in the impact evaluation model to be developed. Figure 3 summarizes the steps involved in identifying the criteria and indicators relevant to the Quebec context.

Fig. 3
figure 3

Identification of criteria and indicators of sustainable development (in the Canadian context)

Full size image

3.3 Development of the impact evaluation model

Various steps of the development of the social and economic impact evaluation model are described in detail below. The principal focus of this effort needs to be on the ease of use of the model and its reliability. The methodological approach adopted is based on a previous study (Yu et al. 2005) for reasons explained elsewhere (Gueye et al. 2020). In brief, the model proposed by Yu et al. (2005) has two major issues that we have improved in this paper: (1) uncontrolled interference between social and economic indicators and (2) absence of a procedure for the selection of criteria and indicators of sustainable development.

The structure of the impact evaluation model is shown in Fig. 4.

Fig. 4
figure 4

Structure of the impact evaluation model

Full size image

3.3.1 Selection and definition of criteria and indicators

The period of evaluation is the latest year considered in the mining company annual report. In the case of indicators not measured during this year, the period during which they were significant was used. This adjustment was necessary in order to take into account the efforts of the mining company in sustainable development, for example, the indicator “investment in education”, the effects of which can be felt over several subsequent years. The indicators were selected to evaluate the social and economic impact of a mine as felt by the local community. According to the GRI (2015, p. 248), the local community is comprised of: “Persons or groups of persons living and/or working in areas affected economically, socially or ecologically (positively or negatively) by the activities of the company. The local community may include persons living near the site of the activities of the company, as well as isolated localities far away but likely to feel the consequences of these activities”.

The following abbreviations were used for codifying purposes: CRT (criterion), ECN (economic) and SOC (social). It should be noted that numbering associated with these abbreviations does not imply any ranking. The definition and method of calculation of the selected criteria and indicators of sustainable development are listed in Table 1.

Table 1 Summary of the relevant socioeconomic criteria and indicators
Full size table

It should be kept in mind that some of the indicators drawn from the literature are not specific for mines and that the way the associated impact was evaluated is not specified. The adjustments made to some of the formulas are indicated in the column “Adapted from” in Table 1. For example, the indicator SOC2 refers to the distance of the communities from the mine, although the distance at which the impact is felt is not specified. Furthermore, it could refer to a place of cultural gathering or to all such places. Regardless of the indicator used, the evaluators need to agree on a definition.

3.3.2 Ranking and weighting of sustainable development criteria and indicators

The importance of a criterion or indicator of sustainability used to evaluate social and economic impact varies from one expert opinion to the next. In this study, the relative importance of each criterion or indicator was set using a weighting factor using the AHP method. Proposed initially by Saaty (1980), this method is applicable to multivariate decisional contexts, for example, as a decision aid in the evaluation of industrial system performance or in risk evaluation (Cherrared et al. 2011). The steps of the AHP method are summarized in the paragraphs below.

  1. (a)

    Reducing a complex problem to a hierarchical structure

In the present case, breaking down the problem of identifying relevant criteria and indicators of sustainable development and assigning weights to these is presented as a structured hierarchy of elements, illustrated in Fig. 5.

Fig. 5
figure 5

A hierarchical structure of elements based on the analytic hierarchy process (Saaty 1980)

Full size image

For reasons mentioned below (Sect. 3.3.2.e), the final hierarchical level (alternatives) is not taken into account in this study.

  1. (b)

    Paired comparisons of elements occurring within the same hierarchical level and having the same hierarchical parent

To establish preferences among the criteria and indicators, these were compared in pairs using a nine-point Saaty (1980) scale of relative importance.

  1. (c)

    Determination of the weighting factors associated with each element

A weight is calculated for each element. This weight represents the relative influence of an element within the element to which it is linked in the level immediately above. A weighting coefficient vector is determined for each group of indicators linked to a criterion.

  1. (d)

    Checking the consistency of the results

In this final step, the AHP method can be used to provide an index of the consistency of the evaluators’ judgments.

  1. (e)

    Evaluation of alternatives

In this final step, the alternatives are ranked (in terms of the overall aim of the study) in order to provide an enlightened choice. However, this step is not considered in our model. As mentioned above (Sect. 2.3), the goal of this study is not to rank the results obtained but rather to indicate the level of performance achieved by a mine, in terms of the economic and social dimensions of sustainable development. To evaluate performance level (once the weightings are established using AHP), a FIJ procedure was used, thus completing the model. In addition to being adapted for evaluating mines based on the relevant indicators, the FIJ presented in Sect. 3.3.3 allows a considerable decrease in the subjectivity associated with the evaluators’ judgments.

3.3.3 Evaluation of social and economic impact from a sustainability perspective

In spite of the wide variety of scientific literature on evaluation of the impact in terms of sustainable development, the definition of this concept remains nebulous (Kommadath et al. 2012) or varies depending on the field in which it is being used. No consensus has been reached on what sustainable development means in practice, making it difficult to evaluate the impact of industrial activities from this perspective (Muñoz et al. 2008). In the case of mining activity, it is often difficult to determine if the impact is positive or negative. An example is the social dimension indicator called respect of cultural heritage. It is impossible to analyse the impact unambiguously using a binary logical value system, that is, sustainable or unsustainable. Persons affected by the impact answer this question as a function of the importance they attach to this heritage, which is seldom a cost–benefit function, and the evaluation model must take this nuance of opinion into account.

In order to achieve its goal, the model must control the subjectivity inherent in opinions. In analyses influenced by ambiguity and subjectivity, fuzzy set theory may provide a solid basis for the development of an evaluation model. Formalized in 1965 by Professor L.A. Zadeh at the University of California, this theory was developed in response to the need to carry out evaluations influenced by human judgment, behaviour and emotions. It may be suitable for a model used to judge whether or not the impact of a mining activity is a contribution to sustainable development. A judgment is considered fuzzy if it is based on vague and imprecise information. One or several key variables called focal factors may influence the judgment of an impact.

In the present case, these variables correspond to the sustainable development criteria and indicators. For example, a loss of confidence in the local authorities may be noted in a mining region. Its impact can be exacerbated if there is an appearance of collusion between mining promoters and these authorities. A judgment is fuzzy if it depends on consideration of more than one key variable (Zhao et al. 1996). As an extension of fuzzy set theory, the FIJ method is designed to maximize the relatedness of an indicator to a judgment, for example, of social and economic impact to the principles of sustainable development. This maximization leads to an integrated solution to a problem influenced by multiple variables in an uncertain environment (Yu et al. 2005). Following a study of the application of fuzzy logic theory to the optimized integration of linguistic judgments based on several factors (Zimmermann 1986), the FIJ method emerged and became useful in industrial settings (Zimmermann 2010), for example, in new product development. By minimizing the uncertainty in the judgments, in-house quality and technical constraints can be matched with client preferences in the form of a diagram defining the relationship between client needs and company capabilities (Karsak and Dursun 2015). FIJ was then combined with AHP to improve energy efficiency through the choice of compressors used in industrial settings (Taylan et al. 2016) and used alone to develop a multi-criteria evaluation framework for determining the most reliable supplier of radio-frequency identification (RFID) service, in this case rejecting suppliers that were judged acceptable using the technique for order preference by similarity to ideal solution (TOPSIS) system (Büyüközkan et al. 2017). FIJ has been used to evaluate strategic decisions of managers operating in nebulous economic environments (Zavadskas et al. 2017), providing better control of the uncertainty in judgments made by stakeholders and increasing their confidence in the solutions identified. In addition to its flexibility and robustness, the FIJ approach was found to provide deeper insight into the decisional problem and therefore a more enlightened and rational solution.

In spite of its effectiveness, the FIJ method is not used widely to evaluate sustainable development in the mining industry. It has been used to evaluate the coordination of sustainability factors in Chinese mining towns (Yu et al. 2005) and to evaluate the degree of sustainable development as a decision optimization aid for stakeholders in the mining sector in India Kommadath et al. (2012). There appears to be an opportunity to study the use of FIJ as method suitable for evaluating social and economic impact in a nebulous framework (sustainable development). The advantages of this method include the possibility of combining quantitative and qualitative variables (Munda et al. 1995) and aggregating judgments while keeping the goal of the evaluation in sight (Zimmermann 2010).

The model presented in this section is based largely on the Chinese mining town study in which the FIJ method was used to determine the degree of coordination of sustainability factors by maximizing the membership function (Yu et al. 2005), which determined a gradation in the relatedness of an impact to the principles of sustainable development. Introduced decades ago (Zadeh 1965), the membership function provides an alternative to Boolean logic and thus allows an element to belong more or less to a subset. In the present project, the maximization approach is used to determine relatedness. The steps of this method are summarized below.

  1. (a)

    Construction of fuzzy sets

Two sets are constructed in this step, one containing the factors and one containing the associated weights. The factors are the economic and social dimensions as well as the sustainable development criteria and indicators. The weights are determined from expert judgments using the AHP method. The term “expert” refers to the participants in the determination of the weights, whereas “stakeholders” refers to participants in the evaluation of impact using FIJ.

  • Let U be a set of factors of level X of the AHP hierarchy, defined as follows:

    $$U = \left( {u_{1} ,u_{2} , \ldots ,u_{n} } \right)$$
    (1)

    where ui (i = 1, 2… n) is the ith factor.

  • Let A be the vector of weighting coefficients of the level X factor set:

    $$A = \left( {a_{1} ,a_{2} , \ldots ,a_{n} } \right)$$
    (2)

    where ai is the weight assigned to factor ui with

    $$\mathop \sum \limits_{i = 1}^{n} a_{i} = 1$$
    (3)

  • Let V be an array of judgments:

    $$V = \left( {v_{1} ,v_{2} , \ldots ,v_{p} } \right)$$
    (4)

The scale (or judgment set) used to evaluate the set of factors for each sustainable development dimension consists of five levels. An example of such a scale used in a nebulous environment is shown in Table 2 (Yu et al. 2005).

Table 2 Example of an evaluation (judgment) scale usable in a nebulous environment
Full size table
  1. (b)

    Construction of the fuzzy judgment matrix

A single judgment value is assigned to factor ui (i = 1, 2… n; j = 1, 2… m), ui being an element of set ui, in order to determine its relatedness \(r_{ijk}\) to judgment \(v_{k}\) (k = 1, 2… p). Thus, \(r_{ijk} = \left( {r_{ij1} ,r_{ij2} \ldots r_{ijp} } \right)\) gives the judgment matrix for the above factor set U:

$$R_{i} = \left( {\begin{array}{*{20}c} {r_{i11} } & {r_{i12} } & \ldots & {r_{i1p} } \\ {r_{i21} } & {r_{i21} } & \ldots & {r_{i2p} } \\ \ldots & \ldots & \ldots & \ldots \\ {r_{im1} } & {r_{im2} } & \ldots & {r_{imp} } \\ \end{array} } \right)$$
(5)
  1. (c)

    Determination of the integrated fuzzy judgment vector

This vector is an aggregate obtained by assigning weights to the judgments attributed to the factors of the next hierarchical level (criteria or indicators). The result of this aggregation represents the evaluation of the factors recorded at this level. With factor set U being composed of n indicators at level X, let B be the integrated fuzzy judgment vector for level X − 1. Vector B thus represents the evaluation of the set of factors recorded at level X − 1.

$$B = \left( {b_{1} ,b_{2} , \ldots ,b_{p} } \right) = A \times R = \left( {\begin{array}{*{20}c} {a_{1} } \\ {a_{2} } \\ \ldots \\ {a_{p} } \\ \end{array} } \right) \times \left( {\begin{array}{*{20}c} {r_{i11} } & {r_{i12} } & \ldots & {r_{i1p} } \\ {r_{i21} } & {r_{i21} } & \ldots & {r_{i2p} } \\ \ldots & \ldots & \ldots & \ldots \\ {r_{im1} } & {r_{im2} } & \ldots & {r_{imp} } \\ \end{array} } \right)$$
(6)

With \(b_{k}\) representing the relatedness of an X − 1-level factor to judgment \(v_{k}\). After normalization (see Formula 5), vector B gives \(\sum\nolimits_{k = 1}^{p} {b_{k} } = 1\). That is,

$$b_{k} = \hbox{max} \left( {b_{1} ,b_{2} , \ldots ,b_{p} } \right)$$
(7)

where kth judgment \(v_{k}\) corresponding to \(b_{k}\) is the final evaluation of the associated factor. This procedure is applied to all levels of the hierarchy obtained using AHP.

3.4 Simulation

Once the social and economic impact evaluation model has been developed, it can be loaded into a spreadsheet (Excel) and its reliability tested using a simulation to evaluate the positive and negative impacts identified in a mine in Quebec. Based on the results of such simulations, possible improvements to the model may be identified.

4 Results

The simulation provided the means of performing a sensitivity analysis on the model.

4.1 Ranking and weighting of sustainability criteria and indicators

4.1.1 Reducing the complex problem to a hierarchical structure

The objective of this simulation was to apply the social and economic impact evaluation model to a realistic situation. To facilitate understanding of the problem and the relationships between the sustainability factors (dimensions, criteria and indicators) identified, AHP was used to rank the elements of the problem, as shown in Fig. 6.

Fig. 6
figure 6

Ranking of criteria and indicators of sustainability by AHP

Full size image

4.1.2 Paired comparison of the elements in a given hierarchical level

The weights of the criteria and indicators were obtained from paired comparisons made using the nine-point Saaty (1980) scale shown in Table 3.

Table 3 Numerical values and verbal equivalents for evaluating characteristics
Full size table

An example of the representation of relative importance based on paired comparisons as judged by 10 experts using the Saaty (1980) scale is illustrated in Table 4.

Table 4 Compilation of paired comparisons of criteria
Full size table

The paired comparisons of the indicators are carried out in the same manner. The tables of weightings of the indicators of the first three criteria (AHP rank) are shown in “Appendix 1”. The corresponding consistency index is shown in “Appendix 2”.

4.1.3 Determination of criteria and indicator weightings

When several experts participate in the procedure, the geometric mean is used to obtain a single value for their evaluations. The “priority vector” represents the weights of criteria (or indicators) obtained by normalizing the geometric means. Table 5 shows the results obtained from the AHP-based criteria weighting method used in the simulation. Values in italics are the reciprocal of those above the diagonal. The priority vector normalized weights are shown in Table 6.

Table 5 Criteria weighting matrix
Full size table
Table 6 Normalized criteria weightings
Full size table

After determining the priority vector, the consistency index (CI) is calculated for each paired comparison. If CI ≤ 0.1, the AHP-based evaluation is considered consistent (Saaty and Vargas 2012). If CI > 0.1, the expert judgments must be reconsidered and the evaluation repeated.

The same procedure is used to calculate the weights of the indicators, with the particularity that a priority vector is obtained for each group of indicators associated with the same criterion (Fig. 6). Table 7, drawn from the simulation, represents the weighting of CRT1-associated indicators having a CI = 0.1 (“Appendices 3–5”).

Table 7 Weighting of CRT1 indicators
Full size table

4.2 Social and economic impact evaluated in terms of sustainability

Once the weights of the criteria and indicators have been obtained, the impact of the mining site from a sustainable development perspective can be evaluated using the FIJ method. The results obtained at each step of the evaluation are summarized below.

4.2.1 Construction of fuzzy sets

In this step, 10 experts (stakeholders) judge the contribution of each indicator to the nebulous concept of sustainable development using the scale described above (Sect. 3.3.3). The tabulation of their judgments is shown in Table 8.

Table 8 Fuzzy judgments based on economic indicators
Full size table

4.2.2 Construction of the judgment matrix

In this step, the proportion of evaluators choosing a given reference level (very strong, strong, moderate, weak, very weak) is determined for each indicator. Table 9 summarizes the weightings for the three CRT1-associated indicators. For example, 20% (0.2) of the evaluators judged the economic performance (ECN1) of the mine as “strong”, 50% (0.5) as “moderate” and 30% (0.3) as “weak”.

Table 9 Indicator weights according to evaluators’ fuzzy judgments
Full size table

The performance of the mine on the basis of each indicator is evaluated using the same procedure.

4.2.3 Determination of integrated fuzzy judgment vectors

The product of the AHP indicator weight and the FIJ-calculated fuzzy judgment matrices provides means of measuring the contribution of the criterion to sustainable development. Figure 7 shows the integrated fuzzy judgment value calculation for criterion CRT1.

Fig. 7
figure 7

Calculation of integrated fuzzy judgment using matrices

Full size image

The integrated fuzzy judgment is determined for each criterion using the procedure represented in Fig. 7. The results are summarized in Table 10.

Table 10 Summary of integrated fuzzy judgments associated with the criteria
Full size table

Once the integrated fuzzy judgment values have been obtained, the importance of the contribution of each criterion to sustainable development becomes apparent. For example, in Table 10, the importance of CRT1 is “moderate” since this is the level for which the value was highest (0.60). The integrated fuzzy judgment of the economic and social dimensions is obtained using the procedure described for criteria. This final step gives the performance of the mine in terms of economic and social sustainability.

However, the same maximal value may be obtained for two or more levels, for example, in the case of CRT4, the importance may be moderate or weak, based on the value of 0.30 (Table 10). The indicators of this criterion must be re-evaluated with evaluators that are apprised of these conflicting values. This situation has not been considered in the literature, and it is not clear exactly how such a re-evaluation should be carried out. The results of the simulation are shown in Figs. 8 and 9, which provide visual representations of the effort of the mine with regard to each criterion of sustainable development.

Fig. 8
figure 8

Performance of a mine in terms of the perceived achievement regarding various criteria of sustainable development

Full size image
Fig. 9
figure 9

Performance of a mine in terms of the perceived achievement regarding two dimensions of sustainable development

Full size image

The uncertainty associated with the model can be gleaned from the sensitivity analysis. The purpose of a sensitivity analysis is to determine, quantify and analyse how the output of a model responds to perturbations of the input variables (Jacques 2005, p. 15). If decisions are going to be based on the study of a phenomenon using mathematical or computerized modelling techniques, it is important to bear in mind that uncertainty is always associated with such models.

It should be mentioned that the goal of this analysis is not to explain the results obtained, since the application is theoretical, but rather to demonstrate that the model can be used to carry out a sensitivity analysis. In this study, the input variables represent various indicators of sustainable development (ECN1, ECN2, etc.) and the output variables represent the perceived performance of the mine in terms of criteria or dimensions of sustainable development. The sensitivity is analysed at two levels, namely the weighting and the judgments. Varying the weighting of the sustainability indicators tests the AHP portion of the model. To simplify the presentation of the results, the first three economic indicators are used, ECN1, ECN2 and ECN3. For each of these, a progressive increment or decrement is applied until a change in the results is noted, using the values in Table 7 as a guide. The variations in weighting are applied while maintaining a CI ≤ 0.1. The results obtained for criterion CRT1 are summarized in Table 11.

Table 11 Results of the sensitivity analysis of the indicator weightings
Full size table

Neither incrementing nor decrementing each indicator while maintaining the CI ≤ 0.1 changed the performance of the mine with regard to CTR1. This criterion is thus relatively insensitive to changes in the judgments of the expert evaluators as long as they are consistent.

Analysing the sensitivity of the fuzzy judgments consists of determining for each indicator what proportion of the stakeholders would have to change their opinion in order to change the perceived performance of the mine from moderate to strong or weak with regard to CRT1. Each indicator is incremented or decremented by 10% until a change is noted. The results of this test are shown in Table 12.

Table 12 Influence of indicator judgments on sustainable development rating
Full size table

At the outset, 20% of the evaluators felt that the mine contributed strongly to sustainable development based on CRT1 (Table 10). Table 12 shows that 95% of the evaluators would have to judge ECN1 as strong for the performance rating of the mine to pass from moderate to strong. For ECN2, 85% of the evaluators would have the same effect. However, CRT1 is much less sensitive to variations in the weight of ECN3. The weight attributed to ECN3 is small compared to those of ECN1 and ECN2.

5 Discussion

The results of this study shed light on the absence of a set of criteria and indicators suitable for exhaustive evaluation of actions having an impact on the sustainability of mining in Canada. In the literature, a few indicators have been adapted somewhat for the purpose of evaluating the impact of mining, and these may reflect partially some deficiencies with regard to sustainable development (Petrie et al. 2007). The systemic definition of criteria and indicators requires adjustment for application to a specific sector (GRI 2015). It has been pointed out also that the indicators used are seldom if ever proactive, in the sense that they cannot be used to prevent any negative impact of a mine (Poveda and Lipsett 2014). Some approaches are clearly reactive, since they are applicable only after the mine has been chosen for study and the project plan has been finalized, for example, in the case of tools based on geographical information systems (Craynon et al. 2016). Analysis of impact evaluation tools has shown that approaches based on a single method are less effective than those based on an integrated model (Petrie et al. 2007). One of the major problems associated with models used in evaluation tools is poor control of the subjectivity inherent in judgments made by the stakeholders, not to mention the absence of a systematic process for selecting the participants (Kommadath et al. 2012).

The problems identified reveal a very real need for a set of criteria and indicators adapted to evaluating the sustainability of the mining industry in Quebec. These would have to meet with the approval of the professional and academic communities. In spite of the efforts of the industry so far, the literature contains no model suitably adapted to evaluating the socioeconomic impact of mining in Quebec or Canada.

In this study, comparison of perceived impact with socioeconomic indicators was used to identify the criteria and indicators best suited to focusing on deficiencies in the Quebec mining industry. Selecting criteria and indicators from case studies is also important, since this confirms the operational aspect of those chosen for the present purposes. Although a few indicators underwent some adaptation, these had been applied in previous studies (Table 1). Using the criteria and indicator identification process adopted for this study, it became clear that a choice based solely on sustainability terms of reference (Global Reporting Initiative, ISO 26000. etc.) is likely to be inadequate in the context of mining in Quebec. Based on the AHP and FIJ methods, the model developed is intended to improve the precision of the evaluation in two ways. The first is weighting the criteria and indicators using AHP in order to check the consistency of the expert judgments. This alone avoids some of the constraints inherent in other methods. For example, the number of judgments does not need to be large, and quantitative as well as qualitative indicators can be considered. The second is using FIJ to determine the level of performance of the mine by evaluating the criteria and dimensions on a scale ranging from very strong to very weak, based on where the expert judgment consensus is maximal. The proposed theoretical model improves control of the subjectivity of the stakeholder judgments by revealing inconsistencies and ambiguous consensus. Based on the results of a simulation, the model appears to be robust.

The simulation showed that the model lends itself to sensitivity analysis on two levels. The effect of varying the weighting of indicators was revealed through AHP, while FIJ allowed testing of the effect of changes in stakeholder opinions. Although the sensitivity analysis was based on a single criterion (CRT1), it could have been carried out using other dimensions. The graphical representation provides deciders with an overview of the performance of the mine in terms of the criteria and socioeconomic dimensions of sustainable development. The other advantage is the temporal comparison, which allows monitoring of performance over the years. The graphs also allow comparisons of different mines on the basis of selected criteria.

Finally, the proposed model is very flexible and easily supports the addition of other factors. Indeed, the mining companies involved have already started to adapt it and integrate other factors. The proposed approach of the tool provides the addition or adaptation of existing indicators and the removal of others depending on the context and the progress of mining projects.

6 Limitations of the study and future research

Those who would attempt to apply to real situations a model such as developed in the course of this research would be well advised to consider the following limitations. The calculation of some indicators required adjustments a posteriori. No systematic process is provided for identifying the most reliable experts or for choosing the participants (stakeholders, members of the community) concerned by the impact evaluation. This aspect is important, since the opinions of all groups are required in order to identify all aspects of the socioeconomic impact associated with mining activities. In addition, the sustainability indicators used in this model are more reactive than proactive. The application of the methods used may require the attention of a specialist in multi-criteria analysis. Furthermore, the AHP method does not eliminate subjectivity, even when combined with FIJ. Unlike other methods of multi-criteria analysis such as the analytic network process (ANP), FIJ does not take into account interactions between different indicators. Another deficiency is the possible ambiguity in assigning importance to criteria. For example, when the final judgment is “moderate” and “weak”, some way of deciding what importance to assign would be preferable to repeating the evaluation. Finally, the manual calculations may be difficult for some users.

These limitations suggest several avenues of research. A set of proactive sustainability criteria and indicators adapted to mining in Canada would make the evaluation more helpful, by anticipating negative socioeconomic impact instead of just noting it after the fact. Interactions between different criteria and between different indicators need to be revealed and taken into account. This would reduce subjectivity even further, since the model would be more capable of detecting biases in the stakeholders’ judgments. A systematic process of choosing the participants in the evaluation would also increase the reliability of the model, by ensuring representation of all aspects of the socioeconomic impact. This would lead to better guidance of efforts to improve performance and contribute to increasing the social acceptability of future mining projects. The potential advantages identified in the simulation carried out using the model need to be confirmed. The tool needs to be made simple and accessible to non-experts in the evaluation of the socioeconomic impact of mines. This means developing software for automating the calculations. The judgment of the importance of criteria could also be improved to yield unambiguous results.

7 Conclusion

The principal goal of this research project was to identify the criteria and indicators of sustainable development best suited to the evaluation the socioeconomic impact of mines in Canada and to use them to develop a theoretical model of evaluation of this impact. A review of the literature and comparison with socioeconomic impact noted in the case of mines in Quebec provided a list of candidate criteria and indicators. A model was then developed using multi-criteria analysis methods.

This study revealed a real need to develop a set of proactive indicators of sustainability in order to anticipate the negative impact of existing and future mines. In a simulation, the model was able to take into account the subjectivity of stakeholder judgments of various aspects of the impact. However, in order for such a model to be truly helpful, the limitations identified above must be addressed in future research. The greatest single improvement would be to provide the model with a systematic process of identifying the stakeholders in order to ensure complete characterization of the impact of mining. The proactive aspect of the model should also be developed as much as possible, since avoiding negative impact on sustainability should be the ultimate goal.