Keywords

1 Introduction: e-Participation Evaluation and Democracy

E-Participation is the use of Internet-based technology to include and empower citizens in government decision making processes. Although e-Participation is associated with the hope that new technology can enhance democracy and contribute to ‘better’ government, the impact on, for example, policy making has so far been limited [1].

As a research field, e-Participation is characterized as multidisciplinary, in that it brings a variety of methods and scholars together [2, 3]. Although there is no unified theory in the field, researchers have made attempts to characterize the field and set out paths for further research. Medaglia [4] argues that e-Participation research needs to move beyond technology to focus on citizens and other stakeholders. Grönlund [5] contends that the current models for describing progress in e-Participation are based on the extensive use of technology and the idea that direct democracy is the most advanced form of democracy.

Other scholars focus on success factors for good e-Participation practice. Jho and Song [6] argue that e-Participation will fail, or even be a menace to democracy, if only technological infrastructure is considered: technology needs to be accompanied by appropriate institutional conditions. Molinari [7] contends that five attributes are crucial to achieving sustainable participation: juridical compliance, legitimacy, social value, efficiency, and productivity. An Austrian case study concludes that to prevail with e-Participation, the users’ specific characteristics must be taken into account, including age, skills, and gender [8]. Skills and gender are also identified as determinants for e-Participation in the Spanish context by Vicente and Novo [9].

The United Nations evaluates e-participation every other year through the UN e-Government Survey (UNES2016) [10]. However, the UN evaluation has been criticized for not taking contextual factors into account, thus allowing non-democratic regimes to gain high positions simply by publishing services on their websites [11]. Linde and Karlsson [12] reveal that positive development in e-participation in non-democratic countries does not lead to positive effects on reducing corruption and quality of government. Karlsson [13] contends that non-democratic states with high levels of e-participation also utilize strategies for controlling citizens’ internet use. Gulati et al. [14] show similar findings, suggesting that authoritarian regimes might utilize e-Government to maintain status quo. Maerz [15] argues that authoritarian regimes utilize e-government to demonstrate modernity and legitimacy, as well as gaining internal legitimacy by pretending to increase transparency and citizens’ engagement. Åström et al. [16] contend that economic globalization is a strong predictor for e-participation development in non-democracies: by implementing ICT-infrastructures, countries make themselves attractive alternatives for foreign investments in technology. Kneuer and Harnisch [17] criticizes the UN surveys for not taking the nations’ motives and strategic purposes of e-Government into account. Sæbø et al. [3] contend that e-Participation evaluation is dependent on articulated objectives, clear democratic ideals, and criteria that can serve as benchmarks.

Against this backdrop, the purpose of this paper is to develop a method for evaluating e-Participation that takes into account a nation’s democratic system. The paper proceeds as follows: Sect. 2 describes the data and research procedure. In Sect. 3, the results are presented and, finally, conclusions, limitations and implications for further research are described in Sect. 4.

2 Methods and Materials

A point of departure in this research is the UN e-Participation Index, which is part of the UNES2016. As literature has pointed out the importance of appropriate democratic institutional conditions in e-Participation initiatives, the Economist Intelligence Unit’s Democracy Index from 2016 (EIUDI2016) [18] was used as a democratic indicator. Furthermore, to account for equality and gender, the Global Gender Gap Index 2016 (GGPI2016) [19] was used. These three datasets were then merged using multi-criteria decision analysis (MCDA), which is a method that allows the evaluation and ranking of different alternatives based on a selected number of criteria that can be individually weighted [20, 21]. A fourth dataset, the Corruption Perceptions Index 2016 (CPI2016) [22], was used to benchmark the index created through the MCDA. The reason for using the CPI2016 was that the level of corruption control in a nation correlates with its degree of civil society participation [23] (Table 1).

Table 1. Description of data
Full size table

2.1 Description of Data

In the EIUDI2016 index, countries are distributed by regime type, as seen in Table 2.

Table 2. Distribution of regimes (EIUDI2016)
Full size table

2.2 Procedure

MCDA is a method for ranking alternatives based on several criteria with assigned weights [20, 21]. The weighted sum model (WSM) is the most straight-forward MCDA model and requires that all the criteria use the same scale. Since the scores could be expressed between 0 and 1, the WSM was utilized. According to the WSM, a decision problem is assumed to consist of m alternatives and N criteria. The relative weight for a criterion \( C_{j} \) is expressed by \( w_{j} \). \( a_{ij} \) is the value of alternative \( A_{i} \) when it is evaluated in terms of a criterion. Then, the total value of \( A_{{i^{Score} }} \) is defined as:

$$ A_{{i^{Score} }} = \mathop \sum \limits_{j = 1}^{N} w_{j} a_{ij} , \,for\;\; i = 1,2,3, \ldots ,m $$

The research was carried out through the following steps:

  1. 1.

    Definition of alternatives:

    As described in Sect. 2.1, the alternatives that are to be ranked are countries listed in the UNES2016 and EIUDI2016 and GGPI2016 and CPI2016.

  2. 2.

    Selection and definition of criteria:

    e-Participation, democracy, and gender equality were chosen as criteria to create a new index (see Fig. 1).

    Fig. 1.
    figure 1

    New index with criteria

    Full size image
  3. 3.

    Assessment of scores for each attribute:

    Scores were extracted from the UNES2016, EIUDI2016 and GGPI2016 datasets. The datasets were formatted in IBM SPSS 23 and Microsoft Excel.

  4. 4.

    Standardization of the scores to make the criteria comparable with each other:

    Each score was expressed on a scale between 0 and 1. Theoretical maximum and minimum were used as a global scale: 1 was treated as the optimal scenario for each criterion, while 0 was considered the worst-case scenario.

  5. 5.

    Weighting of criteria, to assign priorities to them:

    At this point in the research, equal weights were used for all criteria (0.333). A weighting sensitivity analysis for the UNES2016 criteria was performed in step 7.

  6. 6.

    Ranking of the alternatives:

    The UNES2016, EIUDI2016 and GGPI2016 were inserted in a multi-criteria decision matrix created in Excel. The weights for each set were set as equal (0.333) to calculate final scores.

    For example, the total score for Finland in Table 3 is calculated by expressing the criteria between 0–1, followed by multiplying the score for each criterion by its weight, and then summing all the products. In this case, equal weights were applied to the criteria, which gives the following results:

    Table 3. Sample MCDA matrix
    Full size table

    (0.91525 * 0.333) + (0.903 * 0.333) + (0.845 * 0.333) = 0.88686

  7. 7.

    Test the ranking.

    The total score in the new index was then entered into SPSS. To test how the new index fared against the UNES2016, CPI2016 was used as a benchmark using linear regression. The hypothesis was that the total scores from the new index should correlate more strongly with the CPI2016 than would the UNES2016 would. In addition, a weight sensitivity analysis was performed using the Decerns MCDA DE software [24]. Finally, Person’s correlation was utilized to show the correlations between all indexes.

3 Results

Table 4 presents the e-Participation (ePart) rankings of the top 11 countries in the UNES2016 and Table 5 presents the results based on the new index created from the data in UNES2016, EIUDI2016 and GGPI2016. The new index consists of 139 countries. As shown, the most notable difference is that the top 11 countries in the new index are full democracies, while the flawed democracies have been degraded. All the Nordic countries have gained positions in the new index.

Table 4. Top 11 ranking and scores, UNES2016
Full size table
Table 5. Top 11 ranking and scores, new index
Full size table

To test how well the new index fared compared to the UNES2016 in terms of democracy, both indexes were tested against the CPI2016 by using linear regression. Figure 2 contains the results from the UNES2016/ CPI2016 dataset and Fig. 3 from the new index/ CPI2016 dataset. As shown, the new index has fewer outliers and has a higher correlation (see Table 6) with CPI2016 than the UNES2016 has. Furthermore, in the UNES2016 e-Participation Index, 8 authoritarian regimes can be found among the top 50 countries, with one in a top position as number 22. The full democracy with the lowest position in the UNES2016 e-Participation Index is positioned as number 74. In the new index, these rankings are changed, with the authoritarian regime moving from a ranking of 22 down to 75, and the full democracy moving up from 74 to 28. In Figs. 2 and 3, the authoritarian regime (“A”) and the full democracy (“D”) are compared. As shown, they are closer to the regression line when using the new index. Figure 4 reveals that the UNES2016 criteria needs a weight of 59.5%, given that the weights of EIUDI2016 and GGPI2016 remain similar, for regime A to achieve a greater ranking than D. That scenario seems unlikely when considering the trade-offs such a weight would imply.

Fig. 2.
figure 2

UNES2016 and CPI2016

Full size image
Fig. 3.
figure 3

New index and CPI2016

Full size image
Table 6. Correlations between indexes
Full size table
Fig. 4.
figure 4

Weight sensitivity analysis of the UNES2016 criteria

Full size image

Finally, Table 6 shows the correlation between the different indexes. As can be seen, the New index has the strongest correlation with the CPI2016 compared to all included indexes.

4 Concluding Remarks

The purpose of this paper was to develop a method for evaluating e-Participation that takes into account a nation’s democratic system. Multi-criteria analysis was utilized to create a new index by combining datasets ranking levels of democracy and equality with the UN e-Participation index 2016. In the resulting index, a country needs to have a solid democratic base with equality between men and women to rank highly. Building on this foundation, e-Participation can become a deciding criterion, inspiring nations to develop innovative participatory processes. However, no nation will gain a prominent position by focusing on technology and web services alone. By utilizing MCDA, several indexes can be combined and assigned individual weights. The study was not without limitations. In the current research stage, no method for weighting criteria has been utilized. Furthermore, although the suggested method incorporated democratic and gender equality criteria, it does not evaluate whether users actually influence processes and policies through e-Participation, should an index measuring actual user impact be created, it could be incorporated into the model. The results of this study contribute to the improvement of e-Participation evaluation practice by introducing normative criteria for good governance.