1 Introduction

Next to stimulating economic growth and competitiveness, promoting social inclusion has recently gained considerable policy attention in the European Union. Whereas the significant step forward in putting the social integration and development at the head of the EU agenda (next to the realization of economic growth) was already underlined in the Lisbon Treaty in 2000, recently, in 2010, with the implementation of the follow-up to the Lisbon Strategy, the Europe 2020 Strategy, the struggle against poverty and social exclusion was put at the heart of the efforts for smart, sustainable and inclusive growth.Footnote 1 As an illustration of this importance, there was the commitment of the EU Heads of Government to lift at least 20 million people in the EU-region out of a situation of being at risk of poverty or social exclusion until 2020. To increase the chances of reaching this 2020-target, a joint commitment was established among national governments, EU Institutions and key stakeholders. This commitment was institutionalized with the establishment of the European Platform against Poverty and Social Exclusion. Key tasks of this platform are the monitoring of the progress made towards the social Europe 2020 targets and enhancing policy coordination among EU Member States through the use of a (strengthened) social OMC (open method of coordination for social protection and social inclusion) (EPRS 2014) and in collaboration with the Social Protection Committee.

The widespread interest in social inclusion and poverty in EU policy circles has attracted the attention of high-level public servants, policy researchers, academics and other stakeholders (NGOs, etc.). This resulted in the publishing of numerous research reports and academic studies that explored various aspects of the policy areas of social exclusion and poverty in the EU-region. Examples of topics that have been studied are the challenges faced by the EU Member States in the development and implementation of social inclusion and poverty policies (e.g., Atkinson 2002a, b; Atkinson et al. 2005; European Commission 2014; Zeitlin 2008; O’Connor 2005), the role of OMC in promoting effective social inclusion policies (e.g., Bernhard 2007; Ania and Wagener 2014; Mabbett 2007; Daly 2006), data collection on social developments within EU Member States (Atkinson 2003a; European Commission 2013), and the definition of quantitative targets for social inclusion and poverty (e.g., Atkinson et al. 2005; Copeland and Daly 2012). On a more fundamental level, theoretical studies explored the issue of measuring and/or comparing quality of life, social inclusion (or alternatively, social exclusion) and poverty using welfare-theoretic foundations (Atkinson 2003b; Chakravarty and D’Ambrosio 2006; Dowrick et al. 2003; Alkire and Foster 2011a, b; Nolan and Whelan 2010; Whelan et al. 2014).

Only few academic studies explored Member State progress in social inclusion using the set of statistical indicators as laid out by the Heads of State and Government. The papers that did evaluate Member States’ social inclusion performances, advocated the use of ‘benefit-of-the-doubt’ weighted composite indices (CIs) to capture the complex and multidimensional nature of Member States’ social inclusion policies and performances in a context of Social OMC. Examples of such studies include Cherchye et al. (2004) for the period 1995–2001, Lefèbvre et al. (2010) for the period 1995–2006, Caminada et al. (2010) for the period mid-1980s to mid-2000s, and Giambona and Vassallo (2014) for the period 2006–2010. The key reasoning is that CIs, because of their ability of summarizing key information on complex, multivariate tasks and/or policies into a single index, help decision makers by drawing attention to particular issues, identifying benchmark performances and trends, and setting policy priorities in social inclusion.Footnote 2 Several of these papers advocated the BoD-method for constructing CIs because of its flexibility and optimistic stance in the determination of the sub-indicator weights. In a setting in which objective knowledge on the true policy weights is usually lacking or incomplete, the BoD-model derives for each country the set of optimal weights from the observed sub-indicator values themselves. More in particular, the BoD-model defines for each country importance weights such that the impact of sub-indicators of relative strength is maximized and the impact of sub-indicators of relative weakness is minimized in the composite value. This quality explains much of the appeal of the BoD-model: in what is usually a sensitive evaluation environment, with countries often being sensitive about being evaluated, compared and benchmarked against each other, disappointed countries (i.e. countries with CI-values below expectations) can no longer blame a low CI-score on damaging or unfair weights. Particularly in the setting of EU policy making with, on the one hand, a broadly shared EU-wide concern to strive for sustainable and inclusive economic growth (i.e., the core of the Europe 2020 strategy) and, at the same time, the different traditions and instruments to achieve this goal being, under the subsidiarity principle, still largely situated at the national level, the BoD-weighted composite index offers an attractive second best route to measure and evaluate Member State progress in social inclusion. As key research question these studies were primarily interested in whether EU Member States had been able to achieve progress in social inclusion and protection. Another question explored by most of these studies was whether there had been a trend of convergence among EU Member States (what would implicitly indicate a successful implementation of the Social OMC). As the results and data of these studies suggested, there seemed to be considerable convergence in the social inclusion and social protection performance of EU Member States during the period 1995–2010. More in particular, Member States that were initially lagging succeeded in catching up with the better performing Member States in promoting social inclusion.

The present paper contributes to the literature in multiple ways. Firstly, we extend previous empirical studies of social inclusion in the EU both as to the number of Member States considered (the 27 EU Member States whereas previous studies typically only scrutinized the 15 old EU Member States largely due to low data availability for the most recent EU Member States) as well as the time period (the more prolonged time period 2005–2014). Secondly, on the level of methodology, we use the new ‘indirect’ version of the multiplicative BoD-methodology as recently proposed by Van Puyenbroeck and Rogge (2017). As argued and illustrated by these authors (following a comment of Tofallis 2014), this indirect multiplicative BoD-based CIs has some comparative advantages to the direct multiplicative BoD-based CIs used by Giambona and Vassallo (2014) in an analysis of change in social inclusion in the EU (i.e., resulting CIs embedding some intuitive (axiomatic) characteristics and not requiring a (country-specific) scaling factor in order to ensure unit invariance).Footnote 3 Thirdly, we explore the dynamics in EU social inclusion policy in the EU-region before and after the economic and financial crises (period 2005–2009 vs. period 2010–2014).Footnote 4 The key questions are whether we can see progress in social inclusion in the EU since the enlargement of 2004 and whether there were differences between social inclusion performances prior to the economic crises and after the downturn. Thirdly, we examine whether there was a trend of convergence or divergence in the social inclusion performances of the EU Member States during the respective periods (2005–2009 vs. 2010–2014). Finally, we would like to investigate groupings of countries (e.g., the old EU Member States vs. the new EU Member States) to discover differences regarding their social inclusion policy performances.

The remainder of this paper is organised as follows. Section 2 describes the history and evolution of social policies in the EU. Section 3 present the social inclusion and poverty indicators as developed by Eurostat. Section 4 briefly discusses the variant of the BoD-method advocated by Van Puyenbroeck and Rogge (2017) to construct CIs. This section also describes how this method can be used to evaluate Member States’ social inclusion performance at a certain time moment (static) and over time (dynamic). Section 5 presents, discusses and analyses the results. Finally, Sect. 6 concludes and makes some suggestions for further research in the particular topic.

2 Social Inclusion in the EU

The idea of social policy harmonisation within the EU goes back to the very beginning of the Community. The Messina declaration, signed by the six original members in 1955, clearly stressed the importance and desire of a stepwise harmonisation of the Member States’ social policies in order to build a strong and united Europe. However, it was only 20 years later, in 1972 that some initial steps were taken during the Paris Summit (Vanhercke 2012). The Heads of State and Government then agreed to nurture the Community’s social agenda by establishing a European Social Action Plan, aimed at endorsing various measures to combat poverty. Between 1974 and 1994, three so-called anti-poverty programmes were launched by the Commission, all aiming at finding innovative solutions to the increasing problem of poverty and social exclusion in the various Member States. In essence, the three programmes attempted to, and eventually succeeded in, initiating research in the field of social policy, as well as introducing micro-objectives for each Member State with respect to anti-poverty measures. Based on the results of the programmes, the Council of the EU adopted some recommendations, which provided a guiding principle and appealed for regular consultations with Member States regarding their work on social protection.

The next significant steps taken towards harmonisation of national social policies in the EU came with the implementation of the Treaty of Amsterdam in 1999 and the Lisbon Strategy in 2000. After a period of economic crises (early 90’s) with low economic growth and unemployment and social exclusion on the rise in most of the EU Member States, and the enlargement of the EU, with several new Member States from the former Soviet Union, being imminent, Member States recognized the need to implement new strategies in order to strengthen the European economy and its competitiveness, employment, and social cohesion. The Treaty of Amsterdam recognised for the first time that social policy should be considered as a joint responsibility of the individual Member States and the EU itself (Vanhercke 2012). The Lisbon Strategy (period 2000–2010) stressed the importance of coupling social cohesion and economic objectives (Atkinson 2002b). The implementation of the Open Method of Coordination (OMC hereafter) was without doubt one of the most essential novelties (Matsaganis and Sacchi 2002; Heidenreich and Bischoff 2008). Basically, OMC is a EU policy-making procedure which constitutes a method of “soft governance”, meaning that it does not result in any EU legislation (EPRS 2014). Instead, in accordance with the subsidiarity principle, it are the national governments of the Member States which are responsible for designing and implementing social inclusion policy measures. The OMC merely attempts to coordinate and monitor the national social inclusion and social protection policies of the individual Member States by, amongst other things, instituting guidelines, specifying qualitative and quantitative indicators for capturing Member States’ social inclusion policy performances, and sharing benchmarks for social inclusion. Moreover, it attempts to create national targets for the Member States (typically broad goals as agreed upon by the Council of Ministers), which then are backed by periodic evaluations and peer reviews. The resulting evaluations aim at aiding Member States to learn from each other and, in the best case scenario, to review and eventually improve their own domestic policies.

Another important decision of the Lisbon European Council was the establishment of National Action Plans on social inclusion (NAPincls hereafter) that Member States have to submit bi-annually. In these NAPincls Member States outline their specific plans for combatting poverty and social exclusion for the coming two years. Ideally, action plans should be constructed by Member States’ central government (and lower level governments) in close communication with social partners, NGO’s and other organisations in order to disperse knowledge and refine the indicators. As such, these NAPincls play an important role in the coordinating and monitoring process under OMC. Recalling the variety of states composing the EU, it should not come as a surprise that each NAPincls is set up in a unique manner, focusing on different aspects of social policy, depending on the specific policy priorities of the Member State which drew it. Nevertheless, some basic criteria have to be met. First of all, the NAPincls have to take a multidimensional approach, meaning that they have to implement policies in all the different areas that affect the public. Secondly, each NAPincls has to reflect a coherent and planned approach based on in-depth analysis, and objectives need to be linked to the current situation and be clearly defined. Finally, the plans must set very precise quantitative targets to be achieved.

In 2010, the Europe 2020-programme was established as a follow-up to the Lisbon Strategy. Similar to its predecessor, the Europe 2020-programme aimed at providing the Member States with a ten year path for smart, sustainable and inclusive growth and job creation (European Commission 2010). The key priorities of the Europe 2020-programme were put forward by the European Commission in five key targets (to be reached by 2020): (1) 75% of the population aged 20-64 should be employed, (2) the share of early school-leavers should be under 10% and at least 40% of the younger generations should have achieved a tertiary degree, (3) 20 million less people should be at risk of poverty, (4) 3% of the EU’s GDP should be invested in R&D and (5) The “20/20/20” targets for climate and energy should be met (European Commission 2010). Similar to the Lisbon Strategy, a crucial element in the Europe 2020 strategy was the struggle against poverty and social exclusion. With the EU suffering from the consequences of the economic and financial crisis, with over 120 million Europeans being at the risk of social exclusion and poverty (numbers as in 2010), there was the commitment of the EU Heads of Government to lift at least 20 million people in the EU-region out of a situation of being at risk of poverty or social exclusion until 2020. With regard to the implementation, the Europe 2020 programme relies on a two-pillar system: firstly on the thematic approach discussed above, which combines priorities and headline targets, and secondly on country reporting, consisting of both recommendations to the Member States and their own reports on progress made. With the new programme for social policy, the EU decided to largely expand its own competencies for the purpose of greater cohesion of economic and social policy. Member States are now required to report, on an annual basis, on their progress towards the Europe 2020 targets. Related to this, the Council has full ownership of the new strategy, while the Commission is equipped with the competence to make country specific recommendations, monitor progress and enable policy exchange for each individual Member State with regard to their work towards the Europe 2020 targets (through a reinforced social OMC). This monitoring and analysis of the progress made towards the Europe-2020 targets is done in the context of the European Semester, a yearly cycle of policy coordination.

3 Data

Eurostat discloses three levels of statistical indicators for social inclusion and poverty (see Giambona and Vassallo (2014)). The Level 1- indicator, also communicated as the headline indicator, gives a summative picture of the phenomenon by computing the percentage of people who are either at risk of poverty, severely materially deprived or living in a household with very low work intensity. The Level 2 statistical indicators, also known as the operational indicators, reflect the true operational objectives for social inclusion promotion and poverty reduction as designated in the Europe 2020 strategy. Ultimately, Level 3-indicators are more explanatory and associated with the actions as described in the Sustainable Development Plan. In the present analyses, Level 2 statistical indicators are used in the construction of the CI for measuring countries’ social inclusion promotion and poverty reduction for the reason being that these indicators delineate the true operational objectives. In more detail, the Level 2-indicators for social inclusion and poverty reduction comprise the four following statistical indicators:

  1. 1.

    People at Risk-of-Poverty After Social Transfers, which measures the share of people at risk of monetary poverty, and is therefore presented as a percentage of the total population concerned. In order to be considered as being at risk-of-poverty, a person needs to have an equalised disposable income below the risk-of-poverty threshold, normally set at 60% of the national median after any social transfers.

  2. 2.

    Severely Materially Deprived People, which is a statistical indicator providing information regarding economic strains and durables. A person is categorised as severely materially deprived if he or she cannot afford at least four of the following: to pay rent or utility bills; to keep their home adequately warm; to pay unexpected expenses; to eat meat, fish or a protein every second day; a week of holidays away from home; a car; a washing machine; a colour TV; or a telephone.

  3. 3.

    People Living in Households with Very Low Work Intensity, measured as a percentage of the total population. In order to belong to this category of people, a person needs to be aged 0–59 and be living in a household where the working age members worked less than 20% of their full potential during the past year. A social cost, the loss of income for both the individual and the society is evident from this indicator.

  4. 4.

    Early Leavers from Education and Training, defined as the percentage of the population aged 18–24 with at most a lower secondary degree. Low levels of education is often associated with barriers to employment, leading to social exclusion.

The social exclusion and poverty statistical indicator data range from year 2005 up until 2014 and are obtained from the statistical agency of the European Commission, Eurostat. Note that all four statistical indicators are defined such that they measure social exclusion instead of social inclusion. Specifically, for all four Level 2 statistical indicators higher values represent worse performances in terms of social inclusion. In the CI-literature such indicators are typically referred to as ‘bads’. To put all indicators on a common basis so that all measure social inclusion, indicators are adjusted such that higher indicator values correspond to a better Member State performance in social inclusion. The JRC/OECD Handbook on Constructing Composite Indicators (see Nardo et al. 2008) provides an overview of possible transformation or normalization procedures. In the present study, normalized indicators were determined by subtracting the regular indicator values from 100%. As such, redefined indicator values have identical ranges [0,100%] with higher values representing a better Member State performance on that particular aspect of social inclusion. By normalization, the final set of social inclusion indicators used in the construction of the CI are: People not at risk-of-poverty after social transfers (y 1 ), People not severely materially deprived (y 2 ), People not living in households with very low work intensity (y 3 ), and Non-early leavers from education and training (y 4 ).

4 The BoD-Weighted Geometric CIs for Member State Social Inclusion Performance

4.1 Measuring at One Point in Time

We use Van Puyenbroeck and Rogge’s (2017) variant of the BoD CI-framework to weight and aggregate the aforementioned Level 2-indicators for social inclusion into a global CI-score for EU Member States’ social inclusion policy performance. This variant of the BoD-method combines ‘benefit-of-the-doubt’ weighting of the social inclusion statistical indicators with geometric aggregation. This method deviates from the original, and still rather widespread CI-approach to compute CIs as weighted arithmetic averages. The reasons for the use of a geometric version of the BoD-model are multifold. Firstly, there are the recent findings and discussions in the CI-literature that the weighted geometric average is superior to the weighted arithmetic average aggregation procedure (see, e.g., Ebert and Welsch 2004; Van Puyenbroeck and Rogge 2017; Zhou et al. 2010). Secondly, it has been shown by Ebert and Welsch (2004) that, if we use ratio-scale measurement for strictly positive statistical indicators (as in the present case study), the geometric average is valid because the subsequent ordering we attain is independent of the scaling used for each of the statistical indicators. Thirdly, when using the weighted arithmetic average as the functional form of a CI, one implicitly assumes perfect substitutability of the constituting statistical indicators. In the context of measuring a Member State’s social inclusion policy performance, this is a particularly concerning factor as this means that a Member State could perfectly compensate a poor performance on one statistical indicator of social inclusion with a better performance on another statistical indicator of social inclusion. The alternative of multiplicative aggregation (weighted geometric averages) overcomes this issue by making sure that the marginal returns to an increase in a statistical indicator value are diminishing rather than constant. Finally, the use of weighted geometric averages ensures that Member States with more unbalanced performances on the set of social inclusion statistical indicators will receive lower CI-scores, all else equal.

The technique of Van Puyenbroeck and Rogge (2017) basically involves a two-step procedure. In a first step, importance weights are derived for the Level 2-indicators for social inclusion using the traditional BoD-model (Cherchye et al. 2007; Giambona and Vassallo 2013). Formally, this involves solving the following linear programming model:

$$\mathop {\hbox{max} }\limits_{{w_{c,i} }} \mathop \sum \limits_{i = 1}^{m} w_{c,i} y_{c,i}$$
(1)
$$\begin{aligned} & s.t. \\ & \sum\limits_{i = 1}^{m} {w_{c,i} y_{j,i}\, \le\,1\quad \forall j = 1, \ldots ,n} \\ \end{aligned}$$
(1.1)
$$w_{c,i}\, \ge\, 0\quad \forall i = 1, \ldots ,m$$
(1.2)

with \(y_{c,i}\) the performance of the evaluated Member State c on the ith social inclusion statistical indicator (i = 1,…,m; m in casu equal to 4 as there are four statistical social inclusion indicators), \(y_{j,i}\) the performances of the Member State j (j = 1,…,n) on the ith social inclusion statistical indicator, and \(w_{c,i}\) the optimal weight for the evaluated Member State c on the statistical indicator i. Next to the two standard constraints (1.1) and (1.2), we impose the additional minor restriction that the optimal importance weights for each social inclusion statistical indicator should be at least 5%.Footnote 5 Based on the results of the traditional BoD-model, for each EU Member State the optimal importance weights for the statistical indicators (Cherchye et al. (2007) and Van Puyenbroeck and Rogge (2017) use the term “pie-shares”) are computed as follows:

$$\omega_{c,i} = \frac{{w_{c,i} y_{c, i} }}{{\mathop \sum \nolimits_{i = 1}^{m} w_{c,i} y_{c, i} }}$$
(2)

These BoD-based importance weights ω c,i implicitly reflect the particular policy area(s) within social policy that Member States find relatively more (less) important in promoting social inclusion and reducing poverty.

In a second step, these optimal BoD-based importance weights are used to construct a geometric CI for social inclusion. Formally,

$$CI_{c} \left( {y_{c,i} , y_{B,i} ,\omega_{c,i} } \right) = \mathop \prod \limits_{i = 1}^{m} \left( {\frac{{y_{c,i} }}{{y_{B,i} }}} \right)^{{\omega_{c,i} }}$$
(3)

with \(y_{B,i}\) the baseline performance on the ith social inclusion statistical indicator (in casu, the social inclusion statistical indicator values as computed for the EU27) and with \(\omega_{c,i}\)(i = 1, …, m) the relative importance weights for the m statistical indicators of social inclusion, de facto provided by the evaluated Member State’s BoD-based importance weights as derived from (1) and given in (2).Footnote 6

The use of the BoD-based importance weights in the construction of the geometric CI as in (3) is of great importance in the context of measuring social inclusion in the EU Member States. Frequently, poorly performing Member States (i.e., Member States with low CI-values) tend to criticize the reliability of the CI-scores and CI-ranking due to the use of unfavourable weights in the construction of the CIs. The use of BoD-derived importance weights overcomes this issue by assigning for each Member State, within the a priori defined constraints, relatively high (low) importance weights to the statistical social inclusion indicators on which the Member State in question performs relatively well (poorly). Moreover, in a setting in which there is typically disagreement among experts and stakeholders on the true importance of the different aspects of social inclusion (and, hence, the true importance weights of the social inclusion statistical indicators), the use of BoD-based country-specific importance weights enables each Member States to employ different social inclusion policies, focusing on different social policy aspects. This satisfies the notion of Lovell et al. (1995, p. 508) that, in the assessment of country policy performances, endogenous country-specific weighting is more appropriate than, for instance, the more commonly used equal weighting or fixed weighting approach. Particularly in the context of EU social policy with, on the one hand, a broadly shared EU-wide concern to combat poverty and social exclusion and, at the same time, the different traditions and instruments to achieve this goal being, under the subsidiarity principle, still largely situated at the national level, the idea of ‘imposing’ some policy priority weighting scheme fits uneasily and, hence, is overly restrictive.

In addition, the ratio format in (3) with the social inclusion statistical indicators of the evaluated Member State being put into relative perspective to the baseline social inclusion performance values illustrates the benchmarking idea. It enables to identify which Member States are overall doing better, worse, or equally good as compared to the baseline social inclusion performance. In the interpretation of the CI-scores, higher values indicate better global performances of the Member State on social inclusion. More in particular, CI-values lower than 1 denote that the evaluated Member State generally underperformed the baseline social inclusion performance. The opposite holds for CI-values higher than one, i.e., the evaluated Member State overall outperforms the baseline social inclusion performance. A CI-value equal to one indicates that evaluated Member State more or less performs equally well than the baseline social inclusion performance.

4.2 Measuring Over Time

Next to evaluating the social inclusion performances of EU Member States at a particular moment of time (i.e., static setting), we are also interested in measuring Member States’ performance changes in social inclusion. Such a dynamic, or intertemporal analysis, is interesting since it shows how Member States have evolved concerning their CI-index from time t versus time t + 1. In order to measure the global performance change for a Member State, Van Puyenbroeck and Rogge (2017) constructed a ratio of the geometric BoD-weighted CIs for period t and period t + 1, as follows:

$$PC_{c} = \frac{{CI_{c}^{t + 1} }}{{CI_{c}^{t} }} = \frac{{\mathop \prod \nolimits_{i = 1}^{m} \left( {\frac{{y_{c,i}^{t + 1} }}{{y_{B,i}^{t + 1} }}} \right)^{{\omega_{c,i}^{t + 1} }} }}{{\mathop \prod \nolimits_{i = 1}^{m} \left( {\frac{{y_{c,i}^{t} }}{{y_{B,i}^{t} }}} \right)^{{\omega_{c,i}^{t} }} }}$$
(4)

This change metric outlines the progress in social inclusion made by the evaluated Member State between period t and t + 1. PC c -values larger (smaller) than 1 indicate improvement (deterioration) in the Member States’ relative performances in social inclusion. As illustrated by Van Puyenbroeck and Rogge (2017), this performance change metric combines the joint effect of three types of change components. These three performance change components can be easily derived by the following intermediate step:

$$PC_{c} = \mathop \prod \limits_{i = 1}^{m} \left( {\frac{{\frac{{y_{c,i}^{t + 1} }}{{y_{B,i}^{t + 1} }}}}{{\frac{{y_{c,i}^{t} }}{{y_{B,i}^{t} }}}}} \right)^{{\frac{{\omega_{c,i}^{t + 1} + \omega_{c,i}^{t} }}{2}}} \times \mathop \prod \limits_{i = 1}^{m} \left( {\frac{{\frac{{y_{c,i}^{t + 1} }}{{y_{B,i}^{t + 1} }}}}{{\frac{{y_{c,i}^{t} }}{{y_{B,i}^{t} }}}}} \right)^{{\frac{{\omega_{c,i}^{t + 1} - \omega_{c,i}^{t} }}{2}}}$$
(5)

to obtain the following tripartite decomposition:

$$\begin{aligned} PC_{c} & = \mathop \prod \limits_{i = 1}^{m} \left( {\frac{{y_{c,i}^{t + 1} }}{{y_{c,i}^{t} }}} \right)^{{\frac{{\omega_{c,i}^{t + 1} + \omega_{c,i}^{t} }}{2}}} \times \mathop \prod \limits_{i = 1}^{m} \left( {\frac{{y_{B,i}^{t} }}{{y_{B,i}^{t + 1} }}} \right)^{{\frac{{\omega_{c,i}^{t + 1} + \omega_{c,i}^{t} }}{2}}} \times \frac{{\mathop \prod \nolimits_{i = 1}^{m} \left( {\sqrt {\frac{{y_{c,i}^{t + 1} }}{{y_{B,i}^{t + 1} }} \frac{{y_{c,i}^{t} }}{{y_{B,i}^{t} }}} } \right)^{{\omega_{c,i}^{t + 1} }} }}{{\mathop \prod \nolimits_{i = 1}^{m} \left( {\sqrt {\frac{{y_{c,i}^{t + 1} }}{{y_{B,i}^{t + 1} }} \frac{{y_{c,i}^{t} }}{{y_{B,i}^{t} }}} } \right)^{{\omega_{c,i}^{t} }} }} \\ & = \Delta OWN_{c} \times \Delta BP_{c} \times \Delta W_{c} \\ \end{aligned}$$
(6)

The first two factors, \(\Delta OWN_{c}\) and \(\Delta BP_{c}\) look respectively at that part of the performance change that is due to changes in the evaluated Member State’s social inclusion statistical indicators and the part of the Member State’s relative performance change that is due to the changes in the baseline social inclusion performance standards (while averaging out the effects of changes in the BoD-derived sub-indicator importance weights). A \(\Delta OWN_{c}\)-value higher (lower) than one accordingly denotes an aggregate improvement (deterioration) in a Member State’s social inclusion statistical indicators. A \(\Delta OWN_{c}\)-value equal to one implies overall a status quo in the own social inclusion indicator data. A \(\Delta PB_{c}\)-value higher (lower) than one marks regress (progress) in the baseline social inclusion performance values. As to the relative performance change metric of the evaluated Member State, progress (regress) in the baseline social inclusion performance standard negatively (positively) impacts the PC c -value, ceteris paribus. This means that, if everything else is held fixed, and if the baseline social inclusion performance improved (lowered), any Member State’s PC c -score should decrease (increase) as a result. The combination of the two aforementioned change components, \(\Delta OWN_{c} \times \Delta BP_{c}\), provides an answer to the question whether the progress (regress) realized by the evaluated country c on the set of sub-indicators is larger or smaller than the overall progress (regress) in the baseline performance.Footnote 7 The third change component, ΔW c , measures that part in the Member States’ social inclusion performance change metric that is due to changes in the Member States’ BoD-based importance weights for the social inclusion statistical indicators. If the ΔW c -value is higher (smaller) than one, it implies that there has been a change in the BoD-derived importance weights for the evaluated Member State that benefited (lowered) the social inclusion performance evaluation CI of the concerned Member State: social inclusion statistical indicators on which the Member State obtained a relative good performance value are rewarded more (less) generously at t + 1 as compared to t.

5 Results and Discussion

A first objective of the analysis consists in illustrating the situation in social inclusion at the start and the end of the period of study (i.e., the years 2005 and 2014). A second objective is to analyze how social inclusion evolved during this 10-year period. To examine for a potential impact of the financial and economic crisis we distinguish between the changes in social inclusion between the period 2005–2009 and 2010–2014. For both periods we also address the question whether social cohesion increased or not.

Table 1 displays the CI-scores for social inclusion performance of the EU Member States in 2005 and 2014, the respective ranking positions of each Member State (rank values displayed inside the brackets) and the respective BoD-based importance weights (i.e., ω c,i ). For both the years 2005 and 2014, we notice that on average EU Member States in the group of EU-15 countries are performing better in social inclusion as compared to the Member States that only recently became member of the EU (most of them in 2004). This result is expected as the new EU Member States are often considered as still less developed, poorer, and lagging behind in terms of promoting social inclusion. Moreover, as noted in a recent report published by the OECD (2014), the old EU-15 Member States generally devote a higher percentage of their GDP on public social expenditures as compared to the new EU Member States. Even though this difference in public social expenditures is partially due to differences in demographic circumstances and economic conditions, part of the explanation is also situated in the choice for a particular welfare system.

Table 1 Social inclusion CI-scores (ranks) and BoD-based importance weights (year 2005)
Full size table

As to the individual Member State social inclusion CI-scores, not too surprising are the good results (high CI-scores) of the three Scandinavian countries (Sweden, Finland, Denmark), which are known to have advocated strong social inclusion and social protection policies for their citizens. Also Luxembourg and the Netherlands have typically been among the best performing Member States when it comes to the evaluation of social inclusion policies. The high ranking position of Italy is somewhat remarkable. Even though Italy joined the EU in the early days of the Community, it has traditionally not been quite compatible with the social welfare states of Northern and Western Europe. A more detailed analysis of the CI-score of Italy in 2014 and, in particular, Italy’s BoD-derived importance weights for the social inclusion indicators indicates that a very high importance weight (i.e., the maximum possible importance weight of 85% given the a priori defined weight constraints) was assigned to the social inclusion indicator ‘Non-early leavers from education and training (y 4 )’ on which Italy realizes an above-average performance (i.e., a low percentage of the population aged 18–24 with at most a lower secondary degree) as compared to the other Member States. Two other striking results are the social inclusion performance scores of the Czech Republic and Slovakia (both of which entered the union in 2004) with relatively strong social inclusion performances in both 2005 and 2014. Consulting the importance weights for the Czech Republic in 2005 and 2014, we observe that this is what we call a Member State with a CI-score reflecting an unbalanced social inclusion performance, i.e., it gives the maximum importance weight of 85% to one particular social inclusion indicator (the indicator ‘people not at risk-of-poverty after social transfers (y 1 )’) and therefore manages to obtain an above EU-27 average CI-score. As for Slovakia, a somewhat similar evaluation can be made. Both in 2005 and 2014 Slovakia realized high ranking positions. The distribution of the BoD-derived importance weights in 2005 and 2014 shows that this high ranking position is predominantly due to the relatively high importance weight for the social inclusion indicator ‘Non-early leavers from education and training (y 4)’ and, hence, Slovakia’s relatively low percentage of population (aged 18–24) with at most a lower secondary degree as compared to other EU Member States. However, comparing the BoD-based importance weights for Slovakia in 2005 and 2014 shows that during the period 2005–2014 Slovakia shifted towards a somewhat more balanced social inclusion performance, with performances on the social inclusion indicator ‘people not at risk-of-poverty after social transfers (y 1)’ having slightly improved vis-à-vis the other Member States (i.e., a lower share of people at risk of monetary poverty). The 2015 country report for Slovakia (European Commission 2015a) confirmed these findings. According to the European Commission, Slovakia is doing relatively well in combatting social exclusion and poverty, however, regional disparities remain challenging. The report also indicates that the relatively strong performance of Slovakia on the sub-indicator ‘Non-early leavers from education and training (y 4)’ should however be interpreted with caution: “While the early school-leaving rate is low, it has increased in recent years, in particular for the Roma population, putting the national 2020 target of 6% at risk. This is alarming as the employment disadvantage for people without upper secondary education is significantly higher in Slovakia than in the EU in general” and continues with criticizing the educational system: “The quality of teaching and educational outcomes remain poor” which implies challenges for the Slovakian Ministry of Education in the upcoming years.

Among the worst performers, Bulgaria consistently performed poorly in social inclusion with CI-scores well below the EU-27 benchmark in both 2005 and 2014. The 2015 country report for Bulgaria (European Commission 2015b) reveals many concerning factors related to social exclusion, unemployment and poverty. The European Commission is particularly concerned about the slow economic growth, which hampers the labour market recovery. The report also outlays issues with the low levels of unemployment benefits: “The low levels of unemployment benefit coverage can be partly attributed to changes in the composition of the unemployed population. As the unemployment benefit system is mainly focused on unemployment periods of up to one year and full-time employees who were previously insured, certain groups of people, such as young people without an insurance history, long-term unemployed people and part-time or seasonal workers are not eligible for benefits.” This inevitably exposes a large group of people to the risk of poverty and social exclusion. The education system is also underperforming, with early school leaving levels above the new EU Member States’ average level. A large issue, regarding all the dimensions of social inclusion, seems to be the particularly vulnerable Roma population. In order to improve the situation, Bulgaria receives a considerable amount of financial support from the European Social Fund.

Looking at the descriptive statistics for the importance weighting schemes for the social inclusion indicators at the bottom of Tables 1 and 2, one readily observes some general differences between the old and the new EU Member States. Firstly, for both 2005 and 2014, it seems that, on average, old EU Member States have more balanced weighting schemes as compared to new EU Member States. This suggests that there is more cohesion in the social inclusion policies and performances of the old EU Member States. Secondly, the low average importance weight for the social inclusion indicators ‘People not severely materially deprived (y 2)’ and the high average importance weight for the social inclusion indicator ‘People not living in households with very low work intensity (y 3)’, denotes that, as compared to the old EU Member States, new EU Member States are generally performing relatively better on the latter aspect and relatively weaker on the former aspect of social inclusion. On the other hand, a comparison of the average importance weights in 2005 and 2014 shows that for the old EU Member States considerably less weight is assigned to the social inclusion indicator ‘People not living in households with very low work intensity (y 3)’. This suggests that the performance of the old EU Member State on that aspect of social inclusion generally deteriorated during the period 2005–2014. One possible explanation is the impact of the financial and economic crisis on unemployment.

Table 2 Social inclusion CI-scores (ranks) and BoD-based importance weights (year 2014)
Full size table

Prior to the enlargement of 2004, a couple of the EU-15 Member States, in particular Sweden, UK and Ireland, were concerned about the impact the expansion to the east would have on the EU. The main concern was that such an enlargement would put pressure on the social policies and performances of the old EU Member States (due to, e.g., an increase in the labour migrants). Preliminary studies have argued that the impact of the enlargement has been mild (Murphy 2006). Overall previous studies (e.g., Lefèbvre et al. 2010) found that most EU Member States managed to improve social inclusion, with the new EU Member States having benefitted slightly more from joining the Union without hampering social inclusion of the old EU Member States. In what follows, we outline the results of the dynamic analysis of the social inclusion performances of the old and the new EU Member States since 2004. The results are presented in Table 3. We distinguish between the period before and after the economic and financial crises (period 2005–2009 vs. period 2010–2014) in order to check whether the crises impacted social inclusion.

Table 3 Member social inclusion performance change and tripartite decomposition (period 2005–2009 vs. 2010–2014)
Full size table

Looking at the results of the intertemporal analysis, one immediately observes that the different components of the change metrics may, and do, often move in different directions. As for the first time span 2005–2009, we notice for several EU Member States changes in the social inclusion data, in other words, values for PC i that deviate considerably from 1. In particular, in the period prior to the financial crisis, computations clearly show that most of the new Member States (7 out of the EU Member States that joined the EU since 2004) managed to improve their social inclusion performance, while only three of the EU-15 countries (Luxembourg, Finland and the UK) saw an increase in their respective CI-scores (the PC c -values of Belgium and Ireland are too closely situated to 1 in order to speak of proper progress in social inclusion). The higher progress in social inclusion realized by new EU Member States also shows in the descriptive statistics of the PC c -values at the bottom of Table 3 (average PC c -value of 0.9602 for the old EU Member States vs. average PC c -value of 1.0918 for the new EU Member States). The tripartite decomposition of the PC c -values reveals more detailed insights into the social inclusion changes of the EU Member States. Firstly, the ΔOWN c -values higher than 1 denote that the majority of the EU Member States realized social inclusion progress during the period 2005–2009, i.e., the own performance values on the set of social inclusion sub-indicators improved. However, for several of the EU Member States the progress in own social inclusion data was smaller than the overall progress in the baseline EU27 performance (i.e., the \(\Delta OWN_{c} \times \Delta BP_{c}\) is smaller than one). An example of such an old EU Member State is Sweden. The \(\Delta OWN_{c}\)-value of 1.0203 indicates that Sweden slightly improved the own performance on the set of social inclusion sub-indicators in the years leading up to the financial and economic crisis. However, at the same time, the progress in the baseline EU27 performance was more significant (as indicated by the \(\Delta BP_{c}\) value of 0.8865 that is further positioned from the value one). The net result is \(\Delta OWN_{c} \times \Delta BP_{c}\) = 0.9045, hence, a decrease in the gap between the social inclusion performance of Sweden and EU27 (i.e., EU27 catching-up with Sweden). As to the change in the BoD-based importance weighting systems, the average \(\Delta W_{c}\) values close to 1 for the old and the new EU Member States suggest that, on average, sub-indicator importance weights remained more or less unaltered, though, with importance weights being slightly less favourable for the average old EU Member State (\(\Delta W_{c}\) = 0.9790) and somewhat more favourable for the average new EU Member States (\(\Delta W_{c}\) = 1.0387). However, the average values close to one mask considerable differences between Member States. In particular, the range of [0.7130;1.3618] for the \(\Delta W_{c}\)-values of the old EU Member States and of [0.8491;1.3378] for the new EU Member States suggest that for some Member States sub-indicator importance weights changed favourably relative to the other EU Member States (benefiting their PC c -value) whereas for other Member States sub-indicator importance weights became more unfavourable. Overall the \(PC_{c}\)-scores and the tripartite decomposition point out a convergence trend among EU Member States during the period 2005–2009 with both old and new EU Member States realizing generally progress in social inclusion, however, with the new EU Member States on average realizing stronger progress relative to the old EU Member States. Figure 1 provides an overview of the estimated \(PC_{c}\)-scores and the three change factors.

Fig. 1
figure 1

BoD-estimated \(PC_{c}\)-scores and its tripartite decomposition: Period 2005–2009

Full size image

In the second time span, 2010–2014, the estimated \(PC_{c}\)-scores suggest a general trend of progress in social inclusion. More precisely, the average \(PC_{c}\)-values for the old and the new EU Member States (respectively \(PC_{c}\) = 1.2194 and \(PC_{c}\) = 1.1278) specify that both groups of countries realized performance progress as compared to the baseline EU27 social inclusion performance. However, the large standard deviations in the estimated \(PC_{c}\)-values demonstrate that there are considerable differences among both new and old EU Member States in terms of estimated performance change. Particularly the \(PC_{c}\)-values for the old EU Member States differ significantly in the estimated performance change (standard deviation of 1.2752 and range of [0.1822;5.7074]). Nevertheless, the result of general progress in social inclusion in the period 2010–2014 is rather surprising. The consequences of the financial and economic crises in mind we would expect a somewhat opposite result, i.e., (mild) regress in social inclusion. An analysis of the three performance change components in Table 3 shows, however, that the social inclusion change in the period after the economic and financial crises was less pronounced and clear-cut than suggested by the estimated PC c -scores. Specifically, the average ΔOWN c -values of 1.0092 and 1.0404 for respectively the old and new EU Member States point out that the overall progress is social inclusion was only trivial. In fact, somewhat similar to what we observed in the period 2005–2009, for the average old EU Member State the progress in own social inclusion data was smaller than the overall progress in the baseline EU27 performance (i.e., the average \(\Delta OWN_{c} \times \Delta BP_{c}\) is smaller than one). A screening of individual Member State results reveals strong progress in own social inclusion data for Italy (∆OWN c  = 1.6816) and Sweden (∆OWN c  = 1.2573) for the old EU Member States and Latvia (∆OWN c  = 1.3402), Lithuania (∆OWN c  = 1.1940), Poland (∆OWN c  = 1.1413), and Croatia (∆OWN c  = 1.1344) for the new EU Member States. Luxembourg (∆OWN c  = 0.6126) and Cyprus (∆OWN c  = 0.7587) experienced the most severe regress in the own social inclusion data. Regarding the change in the BoD-estimated importance weights for the social inclusion sub-indicators, the average ∆W c -values of 1.1453 and 1.0943 for respectively the old and the new EU Member States indicate that, on average, sub-indicator importance weights became more favourable for the average old EU Member State and (to lesser extent) the average new EU Member States. Two Member States with strongly deviating ∆W c -values are Luxembourg (∆W c  = 0.2921) and Sweden (∆W c  = 4.3104). For Luxembourg BoD-based importance weights became considerably less favourable (as compared to 2010, in 2014 importance weights are much more dispersed with more importance weight, amongst other things, being assigned to the sub-indicators ‘People not living in households with very low work intensity (y 3 )’ where Luxembourg experienced an improvement, i.e., lower percentage of its population living in households with very low work intensity). For Sweden, on the other hand, importance weights in 2014 were much more favourable than the ones estimated in 2010 (in 2014, the maximum importance weight of 85% was assigned to the social inclusion sub-indicator ‘People not severely materially deprived (y 2 )’ where Sweden realized considerable progress, i.e., Sweden has according to data only 0.7 percent of its population that are severally materially deprived, which is very low compared to other EU Member States). In Fig. 2 of the estimated \(PC_{c}\)-scores and the three change factors, the (extreme) results of Sweden and Luxembourg are ignored in order to provide more informative visualizations.Footnote 8

Fig. 2
figure 2

BoD-estimated \(PC_{c}\)-scores and its tripartite decomposition: Period 2010–2014

Full size image

As to the question of whether or not social cohesion increased since the economic and financial crises, the estimations point out somewhat complicated dynamics. Results suggest that, on average, social inclusion has mildly progressed in the EU with both old and new EU Member States having realized small progress. However, on average new EU Member States realized slightly more progress, hence suggesting a slight increase in social cohesion. However, results also indicate that there are significant differences between Member States with countries such as Greece, Cyprus and Luxembourg having experienced substantial regress in social inclusion and other countries such as Sweden, Latvia, and Poland having experienced considerable progress in social inclusion. As such, this result of slightly increased social cohesion since the financial and economic crises should be considered with much caution.

6 Conclusions

In this paper, we focused on a set of statistical indicators as commonly endorsed by the Heads of State and Government in the Europe 2020-program and employed by Social OMC, for the measurement of social inclusion performance in the EU Member States during the period 2005–2014. In order to deliver a comprehensible and easily communicated overall score for social inclusion, we constructed a CI using the recently proposed adjusted version of the benefit-of-the-doubt methodology as enacted by Van Puyenbroeck and Rogge (2017). The key advantage of this version of the BoD-model is that it combines the benefit-of-the-doubt principle in the definition of the country-specific weights for the social indicators with a geometric aggregation procedure in the construction of the CI. Using this new method, we computed CIs to assess the social inclusion performance of EU Member States in 2005 and 2014 as well as decomposed changes in the CIs (tripartite decomposition) to examine how social inclusion evolved over time. Particular attention was given to the assessment of prior- and after crisis social inclusion performance.

The empirical analysis showed a couple of interesting findings. Firstly, the situation in both 2005 and 2014 was such that old EU Member States generally outperformed new EU Member States in social inclusion. Particularly the Scandinavian Member States (Sweden, Denmark, Finland), Luxembourg and the Netherlands appeared among the top performers. However, results also indicated that some of the Southern and Eastern European Member States were not lagging too much behind. Above-average social inclusion performances were for instance observed for Slovakia, Italy and the Czech Republic. However, results showed that for several of these Member States the relatively high CI-scores mask unbalanced performances across the different social inclusion indicators. Concerning the question on whether there has been progress and convergence in social inclusion since the enlargement in the EU and whether or not there were differences in the period prior- and the period after the crisis, results pointed out a trend of slight increase in progress and cohesion in the EU. However, whereas the increase in social progress and social cohesion in the EU was more outspoken in the pré-crisis period, the results for the post-crisis period were more mixed with some EU member States having realized a (slight) increase and other countries having experienced a deterioration in social inclusion. This results largely confirms what was recently noted in the report ‘Social Europe: Aiming for inclusive growth’ (Social Protection Committee, 2014) that the overall situation in the EU with regards to social inclusion and poverty reduction has improved very little, and in fact has been deteriorating in several countries. As remarked in that report, with a total of close to 1 in 4 Europeans living in poverty or social exclusion in 2013, it is clear that the European Union is still far away from reaching its goal set out by the Europe 2020 Strategy to lower the number of people in a situation of poverty and social exclusion or at risk of such a situation by at least 20 million by 2020.

For further research in this topic, it could be interesting to investigate the (possible) relationships between social expenditure (as percentage of a country’s GDP) and social inclusion levels. It is commonly believed that greater spending on a well-developed and well-functioning social policy would contribute to the combatting of poverty and social exclusion, and therefore, implicitly enhance social inclusion performance. A preliminary version of such a “performance versus efficiency”-analysis was already briefly presented by Cherchye et al. (2004). Another interesting application would involve conducting a more in-depth analysis of the effects the accession of the Central and Eastern European (CEE) countries had on social policies in both these new Member States and the original 15 EU Member States. Even with social inclusion policy falling under the principle of subsidiarity, given the extensive policy guidelines, performance monitoring and peer pressure provided by the EU and Social OMC, there is not much doubt that EU Membership impacted social inclusion policies of CEE Member States. Nevertheless, finding such casual relationships requires extensively more work and research than the current analysis has provided.