Introduction

Higher education (henceforth HE) is at the core of economic and social development and many international studies are concerned with the consequences of HE reforms. Indeed, one of the main goals of the Europe 2020 strategy is to have at least 40% of 30–34-year-olds complete HE. A key strategy for achieving this goal is reducing dropout and increasing completion rates in higher education trough proper policies. It is regarded as crucial for creating the high-level skills that Europe’s knowledge—intensive economic sectors need as well as for Europe’s capacity to innovate and foster productivity and social cohesion (EHEA 2012; Vossensteyn et al. 2015; OECD 2016). In this European landscape, Italy fails in improving for years, in spite of a number of successive and disruptive HE reforms. The leading one was launched in 1998 in the vein of the Sorbonne Declaration together with France, Germany and the UK, aiming at adopting a common two-tier structure of study programs and modernizing institutions (Witte 2006). This common initiative began since the traditional single level degree was judged too rigid in front of the expanding and differentiated demand. Moreover, the length of programs was blamed for the high dropout rates in all systems but UK (Ballarino and Perotti 2012). The European HE integration was going on steadily and in June 1999 the so—called Bologna Process was started by 29 countries constituting the European Higher Education Area (EHEA), that counts today 48 members. Italy was therefore among the first countries to change its university system according to European guidelines designed also to easier the recognition of degrees. The reform was introduced in 1999 and reached its implementation in the academic year 2001. The reform canceled the traditional “laurea” replacing it with the “\(3+2\)” structure (bachelor and master degrees). Fifteen years after the launch of the Bologna Process, many studies seek to measure the impact of the HE reform in Europe with respect mainly to participation and graduation success. In the case of Italy, participation is the main goal of the “\(3+2\)” reform, in order to reach a larger diffusion of university education among the population. The increasing college’s number was the new way to enlarge the supply of graduates (Cappellari and Lucifora 2009), whereas in other countries, the result of augmenting participation worsened the social distribution of graduates. Blanden and Machin (2004) analysed the expansion of higher education in the UK and found a widening of participation gaps between rich and poor youngers.

In order to have a better comprehension of orientations in policy-making of higher education institutions, we have to observe indicators regarding not only participation rate and related characteristics but also graduation success measured by retention, completion rate and time-to-degree. It is useful to note that a higher participation rate does not always result in a higher graduation rate (Lassibille 2011; Chen 2012; Meggiolaro et al. 2017), so the determinants of students’ outcome during the university careers are deeply investigated in the recent literature.

The Italian HE reform had different effects on completion probabilities depending on region, organization and institution (Oppedisano 2011). In particular, regions play an important role in determining the enrollment rate and academic performance, which are higher in the Northern and lower in the Southern Italian regions, whereas the opposite happens for the dropout rates. The regional effect is due to culture, local institutions and regional funding in sustaining the equal opportunity and the right to study. The HE organization changes are unevenly distributed among regions, increasing both the number and governance effectiveness of university sites, reorganizing duration and types of curricula and widening the range of degrees. The nationwide effect was an increase of enrollments and retention rates, which increased the equality of opportunities by opening access to more groups in the society. On the other hand, large negative effects on individual performance have been observed in less developed regions, proving only a limited increase in equality of opportunity in terms of completion of tertiary education (Bratti et al. 2008).

With respect to the effect of the Bologna Process in Europe, we observe a similar outcome. Georg (2009) stated that HE reform in Germany reduced the dropout rates of students, whereas Horstschräer and Sprietsma (2010) found no significant effects on college enrollment, nor on the dropout rates. In the analysis for Spain, the main findings are that university features, such as expenditure, student–teacher ratio and financial aid to students are important in accounting for graduation rates, whereas student ability has no significant impact in explaining graduation (García-Estevez and Duch-Brown 2014).

In order to better assess which characteristics are dominant, we need complex data, taking into account all of the aspects examined. Moreover, the temporal dimension cannot be overlooked and the cohort analysis may contribute to give some better insights into the HE changing environment. The aim of this paper is to study how the institutional changing environment affects the characteristics of both degree courses and individuals in one Italian university. With respect to the existing literature, we integrate specific institutional aspects in a longitudinal dataset and we propose a two-step analysis. In the first step, we use matching techniques to build a sample suitable to identify the causal effect of the Bologna Process on graduation and then we estimate a logistic model for the probability of graduating.

The remainder of the paper is organized as follows: Sect. 2 presents evidence from previous studies. Section 3 is devoted to depicting the main features of the Italian university system. Section 4 describes the empirical strategy. In Sect. 5, the data and the sample selection criteria are detailed. Section 6 reports on the effect of the reform on graduation probability. Section 7 concludes the paper with some policy discussions. We also detail in Appendix 1 the balance diagnostics of the matched sample.

Evidence from Previous Studies

Recent research has investigated the extent and the effectiveness of the HE reforms in the European area, particularly in terms of boosting enrollments and lowering both dropout rates and age at graduation. A recent report of the European Commission addresses a comparative study on higher education dropout and completion in Europe (Vossensteyn et al. 2015). It is based on an extensive review of literature and policy documents on study success in higher education in 35 European countries. The main aim is to investigate the available evidence of the effectiveness of policies and good practices in addressing study success at the country level as well as at the institution level.

In this report as in other contributions to HE research, funding is a relevant determinant of graduation success. This involves two main levels: funding of university system and of study grants or loans. As regards funding policies, the observed increase in financial autonomy in HE governance in Europe goes hand in hand with a variety of outcome—based funding models aiming at increasing enrollment or students’ performance (Dobbins and Knill 2017). Financial support also influences student retention and persistence, especially for those from low-income backgrounds (Tinto 2010; Pascarella andTerenzini 2005; Paulsen and Edward 2002). Several studies have found that the Italian reform did partly achieve these goals in particular in terms of graduation and career, even if central budget cuts have slowed the process. Cappellari and Lucifora (2009) using two cohorts (1998 and 2001) of the ISTAT Survey on secondary school graduates, find evidence of a higher rate of access to university, especially for low income and talented students. On the other hand, recent empirical findings by Di Pietro (2012), in line with Boero and Naylor (2005), show that the magnitude of the increase in the enrollment rate is smaller than previously concluded. Bratti et al. (2010) observe, on a different dataset (1993 and 2002 SHIWFootnote 1 waves of Bank of Italy) low access and low graduation rates for students from poor socioeconomic backgrounds.

The poor socio-economic status plays a clear role against participation and retention, whereas gender influences the type of program (technical sciences are chosen mainly by men, while human sciences by women). Triventi and Trivellato (2009), using data from the five waves of the Italian Longitudinal Household Survey, examine the different dynamics of higher education participation and performance among social classes. University attendance did not increase by equal measure among all classes and social groups; people from the bourgeoisie and white collar middle classes enrolling in university outnumber those from the petit bourgeoisie and working classes. Delayed graduations increase for all enrolled students, but the dropout rate rises only among the lower–middle classes. The authors conclude that in Italy inequalities persist over time, reducing from declining performances of the upper classes, and not because of an improvement of the lower classes. In a recent study, Brunori and Serlenga (2012) show a significant improvement in the Italian university system in terms of equality of access after the reform, but they conclude that the long-term effects are less clear. These inconclusive results may be due to high dropout rates at the beginning of the university career, as reported by Cammelli et al. (2011), who highlight the mixed outcomes of the Bologna Process by employing the AlmaLaurea database. On the contrary, ISTAT (2006) reports a reduction in the early dropout rates nationwide from 2001 to 2004, in line with d’Hombres (2007) and Di Pietro and Cutillo (2008) for 1998, 2001, and 2004. They show a reduction in dropout rates following the reform, after controlling for a number of variables that could have influenced the dropout choice. On the other hand, Bratti et al. (2010) using data of a single institution (University of Ancona in Central Italy) estimate no significant changes in the individual probability of dropping out after the implementation of the Bologna reform. Cingano and Cipollone (2007) study the influence of parental educational background on university withdrawal, revealing that the dropout probability is negatively correlated to father’s years of formal education.

There is also another dimension of the reform we are interested in and which we also investigate: the risk that improvements in student performance after the Bologna reform might partly reflect grade inflation. This increased performance may derive from different causes, such as regional habits (Bagues and Zinovyeva 2008), or competition among Higher Education Institutions (HEIs). The literature suggests that the higher competition among HEIs promoted by the Bologna reform might have encouraged universities to compete for as many students as possible to obtain more state funds (Bonaccorsi and Daraio 2007). The university attractiveness may also be pursued by facilitating the students career with a reduction in grading standards rather than by increasing university effectiveness (Bratti et al. 2010). In order to state this issues, it is necessary to assess the dimension of student attraction, which can be done using various measures such as the number of enrolled students, the number of formative credits achieved by students, or the number of graduates. This remark is taken into account in our analysis. On the international level, Cardoso et al. (2008), who compute the effect of the bachelor implementation on the number of university applications in Portugal, find that the number of applicants is significantly higher in the departments that implemented the bachelors degree than in those departments still awarding traditional degrees, suggesting that the Bologna reform increases the attractiveness of university.

Bologna Process in Italy

As seen in the previous paragraph, the aim of the Bologna Process was both to harmonize bachelor and master degrees throughout Europe and to improve the performances of the European universities. The comparability of higher educational degrees across Europe is expected to increase student and labour force mobility, leading to more competition between universities and thus increasing the international attractiveness of the European higher education system, creating the so called European Higher Education Area (EHEA). In line with this view, Italy implemented the Bologna Process in the flow of reforms that have characterized the nineties, when the Italian Parliament passed a series of laws granting Italian universities a substantial autonomy and flexibility. The management of teaching and research activities was allowed starting from 1993 and the flexibility in creating shorter courses, 2–3 years long, called diploma universitari was implemented in 1995, opening the way to the main reform introduced by the Bologna Process. The latter replaced the main university qualification based on one-tier traditional academic degree with an official length of 4–6 years, with a 3-year bachelor degree followed by a 2-year master degree (Law n. 509/1999), the so called \(3+2\) reform. One of the expected outcome of the new programs’ structure was the improvement of the student performance and the increase in the number of applications for Italian universities, which enrollment rates were the lowest in Europe. This was partially due to both the length of study and the dropout rate at the beginning, higher than elsewhere in Europe. Sixty percent of students failed to complete their studies and only 10% managed to complete within the official timespan (Hunter 2015). Therefore, the aim of the reform was to improve enrollment, retention, and progression of students in higher education by decreasing the duration of the degree and consequently the cost of education. Furthermore, by reducing the length of undergraduate degree courses from 4–5 years to 3 years, students should be able to graduate at an earlier age than in the previous system, in line with other European countries, thus reducing the age at first job. Five years after the first reform, Law n. 270/2004 established more freedom for students in choosing the subject of their second-level degree. The objective of this reorganization was both to pursue cross-knowledge and to introduce more open and flexible degree programs. Thereby, the overall reform further improved universities autonomy in determining the curricula, considerably increasing the number of degree courses among which students can choose.

These two recent reforms seem to have reached some positive results according to a recent report by the National Evaluation Agency of Universities (ANVUR 2016). The dropout between the first and second year was on average 22–25% (CNSVU 2002) before 2001, while in academic year 2013/14 is about 13.7% (ANVUR 2016). Tertiary attainment rates among 25–34-year-olds increased from 10% in 2000 to 25.6% in 2016, but Italy remains at the bottom of the ranking: 36th position out of 37 OECD countries (OECD 2017). The quota of students completing a 3-year bachelor’s degree in time increased from 10% of the pre-reform to 26.8% in the academic year 2011/12 (ANVUR 2016). A non-negligible effect on graduation concerns the geographical distribution of student performance. ANVUR reports a graduation rate of 60.2% for the northern macro-region, whereas in Southern Italy only 43.8% of enrolled students reached their bachelor. As regards master courses, the graduation rates are 80.8% in the North and 70.8% in the South. The increase in female participation rates is similar to that in the other OECD coutries. In 2016, the quota of enrolled women was about 55.3%; the bachelors’ graduation rates among women were 57.7% (male 46.7%), whereas for master courses they were respectively 77.2% for females and 75.9% for males (ANVUR 2016). These slightly positive results leave space for analyses aiming to find out which variables and reform aspects are really responsible for the slow success in graduation in Italy. In our study, we used data from the University of Trieste, a middle-size university located in the northeastern part of Italy. It is a highly multidisciplinary university, with courses covering all study fields but agricultural and veterinary.

Empirical Strategy

We collected data on students enrolled prior to the reform, who were given the possibility to either continue their career within the “old” system or switch to the new one, with the aim of assessing the effect of changing the degree course toward a post-reform, 3-year course, on the probability of receiving the degree.

Standard methods with observational data do not allow causal inference on the effect of the treatment on the outcome—here the graduation—because, although both treated and untreated individuals are observed, they may differ for reasons other than the treatment. In other words, any observed difference between the treated and the untreated with respect to the outcome cannot be ascribed to the treatment (is not necessarily an effect of it since it may be due to the differences between the two groups (for instance, less motivated students may be more likely to switch in order to shorten their career, thus leading to an apparent negative effect of switching). In order to overcome this difficulty, and gain insight on the causal effect of the treatment, matching techniques can be used to build a balanced sample. A balanced sample is a subset of the original sample, selected so that it includes a group of treated and a group of untreated units and the two groups are similar with respect to their pretreatment characteristics. Therefore, the conditional independent assumpion (CIA) is satisfied, building up the control group conditionally to the observable X pretreatment variables. In order to build such a balanced sample, matching techniques are used: for each treated unit, one or more untreated units having similar pretreatment characteristics are sought. If such unit(s) are available, the treated unit and its match(es) are included in the balanced sample; otherwise, the treated unit is dropped. Thus, we assume that at least part of the treated group has, in the control group, matching units. This is a crucial implicit assumption, named Common Support, needed to apply the counterfactual approach. When checking the Common Support between the two groups we lose some treated units and this complicates the interpretation of the causal effect, since it represents the average effect of treatment on a subset of treated units. It would therefore be more appropriate to define the effect of treatment as Sample Average Treatment Effect on the Treated (SATT) (Imbens 2004).

Since in observational studies assignment of units to the treatment—here, the reform—is not random, matching methods seem particularly well suited to apply in many educational research contexts. Several studies have sought to identify the causal effect in educational policy research by means of matching techniques. Among these, Alcott (2017), using data from the Longitudinal Study of Young People in England (LSYPE), employs the propensity score matching to investigate whether encouragement influences the likelihood of students enrolling in (i) advanced high school courses and (ii) university degree courses. Brand et al. (2014), with data on postsecondary outcomes of high school graduates of Chicago Public Schools, match community college goers to non-community college goers who have similar propensities to attend, for estimating the effect of community college attendance on bachelors degree completion. Further, McKinney and Burridge (2015) using data from the Beginning Postsecondary Student survey, apply propensity score matching techniques to examine the effects of loans on persistence for students enrolled in degree programs. Finally, Graziosi (2014) and Agasisti and Murtinu (2016) through several matching techniques, estimate the impact of scholarships on students’ careers in Italy [see also Reynolds and DesJardins (2009) for the use of matching methods in higher education research].

Let then, for each unit i, \(Y_i\) represents the outcome (\(Y_i=1\) if unit i graduates, 0 otherwise), \(t_i\) be the treatment (\(t_i=1\) if unit i switch to the post-reform system, 0 otherwise), \({\mathbf{x}}_i\) be the pretreatment characteristics (vector valued, detailed in Table 2).

Let now \(d(\cdot ,\cdot )\) be a distance between units based on the pretreatment variables \({\mathbf{x}}\), \(d({\mathbf{x}}_i,{\mathbf{x}}_j)\) is then the distance between unit i and unit j (more on the choice of d below). The basic matching technique goes as follows.

Let \({\mathscr{T}}=\{i:t_i=1\}\) be the set of treated units and \({\mathscr{C}}=\{i:t_i=0\}\) be the set of control (untreated) units. For each \(i\in {\mathscr{T}}\) the nearest available control unit is singled out: \(c(i)\in {\mathscr{C}}\) such that \(d({\mathbf{x}}_i,{\mathbf{x}}_{c(i)})\le d({\mathbf{x}}_i,{\mathbf{x}}_{j})\) for all \(j\in {\mathscr{C}}\). Some further conditions are often imposed, such as the overall distance between \({\mathbf{x}}_i\) and \({\mathbf{x}}_{c(i)}\) and/or the difference between the values of some particular components of \({\mathbf{x}}\) being below certain thresholds. If these conditions are satisfied, units \({\mathbf{x}}_i\) and \({\mathbf{x}}_{c(i)}\) are matched, that is, they are included in the balanced sample and dropped from \({\mathscr{T}}\) and \({\mathscr{C}}\); otherwise, the treated unit i is dropped from \({\mathscr{T}}\). Then, the same steps are taken for another treated unit in \({\mathscr{T}}\) using the possibly reduced set of controls \({\mathscr{C}}\). The procedure is repeated until all treated units have been dropped or matched.

Note that variants of the above procedure may be used, which would allow for multiple matches.

A common choice for measuring the similarity between units is the propensity score (Rosenbau and Rubin 1983): the distance between two units is given by the difference in the estimated probabilities of being treated as a function of the pretreatment characteristics. Alternatively, Mahalanobis distance (Rubin 1980) could be used: Mahalanobis distance between units i and j is defined as

$$\begin{aligned} d({\mathbf{x}}_i, {\mathbf{x}}_j) = \Big \{({\mathbf{x}}_i-{\mathbf{x}}_j)'S^{ -1 }({\mathbf{x}}_i- {\mathbf{x}}_j) \Big \}^{ 1/2 } \end{aligned}$$

where S is the (sample) variance-covariance matrix of \({\mathbf{X}}\). An improvement over Mahalanobis distance for the purpose of matching and balance was proposed by Sekhon (2011) and Diamond and Sekhon (2013). In particular, a weighted Mahalanobis distance is used

$$\begin{aligned} d_W({\mathbf{x}}_i, {\mathbf{x}}_j) = \Big \{({\mathbf{x}}_i-{\mathbf{x}}_j)' (S^{ -1/2 })' WS^{ -1/2 }({\mathbf{x}}_i- {\mathbf{x}}_j) \Big \}^{ 1/2 } \end{aligned}$$

where W is a \(q\times q\) positive definite weight matrix and \(S^{ 1/2 }\) is the result of the Cholesky decomposition of S. The weights W are chosen to optimise the balance of the covariates observed between the matched treated and control groups obtained by matching units based on \(d_W\). From a computational point of view, a genetic search algorithm (Mebane Jr and Sekhon 2011) is employed to perform the optimisation (Sekhon 2011).

Once a balanced sample is obtained by means of the above procedure, a logistic model is fit (Gelman and Hill 2006, Chap. 10) to estimate the treatment effect.

Let then \(\pi _i=P(Y_i=1)\) be the probability of obtaining the degree, we model \(\pi _i\) as

$$\begin{aligned} \log \left( \frac{\pi _i}{1-\pi _i}\right) = \beta _0+\sum _{j=1}^q \beta _j {{ x}}_{ij} + \gamma t_i \end{aligned}$$
(1)

The logistic model can be estimated using standard techniques (maximum likelihood inference), but, due to the fact that a balanced sample is used, we are allowed to interpret the inference on the model in causal terms. In particular, the coefficient \(\gamma\) in (1) represents the treatment causal effect.

Data and Sample Selection Criteria

Data and Descriptive Statistics

For the purpose of the analysis, we collected administrative individual-level data for the students enrolled at the University of Trieste, located in the Friuli Venezia region (hereafter, FVG) in 2000, the last academic year before the introduction of the reform, and we followed their careers until the 2012 academic year. Of the 25,866 observed students, 64% were female, and a total of 3765 (of whom 60% were female) matriculated for the first time in 2000.

The data include a wide range of students’ information: personal data and previous educational experience, the academic system of enrollment, as well as the department and the course of study. This administrative dataset is merged, and therefore enriched, with information provided by the Regional Job Observatories of both the FVG and Veneto regions on the transitions to work. These archives contain the starting contracts with the following characteristics: type of contract (e.g., open-ended, fixed term, short-term, part-time, full-time) and the number of working days per year. This allows us to identify those students who are also working during their academic career and to include this information as a possible covariate explaining the graduation probability. In particular, we consider the number of days worked during the academic path, weighted according to the part-time or full-time character of the job (the weight of part-time worked days was 0.75).

Table 1 Number of students changing university system and dropping out
Full size table

The final dataset allows keeping under observation the entire academic career of students: the potential change of both the academic system (i.e., from the old to the new one) and the field of study or, alternatively, the withdrawal from the university, the number and the average grade of exams, the graduation year and the graduation mark.

Focusing on students enrolled prior to the reform, who were given the possibility to either continue their career within the “old” system or switch to the new one, we can estimate the effect of changing the degree course toward a post-reform, 3-year course on the probability of receiving the degree. In Table 1 we report, for each observed academic year, the number of students who changed system or dropped out from the university. In parentheses, we also report the number of female students. We registered the highest number of students switching during the first year in which the reform was implemented. More than half of students, both male and female, opted for the new academic system, attracted by the shortened duration of the degree course. Furthermore, the total number of dropouts was higher than the number of students who decided to change academic system, irrespective of their gender.

Sample Selection Criteria

To identify all the variables X that could affect the outcome, we include in the matching algorithm the covariates expressed in Table 2.

We impose exact matching for sex, year of birth, region of residence, type of high school diploma, year of enrollment, department, and range of income, while a maximum tolerance level, the caliper, set at 1 standard deviation, was imposed for the final high school grade, the average mark of exams taken in 2000, and the number of days worked during the enrollment period. In addition, students with no exams were matched only to students with no exams, that is, an exact matching on the variable no exam in 2000 was de facto performed.

Table 2 Student characteristics and matching specifications
Full size table

It is relevant to compare the distribution of the covariates in the whole sample with that in the matched sample to understand to what extent the results can be generalised. In Fig. 1, we compare the distributions of enrollment year and year of birth of students; the evidence shows that the matched sample has a much shorter tail than the whole sample. We notice most students are far behind with regard to graduating in the prescribed time, given the fact that, in Italy, university students are not obliged to graduate within a fixed time frame. From this perspective, the Bologna Process could help students enrolled in the previous years (i.e., those who have not completed the required university exams within a time period set) to reduce the time and effort needed to complete university courses.

Fig. 1
figure 1

Distributions of enrollment year and year of birth of students. The dark grey indicates the distribution of the whole sample, while the light grey refers to the distribution of the matched sample

Full size image

Furthermore, another restriction to generalisation is the fact that almost all matched students were living close to where the university is located: in the FVG region and Veneto region. In particular, almost 79% of enrolled students live in the region where the university is located (FVG), and this value has remained largely stable over time (ISTAT 2016). Of the 21% of students coming from other regions, 19% come from Veneto region, which is the closest region to FVG.

The distribution of the other variables in the matched sample was similar to that in the whole sample. In Appendix 1, the balance diagnostics are reported for all variables (see Table 5), while the quantile-quantile plots are depicted to better capture the differences between the distribution of numeric variables (see Fig. 2).

Results

In this section, we report the estimated probability of obtaining a degree on the balanced sample, in which we included only students enrolled from 1995 and born in 1975 or after, and living in the FVG and Veneto regions. If no covariates were used besides treatment, the estimated effect was \(-\,0.161\) (s.e. 0.112), corresponding to a probability of obtaining a degree of 0.741 for the untreated students, that is students who did not choose the new university system, and 0.709 for the treated students, that is students who opted for the bachelor’s degree.

We also considered more complex logistic models, including all variables of the matched sample (Table 3), and shrinking the analysis to the students enrolled in 2000 (Table 4).

Table 3 Estimates of the matched sample with all variables
Full size table

Table 3 reports the estimates of all the variables used for the matching, excluding the university departments, whose coefficients were not significant. The estimated effect of the treatment is − 0.154, similar to the value obtained with no other covariates besides the treatment. Not surprisingly, the covariates that positively affected the probability of graduating are related to the educational background of students, measured by the final high school grade, and the income, confirming that the degree performance in Italy is strongly affected by both prior educational attainment and the economic background of the student. The higher average of the exams taken in 2000 is positively related to the probability to graduate, while being male decreased the likelihood of degree completion. It is worth noting that the year of enrollment has a not negligible impact on the probability of graduating. This result denotes that the length of registration is associated with a significant negative effect on graduation: for older enrolled students who switched to the new system, we estimate a negative effect on their probability of graduation.

Table 4 Estimates for students enrolled in 2000
Full size table

In Table 4, we restrict the matched sample to the students enrolled for the first time in 2000, the last year before the introduction of the new academic system. The effect of the treatment differs from the previous results: for students enrolled in 2000 changing the academic system increased their probability of obtaining a degree. According to the relevant literature and to the previous results, the educational and economic backgrounds of students as well as their performance during the first year positively affects the probability of graduating. Furthermore, being the probability of graduating decreases for males, their university performance is weaker than for females.

Conclusions

The mixed success of the Bologna Process in the European panorama and especially in Italy opens up room for further research. Most of the literature on Italy focuses on short-term effects, for example, enrollment and dropout, finding positive effects. Moreover, these observed effects are not attributed to the reform. For instance, at the University of Trieste the observed increase in the number of enrolled students from 2001 to 2005 is not attributable to the reform, but is almost certainly due to special admission agreements signed during that period with the Italian Army and the Social-worker Association (Chies et al. 2014), for specific courses (Political and Social Sciences).

In this paper, taking into account this problem, we used pooled data on the 2000 cohort of students enrolled in the same year or in the years before. They were attending a one-tier traditional academic degree “laurea”, but they could switch to the new system. In order to assess the impact of the university system reform (\(3+2\)), we estimate and compare the probability of graduation for students who chose to switch to the new system and for those remaining in the old one. A total of 25,866 undergraduate students were considered, with respect to their probability of completion, that is, obtaining a degree either in the pre-reform system or in the post-reform one, depending on switching.

To isolate the causal effect of the Bologna reform on degree completion, we restricted the inference to a subsample of students first, after the building of a balanced sample with matching techniques.

Our basic idea was to find, in the group of students who did not choose the new university system, those students who were similar to their colleagues who opted for the bachelor’s degree (treated students) in all relevant pretreatment characteristics X. Differences in graduation probabilities of this selected (and thus adequate) control group and those of treated students can be attributed to the university reform. The matching variables were gender, residence in regions close to the university, economic and educational background of students, and their academic performance, measured by the average mark of exams.

Estimates of a logistic model on the balanced sample showed that the effect of treatment (switching to the new academic system) varied according to the year of enrollment of students: for the most recently enrolled students, we estimate a positive effect, meaning that the completion probability increases. As older students were considered, the effect changed sign, progressively increasing in absolute value as the enrolling year went back in time. These results, in line with the previous literature, confirm the negative correlation between duration of enrollment and probability of graduation. This can explain the mixed results emerging from the literature considering the initial short-term effects of the Bologna Process (Horstschräer and Sprietsma 2010; Cammelli et al. 2011).

On the other hand, younger students who opted for the Bologna reform had better performance in terms of completion, in line with the results for Italy reported by ANVUR (2016). These results suggest that the effectiveness of the reform on enhancing completion was confirmed, but we cannot state that the quality of students’ careers was different and, partially, this effect could also be due to the reduction in the number of exams in the bachelor courses.

The previous educational background of students, measured by the final high school grade, and the average of exam marks in 2000 had a positive effect on graduation, according to the relevant literature, but to a lesser extent for the 2000 cohort. As stated by Checchi and Flabbi (2006), our findings confirm that students with a better preparedness usually own a better socioeconomic background. Social origin is still positively associated with success in upper secondary and tertiary education. Furthermore, higher income was positively related to graduation probability, proving that social inequalities are noticeable when the completion rates of tertiary education are considered (Blanden and Machin 2004; Bratti et al. 2008; Checchi et al. 2013).

The lower performance of males in HE reported in the literature is confirmed in the present study. Gender differences in educational careers may be explained by differences in non-cognitive learner characteristics, but also from relevant external factors such as family responsibilities or labour market conditions (OECD 2008; Severiens and ten Dam 2012). Opportunities in the job market may explain the difference between women and men, with respect to the study career. As a result, the opportunity cost of attending university is higher for males than for females. As a matter of fact, the time spent on working during the university may be an indicator of attrition. It seems to have a low but negative effect on performance for the two cohorts. The student in the “\(3+2\)” system have a higher completion rate, whereas the 2000 cohort experiences a negative performance.

Our results draw attention to several contextual factors, which should be taken into account by the policy-makers responsible for educational policies and management. Measures and support services devoted to student career performance have to be tailored according to context and student’s individual characteristics observed preferably in longitudinal data. For males characterized by lower performance rates, both support and counselling services for individual study and the development of appropriate study methodology should be provided. The study should be interpreted with caution when debating its external validity: despite the results having a strong internal validity, strictly based on the selected subsample, and the applied method allows their causal interpretation, the generalisation of the results is quite limited.

An obvious limitation comes from the fact that we consider students form specific university as a consequence, they also come from a small area of the country. Motivational aspects are also of relevance for student’s outcome. Future research should consider also these aspects, in order to better design and implement the HE educational policies.