Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction: The U-Map Instrument

The rationale for developing a European classification of higher education institutions lies in the desire to better understand and use diversity as an important basis for the further development of European higher education and research systems (see also van Vught 2009). In order to reap the full benefits of increasing diversity, a tool is needed to describe this diversity. This is the aim of U-Map – an instrument for mapping the European higher education landscape which enables various groups of stakeholders to comprehend the diverse institutional activity profiles of European higher education institutions. This will contribute to the creation of a stronger profile for European higher education on a global stage and to the realisation of the goals of the Lisbon strategy, the Bologna process, and the Modernisation agenda (European Commission 2011).

2 Classifications and Rankings (Diversity and Transparency)

Global rankings intend to judge higher education institutions and they do so largely by focusing on research performance. They give only limited regard to disciplinary, language and institutional diversity. In addition, global rankings offer composite institutional indicators on the basis of which league tables are constructed. Rankings are instruments to display vertical diversity in terms of performance by using quantitative indicators. Most existing rankings in higher education take the form of a league table. A league table is a single‐dimensional, ordinal list going from ‘best’ to ‘worst’, assigning to the entities unique, discrete positions seemingly at equal distance from each other (from 1 to, e.g., 500).

The critique on rankings is well knownFootnote 1: rankings

  • focus on ‘whole institutions’ (ignoring internal variance)

  • concentrate on ‘traditional’ research productivity and impact and neglect the performance on other dimensions of activity

  • focus on ‘comprehensive research universities’, and neglect the bulk of higher education institutions

  • aggregate performance into composite overall indicators, without clear rationales regarding the weighting of the spate indicators

  • use constructed ‘league table’ that may suggest clear rank orders based on insignificant absolute differences between institutions

  • imply cultural and language biases, favouring English language publications

  • imply bias against humanities and social sciences

Classifications are intended to do something very different. Rather than ignoring or limiting diversity, these instruments intend to make diversity transparent. Classifications are tools that try to describe and visualise the diversity of institutional activity profiles.

3 The Development of U-Map

U-Map was developed in three stages, all subsidized by the European Commission. In the first stage (van Vught et al. 2005), the conceptual framework for the study was drafted and the basic design principles were developed in an intense consultation of various stakeholders. These design principles were in short:

The classification is

  • based on empirical data: it should be based on activities not on normative mission statements

  • based on a multi-actor and multi-dimensional perspective. There is not one single user of the classification but a range of users (policy makers, higher education institutions, employers, students).

  • based on a multi-dimensional perspective. The activities of higher education institutions are too complex to be captured in only one dimension.

  • non-hierarchical; it addresses the horizontal diversity in higher education, the richness of the landscape.

  • relevant for all higher education institutions in Europe; U-Map is focused on individual European higher education institutions that are recognized as separate and legally identifiable organisations in their own national systems and that offer accredited higher education degree programmes.

  • descriptive, not prescriptive; the indicators and the way the results on the indicators are presented should be non-normative, passing no judgment on quality or performance

  • based on reliable and verifiable data

  • parsimonious regarding extra data collection; response fatigue among higher education institutions should be avoided.

Using these design principles, the conceptual frame was further elaborated and dimensions and indicators were selected and defined in the second stage of the development of U-Map (van Vught et al. 2008). In advisory board meetings and stakeholder conferences the users were actively involved in this stage. A first version of the data collection instrument was tested among nearly 75 higher education institutions and the results indicated that U-Map is a feasible transparency instrument. Based on that result, the instrument was further developed. Indicators and dimensions were reassessed and instruments to analyse and visualize the results were developed and tested (van Vught et al. 2010). The current version of U-Map was presented in October 2009.

In the current version of U-Map six dimensions and 25 indicators are used to characterize the activities of a higher education institution (Table 45.1):

Table 45.1 U-Map dimensions and indicators

Once the indicators are defined, empirical information could be collected. The main data-gathering instrument is the on-line U-Map questionnaire for higher education institutions. International databases comprising comparable data at the institutional level do not exist or they cover only a very limited part of the data needed. In Europe the prime data provider will be higher education institutions through country-specific questionnaires that can be pre-filled the information with that is available from national databases. The questionnaire is organized around seven sections:

  1. 1.

    General information

  2. 2.

    Students

  3. 3.

    Graduates

  4. 4.

    Staff

  5. 5.

    Income

  6. 6.

    Expenditure

  7. 7.

    Research and Knowledge Exchange.

An online Glossary and Help Desk are provided to facilitate consistent and comparable data-collection across institutional and national settings.

U-Map offers the option to have the questionnaire partly pre-filled using existing data from national databases provided by statistical agencies, rectors’ conferences or ministries of education. This can reduce institutional data collection burdens and provides standard data definitions.

Institutional data is validated by the U-Map project team in consultation with individual institutions and, where possible, comparing data with existing national and international databases.

The questionnaires have been piloted with more than 50 institutions while the concept of pre-filling has been tested in the case of the Norwegian higher education system. Several other European higher education systems have shown interest in a similar process.

The final step is to determine the position of the institutions on the indicators in the different dimensions. The data provided by the higher education institutions are used to calculate indicator scores. These scores are presented in a categorised way in a graphical chart (a sunburst chart, see Fig. 45.1). The indicator scores are grouped into four categories, where the boundaries between the categories are determined by cut-off points that depend on the distribution of the indicator scores across the sample. At the moment, quartile scores are used to guide the choice of the cut-off points.

Fig. 45.1
figure 1_45

U-Map activity profile (sunburst chart)

To communicate the results of the U-Map classification process, a web-based application was developed that allows the user to explore the results in an interactive way.

The web-based application consists of two instruments, the Profile-Finder and the Profile-Viewer. These tools allow the user of the classification to first select and then compare institutions. Through the Profile-Finder, the user selects HEIs by filtering out those institutions that have the same values on user selected indicators (e.g. the selection of the two institutions in Fig. 45.2 is based on their score on ‘peer reviewed academic publications’ and ‘mature students’). With the Profile-Viewer the user may zoom in on the indicators of the profiles of the institutions selected with the Profile-Finder.

Fig. 45.2
figure 2_45

U-Map profile-viewer

U-Map is a flexible tool continually being improved in a dynamic environment. Indicators, data elements and underlying definitions are held under constant review. User suggestions are collected and improvement options are studied and discussed. This process results in adaptations in the U-Map tool (every 4 or 5 years). Notwithstanding this dynamic evolution, U-Map remains consistent to its founding principles.

4 The Implementation of the U-Map Tool

One of the limitations of the U-Map tools so far has been the number of higher education institutions that have submitted data. Because of this constraint, it is not methodologically sound to present the results in the intended way. The institutional activity profiles are presented in a fully functional way (see www.u-map.eu/finder) but the profiles cannot be linked to a specific institution (i.e. the name and country). This restricted demo mode limits the full potential of the U-Map instrument. Although it allows the user to experience the functionality of the instrument, it does not allow comparison of real identifiable higher education institutions. In order to make the next step towards a fully functional classification tool, the number of data submitting higher education institutions needed to be expanded.

In 2010 a fourth stage was added to the U-Map project sequence. The objective of this project, subsidised by the European Commission under the Lifelong Learning programme, was to further the implementation of the European classification of higher education institutions (U-Map) and to enhance its impact on transparency in the EHEA. The focus of activities was on the expansion of the number of participating institutions.

Two approaches have been used so far for recruiting higher education institutions:

  • A national approach, combined with pre-filling. In this approach, one or more national organisations (be it a ministry of education, or a rector conference) take the initiative to invite all higher education institutions to participate in the classification project. Participation is on a voluntary basis (see Box 45.1: The U-Map Protocol). Analyses are made of existing national data bases. The analyses comprise the coverage, the definitions used and the constraints in data delivery. Relevant data from these national databases are used to pre-fill the on-line questionnaires of the institutions. Relevant data from these national databases are used to pre-fill the on-line questionnaires of the institutions.

  • Bilateral filling refers to the submission of data by an individual higher education institution on the institution’s own initiative. There is no third party involved, other that the institution and the U-Map team, assisting in the process.

More than 70 institutions in two countries (Estonia and Portugal) and 20 individual institutions were recruited. This project, as well as a project initiated by the Dutch government, has provided insights in the feasibility and relevance of U-Map in a cross-national perspective. In the following part the implementation of U-Map in three national systems will be discussed.

6 Estonia

The Estonian higher education system is a small system (around 55,000 students), comprising 6 public universities, 4 private universities and 22 institutions of applied higher education. The larger institutions have branch campuses in different parts of the country. The institutions differ widely in size (from 50 to 17,500 students) and scope of activities and disciplines.

6.1 U-Map Implementation

The recruitment of institutions was organised in close cooperation with the Estonian Ministry of education (department of higher education). Representatives from the Estonian higher education institutions and potential other data providers joined in to discuss procedures regarding pre-filling and the data provision by the institutions. Based on those discussions data were extracted from national existing data sources and used to pre-fill parts of the institutional questionnaires.

After pre-filling, a technical workshop was organised with the participating higher education institutions. Issues regarding the definitions of indicators and data-elements and practical issues regarding the questionnaire were discussed. Based on these discussions the decision was made to change the reference year (from 2008 to 2009), as well as further clarification could be given regarding the definition and use of region and temporary staff data. Data collection using the on-line questionnaires was concluded 4 months after the technical workshop and the verification process took another month.

At the end of the project, 28 higher education institutions (out of 32) provided data, and U-Map profiles were generated.

6.2 The National Policy Context

The Estonian government is rethinking the higher education system. In 2005 a higher education strategy was outlined for the period 2006–2015 (Estonian Ministry of Education and Science 2005). Internationalisation of higher education, increasing participation, especially in science and technology, increasing the production of doctoral graduates, realigning the higher education system to become more compatible with the European systems, reform of the quality assurance system and more attention to the social dimension of higher education were the main objectives in that strategy (Estonian Ministry of Education and Science 2010). Currently (2011), a higher education reform plan is drafted. The aim of that plan is to increase the fairness of the higher education system for students, enhance the efficiency of the system and increase the autonomy and accountability of higher education institutions (see also Estonian Ministry of Education and Science 2011).

The international economic hardship and the national demographic situation (an upcoming drop in secondary school leavers) are major challenges for realising the plans.

Progress is made regarding the legislative frameworks facilitating reform, but the limits on state funding have kept the system from making progress in terms of participation and doctorate production (student support systems and scholarship programs could not be implemented). Important changes have started regarding the institutional landscape and further changes are planned. A few vocational educational institutions have been transformed into institutions of professional higher education and more institutions are to follow. This transformation of the non-university sector is seen as an important way to upgrade participation and quality of education in that sector. There are plans to change the strong central steering of higher education institutions. According to the draft reform plans, the concept of state commissioned education will be abolished and institutions will have to decide how many students places will be created, based on their profile and function. Profile and mission oriented performance contracts between the ministry and individual institutions are envisaged as major instruments for steering the system. As for the internationalisation objectives there are ambivalent results. Estonian students are internationally mobile, Erasmus mobility is well balanced, but there are far more students leaving the country for studying abroad than students coming in.

The dynamic character of higher education in Estonia became apparent in the discussions regarding the reference year. There was a strong feeling among the institutions that the reference year should be as recent as possible, since things changed very rapidly in the Estonian higher education landscape. The fact that there was an intense discussion on staff and how to count staff on short and part time contracts can be related to a particular characteristic of the higher education system: the large number of small institutions.

The resulting U-Map profiles are to a large extent in line with the expectations of the Estonian institutions and the Ministry. The fact that there are five or six larger institutions with significant research involvement and a large number of small highly specialised institutions did not surprise the Estonian audience. The strong regional engagement in most institutions was also not seen as a surprise as Estonia is considered to be one NUTS2 region and most of the larger institutions have branch campuses across the country. Most interesting differences/diversity could be found in the ‘international orientation’ dimension and the dimension ‘knowledge transfer’.

Estonian institutions have started comparing their profiles and questions regarding the various scores have emerged. These questions will be used in a next stage to improve the interpretation of the definitions and the consistency of data provision across the Estonian institutions. It became also clear that some of the small institutions with a ‘small’ profile were already in some way under scrutiny. The doubts regarding their viability were expressed before the profiles were created. This touches upon a crucial point of the use of U-Map. U-Map is an instrument that describes the activity profiles of an institution. If that activity profile is ‘small’ it does not mean that the performance of that institution is low. It may hint at viability issues but that needs to be firmly embedded in the institutional and national context. U-Map may give a signal, but most certainly not the only signal.

7 Portugal

The Portuguese higher education (around 400,000 students in 2010) comprises a large number of institutions: 93 private (thereof 10 universities) with around 88,000 students in 2010 and 40 public institutions (thereof 14 universities) with a total of more than 300,000 students (slightly less than 200,000 in universities).

7.1 U-Map Implementation

In February 2011 meetings were held with the Portuguese Ministry of Education and representatives from the three associations of higher education institutions. During these meetings the procedures regarding pre-filling and the data provision by the institutions were discussed. The databases of the Ministry and the associations were used to pre-fill parts of the institutional questionnaires (mainly student and graduate information). Seventy-five institutions were invited by the associations of higher education institutions to participate in the project. 63 institutions responded and 55 provided a full dataset.

In April a technical workshop was organised in Lisbon, where 53 representatives of institutions, rector conferences and ministry discussed the details of the online questionnaire. Based on the discussions, a country specific FAQ section was created on the website, and a country specific page was created on the website. Main issues at that workshop were the position of the associated research institutes at universities, the exclusion of short post-secondary degrees for the project, the definition of publications and the breakdown of government funding by teaching and research.

The process of data-collection continued into September 2011, whereas the verification process was concluded early October.

7.2 The National Policy Context

Portugal is ‘modernising’ its higher education system. Following up on the 2006 OECD review of tertiary education (OECD 2007), Portuguese government has implemented a number of reforms. The higher education institutions need to become more responsive to the needs of society and the economy. More autonomy and more accountability are keywords in this context. The issues that have been addressed since 2006 are new legislation, system diversity, quality assurance, loans schemes to facilitate more student participation and international partnerships in teaching and research (Ministry of Science, Technology and Higher Education, Portugal 2011).

The new legislative framework is intended to facilitate an outward focus of institutions. Public universities are allowed to acquire an independent legal status. To what extent this will help to create more effectively university industry links is to be seen. Modernisation of the higher education system is interpreted also as strengthening and expanding the polytechnic sector. The main rationale is the stronger orientation of the polytechnic degrees towards the profession. This and the regional dispersion of polytechnics and other non-university HEIs are supposed to enhance knowledge transfer, regional engagement and social inclusion.

U-Map has some clear benefits to offer in the Portuguese policy context. The focus on system diversity is most interesting. Diversity is seen as a strengthening of the binary system. U-Map may help in bringing more nuances to this discussion. Although the responsiveness to the (local/regional) economy and society is a key element in the Portuguese modernisation agenda, there are other dimensions in the reform agenda that go beyond the traditional binary divide. The international orientation and regional engagement are issues that may cut across the binary divide (as becomes apparent in other countries). The divide between both ‘types’ of institutions regarding their activities in these areas may not be as sharp as suggested: universities may be active in ‘professional’ fields, polytechnics (and other non-university institutions) may be more internationally active than some universities etc.

U-Map has a clear potential relevance in the Portuguese policy discussions as there were lively discussions regarding degrees (especially the short technical oriented degrees), discussions regarding the role of professional publications and the emergence and character of research in the polytechnics.

Preliminary results show, not surprisingly, that there is diversity on the relevant issues (regional engagement, international orientation, research orientation), but is also clear that this diversity does not follow the binary divide in all dimensions. The closest ‘fit’ is in the dimensions ‘Research involvement’ and ‘Teaching and Learning Profile’. University have in general a higher involvement in research and have a more doctorate/master level and general formative focus. In other dimensions, like international orientation’ and ‘student profile’ it is quite difficult to find traces of the binary divide. Although it is difficult to predict the impact of the outcomes (as the government has no official part in the project), it is clear that U-Map points out that the current discussions regarding institutional diversity and responsiveness should be broadened beyond the binary divide.

8 The Netherlands

The Dutch higher education system is a binary system with the UAS (40 public ones, 415,000 students) on the one side of the divide and the 14 public universities (165,000 students) on the other side. In addition, there is a private sector where CPD courses and programmes as well as recognized degree programmes are offered. There are more than 60 private institutions, with a wide diversity in size and scope.

8.1 U-Map Implementation

The Dutch U-Map project was initiated by the Dutch Ministry of Education and Science. It was interested in having the U-Map profiles for all Dutch higher education institutions. Early 2010 preliminary meetings with the associations of UAS and universities (HBO-Raad and VSNU) were held, discussing the protocol to follow. All higher education institutions (119: 40 public UAS, 14 public universities) were invited by the Dutch Ministry of Education and Science to participate. Sixty-three responded positively and 46 (of which 13 public universities) provided a full data set. Most of the small specialized teacher training colleges decided not to participate; workload and lack of relevance were reported as the main reasons for that decision. As for the private sector, the umbrella organization of the private institutions informed its members of the project and invited them to participate as well. In total 10 responded and 4 provided a full dataset.

The process of implementation in the Netherlands is still ongoing. Prior to the technical workshop, in which the definitions of the indicators and data-elements were discussed with the participating institutions, the HBO-Raad had started a discussion on the indicators in the research dimension. They argued that the indicators selected did not represent the (practice oriented) research activities that are emerging in the universities of applied sciences. Because of this critique, the U-Map profile was changed and the indicator ‘peer reviewed publications’ was split up in three: ‘peer reviewed academic publications’, professional publications’, and ‘other peer reviewed research products’. This eased some of the pain, but there was still some reluctance to participate. Therefore a discussion with experts regarding the issue was organized, as well as a second workshop in which participating institutions could discuss about alternative indicators in the dimension research and international orientation. These discussions lead to a better understanding of the definitions, but did not lead to another change of the U-Map profile. In the university sector a different discussion emerged: how to deal with teaching hospitals? This issue is a problematic issue for already quite some time. The discussion that U-Map started was welcomed by the relevant universities, but the teaching hospitals did not want to participate due to a problem in the alignment of the timing of the discussions. Eventually the universities decided how to take teaching hospitals into account, but it slowed down the process of data collection and verification significantly. Definition issues dominated the discussions during verification, but political sensitivities had an even more significant impact on the process. Both individual institutions and the associations were during the process reluctant to share the resulting U-Map profiles. The protocol was adjusted so that formal approval of the Board of the institution was needed for publication of the profile on the Dutch U-Map website. This additional ‘hurdle’ was built in on request of the associations and it underlined the political sensitivity of the issue of institutional profiles in the Dutch policy context.

After one and a half year, only 1 out of 4 institutions did allow publication on the Dutch password protected website. Although this may increase in the near future due to an encouragement letter by the associations, it is clear that the policy context has put the U-Map project in a delicate position, forcing the researchers to walk on eggs.

As for the results, they are not very surprising at first glance. The binary divide is clearly visible in the dimensions ‘research involvement’ and ‘teaching and learning profile’. In the other dimensions however, the divide is not that clear and obvious. There are a number of UAS that are equally or even more internationally oriented than the universities. Professional orientation and the indicators in ‘regional engagement’ show also diversity along different, ‘non-binary’ lines.

8.2 The National Policy Context

The Dutch government has presented its latest plans for higher education in the Summer of 2011. In the Strategic Agenda (Ministerie van Onderwijs and Cultuur en Wetenschap 2011), the focus is on raising the quality and efficiency of the higher education system. Enhancing both horizontal and vertical diversity of the higher education system is key to these policy plans. Higher education institutions need to develop their profiles both in terms of activities (horizontal diversity) and their performances (vertical diversity). The government wants to link the performance profile to a very limited part of funding through bilateral performance contracts comprising performance profile related indicators. The activity profiles are seen by the government as a tool that may help higher education institutions in finding and developing their profiles. The government sees U-Map as a promising way to develop institutional activity profiles and has supported the testing and implementation of U-Map in the Dutch higher education system.

The associations and the HEIs have taken a very cautious position. This has most likely to do with the prominent role institutional profiles play in the aforementioned Strategic Agenda and a previous policy document (Commissie Toekomstbestendig hoger onderwijs stelsel 2010). In the latter report there was a strong call for more institutional diversity. U-Map was mentioned as a promising way to illustrate diversity. This report has been well accepted by the higher education institutions (as it also called for substantially more resources for the higher education sector). The report was an important input for the 2011 policy document (Strategic Agenda) that drafts the outline for the higher education system in the years to come. Institutional profiles play an important part in this new policy. Institutions have to decide on their institution profile (mission) and negotiations will be held to draft performance based contracts between individual institutions and the government. These contractual agreements will have potential financial consequences. In this setting, the development of institutional activity profiles (U-Map) is seen by many as confusing. Not all institutions are easily inclined to fully co-operate in drafting and sharing these profiles in current uncertain times. The ministry argues that the U-Map profiles can be used by the institution internally as a tool in the institutional quest for its profile. Even though the ministry stresses that the U-Map profile will not be used in any way related to financial or structural decisions, institutions remain reluctant.

9 Results

In Figs. 45.3 and 45.4 a random selection of institutional profiles from the Estonian, the Portuguese and the Dutch case are presented. Since all three countries have some kind of a binary divide, the profiles are grouped into two groups: the universities of applied sciences (or polytechnics) and the research universities. Within the limited number of 15 institutions per group, the U-Map activity profiles show a remarkable diversity. Least diversity can be seen regarding the teaching and learning dimension and the research dimension. Many indicators in these dimensions reflect the characteristics that are used to demarcate the binary divide. UAS and universities differ most on these indicators, although there are some institutions that would fit in other group quite well.

Fig. 45.3
figure 3_45

Activity profiles of Universities of Applied Sciences in Estonia, Portugal and the Netherlands

Fig. 45.4
figure 4_45

Activity profiles of research universities in Estonia, Portugal and the Netherlands universities

Diversity is much stronger with respect to the other dimensions. This diversity is not only between the two groups, but cuts across the divide. If the focus of analysis shift to another dimension, the grouping of the institution change as well. During the final workshops in Estonia and Portugal, the participants were invited to group the activity profiles into an Estonian/Portuguese classification of higher education institutions. All teams came up with different groupings, using different primary and secondary foci. This hands-on experiment underlined the result that U-Map does not provide one classification of higher education institutions but allows the user to generate a personalised classification of higher education institutions.

10 Discussion

Enhancing transparency in the European Higher Education Area has been a key objective of the Bologna process. Knowing what the vast number of higher education institutions do is a crucial element in that process. Structural reforms to align certain features of higher education institutions (like degree structures, credit systems, quality assurance) have been a powerful way to push that process forward. U-Map is an instrument designed to add new dimensions to the discussions regarding transparency and diversity and transcend the traditional dichotomies that tend to dominate and in many instances stifle the discussions on institutional diversity.

U-Map is not perfect. The implementation of a new transparency instrument is a complex and labour intensive process. The way the process is set-up, the relations between and roles of individual institutions, associations and the policy makers determine to a large extent the speed and success of the implementation.

U-Map is designed as a European transparency instrument. National higher education issues may be addressed in the analyses of the profiles, but it is clear that U-Map as a European tool is not fully aligned with national institutional needs. Its relevance for national policy discussions may therefore vary between countries.

The implementation of U-Map so far has shown that a national approach is the most promising approach. The availability of national databases and frameworks allow for a more in-depth analysis of those existing data structures and for a better alignment of definitions and data both nationally and internationally. It also provides a better base for verification of the data. U-Map has proven to be a viable transparency instrument, that will improve the more institutions and national systems participate. Further development of the instrument, procedures and protocols and the set of indicators will further improve the relevance of the European classification: U-Map.

U-Map is more inclusive than many other transparency instruments: it comprises more dimensions, is open to more ‘types’ of higher education institutions and cuts across national borders. This broadened perspective and the clear orientation towards the user to allow for personalised ‘classifications’ Despite its shortcomings, the U-Map instrument may broaden our understanding of what has happened in the European Higher Education Area since Bologna.