Introduction

A quick review of national and supranational policy documents reveals how, over the last decade, lifelong learning has evolved as the key principle for a comprehensive national education and learning strategy from cradle to grave. Most recently, the United Nations 2030 Agenda for Sustainable Development has identified access to “inclusive and equitable quality education and … lifelong learning opportunities for all” as the fourth of its 17 Sustainable Development Goals (SDGs).Footnote 1 It has also been noted that lifelong learning is critical for the fulfilment of other SDGs like those devoted to health (SDG 3), gender equality (SDG 5), sustainable consumption and production (SDG 12), economic growth and decent work (SDG 8), as well as climate change (SDG 13) (Hinzen and Schmitt 2016; UIL 2016).

Based on the experiences of monitoring an earlier agenda’s Education for All (EFA) Goals 3 and 4,Footnote 2 Aaron Benavot and Ashley Stepanek Lockhart (2016) note that new practices and frameworks are needed for monitoring the parameters and outcomes of lifelong learning, particularly the adult learning and education section in the SDG framework. They observe that there are clear gaps between the level of ambition articulated in SDG 4 and the ability and commitments of national and international bodies to develop an appropriate system able to monitor adult learning and education. The same message is echoed by Irina Bokova, former Director-General of UNESCO:

Because education and learning often happen in undocumented non-formal or informal spaces, [adult learning and education] can be difficult to assess with accuracy. We must continue raising the visibility of learning in all forms and strive for closer monitoring and more accurate data to inform decision-making (UIL 2016, p. 9).

It is only fairly recently that there have been concerted efforts to collect comparative data on adult learning. Information on outcomes of primary and secondary education has been available since the 1960s, and major cross-country comparisons like the International Association for the Evaluation of Educational Achievement (IEEA)’s Trends in International Mathematics and Science Study (TIMMS), first conducted in 1995, and more recently the Organisation for Economic Co-operation and Development (OECD)’s Programme for International Student Assessment (PISA), introduced in 2000, have allowed for benchmarking school performance. However, it was not until the launch of the International Adult Literacy Survey (IALS)Footnote 3 in 1997 that adult learning started to be addressed.

The interest in data on adult learning reflects a growing recognition by policymakers of the necessity to invest in adult learning to achieve economic efficiency and address equity deficiencies. Regardless of their state of development, almost all countries now see knowledge and creativity as the fundamental basis of competitive advantage. There has been an awareness of the relationship between increased years of schooling and economic growth since the OECD’s 1961 ground-breaking report Economic Growth and Investment in Education. (OECD 1961). However, with the exception of a brief interest in recurrent educationFootnote 4 in the 1970s, the role of adult learning for productivity, innovation and individual employability did not begin to come into focus at the OECD until the early 1990s, when there was a growing understanding that learning was important to the economy. The 1996 OECD report on Lifelong learning for all (OECD 1996) and the European Union (EU)’s Memorandum on Lifelong Learning in 2000 (EC 2000) finally moved adult learning and education from the periphery to the centre of the policy discourse. In a concentrated effort on establishing empirical and conceptual linkages between competencies, contexts, policy levers and outcomes, the OECD launched the Programme for the International Assessment of Adult Competencies (PIAAC). The programme’s goal is to:

(a) identify and measure differences between individuals and countries in competencies believed to underline both personal and societal success; (b) assess the impact of these competencies on social and economic outcomes at individual and aggregated levels; (c) gauge the performance of education and training systems in generating required competencies; and (d) help to clarify the policy levers that could contribute to enhancing competencies. (OECD 2005, p. 4)

Despite impressive improvements in addressing adult learning and education in the more widely international PIAAC (its survey is conducted in over 40 countries) as well as in the somewhat smaller-scaled European Adult Education Survey (AES) (Eurostat 2012), it is doubtful how far these surveys are able to respond to the new challenges raised by the aspiration of the United Nations 2030 Agenda to link lifelong learning to the fulfilment of the 17 SDGs. Furthermore, both PIAAC and AES are primarily aimed at exploring the link between lifelong learning and the economy. However, recent studies on the increasing level of inequalities and their detrimental effects on social fabric and economic development suggest that it is important to look more broadly at the outcomes of lifelong learning (i.e. beyond economic gains) and more closely at initiatives like the Human Development Index (HDI).Footnote 5 It is in this context that this article critically examines (a) key international surveys on adult learning and education as well as (b) attempts that have been made to more broadly assess the state of lifelong learning, like the Canadian Composite Learning Index (CLI) on lifelong learning and the European Lifelong Learning Index (ELLI).

Key international surveys on adult learning and education

I will begin the discussion by examining how adult learning and education (ALE) is being conceptualised in the policy discourse and then move to a review of the extent to which this understanding is indeed informing the dominant surveys.

Underlying conceptualisation of adult learning and education

The enormity of the shift from a preoccupation with formal adult education and training to the all-encompassing principle of adult learning, comparable in scale to the Copernican revolution,Footnote 6 has created major challenges for monitoring adult learning and education. Thus, today’s dominant perspective embraces the notion that learning is neither necessarily intentional and structured, nor that it exclusively takes place in formal or non-formal institutional settings. Most national and intergovernmental policy documents currently refer to three basic categories of learning; formal learning, non-formal learning and informal learning. In a nutshell, formal learning refers to intentional and systematic learning in a (state-run) institution which is dedicated to education and provides certificates; non-formal learning generally refers to intentional and systematic learning outside of a state-run institution; and informal learning generally refers to non-intentional and non-structured learning in a life context such as the workplace, the family, etc.

It is important to note that the triad, as presented by the supranational organisations (such as UNESCO, OECD, the World Bank, etc.), is really about the context in which learning takes place, and does not say anything about learning as such. In the scholarly literature, grave concerns are raised about the soundness and utility of this triad of learning. A common argument among those who contest the triad is that recent empirical and theoretical work questions the boundaries between formal, non-formal and informal learning and that the distinctions made among these categories are artificial (Callanan et al. 2011; Hodkinson 2011). So for example, a review of studies addressing the differences between informal learning and formal learning found that different studies used a variety of different criteria, with little overlap, to classify the learning activities (Colley et al. 2003). In fact, according to Helen Colley and colleagues, there did not seem to exist a set of criteria that were universally used by all writers. As a result, they noted that a learning activity that was classified as formal by some scholars would be identified as informal in other studies. According to Phil Hodkinson (2011), the academic confusion around distinguishing between formal and informal learning stems from profound disagreements about the nature of learning itself. It is therefore easy to agree with Günter Hefler (2012), who notes that the triad seems to have been accepted by the supranational organisations as well as national policymakers without any serious reflections of its deeper meaning and implications.

Another major criticism against the triad comes from scholars who in principle accept that learning can be classified according to a triad, but are sceptical about the way this classification is applied and what seems to be ignored. Their main concern is that not enough attention is given to the link between the outcomes of a learning activity and the institutional context in which the activity occurred. Hypothetically, it is possible to imagine a situation where two people have more or less the same competency profile, but where one has acquired the competencies through studies at an elite university while the other has built them up through various non-formal and informal learning activities.

The issue is whether the acquired competencies will have the same value specifically in the marketplace (i.e. in terms of employability) or, invoking Jacques Delors and his seminal report Learning: The treasure within (Delors et al. 1996), more broadly in providing “maps of a complex world in constant turmoil and the compass that will enable people to find their way in it” (ibid., p. 85). Is it the case that the elite university is particularly important for a person’s success on the job market while non-formal institutions like the Nordic folk high schools and study associationsFootnote 7 have important roles to play in providing a citizen with the compass Delors refers to? Embedded in what has been labelled new institutionalism,Footnote 8 this reasoning regards formal education as being a social institution established in wider society, where it has taken on the role of social selection and structuring of opportunity configurations (see e.g. Hefler 2012).

It is interesting to note that the issue Hefler raises can in fact be traced back to some of the debates held in the 1970s around non-formal education and development. Critics of the promotion of non-formal education as the solution to inequalities argued that while students in the non-formal system might acquire knowledge, skills and competencies, they might still not gain the wider benefits that the institutionalised education system delivers for its graduates. At the time, John C. Bock reasoned:

If schools are seen as serving an important mobility management function, strictly controlling access to elite status through the application of certification rules, then non-formal education is viewed as potentially even more inhibiting of the mobility prospects of lower status groups. For, by not providing either the accepted and socially valued certification or the non-cognitive attributes necessary for “promobility”, non-formal education locks workers into the lower segment of the occupational structure (Bock 1976, p. 349, cited in Hefler 2012, p. 42).

Bock’s critique refers to what Pierre Bourdieu (1984) labels symbolic capital and John Meyer (1977) calls the social charter of an institution. This speaks to the potential outcomes that are associated not only with acquired human capital, but also with the symbolic capital that comes with being connected to a specific institution; a perceived asset that needs to be taken into account when discussing various outcomes of formal, non-formal and informal learning and the extent to which the outcomes of the different forms of learning are interchangeable. However, the first question I would like to address here is to what extent the key surveys on participation in adult learning and education indeed provide detailed information on the various aspects of the triad.

Correspondence between conceptualisation and survey data

As mentioned in the introduction of this article, both the OECD and the EU have launched major comparative survey programmes to collect data on participation in adult learning and education within a perspective of lifelong learning. This section considers the strengths and weaknesses of (1) the OECD’s Programme for the International Assessment of Adult Competencies (PIAAC), and (2) The EU’s Adult Education Survey (AES).

OECD-PIAAC

The OECD’s collection of information on adult learning and education is an integral part of the Programme for the International Assessment of Adult Competencies (PIAAC). This is the most comprehensive international survey of adult skills ever undertaken and has come to play a crucial role in countries’ benchmarking of adult learning and education. It is worth noting that the PIAAC survey has meanwhile been expanded beyond OECD member countries.Footnote 9

The PIAAC approach builds closely on the International Adult Literacy Survey (IALS), conducted by the OECD and Human Resources Development Canada (HRDC) between 1994 and 1998, and the Adult Literacy and Lifeskills Survey (ALL), which was almost identical to the IALS, and conducted by a coordinated group of different bodies in a small number of countries between 2003 and 2008.Footnote 10 IALS was the first-ever, large-scale, international comparative assessment designed to identify and measure a range of adult skills in ways that would allow comparing literacy across cultures and across languages. Statistics Canada served as the international coordinating body for both IALS and ALL; the Educational Testing Service in the United States developed the literacy tests, while the OECD provided support for dissemination of IALS and ALL results on the international level and assisted in country recruitment for participation in the studies. IALS and ALL adopted their methodology and scales from the 1992 US National Adult Literacy Survey (NALS).Footnote 11 IALS provided a rich set of information on the literacy skills of adults (aged 16–65) in 22 countries and regions and was used as a blueprint for the development of PIAAC. Trend items from IALS were included in ALL as well as PIAAC, allowing data from IALS to be linked to trend data from participating countries in ALL and PIAAC.

While the document introducing PIAAC’s Conceptual framework (OECD 2009) does not attempt to address the classification of adult learning, it does touch on the issue briefly and claims that participation in lifelong learning is a “crucial” area of concern for governments of OECD member countries. This “cruciality” is informed by a narrow economic perspective:

Formal education, formal training, and informal training all contribute to the stock of human capital, and countries will display different profiles in how the human capital stock is built up. PIAAC will provide a snapshot of human capital investments by the incidence and intensity of training during the previous 12-month period (OECD 2009, p. 5).

The document further stresses that to assist policy development, it will be important to collect “information on how much of this training [i.e. adult learning and education activities] is taking place for work-related reasons” (ibid., p. 5).

While PIAAC never developed a specific classification system for adult learning and education, it is possible to gauge a general understanding of the survey designers’ approach from the instrument as such and the classification options enabled by the data. Learning events are divided into two broad categories: formal learning and other organised learning (i.e. non-formal learning). This could lead one to believe that the OECD is in the camp of those critical of the triad mentioned earlier, but the fact is that OECD policy documents strongly stress the importance of also taking informal learning into account (see e.g. OECD 1996). It is therefore striking that the PIAAC background questionnaire (OECD 2010) makes no attempt to assess informal learning. The questions on adult learning and education reflect a fairly narrow perspective on which learning activities contribute to the generation of human capital. Consequently, there is a strong emphasis on formal educational activities undertaken towards a diploma, certificate or degree. The opening question on non-formal learning activities is broadly cast:

We would now like to turn to other organised learning activities you may have participated in during the last 12 months, including both work and non-work activities (ibid., p. 23, item B_R12).

This gives the impression that adult learning and education (ALE) is broadly conceptualised, but a review of the follow-up questions indicates that this is not in fact the case. Moreover, the follow-up questions have a strong job-related focus. So for example the initial question on motivation asks: “Were the main reasons for choosing to study for this qualification job related [sic]?” (ibid., p. 21, item B_Q05c, emphasis added). If the answer is yes, the respondent is directed to a request to identify in what respect the main reasons were job-related. Other motives for engaging in adult learning and education are not explored. Furthermore, the remaining questions all focus on the learning activity in relation to work. In the OECD world, non-formal adult learning activities that citizens do not engage in for work-related reasons are of no policy interest. In view of the importance given to non-formal and informal learning in the OECD’s seminal report Lifelong Learning for All (OECD 1996), it is puzzling why greater attention is not paid to a broader exploration of adult learning. This reluctance may well be undermining PIAAC’s policy relevance, especially in terms of the relative efficiency of different policy levers applied to lifelong learning. So, for example, PIAAC in its present form is far from ideal when it comes to monitoring the impact of adult learning and education on the SDGs or to contributing to the understanding of how adult learning and education can help in developing the “compass” that will allow a person to navigate a changing world that Delors et al. (1996) talk about.

EU/Eurostat Adult Education Survey (AES) 2011

The European Commission (EC) has conducted the most elaborate work to date on a classification system of adult learning, with the intention of supporting a coherent European survey on participation in adult education and training. As part of the EC’s strategy for the development of skills in EU member countries, Eurostat (the official statistical office of the EU) was requested to launch and coordinate such a survey. The first wave of the Adult Education Survey (AES), called AES 2007, was launched as a pilot project and conducted during the period 2005–2008. Thereafter the survey has been conducted at five-year intervals, with wave number two occurring in 2011–2012, and wave number three in 2016–2017. Today there is a common framework for the AES which is directly applicable by all EU member countries and being enforced by strict EU implementing regulations.Footnote 12

The Eurostat classification system is organised around learning activities, defined as “any activities of an individual organised with the intention to improve his/her knowledge, skills and competence” (Eurostat 2012, p. 20, emphasis added). The typology uses single learning activities as basic building blocks of a classification system that can capture and describe all learning activities. The framework creates a flow chart that classifies activities as formal, non-formal and informal learning respectively, using three key criteria and thus recognising the full spectrum of lifelong learning. The first criterion is “intention to learn”. If there is no intention to learn, the activity is not a learning activity. Consequently, this framework excludes all incidental learning.

Learning activities deemed to be intentional then meet the second filter: institutionalisation. Learning activities are considered institutionalised when there is an organisation providing structured arrangements including a student–teacher relationship especially designed for education and learning. Institutionalised learning activities happen when there is a providing agency/body responsible for: determining the teaching/learning method, scheduling of the learning activity, admission requirements, and location of the learning/teaching facility. Informal learning activities are not institutionalised (ibid., p. 23).

The institutionalised learning activities are then filtered through a third criterion, namely whether or not they are included in the National Qualifications Framework (NQF) of the respective respondent’s country. The NQF could take the form of a regulatory document, which stipulates the qualifications and their relative positions in a hierarchy of learning achievements as well as the awarding bodies that provide or deliver these qualifications. Activities that fulfil this third criterion are classified as formal learning activities, while those not included in the NQF are categorised as non-formal learning. While the Eurostat typology classifies the learning activity according to the official triad, it does not provide sufficient details on the actual nature of the learning event and where it takes place. Thus, in many cases it becomes difficult to assess the benefits of specific informal or non-formal learning activities. Furthermore, while the focus on job-related activities is less pronounced in the AES scheme than in the PIAAC approach, there still remains a certain level of bias towards job-related activities.

Summary

The above brief review of PIAAC and AES points to the fundamental shortcomings in these surveys and the way the data have been classified, particularly in the PIAAC approach. As noted, there is a mismatch between the heavy investment in developing instruments to measure competencies on the one hand, and the lack of focus on the role of different parts of the “adult learning system” in generating and maintaining these competencies on the other. Furthermore, skills are almost exclusively discussed in the context of economic productivity and employability, despite the trend that social and cultural practices are shifting in ways that require higher levels of skills for full participation in democratic processes, cultural life and increasingly complex everyday contexts. The present OECD approach to data gathering on adult and lifelong learning, with its deep roots in the skills agenda, creates a “reality” of adult learning where the broad humanistic traditions of adult education are non-existent. Consequently, it becomes impossible to question the wisdom of present strategies through an evidence-based policy strategy that is being driven by this reality. As Kim Clark “astutely observes, while numbers seem to be a way to avoid ‘the contaminating subjectivity of opinion’” (Milana et al. 2017, p. 341, quoting Clark 1998, p. 185, emphasis added), decisions about what should be counted and how it should be classified are subjective judgement decisions. Michel Foucault’s governmentality thesisFootnote 13 provides a fruitful approach to understanding how the process of collecting and classifying official statistics transforms ideology into discourse which then comes to provide justification for recommending a specific policy agenda (see Naughton 2004, p. 47).

Attempts that have been made to more broadly assess the state of lifelong learning

There have been two serious attempts to develop a comprehensive composite index of lifelong learning that fully embraces the expansive understanding of lifelong learning that is being promoted by the supranational organisations. The first, the Canadian Composite Learning Index (CLI), was launched by the Canadian Council on Learning (CCL) in 2006 (CCL 2006). The intention was to find a way to portray the state of learning across Canada and to monitor its development over time. The prime interest was to create a composite score, like the gross domestic product (GDP) used for measuring economic value, which would capture the outcome of everything that has been learned through participation in formal and non-formal education and training as well as informal learning, including day-to-day life experiences. “A key purpose” of the CLI was “to monitor and report on the progress of learning in Canada, and to draw attention to lifelong learning in a way that is accessible and provocative” (CCL 2006, p. 17). More specifically, the purposes of the Index were:

  • To inform the public about the state of learning in Canada and local communities.

  • To stimulate public discussions about what can be done to continue to improve learning.

  • To provide policymakers with information that can be used to inform the development of policies on learning.

Inspired by and closely building on the Canadian Index, the Bertelsmann Foundation launched the European Lifelong Learning Index (ELLI) in 2010 (Hoskins et al. 2010). The purpose of ELLI was to allow comparisons of the state of lifelong learning between countries, and, if possible, between regions.

Unfortunately, the CLI was dropped when the Canadian Council on Learning was closed down due to a discontinuation of funding in 2012, and ELLI never came to be adopted by the EU. However, as these two indexes provide a state of the art when it comes to assessing the overall status of lifelong learning, it is important to take a closer look at them, since much can be learned from their strengths and weaknesses. In this article, I give special attention to ELLI, as it has benefited from the availability of a richer set of indicators, mainly provided by Eurostat, than what was available for constructing the CLI.

Since ELLI closely builds on the CLI, it is not surprising that the architecture, key components and statistical procedures of these two instruments are, if not identical, at least very similar. In both cases, the Delors report’s four pillars of learning (Learning to know, Learning to do, Learning to be and Learning to live together; Delors et al. 1996) are used as an organising framework to capture learning outcomes and functions. Delors’ urging for a new paradigm of learning was anchored in an understanding that each individual needs to be equipped to take advantage of learning opportunities throughout her/his entire life so as to acquire the knowledge, skills and attitudes that would allow the person to navigate a complex and changing world. To make this possible, Delors et al. argued that throughout a person’s life, education should be organised around four types of learning:

learning to know, that is acquiring the instruments of understanding; learning to do, so as to be able to act creatively on one’s environment; learning to live together, so as to participate and co-operate with other people in all human activities; and learning to be, an essential progression which proceeds from the previous three (Delors et al. 1996, p. 86; emphases in original).

Delors et al. note that formal education tends to focus primarily on learning to know, and to a lesser extent on learning to do, while the two others are largely left to chance.

For each of Delors’ four pillars, a set of indicators and measures were chosen to capture the specific dimension of learning in both of the indexes. Figure 1 shows the dimensions and measures that were included in the CLI.

Fig. 1
figure 1

The Composite Learning Index. Source: CCL (2010, p. 5)

Full size image

The Composite Learning Index is based on elaborate statistical procedures that weight the four pillars in a manner that is consistent with the underlying understanding of how they interact to produce positive socio-economic and economic outcomes. Not all of the CLI and ELLI indicators have the same effect on the overall scores. Thus, each learning indicator has a different degree of importance for a country’s/community’s overall social and economic well-being, and the indexes are designed to reflect this. The index as such statistically determines this level of importance, rather than arbitrarily assigning values based on perceptions of importance.

The conceptual model driving the CLI and ELLI acknowledges that the extent of lifelong learning is being affected by factors like access to childhood education, schools and colleges, libraries, on-the-job training, etc., which are referred to as context of learning. This kind of information is directly incorporated into the index. So for example, each of the pillars (quarters of the circle) in the CLI contains an indicator and measures pertaining to access (see Figure 1).

The criteria for choosing indicators for CLI and ELLI were more or less the same. In the Canadian case, the choice of indicators was based on four considerations. First, they were to measure an aspect of learning related to the conditions, the outcomes and the context of learning. Second, the combined set of indicators was to cover Delors’ four pillars of learning. Third, the indicators were to be sensitive to learning across all age groups and address learning from a life-wide perspective.Footnote 14 Finally, the selection was to be determined by availability of high-quality recurrent data that would allow comparisons, in the case of CLI between communities, and in the case of ELLI between countries. A similar rationale, but from an EU perspective, informed the construction of ELLI.

In sum, while both CLI and ELLI are important contributions to the assessment of the status of lifelong learning, there is a danger to be blinded by their statistical elegance. In fact, the complexity of their statistical analyses makes it difficult for the audience to fully understand what a specific score stands for.

Strengths and weaknesses of the composite lifelong learning index approach

There are several reasons why a well-functioning index like CLI or ELLI could potentially be very attractive to policymakers. In contrast to key comparative surveys like PIAAC and the AES, they are truly based on a broad conceptualisation of lifelong learning that corresponds closely to the high ambitions expressed in core policy documents. The convenience of one composite figure that can provide “the big picture” of the status of lifelong learning in a country/region is tremendously attractive and powerful. However, as recognised by those responsible for CLI and ELLI, the big picture may send misleading policy messages and cannot on its own shed light on specific problems. Moreover, its usefulness of course ultimately depends on the soundness of the variables and the way they have been statistically handled.

The CLI as well as ELLI have been the focus of thorough and highly sophisticated technical analyses by the European Commission’s Joint Research Centre (JRC) (see Saisana 2008, 2010). In the case of ELLI, JRC found the index to be sound; that it has no particular shortcomings in terms of its conceptual structure; and that it is built according to a reliable statistical methodology. JRC states in its conclusion that, with some refinements, ELLI can reliably be used to pinpoint shortcomings in policies and suggest possible helpful actions (Saisana 2010). JRC notes that ELLI is useful for developing national benchmarks in lifelong learning and for more in-depth international comparisons of the underlying dimensions of learning. Finally, it is pointed out that in addition to the outcome factors that were included, ELLI can be used to study the link between lifelong learning and measures such as competitiveness or innovation as well as the links between learning, social cohesion and democracy (ibid.). Similarly, JRC’s technical reviews of CLI (multivariate, dominant and sensitivity analyses) show it is internally sound and robust (Saisana 2008).

Thus, taken together, the two reviews (Saisana 2008, 2010) clearly demonstrate the technical soundness of both ELLI and CLI. However, as Michaela Saisana points out, the technical analyses have taken the underlying conceptual framework for granted:

We would argue, though, that a framework mostly reflects the normative assumptions of its developers, and that as such it can be more appropriately the subject of the critique of experts in the field of lifelong learning (Saisana 2010, p. 41).

It is from this perspective that I would now like to discuss some critical issues regarding the indexes and the data they build on.

While using Delors’ (Delors et al. 1996) four pillars of learning as an underlying structure for assessing the state of lifelong learning is a very elegant way of capturing learning, one has to ask: what is really being measured and how well do the indicators used really capture the richness of Delors’ four pillars?

It is noticeable that the JRC’s technical in-depth analyses of CLI and ELLI did not raise serious concerns about the underlying typology of four clearly distinguishable pillars of learning as presented in Delors’ framework. First, the review of CLI does not corroborate the underlying organising framework of four pillars. Instead, the analysis reveals that the 17 indicators are correlated and share 6 common uncorrelated factors which, according to the review report, “do not have an intuitive interpretation” (Saisana 2008, p. 59).

By contrast, using 36 indicators, the four-pillar structure (termed four dimensions by the JRC) was confirmed in ELLI. However, in the case of ELLI, there is a high correlation between the four learning dimensions, “ranging from [a low of] 0.72 (Learning to Know with Learning to Do) to 0.86 (Learning to Be with Learning to Do or with Learning to Live together)” (Saisana 2010, p. 25). In fact, “the four learning dimensions can be summarized by a single latent factor that captures almost 85% of the variance of the four dimensions” (ibid., p. 20). This would suggest that the four learning dimensions are not separable, and that they account to a considerable extent for similar aspects of learning. It is therefore understandable that a country like Denmark, which has the highest overall ELLI score of all the included European countries, also has the highest score on three of the four pillars and is ranked second on the fourth (Saisana 2008, p. 42, Table 16). This is not to deny that in a few cases, the score on one of the pillars, in most cases “Learning to know”, differs noticeably from the scores reported for the other three pillars.

A key question is how well the measures in ELLI capture Delors’ understanding of the four dimensions of learning. A quick glance at the measures presented in Table 1 would indicate that ELLI has done a rather good job in finding variables that intuitively seem relevant to capture the intended essence of the four learning dimensions. So for example the nine measures chosen for “Learning to be” at the end of the table speak to Delors’ notion of the qualities of imagination and creativity through participation in “aesthetic, artistic, sporting, scientific, cultural and social” activities (Delors et al. 1996, p. 95). Turning to the measures covering “Learning to do”, one may conclude that they are narrower than “to be able to act creatively on one’s environment” (ibid., p. 86), as suggested by the Delors report. However, from Delors’ more detailed description of the four pillars, it becomes evident that it is the more narrow work-related understanding that has been captured and is really being promoted in ELLI. While the measures chosen to capture a certain dimension make sense, the problem is that they are highly correlated not only with the dimension they are supposed to cover but also with the other three learning dimensions (see Table 1).

Table 1 Correlations between ELLI indicators and dimensions/pillars
Full size table

As could be expected, Table 1 shows that most measures have a higher correlation with the specific learning dimension (pillar) they were supposed to be part of, than with any of the other three dimensions of learning. However, while this may be the case, Table 1 also shows that in many instances a specific indicator is highly related not only to its “own” dimension, but also to the other three learning dimensions. So for example, learning new things at work has a correlation with “Learning to do” of 0.83, but is almost as highly correlated with the three other learning dimensions (0.72, 0.78 and 0.80). Looking at the correlations in Table 1, one begins to wonder what the underlying factors really are. As an example, using the Internet at work is highly correlated with all four learning dimensions ranging from a high of 0.92 with “Learning to know” to a low of 0.80 with “Learning to live together”. Most likely it is not the actual use of the Internet at work that is behind the high score, but that the measure captures a general factor that is considered important, e.g. the person’s overall status. The same thing is true for many, if not most, of the 36 indicators used in ELLI. Consequently, the ELLI model built a set of indicators and measures which are unique to a specific pillar, and this does not seem to make much sense. The same is of course true for the CLI model in Figure 1.

A big advantage of ELLI is that it was not only built with a focus on economic benefits of learning, but that it looks more broadly at “social benefits of learning, such as income, employability, population health, life satisfaction, voters’ participation and trust in political institutions … The results show a high linear relationship [r=0.913] between lifelong learning conditions and the economic and social well-being in EU Member States” (Saisana 2010, pp. 26–27). While there is a danger of drawing far-reaching conclusions in terms of causality, the strength of ELLI is that it provides a way of widening the often narrow economic perspective that has been driving the focus of measuring outcomes of lifelong learning.

An intriguing finding in ELLI is that the measures of

museums/galleries (ELLI index score 0.83);

learning new things at work (0.84);

opinion that gay men and lesbians should be free to live their own lives as they wish (–0.84);

personal use of Internet (0.85);

participation in sports (0.86);

participation in lifelong learning and training (0.86);

Internet access in households (0.89);

enterprises providing continuing vocational training (CVT) courses (0.9);

trust in other people (0.9); and

using Internet at work (0.91),

most of which belong to the (learning to) BE dimension, showed the highest correlation to the ELLI index scores, 0.83 or higher. Interestingly, PISA scores (in reading, mathematics and science) and other indicators in the “Learning to know” dimension seem less influential, with scores of 0.74 or lower. This may be due to the nature of the measures used to capture the “Learning to know” dimension, but might also suggest that non-formal and informal learning are very important for the status of lifelong learning.

In sum, while a composite index has many attractive features, closer examination of CLI and ELLI reveals serious problems of using this approach. Not only must one question the underlying framework based on Delors’ four pillars of learning, but also its practical use for directly assessing the impact of the various aspects of lifelong learning and education.

Concluding comment

As discussed above, recent policy initiatives by individual countries, and by supranational organisations such as the OECD and the United Nations/UNESCO, all come with a growing demand for data on participation in adult learning and education which far surpasses the present capacity to produce the information needed for any form of evidence-based reform strategy. A core issue in moving forward is how to capture the richness of adult learning and education in ways that are theoretically sound, policy-relevant and allow a full recognition of the many desired outcomes of adult learning and education in a broad way.

While there are several examples of well-functioning composite indexes, such as, for example, the Human Development Index (UNDP 2016), the benefits of using composite indexes to portray the status of lifelong learning, or its outcomes, are questionable. The two serious attempts that have been made to construct a composite index that would capture the overall status of lifelong learning are highly problematic. Not only is the underlying framework, based on Delors’ (Delors et al. 1996) four pillars of learning, debatable; a composite score also masks many important aspects of adult learning and education. The focus is on the total score and there is a failure to adequately address the composition of the score and how this impacts on the various outcomes. One cannot assume that one form of learning can substitute another form and result in the same or similar outcomes. Consequently, it would seem more fruitful to concentrate on first identifying some key forms of adult learning and education and then closely studying their impact on what has been identified as core outcomes of adult learning and education, e.g. those identified in the United Nations 2030 Agenda for Sustainable Development.

The Agenda for Sustainable Development refers to lifelong learning in four of the ten targets associated with SDG 4, which pertains to “Quality Education”:

  1. 4.5

    By 2030, eliminate gender disparities in education and ensure equal access to all levels of education and vocational training for the vulnerable, including persons with disabilities, indigenous peoples and children in vulnerable situations

  2. 4.6

    By 2030, ensure that all youth and a substantial proportion of adults, both men and women, achieve literacy and numeracy

  3. 4.7

    By 2030, ensure that all learners acquire the knowledge and skills needed to promote sustainable development, including, among others, through education for sustainable development and sustainable lifestyles, human rights, gender equality, promotion of a culture of peace and non-violence, global citizenship and appreciation of cultural diversity and of culture’s contribution to sustainable development

  4. 4.A

    Build and upgrade education facilities that are child, disability and gender sensitive and provide safe, non-violent, inclusive and effective learning environments for all (UN 2016).

While monitoring SDG 4 is a daunting task in the context of the present availability of data, the need is very much bigger than this goal’s targets. Looking at the other 16 SDGs, few people would deny that lifelong learning and education have a fundamental role to play in addressing goals like Good Health and Well-being (SDG 3), Clean Water and Sanitation (SDG 6), Gender Equality (SDG 5), Decent Work and Economic Growth (SDG 8), Reduced Inequalities (SDG 10), Responsible Consumption and Production (SDG 12), Climate Action (SDG 13), to mention the obvious ones. The 3rd Global Report on Adult Learning and Education (UIL 2016) provides a thorough review of the economic and wider benefits of adult learning and education in relation to health, economic prosperity, social cohesion and democratic traditions. Some of the findings demonstrate how a specific form of adult learning and education can result in positive outcomes. This speaks to the importance of having data on participation which are detailed enough to capture what form of learning activity a person has been involved in, something that is partly missing today. However, it would be possible without overly extensive work to tweak surveys like PIAAC and AES to allow for richer information on the nature of a specific adult learning and education activity a respondent has engaged in.

The United Nations Sustainability Agenda and the OECD’s Skills StrategyFootnote 15 provide different perspectives on outcomes of adult learning and education. PIAAC, which is part of the OECD’s Skills Agenda, has its roots in a rather simplistic human capital framework where individual skills are the primary outcome. As pointed out by Steven Klees (2016), a core assumption in this strategy is that skills are the answer to all major problems facing the global community. While no one can deny that skills are important for the economy, what tends to be missed, according to Klees, is that the human capital logic does not address fundamental issues like how can good jobs be created which require the skills being promoted, what is really driving inequality, how to address the democratic deficit, etc. What tends to be left out is how people can be in command of their own life and, through individual and/or collective action, influence the structures that govern their opportunities to live a full life.

Although less explicit than the OECD Skills Strategy, the United Nations Sustainability Agenda can also be linked to a specific development perspective, namely the United Nations human development approach, anchored in Amartya Sen’s capability framework (Sen 1992, 1999).Footnote 16 The 2016 Human Development Report argues that the “conceptual foundation of the 2030 Agenda” can be “strengthened by the analytical elements of the human development approach” (UNDP 2016, p. 45). This approach sees human development as acquiring the capabilities that will bring about expanding choices and opportunities. It is premised on expanded human rights and peoples’ active participation in the processes that influence and shape their lives; the “ultimate objective [of human development] is to enlarge human freedoms” (ibid., p. 25). The capability approach helps us draw attention to the fact that dispositions and preferences are not independent of economic and social conditions (Nussbaum 2002). Sen (1999) discerns a dynamic process whereby the development of an individual’s capabilities under certain circumstances can impact structural conditions through individual and/or collective action.

A combination of Sen’s (1999) framework and the United Nations Development Programme (UNDP)’s human development approach could be a very fruitful starting point for exploring outcomes of adult learning and education, since Sen’s framework adds several crucial elements to the human capital approach and seems more attuned to the broader aspirations of adult learning and education as expressed both by experts in the field and by the 2030 Sustainability Agenda. Thus, adult learning and education can be instrumental in fostering capabilities, but capabilities can, in turn, play a crucial role in the decision to participate. From this perspective, the issue becomes: Which forms of adult learning and education help build which capabilities and, in Sen’s words, freedom of opportunities?

In conclusion, the core message from the discussion above is that concerted efforts are needed to arrive at a better understanding of how specific forms of adult learning and education impact on various dimensions of peoples’ well-being. To be successful, this work will require more detailed data on participation that are theoretically driven and build on knowledge, not only from the field of adult education, but also drawn from other relevant core fields and disciplines. During the past two years, we have once again been reminded that while formal education is most important for developing skills for the economy and the labour market, it is less clear what kind of adult learning and education would be particularly suited for addressing democratic deficit and for providing the readiness and understanding needed to be an active citizen. A general impression from the so-called “Brexit plebiscite”Footnote 17 is that there was a lack of access to robust information about the issues concerned, making it difficult for people to educate themselves about the likely consequences of voting yes or no.

This is very different to what happened when Sweden voted (in 1994) on joining the EU or (in 1980) on the abolishment of nuclear power stations. In these instances, there had been an intensive study circleFootnote 18 campaign to help people think through the options. It is interesting to note that there is some research evidence from the Nordic countries on the role that popular adult education can play in developing citizens’ capacity for democratic deliberation (see e.g. Laginder et al. 2013). This is just one example from the knowledge base that is available when addressing how to identify the key areas of adult learning and education that need to be included in future surveys and analyses of outcomes. Up to now, the OECD has provided important intellectual work in connection with its Skills Strategy and the development of PIAAC, but as discussed above, the OECD almost exclusively focuses on one aspect, namely the economy. The same intellectual and practical effort is now needed to address broader outcomes of adult learning and education, and perhaps this is where the UNESCO Institute of Lifelong Learning could take on a leadership role.